Modified post written to entrepreneurs by Eric Ries  "Beware of Vanity Metrics"

The only metrics that physicians should invest energy in collecting are those that help them make decisions. Unfortunately, the majority of data available in off-the-shelf analytics packages are what I call Vanity Metrics. They might make you feel good, but they don’t offer clear guidance for what to do. 

To avoid falling into this trap, I recommend you follow the three A's of metrics. All metrics should be: actionable, accessible, and auditable.
Actionable: 
When an employee sees a report about a specific metric, it's essential that they have some idea how to replicate the result in the report. 

Accessible: Most data warehousing systems provide reports that are too complicated to read and take too long to generate. As a result, many teams don't get any benefit out of them. In order for reports to be worthwhile, it's essential that:

• Everyone in the company understand how to read them
• Everyone in the company has easy access to the latest data

Additionally, to keep accessibility high, the feedback loop between taking an action and seeing the results should be as short as possible. In my experience, waiting for even a weekly round-up or having to ask a data warehousing expert to run a report creates too much friction. Key data should be available to any employee, anytime, in a matter of minutes. In order to achieve that goal, the reports themselves have to be extremely simple.

Auditable: The biggest benefit of actionable metrics is that they can be used to reap the biggest savings in all of product development, when they tell you that you don't need to do something. Metrics have to be credible to the people who drive the product vision, including company founders. Even if they understand a report, and know what caused it, that doesn't mean they will actually learn. Is the report accurate? Believe me, when it's your idea on the line, it's much easier to believe the report is the problem rather than the idea.

Thus, it's important that skeptics can audit a report. When possible, this should mean that report generation is simple. Hopefully it can be created with direct access to primary data. For example, a report about revenue that's generated directly from the master orders database is more credible than one that requires several intermediate steps. Most importantly, it should be possible to translate the summary numbers in the report back to the actual customers who generated them.

Remember, metrics are really reports on people. This is where most off-the-shelf metrics packages fail.

Response to a great blog post by Michael Planchart @TheEHRGuy Can see his original post on From Little Data to Big Data

Michael Planchart recently wrote about getting to Big Data and using all the data that is currently being collected to provide more actionable metrics- to improve performance- both clinical and financial. I want to take it a step further. Michael's post is very practical on how to get to point B. Mine is an unrealistic post- but are my thoughts on how healthcare could actually start to lead in technology.

First things first:

What is Big Data? 
Best to give some examples to know what is Big Data. Because it is more than just a lot of information.
Some examples include- per Wikipedia: Examples include web logs, sensor networks, social networks, social data, Internet text and documents, Internet search indexing, call detail records, astronomy, atmospheric science, genomics, biogeochemical, biological, and other complex and often interdisciplinary scientific research, military surveillance, medical records, photography archives, video archives, and large-scale e-commerce.


Why are we seeing more info about Big Data now? 
Internet. See some of the examples above- but basically, anything you do on the web can be tracked in a database. Any site you visit (especially in the same session) provides value to the sites. What browser do you use, where did you come from, what OS do you use and other metrics help websites understand their users. Additionally sites can perform A/B testing, for example: if you change the color of the "Buy Now" button, do you have more people click the button? What if it is on the right or left? What if you change the button to "Watch a video to learn more"? All of these things can and ARE tracked. Everything you put into a search engine can be researched. 


What makes the web different?
Everything can be tracked. This is why Facebook has so much potential. All sorts of info is stored- and you volunteer all that info for FREE! Awesome for Facebook. Bad for you. Maybe. See, the web provides so much information not only about you, but people like you and not like you. It captures everything that is happening in real time that gives a lot of information about what is happening offline.


Well, this is good right?
Well, depends on which side you are on. This data can be used for good or evil. Do you want Facebook to know where you are right now? Well, if you have a smartphone and have their app, then they do. Maybe that isn't weird. But, they know where you live- because they can see where your phone is at night, for instance. They know what kind of site you click on at home, at work, at the beach, on vacation (when you click on links in their site). Maybe that pales in comparison if you use Siri on your iphone. That program sends all your conversations with Siri back to Apple. Oh, and they definitely know where you are at all times. 


So, what does this have to do with HealthCare? 
We now have the ability to capture so much more data about ourselves and each other. Let me tell you a story where I think we are going with Healthcare- some of this technology exists already- all of it could be done today. (technically speaking)

You go to bed at night and your phone can track your sleeping habits- when did you go to bed, did you wake up at night, when did you wake up in the morning etc... You go for a morning run, with your phone and your speed and distance is tracked and reported back to the web. You have breakfast and you take a picture of your meal, which is automatically uploaded to the web, where it tracks the meal, the caloric intake and nutritional value. (alternatively this could be automated when your credit/debit transactions are tracked and coded- remember that Big Mac the other night? Part of your medical record with no additional effort)

This morning you have a doctors appointment. When you arrive and check-in, the office has motion sensors that track your movement throughout the office. This is tracked to determine wait times, movements, nervous behaviors and reactions to internal stimulus. Does the waterfall help relieve stress in the office? In the meetings with the doctors and nurses, each conversation is recorded- transcribed and searchable. Attached to this visit are all the expected tests that you had that day, from weight, urine, xray, blood, and blood pressure. (all of these are recorded digitally)

Inside the medical office all of this information is available and searchable on a tablet by the providers. A manager can receive real-time clinical and financial metrics. The same motion sensors are able to track a physicians movements. Now the physician can get feedback about her movements in the clinic or hospital. Room assignments can be modified because now there is evidence that those few extra feet that the physician walks each day can be reduced. The result is less wait time and length of stay, higher physician productivity and more capacity for the clinic. Other benefits from the system include better hand-washing rates, which reduces hospital acquired infections and lower prescription drug errors. 

Can you summarize please?
Yes. Two main points:
Everything we do in life affects our health. We are our bodies and that is what we should track. You want to cure cancer or find the causes of Alzheimer's? You need a lot more information. More than that, we need everything. All information. And we can now capture it.

In Healthcare, we need to do the same type of data capture- operationally. Capture as much data as we can. Make sense of the data. Make the data available. Open the gates and let the data run wild! This will make us so much better at treating illness and help us run a more sustaining system. 

Is this data collection and analysis going to be used for good or evil?
Probably both, since that is human nature. I don't know what the solution is exactly, but it has to include a market to be successful. It will fail and be too expensive if it is managed centrally. It needs to look like the internet. It needs to be so chaotic and poorly designed that it allows a lot of people to come in and contribute their ideas and solutions. Out of that chaos something will come out, something beautiful and manageable. I wish I knew what that meant, because then I would solve the problem myself. But, while this data is so valuable- it has to be available. The only way to do that is have a somewhat chaotic and open system. This process fixes and perfects ideas and solutions, quickly. I know it seems counterintuitive- but it has been said a camel is a horse built by committee. 

Steward Brand once told Steve Wozniak:
"On the one hand information wants to be expensive, because it's so valuable. The right information in the right place just changes your life. On the other hand, information wants to be free, because the cost of getting it out is getting lower and lower all the time. So you have these two fighting against each other."

Like I said at the beginning, this isn't a practical method of getting to point B with Big Data in Healthcare. But, imagine the possibilities. Imagine the benefits. Imagine the improvements. To get there we have to capture all sorts of information. It needs to be an open system. Somehow.

Here is the latest post on Diagnostic Imaging, from David Fuhriman

Mediocrity is ubiquitous. Whether we are shopping, working, driving, dining, or discussing phone business, we are frequently confronted by those who don’t know, don’t care, don’t know that they don’t care, or don’t care to know the difference. 

It’s not so much that they don’t like us or want us to get lost. It’s more like they just don’t get it. Or that they’re obtuse or too dense to fully understand how to do it or get it right (whatever “it” is).

It’s likely that we have encountered this in our personal lives and perhaps even considered it with certain of our referring or clinical colleagues. However we should all periodically self-evaluate to make sure others would not similarly characterize us.

It’s been nearly three decades since I finished my residency and though I’ve continuously maintained privileges at one institution, I have experienced various practice models including an independent private group, locums, teleradiology and most recently hospital employment. Regardless of our practice situations, we should always encourage each other to avoid the “M” word — mediocrity.

It is so easy to start sliding. We’ve just been on call so we are tired, not as sharp as usual and just want to finish and get home. We don’t want to take the time or expend the extra effort to be as compulsive as our patients and referring physicians deserve.

You know the drill. Abdomen/pelvis CT for abdomen pain. We go through our standard search pattern and discover an incidental pulmonary nodule. Since most of us read from a PACS work station, we expect comparison studies to automatically load onto the comparison monitor, and when we don’t see a comparison, we assume there is none.

However, we need to be smarter than our machines and remember that they cannot think and will not load the four-year-old HRCT at the bottom of the prior exam list. For any number of reasons, we do not look through that list, so now we cannot confirm stability. So instead of finding the “comparison” that confirms stability, we recommend a follow-up CT according to the Fleischner Society guidelines and feel like we’ve done our job. Sound familiar?

Or maybe you’ve seen a different scenario. PACS work stations are pervasive and well understood by virtually all of us. Likely we’ve all seen a difficult case remain on the work list for hours or until the next day since everyone seems to skip over it. Finally one of the rads that can always seem to get things done will pick it off and then the case gets read. Should not happen, but of course we know it does.

Or maybe it’s something not related to interpreting images, but more with style or lack of consideration. It’s been a busy day and we decide that we’ve had enough. So we ask our colleagues if it would be ok if we left early since we’re not feeling well, have an appointment, or are just tired. We put them in a tough spot since it’s hard to say no to an associate. Over coffee or in the lounge, we frequently talk about lack of personal responsibility across America, and yet do we ever find it creeping into our workplace? If everyone did what we are doing, would the entire group be better off?

We should always be vigilant, periodically self-examine our own habits, and be ready to take evasive action to avoid mediocrity.

Ken Keller, MD, FACR, serves as medical director of the Department of Radiology at Trinity Health in Minot, ND, where has been since an attending radiologist since starting practice in 1983. He is particularly interested in interventional and breast imaging.