First things first:
What is Big Data?
Best to give some examples to know what is Big Data. Because it is more than just a lot of information.
Some examples include- per Wikipedia: Examples include web logs, sensor networks, social networks, social data, Internet text and documents, Internet search indexing, call detail records, astronomy, atmospheric science, genomics, biogeochemical, biological, and other complex and often interdisciplinary scientific research, military surveillance, medical records, photography archives, video archives, and large-scale e-commerce.
Why are we seeing more info about Big Data now?
Internet. See some of the examples above- but basically, anything you do on the web can be tracked in a database. Any site you visit (especially in the same session) provides value to the sites. What browser do you use, where did you come from, what OS do you use and other metrics help websites understand their users. Additionally sites can perform A/B testing, for example: if you change the color of the "Buy Now" button, do you have more people click the button? What if it is on the right or left? What if you change the button to "Watch a video to learn more"? All of these things can and ARE tracked. Everything you put into a search engine can be researched.
What makes the web different?
Everything can be tracked. This is why Facebook has so much potential. All sorts of info is stored- and you volunteer all that info for FREE! Awesome for Facebook. Bad for you. Maybe. See, the web provides so much information not only about you, but people like you and not like you. It captures everything that is happening in real time that gives a lot of information about what is happening offline.
Well, this is good right?
Well, depends on which side you are on. This data can be used for good or evil. Do you want Facebook to know where you are right now? Well, if you have a smartphone and have their app, then they do. Maybe that isn't weird. But, they know where you live- because they can see where your phone is at night, for instance. They know what kind of site you click on at home, at work, at the beach, on vacation (when you click on links in their site). Maybe that pales in comparison if you use Siri on your iphone. That program sends all your conversations with Siri back to Apple. Oh, and they definitely know where you are at all times.
So, what does this have to do with HealthCare?
We now have the ability to capture so much more data about ourselves and each other. Let me tell you a story where I think we are going with Healthcare- some of this technology exists already- all of it could be done today. (technically speaking)
You go to bed at night and your phone can track your sleeping habits- when did you go to bed, did you wake up at night, when did you wake up in the morning etc... You go for a morning run, with your phone and your speed and distance is tracked and reported back to the web. You have breakfast and you take a picture of your meal, which is automatically uploaded to the web, where it tracks the meal, the caloric intake and nutritional value. (alternatively this could be automated when your credit/debit transactions are tracked and coded- remember that Big Mac the other night? Part of your medical record with no additional effort)
This morning you have a doctors appointment. When you arrive and check-in, the office has motion sensors that track your movement throughout the office. This is tracked to determine wait times, movements, nervous behaviors and reactions to internal stimulus. Does the waterfall help relieve stress in the office? In the meetings with the doctors and nurses, each conversation is recorded- transcribed and searchable. Attached to this visit are all the expected tests that you had that day, from weight, urine, xray, blood, and blood pressure. (all of these are recorded digitally)
Inside the medical office all of this information is available and searchable on a tablet by the providers. A manager can receive real-time clinical and financial metrics. The same motion sensors are able to track a physicians movements. Now the physician can get feedback about her movements in the clinic or hospital. Room assignments can be modified because now there is evidence that those few extra feet that the physician walks each day can be reduced. The result is less wait time and length of stay, higher physician productivity and more capacity for the clinic. Other benefits from the system include better hand-washing rates, which reduces hospital acquired infections and lower prescription drug errors.
Can you summarize please?
Yes. Two main points:
Everything we do in life affects our health. We are our bodies and that is what we should track. You want to cure cancer or find the causes of Alzheimer's? You need a lot more information. More than that, we need everything. All information. And we can now capture it.
In Healthcare, we need to do the same type of data capture- operationally. Capture as much data as we can. Make sense of the data. Make the data available. Open the gates and let the data run wild! This will make us so much better at treating illness and help us run a more sustaining system.
Is this data collection and analysis going to be used for good or evil?
Probably both, since that is human nature. I don't know what the solution is exactly, but it has to include a market to be successful. It will fail and be too expensive if it is managed centrally. It needs to look like the internet. It needs to be so chaotic and poorly designed that it allows a lot of people to come in and contribute their ideas and solutions. Out of that chaos something will come out, something beautiful and manageable. I wish I knew what that meant, because then I would solve the problem myself. But, while this data is so valuable- it has to be available. The only way to do that is have a somewhat chaotic and open system. This process fixes and perfects ideas and solutions, quickly. I know it seems counterintuitive- but it has been said a camel is a horse built by committee.
Steward Brand once told Steve Wozniak:
"On the one hand information wants to be expensive, because it's so valuable. The right information in the right place just changes your life. On the other hand, information wants to be free, because the cost of getting it out is getting lower and lower all the time. So you have these two fighting against each other."
Like I said at the beginning, this isn't a practical method of getting to point B with Big Data in Healthcare. But, imagine the possibilities. Imagine the benefits. Imagine the improvements. To get there we have to capture all sorts of information. It needs to be an open system. Somehow.