The Difference Between Big Data and Smart Data in Healthcare

“Physicians are baffled by what feels like the ‘physician data paradox,’” Slavitt said earlier this spring.

“They are overloaded on data entry and yet rampantly under-informed. And physicians don’t understand why their computer at work doesn’t allow them to track what happens when they refer a patient to a specialist when their computer at home connects them everywhere.”

Spotty health information exchange and insufficient workflow integration are two of the major concerns when it comes to accessing the right data at the right time within the EHR.

A new survey from Quest Diagnostics and Inovalon found that 65 percent of providers do not have the ability to view and utilize all the patient data they need during an encounter, and only 36 percent are satisfied with the limited abilities they have to integrate big data from external sources into their daily routines.

On the surface, it appears that more data sharing should be the solution.  If everyone across the entire care continuum allows every one of its partners to view all its data, shouldn’t providers feel more equipped to make informed decisions about the next steps for their patients?

Yes and no.  As the vast majority of providers have already learned to their cost, more data isn’t always better data – and big data isn’t always smart data.  Even when providers have access to health information exchange, the data that comes through the pipes isn’t always very organized, or may not be in a format they can easily use.

“We’re going through very profound business model changes in healthcare right now, and providers are targeting  processes that will help them with the transition from volume to value.”

Scanning through endless PDFs recounting ten-year-old blood tests and x-rays for long-healed fractures won’t necessarily help a primary care provider diagnose a patient’s stomach ailment or figure out why they are reacting negatively to a certain medication.

Actionable insights are the key to using big data analytics effectively, yet they are as rare and elusive as a patient who always takes all her medications on time and never misses a physical.


“Big data” is one of those terms that gets thrown about the healthcare industry – and plenty of other industries – without much of a consensus as to what it means.  Technically, big data is any pool of information that is compiled from more than a single source.

For healthcare organizations, this could mean creating a database that takes patient names and addresses from one system and matching it up with scheduled appointments from another, or integrating claims data with clinical notes from the EHR.

Stitching multiple sources of information together into a centralized databank accessed by reporting or a query system can provide a more in-depth and actionable snapshot of a patient’s history, diagnoses, treatments, socioeconomic challenges, and risk profiles.

But leveraging these disparate data sources requires the right tools and competencies, which aren’t always easy to develop.

Electronic health records are starting to take big data analytics seriously by offering healthcare organizations new population health management and risk stratification options, but many providers still turn to specialized analytics packages to find, aggregate, standardize, analyze, and deliver data to the point of care in an intuitive and meaningful format.

These tools may include quality benchmarking and performance measurement systems, clinical analytics algorithms that monitor patients in real-time, revenue cycle and claims analytics, and population health management packages that foster engagement, deliver alerts and reminders, stratify beneficiaries, or gauge risk of a certain disease.

In addition to the right technologies, providers must invest time and manpower into acquiring the competencies to make analytics work for them.  This includes crafting a dedicated team of experts to oversee big data projects, implement and optimize software, and convince clinicians that these new strategies are worth their while.


That last step is often the hardest.  Clinicians who are already disillusioned with their sub-par EHRs may not be eager to embrace yet another interface that will pour even more data down their throats without any demonstrable benefits.

Health IT users are rightly demanding that vendors step up their game when it comes to helping clinicians apply big data to decision-making, largely because those organizations that have been successful with their efforts tend to reap significant rewards.

In addition to cutting-edge innovations like the Precision Medicine Initiative, which relies wholly on massive collections of big data to tease out the genetic roots of cancer, diabetes, autism, and other conditions, providers are using big data to achieve a variety of everyday goals, including:

  • Cutting costs through improved care coordination founded on increased visibility of patient activities, reducing unnecessary service utilization, preventing the development of chronic disease, and allocating internal resources more efficiently
  • Meeting value-based reimbursement goals and accruing shared savings in accountable care organization (ACO) arrangements by reducing unnecessary utilization, using big data to flag opportunities for preventative care, and managing operational efficiencies
  • Improving the monitoring of patient activities outside the traditional care setting, including medication adherence management, chronic disease management support, and home-based monitoring and other interventions for patients with advanced needs
  • Integrating mental healthcare into the traditional clinical setting in an effort to address patient care more holistically, connect patients with services within their communities, and provide support for patients with socioeconomic challenges.



Many of these potential benefits for patients fall under the broad umbrella of population health management, a critical competency for providers who are facing a seismic shift in their reimbursement landscape.

Value-based reimbursement is based on the idea that providers are responsible for the overall well being and outcomes of a defined patient population.

“We need tools that fit into the workflow of the physician or the care manager or the nurse to help them make sure that they are on top of the people they’re responsible for.”

When these patients achieve better outcomes with less spending on the provider’s part, participating healthcare organizations often receive a financial bonus or share in the rewards of a savings pool.

“We’re going through very profound business model changes in healthcare right now, and obviously providers are targeting areas of their care processes that will help them with the transition from volume to value and with reducing costs,” said John Glaser, Senior Vice President of Population Health and Global Strategy at Cerner Corporation.

“We need tools that fit into the workflow of the physician or the care manager or the nurse to help them make sure that they are on top of the people they’re responsible for.”

Big data is the key to visualizing these trends, predicting areas of the greatest risk, and tracking the activities of patients and their providers to keep costs as low as possible.

“What we really need in order to improve outcomes is end-to-end visibility across the spectrum of care at a service line level,” said David Delaney, MD, Chief Medical Officer at SAP.

“As the healthcare system starts to realize the importance of using data analytics to develop this visibility into value, the marketplace will generate the tools required to innovate and make sure that every patient is receiving the best possible care along the right service lines.”

“That’s when big data will really start to line up with patient interests,” he added.  “Ultimately, patients want to get healthier and stay healthier in a safe and cost-effective fashion.  It’s the healthcare industry’s job to figure out how we can do that at scale for some very complex cases across the population.”


Designing and implementing a big data analytics program that can achieve some of the results mentioned above is a complicated and challenging undertaking.  It requires advanced planning, unanimous buy-in, an intelligent choice of vendors, and lots of patience.

But most providers haven’t started their population health management or predictive analytics activates from scratch.  They have moved slowly into building new infrastructure, one system at a time, piling their latest acquisitions on top of a teetering tower of legacy software and existing workflows.

“Despite the CIOs drowning in data, [end users are] starving for contextual information to help them make better decisions doing their job.”

Not only does this cause the technical interoperability problems that make health information exchange so frustrating, but it also makes it difficult to know what data is available, what format it’s in, how to synthesize old and new information, and how to architect a sensible way for end-users to interact with big data tools.

“We’ll often talk to CIOs, and they might say that they have a petabyte of information,” said Delaney in 2014.  “But when you drill down a little further, it turns out that in terms of unique information, they might only have about 200 terabytes.  And that bloats up to a petabyte because they copy all that data into a staging area, and then they copy that into data warehouse, and then they create data marts on top of that.”

“The other side of the coin is with the end users: the clinicians and the administrators.  Despite the CIOs drowning in data, they’re starving for contextual information to help them make better decisions doing their job.

“And when they do manage to get information from their analytics, most dashboards and reports are backward looking.  They can tell you what happened last quarter or last month, but they really aren’t driving real-time decision making.”

To achieve that goal, providers must invest in big data analytics infrastructure that relies on common data standards, such as HL7, and gives users access to analytics generated from real-time data sources, such as directly from the EHR.

A recent study from the University of Texas Southwestern suggests that achieving those goals may not be as difficult as it seems.  Targeting the right data, instead of trying to combine all the available data, can give clinicians just as much insight into a critical piece of the care puzzle.

The study found that providers could predict 30-day readmissions to the hospital with only a few elements of information from the first 24-hours of a patient’s hospital stay, and that using data from a longer period of time had little measurable effect on the accuracy of the analytics.

“Our group’s previous research found that using clinical data from the first day of admission was more effective in predicting hospital readmissions than using administrative billing data,” said lead author Dr. Oanh Nguyen, Assistant Professor of Internal Medicine and Clinical Sciences at UT Southwestern.

“We expected that adding even more detailed clinical data from the entire hospitalization would allow us to better identify which patients are at highest risk for readmission. However, we were surprised to find that this was not the case.”

The research indicates that providers may need to target and tailor their big data analytics efforts instead of taking a scattershot approach to hitting every possible element in their possession.

“Raw data alone cannot lead to systematic improvement,” said the National Quality Forum in a white paper.  “It has to be turned into meaningful information.  Institutional leadership and culture have to support improvement efforts, and clinicians and healthcare staff need the skills to analyze and apply data.”

The NQF suggests that providers start to cultivate smarter data by defining their goals and use cases before investing in technologies, assessing their available information and its integrity, identifying systemic challenges that may present roadblocks, leveraging existing resources to build analytics competencies, and taking into consideration the needs, preferences, and frustrations of end-users when designing interfaces.

Healthcare organizations should also work with their business partners, including other providers, payers, and public entities, to increase data transparency, foster interoperability, and share best practices for developing meaningful insights.

Organizational policies and frameworks should focus on cultivating a culture of improvement that relies on smart data for the delivery of evidence-based care.  This strong foundation for utilizing big data analytics will help providers achieve the cost reductions and quality improvements that are so important for success in a rapidly changing delivery environment.



Leave Your Comments

Your email address will not be published. Required fields are marked *