Epidemiologists’ Example: 3 Lessons For Insights Professionals
By Andrew Grenville, Chief Research Officer | April 21, 2020
Epidemiologists are in the spotlight these days. After toiling away in obscurity for many years, they are suddenly revealed as the public health heroes they are. Their analyses and recommendations are saving millions of lives. As insights professionals, there is much we can learn from their efforts and approaches.
Epidemiology and the insights industry come from a common root and are more similar than might be immediately apparent. Epidemiologists wrangle data and produce predictive and descriptive models. Methods of surveying and sampling are the same (through epidemiologists are more careful with their methods of sampling—sample quality being the Achilles heel of the insights industry). Ultimately, they make recommendations based on their analyses and insights. And we count on them to make excellent and actionable recommendations—because it is now acutely apparent just how high the stakes can be.
Here are three key lessons from epidemiologists’ thoughtful approach to their work:
Be cautious of the limitations of your data, but don’t be afraid to be boldly prescriptive.
Learn from the past and from other markets—even imperfect comparisons can be incredibly instructive.
Don’t be overly reliant on one data source and don’t take results at face value—look for what might be missing, or mismeasured.
Be cautious but bold
Epidemiologists know their data is never perfect, and their papers are suitably full of qualifiers and disclaimers. But that does not stop them from make bold recommendations—recommendations they know will be unpopular.
In a paper that prodded a reluctant US and UK to close workplaces and schools and to embrace social distancing, Neil Ferguson and colleagues produce models which indicate “in an unmitigated epidemic, we would predict approximately 510,000 deaths in GB and 2.2 million in the US, not accounting for the potential negative effects of health systems being overwhelmed on mortality.”
After this sobering statistic, they show models of how alternative interventions could dramatically reduce the death rates. Their recommendation is bold and crystal clear: “…epidemic suppression is the only viable strategy at the current time.” At the same time, they admit, “The social and economic effects of the measures which are needed to achieve this policy goal will be profound.”
They also qualify that the outcome of their recommendation is far from certain. “However, there are very large uncertainties around the transmission of this virus, the likely effectiveness of different policies and the extent to which the population spontaneously adopts risk reducing behaviors…. Future decisions on when and for how long to relax policies will need to be informed by ongoing surveillance.” This admission of uncertainty fits with statistician George Box’s observation that “All models are wrong, but some are useful.”
Ferguson and colleagues acknowledge that their unambiguous recommendation—which has profound implications for the world economy—is far from being guaranteed to be effective and is subject to revision. But they still stand behind it as the best course of action, because they know how many lives are at stake. This is a great example of a recommendation that is both suitably cautious and bold.
Learn from comparisons—past and present
Insights professionals often assume their market is unique and without precedent. Its not uncommon to hear researchers say, “We can’t learn from country X, because their system is so different.” Or “The other division’s customers are really distinct, there is not much we can discover from them.”
It’s easy to fall prey to what Freud called the “narcissism of small differences”—a tendency to exaggerate and be bothered by relatively minor variations. But epidemiologists battling a pandemic don’t have that luxury.
Even though each country, and regions within countries, have a different spread history, varying testing rates, and diverse rules, epidemiologists are drawing data from everywhere to inform their models and recommendations. The inputs are imperfect, but they are informative.
Epidemiologists are not just looking to other countries for insight, they are also looking to the past. A 2007 paper by Howard Markel and others with the boring title “Nonpharmaceutical Interventions Implemented by US Cities During the 1918-1919 Influenza Pandemic” is suddenly in hot demand. Its analysis of how different cities handled interventions—including closing workplaces and schools and encouraging social distancing—concluded that there is “…a strong association between early, sustained, and layered application of nonpharmaceutical interventions and mitigating the consequences of the 1918-1919 influenza pandemic in the United States.”
While there are differences between the 1918 influenza pandemic and what we face today, epidemiologists had already looked to history for lessons that could inform lawmakers today. Sometimes you must look backward to move forward.
Don’t assume the number is the answer
When we get the results of a survey, we often take it at face value. We don’t necessarily think too much about how considering an additional source might reveal that we are missing part of the picture.
You’d think death rates would be a pretty reliable indicator of, well, the death rate. But when it comes to counting how many people are being downed by COVID-19, it is important to look beyond that number and consider other data points.
While some places only count deaths with positive tests for COVID-19, other regions are including presumptive cases. That makes a big difference, as New York City recently demonstrated. But beyond that, when trying to calculate the full impact of a disease, epidemiologists look to “excess deaths,” which is the rate of death beyond the average for that time of year. These would be deaths would not appear in official COVID-19 death counts but would likely be due to it.
Data coming out of Europe suggests that when you look at excess deaths, the number of deaths due to COVID-19 may be double what the official death rate indicates. This underscores the value of looking at multiple data sources, and not taking a number at face value.
Learning from unsung heroes
Epidemiologists might seem unlikely heroes, but the value of their methods are now abundantly clear. Through their work we can see the importance of being cautious but bold, of learning from other places and times, and of not taking numbers at face value. Next time you meet an epidemiologist, thank them.