Skip to main content.

What’s the true toll of the epidemic? Here’s how we estimated it for Florida.

What’s the true toll of the epidemic? Here’s how we estimated it for Florida.

Picture of Kathleen McGrory
(Photo by Dan Mullan/Getty Images)
(Photo by Dan Mullan/Getty Images)

In recent weeks, The Washington Post, The New York Times and FiveThirtyEight have reported that the true toll of the coronavirus epidemic is much higher than official death count. 

Curious what’s happening in your state? So were we.

I write for The Tampa Bay Times. Here in Florida, the death count has been controversial and closely watched. The state Department of Health maintains the official tally. But the state’s medical examiners, who must certify all COVID-19 deaths under Florida law, have their own count. In April, a colleague and I reported that the two counts were different, with the official tally missing some cases. Then the state stopped releasing the medical examiners’ data.

We knew the discrepancy explained only part of the undercount. Experts were telling us that a number of COVID-19 deaths had almost certainly been misclassified in March and April. They also believed some deaths from other causes happened because people were scared to seek care.

Reporting in The Washington Post, based on a Yale-led analysis of federal data, suggested the undercount in Florida was small. Still, we were curious what the data could tell us about the epidemic in our state.

Two of my colleagues started by requesting monthly mortality data from Florida’s Bureau of Vital Statistics. Around the same time, I connected with a health researcher at the University of South Florida who was looking at the weekly death counts published by the U.S. Centers for Disease Control and Prevention. I asked if he would be open to a collaboration. He was.

Our goal was to compare the total number of deaths in Florida to what we would expect in a normal year based on the last few years of data. Public health experts call this an excess death analysis. We looked at deaths from all causes, not just COVID-19, because we knew that some epidemic-related deaths had been attributed to other things. 

We used the federal data instead of the state data, partly because the former included a field for COVID-19 deaths. We did, however, look at both datasets to make sure there were no inconsistencies.

Troy Quast, the USF researcher, used two different methods to calculate the number of expected deaths for each week of the year. First, he took the total number of deaths for each week dating back six years and adjusted for population growth using U.S. Census figures. Then, he calculated the average. 

He also used a more conservative measure preferred by the CDC that involved calculating a 95% confidence interval, or a range of values that had a 95% probability of containing the true value. We used the highest number in the range as our more conservative estimate for the expected number of deaths. (The approach is similar to the one used in this JAMA article that estimated excess deaths in Puerto Rico after Hurricane Maria in 2017. I wouldn’t have been comfortable doing this calculation on my own, so I was glad to have an expert spearheading the work.)

We took our estimates of expected deaths and subtracted out the actual number of deaths for that week. We presented “excess deaths” as a range between the two figures. Then, we subtracted the number of COVID deaths to show how many could not be explained by the official death count. 

By some measures, we found hundreds more deaths in Florida than the official count. It wasn’t the thousands that researchers and journalists identified in coronavirus hotspots like New York. But given the smaller size of the epidemic in Florida, it felt meaningful. 

The analysis got more interesting when we dug deeper. We repeated the exercise for two subsets of the data: deaths by natural causes (i.e. infection and disease) and non-natural causes (i.e. accidents, homicides and suicides). When we looked specifically at deaths by natural causes, we found even more deaths than expected. The analysis suggested the true count of epidemic-related deaths could be between 17% and 58% higher than the published death figures. 

Our analysis of non-natural causes, on the other hand, found fewer deaths in 2020 than expected for a normal year. That aligned with anecdotes from medical examiners, who said they were seeing fewer motor-vehicle deaths and homicides this year than in the past.

We made charts for the story using Datawrapper, a data visualization program that is free and easy to use. We considered publishing bar charts that would show the difference between the expected and actual deaths, but ultimately chose line graphs that relayed more information. We plotted the confidence interval, the mean number of deaths over the last six years and the actual number of deaths in 2020. We also showed the COVID-19 death count.

The analysis could be replicated for any state, though I would recommend working with a public health researcher or health economist to calculate the confidence interval. Look for someone who both understands the theory and has experience crunching numbers. When you make your initial approach, explain what hope to accomplish from the collaboration and provide examples of your work. Be sure to explain your process and anticipated timeline. Most news outlets work faster than academics. 

I would also recommend showing the methodology and findings to as many experts as possible before publishing, even if you take the simpler approach. The more eyes, the better.

There’s reason to be cautious. In late May, a Reddit post claimed there had been thousands more pneumonia deaths in Florida than usual. Several high-profile politicians jumped on the theory, using it as evidence that health officials had been dishonest about the coronavirus death toll. But it was unclear what the numbers in the post were describing. And top experts in morbidity statistics told my colleague Zack Sampson the theory was not supported by facts

Jeffrey Howard, an assistant professor at the University of Texas at Austin, made an important point: “People who aren’t trained and don’t have experience with assessing these data from the (U.S. Centers for Disease Control and Prevention) can easily come away with very inaccurate conclusions.”

Comments

Picture of

I just read your article from start to finish, and there are no charts or graphs. So where are they? I would love to see your data and calculations have exactly how you came to your conclusions, including charts and graphs, etc.

Picture of

yes why are there no charts ?

Picture of

The basics aside, Floridas death rate (pre-July) is vastly lower than the USA as a whole, Germany, Italy, France, UK, the healthiest states, etc.
Even if one increases the mortality to 2.5% to allow for undercount - how can that be when states with vastly healthier populations and better health care access and younger average populations are often double that?

Ask your data and stats people what they tend to think when something falls far outside an expected range? for example, we know Pat Buchanan didn't get 3K votes in Palm Beach County in 2000- the odds were 1 in 2 BILLION against that. So if we accept such things on their face, we pretty much have to be mistaken.
Ohio has a 6% death rate. If Florida well over twice as resistant to COVID?

I know this is dynamic data and more to come - still, when something seems not to fit the other known data (Florida poverty, lack of health care access, older population, etc.), it's always of some concern.

Picture of

Thank God for yet another publication that tells us to fear the worst and dont believe the "other" side. In it's race to gain readers and ultimately ad space, media outlets race to spit out any number that will make a person stay tuned.

How many journalists do you think there are in the usa? Does a journalist tell the truth first or pay his or her mortgage first? It's far easier to write a word than turn a wrench but how do you get paid for it? Sell fear.

Picture of

One graph with no legend or sources Epidemic Related Deaths? suicides Drug overdoses Have you considered GIGO (garbage in: garbage out) in your analysis. I.e. MEs miscategorizing deaths as Covid Related. Tests that give are positive for all forms of Covid related viruses when determining COD.?CDC guidance that counts any death as Covid related so long as deceased tests positive, motorcycle deaths etc etc. even grading on a curve C- for this article.

Picture of

Start by comparing grand total Florida deaths from all causes for January through June 2020 to grand total Florida deaths from all causes for January through June 2019 (and prior years). This should be quite easy...and revealing.

Picture of

Where are your charts and the comparative results

Picture of

Statistically, 550 people a day die in Florida of all causes. If the writer has the data for daily death toll for 2020 all she had to do was post it. Instead, she fills the page with double talk and percentages.

Picture of

I completed an analysis of Florida morbidity to identify excess deaths in a very similar manner. However, I went further down into an age demographic analysis so in addition to seeing in what age group the excess deaths are expected. My data findings were very encouraging and I've concluded the excess death rate is about 6% higher than what is expected and the affected population is the 60+ age band. It's time to stop this fear conspiracy and let America get back to work.

Picture of

From all the data I find in the previous 3 years January - September FL deaths are between 151K - 155K seems we are on that same target. Can't explain NY and their flood of deaths earlier this year, but I'm pretty sure that Covid 19 is not the whole blame. It is just so wide spread that it was a common factor and an easy connection in the dead, lots of people are testing positive and have absolutely zero symptoms. This should be top most for scientist, my guess is we are chasing the wrong bug!

Leave A Comment

Announcements

A global pandemic, a national reckoning with racism, botched school reopenings and leadership vacuums — it's not an easy moment to be starting out as a journalist. Join us as we hear from three youth journalists from around the country as they discuss the massive challenges confronting their generation. Sign-up here

Ready to take your journalism to a new level by honing your data analysis and visualization skills?  We're offering our highly acclaimed annual Data Fellowship through Zoom from Nov. 30-Dec. 4.

Do you have a great idea for a potentially impactful reporting project on a health challenge in California?  Our 2020 Impact Fund can provide financial support and six months of mentoring.

CONNECT WITH THE COMMUNITY

Follow Us

Facebook


Twitter

CHJ Icon
ReportingHealth