So if some countries only test patients ill enough to go to hospital – and don’t test the less-ill (or even asymptomatic) Covid-19 patients who don’t get to hospital (which is what the UK is currently doing) – the death rate can appear higher than in countries where testing is widespread (such as Germany or South Korea).

Impact of testing

Even if you’re careful to compare the same type of fatality rate across countries, it’s easy to see how testing more, or fewer, people would change the results.

In fact, the lack of widespread, systematic testing in most countries is the main source of discrepancies in death rates internationally, says Dietrich Rothenbacher, director of the Institute of Epidemiology and Medical Biometry at the University of Ulm in Germany.

As a result, the current figures are “not at all” directly comparable between countries, he says. This is because, to get an accurate figure across a population, it is necessary to test not just symptomatic cases, but asymptomatic people too. Having that data would give an accurate picture of how the pandemic is affecting whole populations, not just the sick.

“Currently we have a huge bias in the numbers coming from different countries – therefore the data are not directly comparable,” he says. “What we need to really have valid and comparable numbers would be a defined and systematic way to choose a representative sampling frame.”

The village of Vò in northern Italy is an example of why testing is important not just to get accurate data, but to contain Covid-19. When the first Covid-19 case in Vò was confirmed, testing was rolled out to the entire village of 3,300 people. The results showed that at the time of the “first case”, 3% of the village was already infected, but showing no or few symptoms.

A widespread testing programme in Iceland shows a similar picture. Iceland has tested more than 3% of its population of around 365,000 people so far, both those who show symptoms and those who don’t. By extrapolating the results, the testing programme estimated that 0.5% of Iceland’s population was likely to have had Covid-19. But even this figure may be slightly low because asymptomatic people are less likely to seek out testing, notes Heneghan. It is thought that the actual figure could be closer to 1% of Iceland’s population, which would mean roughly 3,650 infections.

An added difficulty is that this data is not from peer-reviewed research, but rather is almost real-time clinical data – which can be messy and come with many caveats. What figures like these do underline, says Sheila Bird of the University of Cambridge’s MRC Biostatistics Unit, is the importance of widespread testing to help inform public health measures.

“If you never actually develop symptoms but had encountered the virus, that would be an infection, but it is an ‘uncountable’ one – uncountable until the time that we have an antibody test,” says Bird.

Antibody tests detect the traces of an immune response to the virus and reveal who has had an infection. Those tests are the game-changers that could reveal who has developed immunity to the virus and can safely return to daily life without risk of infection or of spreading the virus. “That’s why the development of that test and its deployment is so terribly important,” says Bird.

In Vò, the spread of Covid-19 was halted after two weeks, as both the widespread testing and strict follow-up measures allowed for the targeted and effective containment of infections. Iceland, so far, has had only two Covid-19 deaths.

What counts as a Covid-19 death?

There are other factors that alter the death rate, too.

One of them is what doctors actually count as a Covid-19 death. At first it might seem simple enough: if a patient dies while infected with Covid-19, they died of Covid-19.

But what if they had an underlying condition, such as asthma, which was exacerbated by Covid-19? Or what if the patient died from something seemingly less related to Covid-19, which is a respiratory disease – such as, say, a brain aneurysm? Which condition should be considered the cause of death?

Even within a country, official statistics can vary according to what you count. In the UK, for example, the Department of Health and Social Care releases daily updates on how many people who tested positive for Covid-19 died that day. This includes any patient who tested positive for Covid-19 but who might have died from another condition (for example, terminal cancer). But the UK’s Office for National Statistics counts all deaths as Covid-19 where Covid-19 was mentioned on the death certificate, regardless of whether they were tested or if it was merely a suspected case of Covid-19. Adding to the complexity of trying to understand the death rates is that the two are out of sync, since the ONS way of counting can only happen after a death certificate has been issued, so takes longer.

“The issue is not really about right or wrong, but about each source of data having its own strengths and weaknesses,” Sarah Caul, head of mortality analysis at ONS, writes in a blog post on the different ways of counting deaths.

This is not necessarily a source of discrepancy between most countries, though, as many are counting deaths in the same way. Italy counts any death of a patient who has Covid-19 as a death caused by Covid-19; so does Germany and Hong Kong.

In the US, doctors have more discretion: they are asked to record whether the patient died “as a result of this illness” when reporting Covid-19 deaths to the Centers for Disease Control and Prevention. It could be easy to see how a physician might believe that a Covid-19 patient who died of, say, a heart attack or brain aneurysm didn’t die as a result of Covid-19, and so wouldn’t report accordingly.

Importantly, though, while this might make a difference when the data is analysed months or years from now, this doesn’t translate into any difference in the death statistics at the moment. At present in the US, any death of a Covid-19 patient, no matter what the physician believes to be the direct cause, is counted for public reporting as a Covid-19 death.

“I expect that the final death certificate will have Covid along with pre-existing conditions, should there be any,” says Cécile Viboud, an epidemiologist at the Fogarty International Center at the National Institutes of Health. “But at this point, any Covid-positive case who dies will be tallied in the US death count.”

So what counts as a Covid-19 death will affect our overall understanding of the lethality of the disease in the long run, but it most likely isn’t playing a huge factor between countries.

Complex causes

The picture is murkier still when patients have not had a Covid-19 test, but are a suspected case. Given that many deaths from Covid-19 are in people who have underlying health issues, doctors still have to make the call on the cause of death.

During an epidemic, doctors are more likely to attribute a death with complex causes as being caused by the disease in question – a trait known as ascertainment bias.

“We know, during an epidemic, people will call every death as though it’s related to Covid-19. But that is not the case,” says Heneghan. “Always, when people look back at the case notes and assign causation, they realise they will have overestimated the case fatality in relation to the disease.”

The reason for the bias is that “there’s a tendency to focus on the worst-case scenario”, says Heneghan. “That’s the only message that gets out there.”

One example is the H1N1 pandemic of 2009, known as swine flu. Early case fatality rate estimates were inflated by a factor of more than 10. Even 10 weeks into the epidemic, estimates varied widely between countries, coming in between 0.1% and 5.1%. When medics later had a chance to go through case documents and evaluate cases, the actual H1N1 case death rate was far lower, at 0.02%.

That’s not a cause for complacency, says Heneghan. But it could be an antidote to some of the alarm at the very high reported death rates in some countries.

Hidden deaths

While over-counting deaths among recorded Covid-19 cases may lead to overestimation of the death rate, there is another factor that could mean the death rate is also – confusingly – being underestimated at the same time.

This is the problem of hidden deaths from Covid-19: those people who die from the disease who are never tested. This comes into play when health services are overwhelmed and even those patients who have severe symptoms of the virus are not taken into hospital to be tested and treated, simply because there isn’t capacity.

In the small Italian town of Nembro, in Lombardy, only 31 people have officially died from Covid-19. But one preliminary study has found that it’s likely that far more people have died from Covid-19. That’s because the overall death rate – not just from Covid-19, but from all causes – was four times higher this year than in the same period last year. Ordinarily, around 35 people die in the first months of the year in Nembro. This year, 158 people were registered to have died.

This jump in excess deaths is speculated to be down to undiagnosed and untested cases of Covid-19.

The number of hospital beds available could also play a role, as countries with lower capacity in their health services may have to start making decisions sooner about which Covid-19 cases to prioritise for treatment. This could lead to more Covid-19 deaths in the community that go untested (and uncounted), as people with symptoms stay away.

While this might lead to more uncounted deaths, it may not necessarily translate into to more deaths overall. “There’s nothing to say that going into an intensive care unit is going to have a better outcome compared with care in the community,” says Heneghan.

More important than the number of beds available in hospitals, he says, is the way the beds are organised. If Covid-19 patients are in close proximity to other patients – or if doctors move between Covid-19 and non-Covid-19 wards – it heightens the risk of spreading the disease.

“That is why you need separate hospitals for separate infection teams,” he says.

What role does age play?

As well as differences caused by clinical definitions of what counts as a Covid-19 death and the numbers of people going untested, there are other factors that mean the virus hits some countries harder than others.

One that has been put forward by Italian doctors is the age profile of a country. In 2019, nearly a quarter of the Italian population was 65 years or older, compared to only 11% in China. The overall case fatality rate in Italy as of mid-March was 7.2% – much higher than China’s rate of 2.3% at a comparable stage of its epidemic. But from the ages of zero to 69, the two country’s case fatality rates are comparable, note researchers from the Istituto Superiore di Sanità in Rome.

Among the oldest patients, though, Italy and China part ways. The 70-79 age group in Italy had a case fatality rate of 12.8%, while China’s was 8%. For the over-80s, the difference was more pronounced still: Italy’s was 20.2% and China’s 14.8%. The reason for this discrepancy is still a bit of a mystery, the researchers note.

Heneghan suspects that one factor at play in Italy’s high figures could be not to do with the virus itself, but with bacteria. The country has the highest numbers of deaths due to antimicrobial resistance in the EU – in fact, a third of all EU deaths from antimicrobial resistance happen in Italy. While antibiotics do precisely nothing to tackle a virus, a viral infection can often open the way for secondary infections or complications like bacterial pneumonia. If that then can’t be treated properly with antibiotics because the bacteria is resistant, then this can be what kills the patient, not the virus itself.

“This is an incredibly important part of the whole story,” says Heneghan. “And it is particularly prevalent in the elderly.”

As well as age, the overall health of the population has been highlighted as another contributing factor – particularly given that people who have underlying health problems are more vulnerable. While that might play a contributing role, this doesn’t explain why some countries are reporting more Covid-19 deaths than others: Italy is consistently ranked as one of the world’s healthiest countries, for example, and has a longer healthy life expectancy than China.