The Unknown Unknowns: Diagnosing the New Coronavirus

As countries across the world struggle to contain the spread of the novel coronavirus, rapid and accurate diagnosis has rarely been so important.

Or so controversial. 

Over the past few weeks, diagnostic tests for the new coronavirus have been developed, authorized and distributed at an unprecedented speed. But questions about the reliability and accuracy of diagnostic data have only multiplied:

• In China, changes in diagnostic protocol confounded global health experts and spurred fears of information suppression. 
• In the UK, National Health Service doctors have described the government’s testing protocol  “as leaky as a sieve.” 
• Manufacturing flaws in a CDC test granted rapid emergency approval by the FDA have attracted harsh criticism in the US. 
• Initial relief at low numbers of infections in sub-Saharan Africa have quickly turned to fear that the “hidden” virus may already be spreading there undetected.  

Media criticism is adamant: There are not enough tests, and they aren’t good or fast enough. But perhaps the issue is different. Perhaps our expectations of diagnostic tests are too high.

No diagnostic test ever provides complete certainty. Results exist within a margin of error, expressed as the percentage of true positive cases that are detected (sensitivity), the percentage of true negative results that are detected (specificity), and the lowest detectable concentration at which 95% of all true positive cases are detected (limit of detection). The notion of a “confirmed test” is a myth; the veracity of diagnosis is always an approximation, reflecting the degree of uncertainty public health authorities, clinicians and, in the case of an emergency, lawmakers, are willing to accept. 

But in the midst of a rapidly evolving emergency like COVID-19, when fears and anxieties thrive, we turn to diagnostic tests to provide some degree of assurance. Yet what we expect from diagnostic tests is often unreasonable. 

In an emergency scenario, routine diagnostic uncertainty no longer exists within a threshold but rather becomes the province of “unknown unknowns.” We don’t know what we don’t know, as Donald Rumsfeld would say.

The reduced evidentiary requirements in accelerated regulation mean margins of error remain unclear. For example, how soon after infection a biological sample is taken and from which part of the body appear to impact on the accuracy of new COVID-19 tests in as yet unverifiable ways. In some cases infective patients have tested negative multiple times before receiving a diagnosis, including a woman in San Antonio who was released from quarantine by the CDC after two negative tests, only to later test positive for the virus.

Even when a test performs well in a laboratory, new uncertainties emerge as soon as the test enters the field. Diagnostic tests are highly sensitive to the way they are used—functionality often requires rigid testing protocols and presume a level of contextual consistency. Operational limitations in under-resourced health systems escalate the challenges. Here, the dizzying number of scenarios that can impact diagnostic accuracy—from tests left on the hot tarmac while awaiting customs declarations to electricity outages and impassable roads—suggests that a standard margin of error can no longer be presumed. 

Accelerated diagnostic development has focused on stand-alone tests, but even self-contained, highly portable tests depend on supply chains, some form of training, and proper integration into health system infrastructures.  Without a fully functional multi-tier diagnostic system, stand-alone tests have only limited utility in an outbreak—limitations that were thrown into stark relief during the Ebola outbreak.  

What we want is a test that can be immediately scaled up without manufacturing quality being compromised, a test that can provide 100% accuracy, a test that can be used anywhere and by anyone. But it is important that authorities do not portray novel diagnostics as a panacea. To do so raises expectations of diagnostic certainty that are impossible to meet—as when patients who are discharged are later found to be infected—and potentially erode faith in the health system and government response. 

Alice Street, PhD, is a senior lecturer in Social Anthropology at the University of Edinburgh. Ann H. Kelly, PhD, is an associate professor in Global Health at King’s College London.

Street and Kelly are supported by, DIADEV, a project funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme under grant agreement No 715450.

- - -

For the latest, most reliable COVID-19 insights from some of the world’s most respected global health experts, see Global Health NOW’s COVID-19 Expert Reality Check.

Join the tens of thousands of subscribers who rely on Global Health NOW summaries and exclusive articles for the latest public health news. Sign up for our free weekday newsletter, and please share the link with friends and colleagues: https://www.globalhealthnow.org/subscribe