New Cochrane review assesses how accurate antibody tests are for detecting COVID-19

Today Cochrane, a global independent organization that reviews evidence from research to inform health decision-making, publishes a review of studies looking at the accuracy of COVID-19 antibody tests.

The review shows that antibody tests could have a useful role in detecting if someone has had COVID-19, but that timing is important. The tests were better at detecting COVID-19 in people two or more weeks after their symptoms started, but we do not know how well they work more than five weeks after symptoms started. We do not know if this is true for people who have milder disease or no symptoms, because the studies in the review were mainly done in people who were in hospital. In time, we will learn whether having previously had COVID-19 provides individuals with immunity to future infection.

Antibody tests are an important public health tool to identify individuals with previous COVID-19 disease. This enables assessment of the spread of infection and the need for public health interventions. The review summarizes research evidence available up until the end of April 2020 to see whether antibody tests:

are accurate enough to diagnose disease in people with or without symptoms of COVID-19, and
can be used to find out if someone has already had COVID-19.

The immune system of people who have COVID-19 responds by developing proteins in the blood called antibodies that attack the virus. Detecting antibodies in people’s blood may indicate whether they currently have COVID-19 or have had it previously. Whilst detecting current infection is usually done using swab tests within the first 5 days of illness, they may miss infection and are not available to all.

Cochrane researchers from universities across the world led by experts from the University of Birmingham searched through the 11,000 publications on COVID-19 available at the end of April to find studies that reported results of antibody tests in groups of people known to have (or have had) COVID-19 and others known not to have had COVID-19 based. They found a total of 54 relevant studies reporting test results for nearly 16,000 samples. The majority of studies were from China and were carried out in people who had been admitted to hospital and likely to have had severe disease.

The studies looked at three types of antibody, IgA, IgG and IgM. Most tests measured both IgG and IgM, but some measured a single antibody or combinations of the three antibodies. Data were only available for 27 tests, a small fraction of the over 200 tests on the market. Data were available on both laboratory based tests, which require blood samples taken from the veins, and point-of-care tests, which can use finger-prick blood samples. There were not enough data to compare the accuracy of different tests. The authors will continue to update this review over the next few months to provide a more complete summary of the research evidence as it accumulates.

The researchers found that the sensitivity (the proportion of the people who have had COVID-19 that the test can detect) of antibody testing is very closely related to when the test is performed. Tests of the IgG and IgM antibodies at 8 to 14 days after onset of symptoms correctly identified only 70% of people who had COVID-19. However, when the researchers looked at data reported at between 15 and 35 days after symptoms first began, antibody tests accurately detected over 90% of people with COVID-19. There are insufficient studies to estimate the sensitivity of antibody tests beyond 35 days after the beginning of symptoms. The tests only wrongly diagnosed COVID-19 in 1% to 2% of people without COVID-19.

To illustrate what these accuracy figures mean, in a sample of 1000 people where 200 people (20%) really have COVID-19, typical of workers in a hospital setting where COVID-19 patients have been treated:

193 people would receive a positive test result but 10 (5%) of those people would not have COVID-19 (known as a false positive result);
807 people would receive a negative test result but 17 (2%) of those people would have COVID-19 (known as false negative result).

In a population where COVID-19 was more common there would be more false negatives and fewer false positives.

Studies showed that antibody tests may have a role in diagnosing COVID-19 in patients who have had COVID-19 symptoms for two or more weeks but who have not had a swab (PCR) test or tested negative despite COVID-19-like symptoms.

Cochrane Rapid Review

Professor Jon Deeks, Professor of Biostatistics and head of the Test Evaluation Research Group at the University of Birmingham, explains: “We’ve analyzed all available data from around the globe - discovering clear patterns telling us that timing is vital in using these tests. Use them at the wrong time and they don’t work. While these first COVID-19 antibody tests show potential, particularly when used two or three weeks after the onset of symptoms, the data are nearly all from hospitalized patients, so we don’t really know how accurately they identify COVID-19 in people with mild or no symptoms, or tested more than five weeks after symptoms started.

The researchers also had several concerns about the quality of the studies they found. Studies were small and did not report their results fully. Many papers included multiple samples from the same patients. More than half of the studies were made available before they had been through peer review (publications known as ‘preprints’). In one important UK study the biomarker manufacturers did not approve the identification of the tests that had been evaluated.

Dr Jac Dinnes, who worked on the review with the University of Birmingham team commented, “The design, execution and reporting of studies of the accuracy of COVID-19 tests requires considerable improvement. Studies must report data broken down by time since onset of symptoms. Action is needed to ensure that all results of test evaluations are available in the public domain to prevent selective reporting. This is a fast-moving field and we plan to update this review regularly as more studies are published.”