Tag Archives: Rapid Tests

COVID Testing: Cycle Thresholds and Coffee Grounds

19 Saturday Dec 2020

Posted by in Coronavirus, Public Health, Uncategorized

≈ Leave a comment

Tags

, , , , , , , , , ,

Here’s some incredible data on PCR tests demonstrating a radically excessive lab practice that generates false positives. I’m almost tempted to say we’d do just as well using a thermometer and the coffee ground test. Open a coffee tin and take a sniff. Can you smell the distinct aroma of the grounds? If not, and if you have other common symptoms, there’s a decent chance you have an active COVID infection. That test is actually in use in some parts of the globe!

The data shown below on PCR tests are from the Rhode Island Department of a Health and the Rhode Island State Health Lab. They summarize over 5,000 positive COVID PCR tests (collected via deep nasal swabs) taken from late March through early July. The vertical axis in the chart measures the cycle threshold (Ct) value of each positive test. Ct is the number of times the RNA in a sample must be replicated before any COVID-19 (or COVID-like) RNA is detected. It might be from a live virus or perhaps a fragment of a dead virus. A positive test with a low Ct value indicates that the subject is likely infected with billions of live COVID-19 viruses, while a high Ct value indicates perhaps a handful or no live virus at all.

The range of red dots in the chart (< 28 Ct) indicates relatively low Ct values and active infections. The yellow range of dots, for which 28 < Ct <= 32, indicates possible infections, and the upper range of green dots, where Ct > 32, indicates that active infections were highly unlikely. It’s important to note that all of these tests were recorded as new COVID cases, so the range of Ct values suggest that testing in Rhode Island was unreasonably sensitive. That’s broadly true across the U.S. as well, which means that COVID cases are over-counted by perhaps 30% or more. And yet it is extremely difficult for subjects testing positive to learn their Ct values. You can ask, but you probably won’t get an answer, which is absurd and counterproductive.

Notice that the concentration of red dots diminished over time, and we know that the spring wave of the virus in the Northeast was waning as the summer approached. The share of positives tests with high Ct values increased over that time frame, however. This is borne out by the next chart, which shows the daily mean Ct of these positive tests. The chart shows that active infections became increasingly rare over that time frame both because positive tests decreased and the average Ct value rose. What we don’t know is whether labs bumped up the number of cycles or replications to which samples were subjected. Still, the trend is rather disturbing because most of the positive cases in May and the first half of June were more likely to be virus remnants than live viruses.

It’s also worth noting that COVID deaths declined in concert with the upward trend in Ct values. This is shown in the chart below (where the Ct scale is inverted). This demonstrates the truly benign nature of positive tests having high Ct values.

This is also demonstrated by the following data from a New York City academic hospital, which was posted by Andrew Bostom. It shows that a more favorable “clinical status” of COVID patients is associated with higher Ct values.

It’s astounding that the U.S. has relied so heavily on a diagnostic tool that gets so many subjects wrong. And it’s nearly impossible for subjects testing positive to obtain their Ct values. Instead, they are subject to self-quarantine for up to two weeks. Even worse, until recently there were delays in reporting the results of these tests of up to a week or more. That made them extremely unhelpful. On the other hand, the coffee ground test is fast and cheap, and it might enhance the credibility of a subsequent positive PCR test, if one is necessary … and especially if the lab won’t report the Ct value.

The PCR test has identified far too many false infections, but it wouldn’t have been quite so damaging if 1) a reasonably low maximum cycle threshold had been established; 2) test results had not been subject to such long delays; and 3) rapid retests had been available for confirmation. The cycle threshold issue is starting to receive more attention, quite belatedly, and more rapid tests have become available. As I’ve emphasized in the past, cheap, rapid tests exist. But having dithered in February and March in approving even the PCR test, the FDA has remained extremely grudging in approving newer tests, and it persists in creating obstacles to their use. The FDA needs to wake up and smell the coffee!

False Positives, False Cases, False Deaths

14 Monday Sep 2020

Posted by in Coronavirus, Pandemic

≈ Leave a comment

Tags

, , , , , , , , , , , , , , , ,

The tremendous increase in testing for COVID-19 (C19) this summer was associated with an increase in cases. Most of these tests were so-called PCR tests with samples collected via deep nasal swabs. More testing did not fully explain the increased case load, but false positives (FPs) still accounted for a substantial share. That’s especially true in light of the decline in positivity rates, which reflected a decline in the actual prevalence of active infections. FPs also account for a substantial share of the deaths attributed to COVID, which are obviously cases of false attribution. If a test for C19 is positive, it will be listed on the death certificate.  

COVID Case Inflation

The exaggeration of confirmed cases due to FPs is more substantial as the prevalence of active infection declines. That’s because the share of true positives in the tested population declines, while the share of false positives must rise due to the greater share of uninfected individuals in the population.

Now, as the contagion is waning in former hot spots, there is a danger that FPs create the impression of persistence in the case counts. That’s costly not just for those incorrectly diagnosed, but also in terms of medical resources, for communities subject to excessive public intervention, such as inappropriate lockdowns, and in terms of the fear promoted by these inaccuracies.

FPs are extremely disruptive when testing is relied upon in critical situations such as health care staffing, or even among sports teams. For example, at the University of Arizona, out of 25 positive tests on September 3, only 10 were confirmed as positives in later tests. The NFL has also had its share of false positives. 

Lax Testing Standards

There is evidence that testing standards under CDC guidance are so broad that a large number of inactive, non-infectious cases are being flagged as positives (see the chart above for the intuition, as well as the graphic at the bottom of this post). The tests sometimes amount to a coin flip when it comes to evaluating positives; some of the positives might even come from non-novel coronaviruses such as the common cold! This paper by Andrew N. Cohen, Bruce Kessel, & Michael G. Milgroom – CKM) questions the guidance of public health authorities on testing more generally. From the abstract (my emphasis):

“Unlike previous epidemics, in addressing COVID-19 nearly all international health organizations and national health ministries have treated a single positive result from a PCR-based test as confirmation of infection, even in asymptomatic persons without any history of exposure. …  positive results in asymptomatic individuals that haven’t been confirmed by a second test should be considered suspect.”

False Positive Math

When I wrote about “The Scourge of False Positives” in July. I noted that a test specificity of 95% implies that 5% of uninfected individuals will falsely test positive. Unfortunately, that still produces a huge number of FPs when testing is broad. That’s NOT a good reason to avoid broad testing; it just means that positive tests should be confirmed by another test. (In this case, two tests with the same specificity reduce a 5% false positive rate to 0.25%. That’s why fast, cheap tests are necessary for confirmation.

Again, exaggerated case counts due to FP’s become more severe as a contagion wanes. That’s because FPs become an increasingly large share of positive test results and overstate the persistence of the virus. If active infections fall to 1% of 750,000 daily tests, or 7,500 true cases, the 5% specificity implies 37,125 FPs: true positives would be only 17% of positive cases. Much worse than a coin flip! And again, which cases are infectious?

How Bad Are FPs, Really?

This recent research, also authored by CKM, explains the reasons why FPs are usually an issue in the real world, despite the tests’ reportedly perfect reactivity to anything other than the virus’ genetic fragments. CKM find that the median FP rate in their sample of “tests of tests” was 2.3%. That means 23 out of every 1,000 uninfected people tested will test positive.

If that seems small to you, suppose the true prevalence of active infection in a population is 4%. If 1,000,000 people are tested and there are no false negatives (unlikely), then 40,000 infected people will be identified by the test. However, another 22,000 uninfected people will also test positive ((1,000,000 – 40,000 infected) x 0.023). That means the number of positive tests will be inflated by 55%. They’ll all receive some form of treatment or ordered into quarantine. Expanded Testing and FPs This summer, the volume of daily tests increased from about 150,000 a day in early April to more than 750,000 a day in July. That’s a 400% increase, but the true prevalence of active infection in the expanded test population during the summer was almost certainly lower than in the spring. Suppose active infections fell from 10% of the test population in the spring to 5% in the summer. That means the daily number of “true positives” would have risen from 15,000 to 35,000 in the expanded test population (and again I assume no false negatives for simplicity). The number of FPs, however, would have risen from 3,105 to 16,445. Therefore, FPs would have accounted for 40% of the increase in “confirmed” cases between spring and summer.

False COVID Deaths

FPs are also inflating COVID death counts. PCR tests are routinely given at hospital admission for any cause, and even after sudden death, especially as the availability of tests increased late in the spring. This subset of the tested population will certainly have its share of FPs. If such a patient dies, regardless of underlying cause, it might well be attributed to COVID-19 as it will still appear on the death certificate. The same has occurred in the case of traffic fatalities, suicides, and other sudden deaths.

Antibody Tests

The FP problem also plagues tests of seroprevalence, which determine whether an individual has had the virus or is cross-protected against the virus by antibodies acquired via non-novel coronavirus infections. The consequences of these antibody FPs can be serious as well, because it means a positive test might not ensure immunity. As the exposed share of the population increases, however, the FP share of antibody tests is diminished.

Conclusion

As long as testing is required, dealing with FPs (and false negatives, of course) requires repeated testing, as CKM state unequivocally. And the tests must be fast to be of any use. The current testing regime must be overhauled to prevent false positives from costly impositions on the lives of uninfected patients, consuming unnecessary medical resources, making unrealistic assessments of cases and deaths, and unnecessary suspensions of normal human social activity and liberty.

The FDA Can Put Virus Behind Us, Sans Vaccine

19 Wednesday Aug 2020

Posted by in Liberty, Pandemic, Vaccinations

≈ 1 Comment

Tags

, , , , , , , , , , , , , , , , , , , , , , , , , , ,

Most of the news about COVID vaccine development is positive, but there are still huge doubts about 1) whether an effective vaccine(s) will ever be available; 2) when it will be available; 3) in what quantities (supply chains for vaccines present issues that most lay persons would never imagine) ; 4) the best approaches to allocation across young/healthy vs. old/vulnerable; 5) how long it will provide protection (the news is good on lasting immunity as well); and 6) whether people will actually take it. Given all these uncertainties, it’s worth considering an approach to stanching the coronavirus that won’t require a vaccine while still allowing a return to normalcy: cheap, rapid tests available to consumers on a daily basis in their homes or in businesses.

The full benefits of cheap, rapid tests can take people a while to wrap their heads around. In fact, there are skeptics who’s views on any and all testing are colored by suspicions that increased testing is some sort of conspiracy to spread fear and keep the economy hobbled. It’s true that increased testing drove much of the increase in COVID cases this summer, which caused the mainstream media to delight in spinning alarmist narratives. Fair enough, but that misses the point, which I’ll try to elucidate below. I credit a John Cochrane post for bringing this to my attention.

A successful vaccine breaks the so-called “transmission chain”, but so does frequent testing to identify infectious individuals on an ongoing basis so they can self-quarantine. As Alex Tabarrok has emphasized, we should worry about identifying infectious individuals, as opposed to infected individuals. They are not the same. Cheap, rapid, and easy-to-administer tests have already proven to be fairly accurate during the infectious stage. The idea is for individuals to self-test every day and stay home if they are positive. Or, employers can test workers every day and send them home if they are positive. Frequent testing also makes it simpler to trace the source of an infection and may reduce the importance of tracing.

To those who say this represents an affront to personal liberty, and I’m very touchy on that subject myself, recall that even now people are being screened in their workplaces using thermometers, questionnaires, or on the basis of any frogginess perceived by supervisors and co-workers. Those “tests” are far less accurate in identifying COVID-19 contagiousness than the kinds of cheap tests at issue here, and they are certainly no less intrusive. Then there are the many businesses facing restrictions on their operations: how “accurate” is it to keep everyone at home by locking down places of business? How intrusive is that? Those restrictions are indefensible, and especially with the advent and diffusion of cheap, rapid tests.

Of course, people might cheat and not report positives. Tests could be administered at workplaces to avoid that possibility, or at points of admission to businesses and facilities, but a few minutes of delay would be necessary. I would not support a centralized database of daily test results. If nothing else, relying on the good faith of individuals in reporting their results would be a giant leap forward in breaking the transmission chain now, rather than counting on the possibility of a successful virus in the indefinite future. And we might then avoid the whole pro-vax/anti-vax imbroglio that already foments, which raises major questions bearing on individual liberty.

Then there is the question of positive tests within multi-person households. Should the entire family or household self-quarantine? I say no, not if the others are negative, but then the others should test twice before going out, which dramatically reduces the probability of a false negative, and they should probably test more frequently, perhaps several times a day.

There are other important details to address: Who will pay for the tests? Will workers be paid to stay home if they test positive? How long will they be required to stay home? How will repeated tests be treated? I don’t want to get into detail on all of these points, but cheap, fast tests can help overcome many of these difficulties, and I believe many of the details can and should be worked out privately.

Unfortunately, the FDA has approved only two rapid tests, and they are not very rapid and not cheap enough. Only one had been approved up until last weekend because the FDA found the accuracy to be lacking … compared to PCR tests! But the FDA finally issued an Emergency Use Authorization for a saliva-based test (SalivaDirect) developed at Yale, partly funded by the NBA and the Players Association. The test still requires processing at a lab, so it’s really not convenient enough and not fast enough. Here is Zach Lowe on the cost:

“The cost per sample could be as low as about $4, though the cost to consumers will likely be higher than that — perhaps around $15 or $20 in some cases, according to expert sources.”

Not bad, but it’s much higher than more rapid, paper tests developed by Harvard’s Wyss Institute for Biologically Inspired Engineering and a company called E25Bio. Both of those are expected to cost about $1 per sample and can be completed anywhere. That’s a price that can work. And there are other promising candidates.

The benefits of tests that are rough, ready, and cheap will be huge. Such tests will also enable retesting, which helps to overcome the dilemmas of false positives and negatives. False negatives might be of greater concern to the FDA, but again, false negatives are less likely during the contagious stage of an infection, and the tests will be accurate enough that transmission risk will be drastically reduced.

The FDA needs to move beyond its stodgy insistence on achieving laboratory levels of accuracy. It’s unlikely that a single test source will be adequate to stanch the transmission chain, so the agency should rush to approve as many cheap, rapid tests as possible, with as many advisories and patient warnings regarding test results and follow-up instructions as it deems necessary. Remember, these tests are much better than thermometers!