Why Imperial College’s REACT study is so problematic

There was very gloomy news this week. ‘Coronavirus infections are not falling in England, latest REACT findings show,’ said a press release from Imperial College. It was widely covered in the press in this vein: Covid levels ‘may even have risen’ since the latest lockdown, BBC news reported. This reignited fears that further tighter lockdown measures might be needed to contain it. It was all a result of Imperial College's latest REACT study of Covid-19 infections, a massive study of 143,000 people and one of the biggest Covid surveys around. So its findings - and talk of rising cases - were taken very seriously. And understandably so. 

The study’s author, moreover, was adamant that the virus has had a resurgence. ‘Across this round of the study we’ve seen that the number of infected people has remained high and we have no good evidence that infections are falling in England,’ Professor Steven Riley said. ‘We are working to better understand why we are seeing these trends when the country is in lockdown, including studying the new variant, so that policymakers can respond urgently to help bring infections down and save lives.’ 

Prof Riley’s words were quite clear. The notion that the new mutation is so potent that it has made lockdown pointless is frightening and depressing in equal measure. But does it stand up to scrutiny? No. The opposite, in fact, is the case: there is good evidence that the current lockdown is working better than the second lockdown in November. 

The latest results from the REACT study fit in with modelling we carry out here at Bristol University showing that active infections in England have been continuously falling since the third lockdown was introduced on 6 January. This all throws an interesting light on the reporting of the pandemic: why did no one spot, earlier, that Imperial’s conclusion is clearly contradicted by the falling trend of new cases shown on the government's coronavirus dashboard? 

The Imperial study is extensive, and clearly expensive. The 143,000 people were given throat and nose swabs over a period from 6 to 15 January. Of these, just under 2,000 tested positive. On this basis, researchers calculated that 1.58 per cent of England's population had an active infection during the time of their study. Which, as a share of the general population, would be 893,000. This is well below the 1.12 million that were estimated to have the virus on 30 December by the Office for National Statistics (ONS) – in other words, this indicates a fall. 

The authors of the REACT report say they saw ‘worrying suggestions of a recent uptick’ in infections. But identifying a trend within a sample that contains only a few days' data is deeply problematic. Why? Because short-term, random fluctuations will disguise any consistent long-term movement, as the figures are moving up and down continually anyway. The authors themselves admit this limitation. There are now a few studies estimating Covid levels. We have figures for people who get tested, but REACT and the ONS do a kind of viral opinion poll, testing a huge sample and using that to estimate levels in the whole population. Crucially, these samples are taken a month (sometimes more) apart. So the REACT survey that made so many headlines missed out the period between 3 December and 6 January. 

As the above chart shows, the REACT lack of coverage coincided with the more infectious new B.117 variant getting into its stride and, of course, it included the Christmas period, which experts forecast would be a time of high transmission. So while the ONS and REACT surveys provide very useful data, the value of the information is reduced because of its intermittent nature. The measurements, moreover, are typically ten days out of date when they land. This makes both sets of data unreliable for tackling a virus that, as we have constantly seen, can grow extremely rapidly in a very short space of time. 

Real-time monitoring of Covid-19 

There is an alternative. At the University of Bristol, we have developed a predictor-corrector coronavirus model, known as the PCCF. It has been going for a while now and its track record is pretty impressive: it’s the kind of thing you can consult to help work out whether the virus level is going up or down. Using fairly simple mathematical models, the PCCF reconciles the official, daily figures for ‘cases by date reported’ with the survey figures that are produced at intervals by the ONS and the REACT studies. The PCCF is then able to produce a continuous trace of how many active infections there are in England on any date, as shown on the Chart below. 

The chart shows that the ONS numbers, the REACT figures and the PCCF curve all tell pretty much the same story: active cases plateaued in early November; then they fell in the second half of the month, but picked up strongly in December as England came out of its second lockdown and the effects of the B.117 strain kicked in. Cases peaked in early January - and then started a long fall during our third lockdown. All of the data - even the REACT study - appears to support this conclusion. A peak in infections around the time of the new year and a steady fall thereafter. Covid levels are high: there is nothing in these figures to give anyone reason to drop their guard. But the real story is the opposite of that in the Imperial/React headline: Coronavirus infections are falling in England. Let us hope that this trend continues.