Jan M Brauner*, Sören Mindermann*, Mrinank Sharma*, David Johnston, John Salvatier, Tomáš Gavenčiak, Anna B Stephenson, Gavin Leech, George Altman, Vladimir Mikulik, Alexander John Norman, Joshua Teperowski Monrad, Tamay Besiroglu, Hong Ge, Meghan A Hartwick, Yee Whye Teh, Leonid Chindelevitch, Yarin Gal, Jan Kulveit
Inferring the effectiveness of government interventions against COVID-19
INTRODUCTION
Governments across the world have implemented a wide range of nonpharmaceutical interventions (NPIs) to mitigate the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Given the increasing death toll of the pandemic and the social cost of some interventions, it is critical to understand their relative effectiveness. By considering the effects that interventions had on transmission during the first wave of the outbreak, governments can make more-informed decisions about how to control the pandemic.
RATIONALE
Rigorously studying the effectiveness of individual interventions poses considerable methodological challenges. Simulation studies can explore scenarios, but they make strong assumptions that may be difficult to validate. Data-driven, cross-country modeling comparing the timing of national interventions to the subsequent numbers of cases or deaths is a promising alternative approach. We have collected chronological data on the implementation of several interventions in 41 countries between January and the end of May 2020, using independent double entry by researchers to ensure high data quality. Because countries deployed different combinations of interventions in different orders and with different outcomes, it is possible to disentangle the effect of individual interventions. We estimate the effectiveness of specific interventions with a Bayesian hierarchical model by linking intervention implementation dates to national case and death counts. We partially pool NPI effectiveness to allow for country-specific NPI effects. Our model also accounts for uncertainty in key epidemiological parameters, such as the average delay from infection to death. However, intervention effectiveness estimates should only be used for policy-making if they are robust across a range of modeling choices. We therefore support the results with extensive empirical validation, including 11 sensitivity analyses under 206 experimental conditions. In these analyses, we show how results change when we vary the data, the epidemiological parameters, or the model structure or when we account for confounders.
RESULTS
While exact intervention effectiveness estimates varied with modeling assumptions, broader trends in the results were highly consistent across experimental conditions. To describe these trends, we categorized intervention effect sizes as small, moderate, or large, corresponding to posterior median reductions in the reproduction number R of <17.5%, between 17.5 and 35%, and >35%, respectively. Across all experimental conditions, all interventions could robustly be placed in one or two of these categories. Closing both schools and universities was consistently highly effective at reducing transmission at the advent of the pandemic. Banning gatherings was effective, with a large effect size for limiting gatherings to 10 people or less, a moderate-to-large effect for 100 people or less, and a small-to-moderate effect for 1000 people or less. Targeted closures of face-to-face businesses with a high risk of infection, such as restaurants, bars, and nightclubs, had a small-to-moderate effect. Closing most nonessential businesses delivering personal services was only somewhat more effective (moderate effect). When these interventions were already in place, issuing a stay-at-home order had only a small additional effect. These results indicate that, by using effective interventions, some countries could control the epidemic while avoiding stay-at-home orders.
CONCLUSION
We estimated the effects of nonpharmaceutical interventions on COVID-19 transmission in 41 countries during the first wave of the pandemic. Some interventions were robustly more effective than others. This work may provide insights into which areas of public life require additional interventions to be able to maintain activity despite the pandemic. However, because of the limitations inherent in observational study designs, our estimates should not be seen as final but rather as a contribution to a diverse body of evidence, alongside other retrospective studies, simulation studies, and experimental trials.