Introduction

Beginning in March 2020, government entities began to respond to the COVID-19 crisis by shutting down businesses and many social activities. The early business restrictions were simple; establishments were open or closed. Consequently, the early literature focused on full closures and their impact [1–9]. Once cases started to decline, governments often responded by letting businesses partially reopen. The hope was to mitigate economic costs while keeping COVID-19 under control. How effective were partial shutdowns compared to full closures? This question is of considerable interest since incomplete vaccine rollouts and limited vaccine availability to some segments of the population (e.g., children) may lead policymakers to consider imposing new restrictions to help reduce fatalities. This paper uses hand-collected county-level data to analyze the impact of partial shutdowns of restaurants, bars, gyms, spas, retail establishments, and movie theaters on the growth of COVID-19 fatalities. (We use the term “spas” to refer to all barbershops, salons, and other personal services.) The data that we use offer far greater granularity relative to datasets used by others studying the effectiveness of non-pharmaceutical policy interventions (NPIs) on COVID-19 deaths in the U.S.

We begin with the database used in [8]. It contains U.S. county-level policy restrictions for the period March 1, 2020 through December 31, 2020. The database captures full closures and lockdowns, including: general business closures; specific closures targeting bars, restaurants, gyms, and spas; no visitation policies at nursing homes; mandatory mask orders; park and beach closures; and limits on the size of gatherings. Because many policies have also targeted retail establishments and movie theaters, we add those sectors to the database. In total, we track six specific sectors: restaurants, bars, gyms, spas, retail establishments, and movie theaters. These are the businesses for which capacity restrictions are common and where understanding whether it is possible to limit the spread of COVID-19 while keeping businesses at least partially open is of particular interest. To examine the impact of partial closures, we introduce a range of sector-specific capacity limits. For each county or state government mandate, we record a restriction level and a start date. Table 1 contains a list of key terms that describe the restrictions that we analyze. Data access information can be found in S1 Methods.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Restaurant, bar, gym, spa, retail, and movies restrictions.

https://doi.org/10.1371/journal.pone.0262925.t001

As Table 1 indicates, we do not track outdoor regulations for gyms, spas, retail establishments, and movie theaters. In the rare instances in which these businesses are restricted to outdoor activities, we classify them as closed. During our sample period, restrictions on bars are always as strict as or stricter than those on restaurants. As such, we can only study the impact of restricting bar capacity, given a specific restriction on restaurants. To account for this, we focus the analysis on restaurant-bar restriction pairs (Bars X, Rest Y). The S1 Table shows summary statistics for the policy variables on which we focus the analysis. The S2 Table Panel A summarizes the number of weeks each policy remains in force.

Empirical analysis

Our regressions forecast the week t+k rate of fatality growth based on data as of period t: (1) In Eq (1) p is an indicator for the policy restriction put into place in county c at date t. The β are estimated coefficients and D are policy dummies. Dummies equal 1 if a particular policy is in place at time t and are zero otherwise. If multiple restrictions are imposed during a particular week, the restriction covering the most days is set to 1 and the others to 0. For gyms, spas, retail establishments, and movie theaters, capacity limits of over 50% are the omitted variable. For restaurants and bars, the omitted variable is one indicating that both can open beyond 50% of capacity. Standard errors are clustered at the county level and are robust to heteroskedasticity

Interpreting the coefficient estimates from Eq (1) is straightforward. We hold constant an area’s demographics, weather, past COVID-19 fatality growth and other policies in place as of date t. The β_p,k coefficients then imply that, k weeks after policy p is in place, the county’s fatality growth rate will, on average, differ by β_p,k relative to an identical county without the policy in place.

Potential false positives and false negatives are natural concerns when evaluating policy data. Suppose government entities introduce rules near the natural peak of fatality growth.¹ In this case, ineffective or even somewhat counterproductive restrictions can appear to be beneficial–a false positive. False negatives are also possible when effective policies are introduced as fatalities are accelerating. To combat these twin endogeneity problems, we take four approaches.

First, estimates of Eq (1) include a wide array of controls. These include county demographics, weather, and a variety of non-business restrictions, such as limits on gathering sizes, park closures, and mask policies. Earlier studies of NPIs during the COVID-19 pandemic have shown that these variables are closely associated with COVID-19’s spread [3–7, 10–13] and result in reduced population movement.[14–16] Similar to [8], we analyze counties rather than states (or countries). While many orders originate at the state level, they often apply to just a subset of counties. Conversely, some counties instituted policies apart from those issued by the state. County data more accurately reflect the policies in place and provide considerable statistical power relative to state our country level data others have used.[10, 11, 13, 14]

Every test controls for the total number of fatalities to date, the time since the first reported fatality in the county, the time since March 1, 2020, and six lags of weekly fatality growth. These help identify which policies alter the trajectory of transmission and death versus those that happen to be implemented during its natural rise or fall within a community.

Our second strategy for mitigating the endogeneity issue uses the trend in fatalities at the time a policy is enacted. We only consider a policy effective or counterproductive if the trends in fatality growth at the time of policy introduction (i.e., during the first and second week after a policy’s enactment) are either both insignificantly different from zero or, if significant, have a sign opposite that of the forecast horizon effect.

Third, we try to identify potential false positives by removing each state’s most populous counties and then repeating the regression analysis. Presumably, state governments focus their attention on more populated areas. When counties fall under orders optimized for other more populous parts of the state, we can estimate the impact of “out-of-equilibrium” policies on future fatalities. This should help identify false positives (e.g., policy introductions at the county’s natural fatality peak).

Fourth, we conduct “near-border” tests that are similar to the standard nearest neighbor analysis to further mitigate endogeneity problems. Instead of examining matched pairs of counties that lie on state borders (as a traditional nearest neighbor pairing would), we require at least a one-county buffer between a near-border county and a state line. Given the way that COVID-19 spreads, a policy that affects transmission in one county is likely to have a direct impact on an adjacent one. For example, different regulations for dining in between Queens County and Nassau County in New York caused some patrons to dine in Nassau County [17]. The same happened when Georgia opened its restaurants and its neighboring states did not. Similarly, Pennsylvania residents traveled to Ohio to purchase alcohol when the two states had differing restrictions [18]. Researchers, using more formal tests, have also found that stay-at-home orders in one area affected movement in others [6]. The geographic buffer in the near-border tests help address this issue while also using geographic proximity to control for the possibility that policies simply reflect local trends in the trajectory of the virus.

Results and discussion

The tables that follow include results of estimating Eq (1) when k is set to 4 and 6, corresponding to the growth rate in fatalities k weeks after a policy is put in place. The CDC reports a median incubation period from exposure to symptom onset of 4–5 days. Among people with severe disease, the median time to ICU admission from the onset of illness or symptoms ranges from 10–12 days [15]. For patients admitted to the hospital and who do not survive, [19] report a median duration of hospital stay of 12.7 day. Based on these estimates, it likely takes a policy approximately four to six weeks to influence the COVID-19 fatality growth rate.

Table 2 shows the results from restricting restaurant, bar, gym, spa, retail and movie theater operations for the baseline and low population databases. Some partial closures appear to be at least as effective as full shutdowns, especially in the bar and restaurant sector. Among them, letting restaurants and bars open at 25% of capacity appears to be the most effective. Baseline estimates imply that, counties imposing this limit combination see 6-week ahead fatality growth that is 3.37% lower in level than a county which keeps both open without any restrictions. This magnitude is meaningful. During the sample period, counties saw a mean growth rate in the weekly fatalities of 7.84%. The 3.37% figure implies a 43% reduction from the mean rate of 7.84%. By comparison, the estimates imply that locales which completely closed both restaurants and bars saw a subsequent fatality growth reduction of only 16%. As shown in the S3 Table, this difference is statistically significant. Note that, although the estimated coefficient on the Bars Closed, Restaurants 25% combination is even larger than that on Bars 25%, Restaurants 25% we cannot draw strong conclusions because the pre-trend analysis in the S4 Table shows significant pre-trends that indicate declining fatality growth near Bars Closed, Restaurants 25% policy introduction. The other two bar and restaurant combinations that produce consistently negative and statistically significant estimates across forecast horizons and databases are restricting bars to outside service while allowing restaurants to open to 50% of capacity and closing bars while allowing restaurants open to 50% of capacity. (The estimated coefficients on both these policies are significantly smaller than what we observe for full closures. See the S3 Table.) The one policy that appears to be counterproductive is letting bars open to 25% of capacity and restaurants to 50%. However, based on the pre-trends analysis in the S4 Table, this policy combination also tends to follow large growth in rate of COVID-19 fatalities, which violates one of the conditions necessary for us to interpret the estimates.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Baseline and low population counties forecast regressions 4 and 6 weeks ahead.

https://doi.org/10.1371/journal.pone.0262925.t002

The retail sector is another area where partial closures are associated with lower COVID-19 fatality growth rates than full lockdowns. While closing them appears ineffective, introducing 50% capacity limits future fatalities. Indeed, a 50% capacity limit appears to limit future fatality growth more than even a 25% limit does. It may be that policy makers had some idea that this might be true. S1 Table indicates about 30% of all the county weeks saw capacity restricted to 50% while only 7% and 12% include a total closure or a 25% capacity limit respectively.

For gyms, Table 2 indicates that full closures reduced the COVID-19 fatality growth rate. Both the 4 and 6-week forecast horizons in the baseline and low population data produce negative and statistically significant estimates. Importantly, the pre-trend analysis in the S4 Table shows that fatalities were increasing prior to the introduction of gym restrictions. This implies that the reduced fatality growth rate estimates from gym closures are not simply reflecting an existing trend. Other gym constraints, however, appear to be either ineffective or even counterproductive. Based on the estimates, one would conclude that gyms should either be closed altogether or allowed to reopen to over 50% of capacity. Unlike restaurant, bar and gym restrictions, the results in Table 2 imply that all restrictions on spas and movie theaters are either ineffective or counterproductive. Spa closures appear particularly counterproductive. However, policies were not as harmful as the coefficients in Table 2 imply because spa closures were introduced when fatality growth rates were abnormally high.

The S5–S7 Tables show a variety of robustness checks. In the S5 Table, we remove all lagged dependent variables from the specification. The estimated coefficients on the policy variables are similar to those shown in Table 2, suggesting that including controls for recent fatalities does not bias our main findings. In the S6 Table, we winsorize the fatality growth variable to ensure that results are not driven by outliers. Again, the findings are similar to what we observe in Table 2. The S7 Table shows results when we include county fixed effects. This specification absorbs some important variation in the data since it does not allow us to exploit cross-county variation for areas where policies are in effect for most of the sample period. Still, we continue to find that restricting bars and restaurants to 25% capacity is even more effective than closing them completely. We also continue to find that many restrictions are ineffective or counterproductive.

Table 3 shows results for the near-neighbor samples. Overall, the estimates reinforce the earlier findings. In Table 4, we summarize all of the findings in Tables 2 and 3 and we also compare them to the pre-trends analysis. We base our main conclusions on the “Overall” column, where we indicate a (-) or (+) if two conditions are met: (1) the Tables 2 and 3 columns show that the policy is helpful (-) or harmful (+); and (2) the pre-trends analysis indicates no trend or a trend of the opposite sign. In support of the earlier discussion, several combinations of partial capacity restrictions for bars and restaurants are as effective as full shutdowns. For gyms, while full closures are followed by a lower COVID-19 fatality growth rate, partial closures may be counterproductive relative to leaving capacity unrestricted. For spas and movie theaters, we find that constraints are either ineffective or counterproductive.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Near neighbor regressions.

https://doi.org/10.1371/journal.pone.0262925.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Summary of coefficients.

https://doi.org/10.1371/journal.pone.0262925.t004

Conclusions

Government entities initially responded to the COVID-19 pandemic with policies that shut down an array of businesses. The literature indicates that some of these policies likely helped curb future fatalities. A natural question is whether it is possible to achieve similar results with less stringent policies. This paper uses data on partial openings of restaurants, bars, gyms, spas, retail establishments, and movie theaters to shed light on that question.

For gyms, the evidence indicates that shutting them down completely is beneficial, but other restrictions are ineffective. Counties that did not limit gym capacity below 50% saw no faster growth in COVID-19 fatality growth than those that did; unless they closed them entirely. When it comes to movie theaters, spas and other personal care services, all restrictions appear to be unhelpful or even counterproductive.

The fact that no county has placed tighter restrictions on restaurants than bars means that we must consider policy pairs for these establishments. We analyze the effectiveness of policies that restrict bars, given a particular restriction on restaurants. As one might expect, we find that closing them completely is then followed by a reduction in COVID-19 fatality growth. However, less restrictive pairs that limit bars to outdoor service generate similar results. For example, limiting bars and restaurants to 25% of capacity is followed by greater estimated reduction in COVID-19 fatality growth than full closures. Closing bars or restricting them to outdoor service while letting restaurants open to 50% of capacity generally also reduces fatality growth, though somewhat less than closing them completely. Why these pairs seem particularly effective at slowing deaths due to COVID-19 is a question we leave open to future research.

It may seem surprising that some capacity restrictions fail to reduce fatality growth due to COVID-19, or that some partial restrictions do more to reduce it than full shutdowns. However, these findings may reflect unintended consequences, where rules meant to reduce risks have the opposite effect. Although difficult to test without detailed location data, it is possible that people respond by substituting into riskier activities. The fact that these types of patterns have been documented in other safety settings [20–22] suggests that substitution effects could be at work. A definitive answer will, of course, require additional data and analysis.

Whenever one undertakes analysis of policy effectiveness, potential endogeneity issues arise. For example, [23] report large voluntary reductions in economic activity at the beginning of the pandemic. Our tests include a number of control variables and filters, as well as the near-neighbor analysis, to mitigate these concerns. Nevertheless, no set of controls or tests can fully eliminate the chance that regression results are coincidental (i.e. due to endogeneity) rather than causal. In the end, we can state whether fatality growth rose or fell after initiating various policies while controlling for a wide range of factors that might support alternative hypotheses. The results suggest that, while a subset of the policies put into place worked, others were ineffective.

This paper focuses on one particular benefit of business restrictions: slowing the rate of growth in COVID-19 fatalities. Future researchers might want to examine the costs. Clearly, higher capacity constraints do less economic and social damage than lower constraints. But, just how large of a difference is an issue that future articles should address. On the economic front, closures and restrictions lead to business bankruptcies and those often lead their owners and employees into financial distress. Beyond these economic costs there are social ones as well. To what degree did partial restrictions help reduce social problems like family breakups, drug addiction and other issues tied to financial hardship and social isolation. Before we can fully assess the value of full or even partial capacity restrictions we will need answers to all of these questions.

We acknowledge that there are some are natural limitations in applying the results of this study to policymaking. For example, we cannot be sure that our results apply outside of the United States or to communities with high vaccination rates (because our sample ends in December 2020). However, given the reduction in economic activity that shutdowns cause, one clear policy implication is that businesses should never fully shut down if partial opening does at least as well. The evidence in this paper shows that, in many cases, full shutdowns are unnecessary. Selective use of restrictions based on evidence regarding their efficacy might help policymakers gain trust from the public that some interventions can be helpful. This is especially important given the wide dispersion in public trust during the pandemic.[24] Of course, vaccines, therapeutics, and the virus itself may change over time. In devising policy responses based on the estimates produced here, policy makers should also pay close attention to the impact of these types of developments.

Materials and Methods

The methods section is organized as follows. The “Data” section describes the datasets that we use in the analysis, the construction of the weekly COVID-19 fatality growth rates that we use in the regressions, and filters that we impose. The “Empirical Model” section describes the basic regression model that we use to estimate the relationship between fatality growth due to COVID-19 and the policy variables for the full dataset, as well as low population counties. The “Near-Neighbors” section describes the method used to identify counties for the near-neighbor regressions.

Supporting information

Acknowledgments

We would like to thank Timothy Akintayo, William Babalola, William Cook, Josephine Cureton, Alec Dai, Renee Dauerman, Nora Draper, Golden Gao, Patrick Hayes, Kevin Hong, Nina Huang, Aykhan Huseynov, Ryan Jennings, Alex Liang, Christine Liaw, Gen Li, Emily Lin, Natalie Lord, Vanika Mahesh, Stephen Martinez-Hernandez, David Mason, Paul Nash, Daniel Nguyen, Joojo Ocran, Sean-Michael Pigeon, Elissa Prieto, Preston Smith, Mingjun Sun, Crystal Wang, Joanna Wrobel, and Charlotte Zimmer for excellent research assistance.