A rainbow is on the horizon, as we anticipate that the FDA will issue an emergency use authorization (EUA) for the Pfizer-BioNTech candidate COVID-19 vaccine, following an advisory committee meeting scheduled for Dec. 10. Based on data from the phase III placebo-controlled trial, the Pfizer-BioNTech vaccine is reported to have an overall efficacy rate of 95%, consistent across age, gender, race, and ethnicity demographics; and the observed efficacy in adults over 65 years of age was over 94%. The trial involved more than 43,000 participants (with a 1:1 randomization to vaccine or placebo) and a total of 170 confirmed cases of COVID-19 were evaluated, with 162 observed in the placebo group versus eight in the vaccine group.
Vaccine efficacy is a measure of the performance of the vaccine under "ideal and controlled circumstances" as in a research study -- a randomized controlled trial, whereas effectiveness refers to its performance under "real-world" conditions, where other variables come to bear, allowing for a pragmatic evaluation. A critical consideration is that the primary efficacy endpoint of this phase III trial was the development of symptomatic COVID-19 infection; therefore, participants who may have developed asymptomatic coronavirus infection (after receiving the vaccine or placebo) were not identified or accounted for in this study. Consequently, if an EUA is issued, the FDA will have to be clear about the spectrum of COVID-19 infection that this vaccine is efficacious against, noting that asymptomatic transmission is a key driver of the spread of COVID-19.
In deciding whether to recommend authorization, advisory group members will address the following overarching questions:
1. Based on the totality of scientific evidence available, including data from adequate and well-controlled trials, is it reasonable to believe that the candidate vaccine may be effective to prevent SARS-CoV-2 infection?
2. Do the known and potential benefits of the candidate vaccine outweigh the known and potential risks of the candidate vaccine?
The adequacy and overall quality of the evidence will come under scrutiny from multiple angles, including attention to statistical power, which refers to the probability that the trial will detect a particular effect, if it truly exists. Power is a function of multiple factors but strongly linked to sample size, and adequate power is necessary in order to differentiate between an actual vaccine effect and a difference occurring by chance.
According to the Pfizer-BioNTech study protocol, the sample size is sufficient to provide 90% power to conclude that the true vaccine efficacy is greater than 30%, which is the lower bound required by the FDA for estimating vaccine efficacy using appropriate confidence intervals. Of note, however, is that the power of this phase III trial was calculated to detect an overall effect for vaccine efficacy, and does not specifically account for subgroup differences by age, gender, race, and ethnicity. Such subgroup analyses will require multiple comparisons, and if this component of the study is underpowered, that could increase the probability of obtaining biased and spurious subgroup differences in vaccine efficacy, even with multiplicity adjustment.
Regarding vaccine reactogenicity, the companies' topline announcement said the vaccine was well tolerated across all populations with no serious safety concerns. The only grade 3 (severe) adverse events or side effects were fatigue at 3.8% and headache at 2.0%. No statistical data were reported for the frequency of mild (grade 1) and moderate (grade 2) adverse events. A previous, smaller study of this vaccine noted frequent occurrences of adverse effects encompassing injection site pain, fever, chills, headaches, and fatigue, which were primarily mild or moderate, and these are recognized as common side effects of vaccines.
This phase III trial sought to blind participants to whether they received the vaccine or placebo. However, it is reasonable to assume that given the widespread awareness of common vaccine side effects, those who had that experience may have associated it with receiving the vaccine and hence become unblinded, and this could potentially confound the study outcome, resulting in biased estimates of vaccine efficacy. Those in the placebo group could have been similarly affected by associating the absence of vaccine side effects with receiving the placebo. Experimental studies without adequate and appropriate blinding tend to show larger treatment effects than studies with proper blinding. Therefore, performance and observer bias cannot be ruled out.
Based on the evidence available in the public domain, at this time, the general expectation is that the FDA will issue an EUA for this candidate COVID-19 vaccine, with the caveat that the evidence supports that the vaccine can protect against symptomatic disease, but it is not known if it prevents infection and transmission. The FDA may be less inclined to make any recommendation or determination regarding vaccine efficacy for specific subgroups. Additional prospective data are required to determine the level of effectiveness of this vaccine including subgroup differences, the durability of its protection, as well as to monitor safety and tolerability.
Rossi A. Hassad, PhD, MPH, is an epidemiologist and professor at Mercy College, in Dobbs Ferry, New York. He is a member of the American College of Epidemiology and a fellow and chartered statistician of Britain's Royal Statistical Society.