Effectiveness of mRNA COVID-19 Vaccines

Against the Delta Variant

Among 5.6M Medicare Beneficiaries 65 Years and Older
Weekly update of September 28, 2021

Project Salus
 Executive Summary
Project Salus Project Salus provides answers
to these questions

Basic questions which require ▪ VE of both mRNA vaccines appears to wane over
data-driven answers time in this large 5.6M US-based 65 & over vaccinated cohort

▪ Risk of breakthrough hospitalization increases with time

elapsed since mRNA vaccination with odds ratio increasing to
2.5 at 6 months post vaccination

▪ VE against Delta breakthrough hospitalization (62%) exceeds

VE against Delta infection (41%)

▪ Prior COVID-19 infection has a major protective effect

against breakthrough hospitalization

▪ Older age groups (75-84 & 85 and older) experienced further

reduction in vaccine protection against hospitalization

▪ Hospitalization rate (21% vs 32%) and death rate (4% vs 12%) of

breakthrough infections lower than rates observed in Covid-19
Graphic adapted from CDC Presentation ACIP Meeting August 30, 2021 cases in pre-vaccination pandemic phase in 2020
Oliver, S. Framework for Booster Doses of COVID-19 Vaccines
2
JAIC is the US Department of Defense Joint Artificial Intelligence Center
 Salus Platform for COVID-19 Analyses
Project Salus

VE Study Attributes Other Platform Applications

Cohort
20M Medicare beneficiaries nationwide
with 16M individuals 65 years and older
100+M weekly SVI data
Medicare claim records
Exposure
Nationwide Mapping of COVID-19 Outcomes
5.6M fully vaccinated with
Hospitalizations, ICU, Ventilator Rx, Deaths
2.7M Pfizer and 2.9M Moderna

Terminology & Analytics

Period of study
Database: 20M individuals with
January - August 21 2021
claim data Oct 1, 2019 - present

Breakthrough Key Metrics Disease Risk Models with Population Risk

Profiling: Severe COVID-19 risk with
161K Breakthrough cases Validation with Hospitalization Rates

33K Breakthrough hospitalizations

10.4K requiring ICU admissions

Vaccination Mapping overlaid on severe
COVID-19 risk 3
* Medicare data and Humetrix software are hosted in a secure government enclave of the Department of Defense
 Salus Breakthrough Analysis Methodology and Limitations
Project Salus

• Breakthrough case definition: new COVID-19 diagnosis (by COVID-19 ICD-10 code) occurring no earlier than
2-weeks post the second vaccine dose (see appendix for more details on case definition)

• Breakthrough analysis methodology: to estimate weekly breakthrough cases and hospitalizations we

multiplied our Medicare claim-based weekly breakthrough case counts and hospitalization counts by the
corresponding weekly ratio of the claims-based vaccination rate to the CDC vaccination rate to compensate
for missing COVID-19 vaccination data from Medicare claim data (Medicare claims only provide ~45% of the
published CDC vaccination rate in the 65 and over age group)

• Breakthrough data limitations:

• Possible overestimation of breakthrough rates due to breakthroughs clinically defined with a COVID-19
diagnosis but not confirmed by PCR or antigen test (unavailable in claim data)
• Possible overestimation of breakthrough rates due to assuming identical breakthrough rates between
individuals with claim-based vaccination data and those lacking vaccination data in their claims
• Overestimation of breakthrough rates would lead to underestimating vaccine VE against breakthrough
infections and breakthrough hospitalizations
4
 COVID-19 Case Definitions
Project Salus

▪ COVID-19 case definition: COVID-19 ICD-10-CM code U071 found in any claim type. Date of diagnosis
based on first claim with U071. Note: 29% have either a COVID-19 PCR or antigen test in a claim.
▪ COVID-19 breakthrough infection definition: COVID-19 diagnosis more than 2 weeks after second dose
of mRNA vaccine or single dose of J&J vaccine with no COVID-19 ICD-10 code U071 between first and
second dose of mRNA vaccine. Note: 36% of breakthrough cases have either a COVID-19 PCR or antigen
test in a claim.
▪ COVID-19 hospitalization definitions: (1) Inpatient claim with primary admitting diagnosis ICD-10-C
code U071 with data of admission within 14 days after COVID-19 diagnosis or date of discharge within
10 days of post hospitalization COVID-19 diagnosis OR (2) Carrier claim with ICD10 code U071 and place
of service code = 21 and date of service either 14 days after COVID-19 diagnosis or 10 days before
COVID-19 diagnosis.
▪ COVID-19 associated death definitions: (1) Inpatient claim patient discharge status code = 41 (expired
in facility) OR (2) MBSF file Date of Death are within 60 days of COVID-19 diagnosis. 85% of COVID-19
deaths using this definition occurred within 30 days and 72% within 20 days of COVID-19 diagnosis

5
 Key Breakthrough vs. Pre-Vaccination COVID-19 Metrics
Project Salus

Among 5.6M fully vaccine immunized Salus cohort members aged 65 and older
(2.7M Pfizer and 2.9M Moderna), as of September 10, 2021:
• 2.9% cumulative breakthrough rate
• 21% hospitalization rate in breakthrough infections, reduced three-fold from
32% hospitalization rate March – December 2020
• 31% breakthrough hospitalizations include ICU care, equivalent to
32% ICU rate March – December 2020
• 4.0% death rate in breakthrough infections, reduced three-fold from
12% death rate March – December 2020

6
 Total & Breakthrough Cases in the 65 Years and Older Salus Cohort
Project Salus

▪ As Delta variant became

predominant, COVID-19 cases
increased five-fold in the >=65
Aug 28
97%
population
June 26
May 22 53% ▪ In this 80% vaccinated >=65
4% Delta variant population, an estimated 71%
of COVID-19 cases occurred in
fully vaccinated individuals

Breakthrough cases =
71% of total Covid-19
cases in cohort

Week ending 08/28/21, data

incomplete due to lag in claims
processing

7
*CDC data: % of SARS-CoV-2 genomes sequenced
 Is mRNA Vaccine Effectiveness Against Delta Breakthrough
Project Salus
Infection Waning Over Time in 65 Years and Older Salus Cohort?

☥ ▪ Breakthrough infection rates

☥ 5-6 months post vaccination
are twice as high as 3-4
August 21
☥ months post vaccination
July 31
June 19
July 10 96%
98%
☥
82%
36%
Delta variant ☥
95% CI

☥
☥ Breakthrough infection rates 5-6
☥ ☥ months since vaccination > 3-4
☥ ☥ months since vaccination
P < 0.001

8
 Age Distribution of Vaccinated Groups in the
Project Salus 65 Years and Older Cohort
▪ Could higher proportion of
85 years and older members
in first vaccinated group
explain reduced VE?

9
 Does Age Affect Vaccine Effectiveness Against Breakthrough
Project Salus
Infections in the 65 Years and
-
Older Cohort?
• Age has a minor contribution
to the reduced vaccine
+
NS protection seen in the group
-
NS vaccinated 5-6 months ago
+ # #
NS
August 21 ++
July 31 98% **
July 10
June 19 96% #
82%
36%
Delta variant ++ #
+++ ***
***
#
+++ 95% CI
++
+ *** *** ++
** #
**
# # Over 85 > Over 85 > 75 to 84 >
# # 75 to 84 65 to 74 65 to 74
P < 0.001 +++ *** none

P < 0.01 ++ ** none

P < 0.05 + none none

P > 0.05 - NS #

10
 Are There Differences in Waning Effectiveness Between Pfizer-
Project Salus BioNTech and Moderna Vaccines in the 65 Years and Older Cohort ?

NS
▪ Waning immunity are seen with both
Pfizer-BioNTech and Moderna vaccines
during Delta phase of the pandemic
NS ▪ Moderna vaccine offers better
August 21 NS
protection than Pfizer vaccine for
July 31
July 10 96%
98%
NS
individuals vaccinated 4 months prior for
June 19
36%
82% weeks ending after July 31
Delta variant #
*
* 95% CI
NS
*

# * Breakthrough infection rate

☩
☩ ☩ Pfizer > Moderna
NS
P < 0.001 *
NS
# * NS ☩
P < 0.01 #

P < 0.05 ☩

P > 0.05 NS

11
 Total & Breakthrough Hospitalizations in the
Project Salus 65 Years and Older Cohort
▪ As Delta variant surged to over 50%
in June, COVID-19 hospitalizations
more than doubled, reversing the
August 14
98% prior trend of decreasing
June 26
May 22 53% hospitalizations since April
4% Delta variant ▪ In this 80% vaccinated 65+
population, an estimated 60% of
COVID-19 hospitalizations occurred
in fully vaccinated individuals in the
week ending August 7th

60% of COVID-19
hospitalizations are in
vaccinated individuals

On 08/14/21, data incomplete

due to lag in claims processing

12
 Is Vaccine Protection Against Breakthrough Hospitalization
Project Salus
Waning Over Time in the 65 Years and Older Cohort?

▪ VE against breakthrough
hospitalization is significantly
lower 5-6 months post
vaccination than 3-4 months
☥ ☥
post vaccination
August 7
July 10 96%
June 19 82%
36%
Delta variant
☥
☥
☥
☥
95% CI
☥
☥
☥ Breakthrough hospitalization rate for
5-6 months since vaccination > 3-4
months
P < 0.001

13
 Are there Age Differences in Vaccine Protection Against
Project Salus
Breakthrough Hospitalizations in the 65 Years and Older Cohort?

+++ +++
***
*** ▪ Older age associated with
increased breakthrough
hospitalization rates
August 7
July 10 96%
June 19 82%
36% ++
Delta variant xxx #
***
++
*** 95% CI

+++ #
*** - Over 85 > Over 85 > 75 to 84 >
*** xx
- 75 to 84 65 to 74 65 to 74
*** xx
-
xx NS P < 0.001 +++ *** xxx
xx
#
P < 0.01 ++ none xx
P < 0.05 + none x
P > 0.05 - NS #

14
 What is the Vaccine Effectiveness Against the Delta Variant in
Project Salus
the Salus Cohort? – Using the CDC Screening Approach
▪ 41% calculated VE against
infection

80% of Salus cohort

are fully vaccinated ▪ 62% calculated VE against
hospitalization
71% COVID-19 cases are breakthroughs
in week after Delta variant > 90%

60% hospitalized cases are

breakthroughs in week after VE Screening method
Delta variant > 90%
VE = 1- [(PCV/(1-PCV))((1-PPV)/PPV)]
PCV = proportion cases vaccinated
PPV = proportion population
vaccinated

Graphic adapted from CDC Presentation July 30, 2021

Improving communication around vaccine breakthrough and vaccine effectiveness
15
 How Does mRNA Vaccine Effectiveness in 65+ Salus Cohort
Project Salus with 5.6M Vaccinees Compared to Published Estimates?
VE %

Salus >65 years cohort

Pfizer-BioNTech or Moderna
against Delta VE = 62%

Salus >65 years cohort

Pfizer-BioNTech or
Moderna
against Delta VE = 41%

▪ VE of both mRNA vaccines in this 65+ cohort is lower than previously reported in smaller study sizes for both
COVID-19 infection and hospitalization
▪ VE for mRNA vaccines is higher against hospitalization than against infection
Graphic adapted from CDC Presentation ACIP Meeting August 30, 2021, Oliver, S. Framework for Booster Doses of COVID-19 Vaccines 16
 Risk Model for Breakthrough Hospitalization
Project Salus

▪ Risk of breakthrough hospitalization increases with

time elapsed since mRNA vaccination with odds
ratio increasing to 2.5 at 6 months post vaccination
▪ Prior COVID-19 infection has a major protective
effect against breakthrough hospitalization
▪ There is a step up in risk in the 75-84 and again in the
over 85 age categories compared to the 65-74
category
▪ Risk model can be used to stratify the over 65
population to best select those in most need of
booster vaccine dose

Logistic Regression Model performance:

AUROC 0.73, balanced accuracy 0.67
0.2 0.4 0.6 1.0 2.0 3.0

17