COVID-19 forecasting in the era of vaccines and variants

# COVID-19 forecasting in the era of vaccines and variants
### Irena Papst, PhD Postdoctoral Fellow Department of Mathematics & Statistics McMaster University

---

These slides are available at

[papsti.github.io/talks/2021-09-29_EEB-seminar.html](https:/papsti.github.io/talks/2021-09-29_EEB-seminar.html)

???
- in case you want to **follow along** 
- there are **hyperlinks** that may be of interest sprinkled throughout the talk
- in case i go though too quickly and you want to **flip back** to a previously shown slide

---

I would like to begin by giving honour and thanks to the **Haudenosaunee** and **Anishinaabe** nations as the traditional inhabitants of the lands where McMaster stands.

To say that is to acknowledge **a debt to those who were here before us** and to recognize our responsibility, as guests (both physical and virtual), to **respect and honour** the intimate relationship Indigenous peoples have to this land.

???
- Indigenous land acknowledgment statement adapted from the McMaster Student Success Centre [guide to land acknowledgements](https://healthsci.mcmaster.ca/docs/librariesprovider59/resources/mcmaster-university-land-acknowledgment-guide.pdf?sfvrsn=7318d517_2)

---

## Collaborators

???
- the work i'd like to talk to you about today is part of an ongoing project with these wonderful individuals, almost all of whom have either present or past connections with McMaster and you may recognize some names
- top row are people that currently work at PHAC, the Public Health Agency of Canada
- bottom row are McMaster faculty

---

## Forecasting is an important tool in the fight against COVID-19

???
- short-term forecasting of COVID-19 has proven itself to be an **enormously useful** tool for **public health planning** throughout this pandemic

Source: [Public Health Agency of Canada](https://www.canada.ca/content/dam/phac-aspc/documents/services/diseases-maladies/coronavirus-disease-covid-19/epidemiological-economic-research-data/update-covid-19-canada-epidemiology-modelling-20210903-en.pdf)

???
-----
- here's a **recent example** from a public briefing by **Dr. Theresa Tam, Canada's Chief  Public Health Officer**, showing forecasts for the **late summer** and **early fall** in Canada
- looking at the **number of reported cases we can expect** in Canada in the short-term, depending on whether we **increase our contacts with other people**, or **maintain the current level of interaction**
- the red points overtop are case reports since the forecast was made, highlighting its **accuracy** in predicting the **fourth wave**, primarily driven by the Delta variant of concern
- this forecast was prepared by one of my collaborators, Mike Li, at the Public Health Agency of Canada, using the **model that i'm talking about today**

---

## How do we make these forecasts?

---

## Base model

???
- let's start with the **base model** that we've been using to make COVID-19 forecasts
- this model was developed in the summer of 2020, so it was very much tailored to the **initial pandemic period**
- going into how we've adapted this to this new era of vaccines and variants later on

We use a **compartmental** epidemiological model (SEIR framework).

???

???
- SEIR refers to the main compartments in this model: susceptible, exposed, infected, recovered
- the population is **partitioned** into one of these states
- **everyone starts in the susceptible class **and then individuals can get **exposed** to (and therefore infected by) SARS-CoV-2
- but they aren't infectious until they enter **one of the four infectious classes** representing different infection types important for COVID-19
- the first two are:
1. **asymptomatic**, where you don't show symptoms but can still infected others (albeit in a reduced capacity)
2. **pre-symptomatic**, where you will eventually develop symptoms
- asymptomatic infections simply recover
- while presymptomatic infections turn into either mild or severe infections 
- **mild** infections recover and never result in hospitalization or death
- while **severe** infections always result in either hospitalization 
or death before hospitalization
- and then we have some downstream outcomes important to modelling the healthcare system, which we won't focus on today

-----

- each of these arrows represents a **flow rate** specified with various parameters

**Deterministic** with **stochastic elements** in simulation.

???
- i will also mention that, while the underlying model is deterministic, we add **stochastic elements** to the simulation software to be able to estimate confidence intervals around our forecasts
- so that's the **abstract model**, but how do we actually use to it forecast?

---

## Calibrating transmission

???
- the key is to understand what's **happening with transmission** by calibrating, **or fitting**, a transmission rate

How many **new infections** will we *observe* each day?

Infections are _observed_ via COVID-19 **infection reports**

???
- "infection reports" are sometimes referred to as cases in the media... its the infections that the province or other jurisdiction is reporting based on detection through testing 
- important to remember that these are just a *sample* of infections

- **underestimate** of actual infections
  - **delay** of 2-3 weeks
  - depend on **testing availability** and **behavior**

New infections are governed by **transmission**:

$$
\quad  S(t)  \xrightarrow{(\color{#7A003C}{\text{force of infection}}) \times S(t)} E(t)
$$

???
-----
- transmission converts susceptibles to exposeds
- the **per capita rate** with which this **conversion** happens is called the **force of infection**

$$
\color{#7A003C}{\text{force of infection} = \beta(t) \times \frac{I(t)}{N}}
$$
???
-----
- the **basic structure** of the force of infection in our model

- `$\color{#7A003C}{\beta(t)}$`: time-dependent transmission rate (# of **transmission opportunities** per individual per day, depends on the average **contact rate** between individuals)

???
-----
- **transmission opportunities**

- `$\color{#7A003C}{\frac{I(t)}{N}}$`: probability of **encountering an infective** during a transmission opportunity at time `$\color{#7A003C}{t}$`, under the assumption that the population is **homogeneously mixed**

???
-----
- probability of encountering an infective (**encounters are random**)
- this is a **simplified version** of the force of infection used in the base model
- we also **facet by infection type** (asymptomatic, presymptomatic...)
- i'm just **bundling** all infection types into this `$I(t)$` term - total infectives at time `$t$` - for simplicity right now

---

## Calibrating transmission

We **fit `$\beta(t)$`** as a **piecewise-constant** function to available infection report data.

???
- we fit...
- to make transmission **time-varying**, we

We re-estimate `$\color{#7A003C}{\beta}$` when there has been a **significant change** in transmission-related behavior (e.g. restrictions loosening, stay-at-home order):

--
 
<img src="data:image/png;base64,#figs_files/figure-html/betademo-1.png" width="1344" style="display: block; margin: auto;" />

???
-----
- this is a **demo** of what beta might look like
  
---

## Forecasting transmission

We make **assumptions** on `$\color{#7A003C}{\beta(t)}$` in the forecast period.
  
???
- we make...

---

## Forecasting transmission

We make **assumptions** on `$\color{#7A003C}{\beta(t)}$` in the forecast period.

Start with **latest** `$\color{#7A003C}{\beta}$` (from last estimation window).

???
- start with...

---

## Forecasting transmission

We make **assumptions** on `$\color{#7A003C}{\beta(t)}$` in the forecast period.

Start with **latest** `$\color{#7A003C}{\beta}$` (from last estimation window).

In **forecast period**, either:

???
- in forecast period...

---

## Forecasting transmission

We make **assumptions** on `$\color{#7A003C}{\beta(t)}$` in the forecast period.

Start with **latest** `$\color{#7A003C}{\beta}$` (from last estimation window).

In **forecast period**, either:

1. use latest `$\color{#7A003C}{\beta}$` (**status quo** forecast)

???
- use latest...

---
  
## Forecasting transmission

We make **assumptions** on `$\color{#7A003C}{\beta(t)}$` in the forecast period.

Start with **latest** `$\color{#7A003C}{\beta}$` (from last estimation window).

In **forecast period**, either:

1. use latest `$\color{#7A003C}{\beta}$` (**status quo** forecast)

2. increase (or decrease) transmission by **x%** relative to latest `$\color{#7A003C}{\beta}$`
  
???
- increase (or decrease)...
- that's the general procedure we employ
- and this base model worked really well for a while... but

---

## The COVID-19 pandemic has changed

???
- a lot has changed about the COVID-19 pandemic in canada, since its initial phase in 2020
- and it continues to change!
- two big drivers of the most recent changse change are

---

## Vaccination

???
- vaccines!
- in canada, we've had ready access to vaccines for months
- Photo by Artem Podrez from Pexels: https://www.pexels.com/photo/a-close-up-view-of-a-covid-19-vaccine-vial-on-blue-background-5878516/

---

## Variants of Concern

???
- but we also have emerging variants of concern that are changing the epidemiology of the virus in important ways
- Screenshot of a page on the World Health Organization website: https://www.who.int/en/activities/tracking-SARS-CoV-2-variants/

---

# How do we ensure forecasts remain useful and informative?

???
- let's look at two past forecasts made by our group that highlight the importance of keeping our ears to the ground as modellers

---

# Forecast from 21 Feb 2021 in Ontario

???
- examples of some forecasts that we've been making for ontario
- this way, i can **highlight the value** of the extended model **before delving** into the details
- the first forecast is from **late february**
- **what was going on** in Ontario at the time?

---

???
- in the **lead up** to the forecast date, marked here with a **dashed line**, we saw a slow and steady increase in **infection reports** 
- and while **2000/3000 infections** reported in a province of almost **15 million** may not seem like much, the **strain** on the healthcare system was beginning to show

---

???
-----
- you can see **ICU occupancy** had also been rising with infection reports

---

???
- and we had actually surpassed the **pre-COVID ICU capacity** threshold for the entire province
- while we did expand ICU capacity in response to the pandemic, it was achieved through **hugely disruptive measures**, including **cancellations of "elective" surgeries** that sound optional, but may actually be lifesaving down the line
- this was the beginning of a **potential crisis** in our hospitals, and we had to act

---

???
- in response to rising infection reports and ICU occupancy, the government implemented **several interventions**, namely
- lockdowns in Toronto and Peel in late november, the **two most populous health regions in the province** (representing about 30% of the provincial population)
- followed by a **province-wide shutdown** on 26 december 
- and then ultimately a **province-wide stay-at-home order** in mid-January

---

???
- the combination of these interventions finally began **decreasing** infection reports and **pretty sharply** compared to the original incline, though **decline in ICU occupancy** was **slower**

---

???
- the stay-at-home order was **set to expire** in early march, and since things were looking good, it was looking **likely** that the province **wouldn't extend it**
- so the **question** at the time (as it usually is in these forecasts) was

---

# Is it safe to loosen restrictions?

---

???
- **just** looking at the trends in infection reports and ICU occupancy, you may think *"yes, it should be safe to reopen!"*
- ICU occupancy was **still high** but **trending downward**, and we know the ICU trend is **lagged** from infection reports---people enter the ICU some time after testing positive and then they may stay in the ICU for a while
- but there's **another piece of the puzzle** that isn't clear from infection reports or ICU occupancy

---

???
- **under the hood**, the Alpha varaint (aka strain **B.1.1.7**) had made its way into the province
- this variant is about **50% more transmissible** compared to resident strains at the time
- it was first confirmed in the province **coincidentally** on the same day as the province-wide shutdown, **26 december**
- by the forecast date, it represented about **20%** of infection reports
- we were particularly concerned about Alpha because **despite** the fact that our public health interventions were **successfully driving infection reports down pretty rapidly**, Alpha was finding a way to **increase its frequency** over this period... _scary!_

Trends in infection reports and ICU occupancy favored reopening, while Alpha threatened it.

---

???
- so we fit transmission to the infection report data (the **black curve**) and then
- so we made **two forecasts** at the time:
  1. one where we **explicitly modelled the increasing frequency** of the Alpha variant through both the fit and the forecast period (**extended model**)
  2. another where we detected **whatever effect** the variant was having on infection reports by fitting the transmission rate as usual, then we just **propagated that effect forward**, with no change to Alpha frequency going forward (**base model**)
- the curve is the median forecast in ecah case and the band around each curve represents 95% confidence intervals
- and here is **what happened** actually happened

---

???
- these **hollow points** are still infection reports, just those observed after the forecast date
- you can see that the forecast using the **extended model**, with the effect of the variant, **caught the third wave takeoff**
- while the forecast using the base model showed some take-off, but **grossly underestimated the magnitude of the uptick** resulting from the measures relaxing

Infection reports *alone* could not signal the start of the third wave.

???
- infection reports alone...
- and that **including the variant** in our model was **crucial** for this forecast

---

# Forecast from 5 Jun 2021 in Ontario

---

???
- so **fast-forward** now to another forecast in **june**
- for reference, here is the **previous forecast date**
- this was the start of the **third *wave** of infection in the province, **driven by** the Alpha variant
- by the beginning of april, we had almost hit the **previous peak** of infection reports from January, which was around 4000
- but even worse, **ICU**s didn't get chance to empty out from the second wave and the numbers just kept climbing

---

???
- so the province instituted another stay-at-home order in the **beginning of april**

---

???
- which again succeeded in bringing infection reports down, though the **ICU debt** was much more severe
- we ended up at **triple the pre-COVID capacity**, and the situation was very dire
- doctors were **on the verge** of having to decide **who would get care and who wouldn't**
- patients were being **flown out of province** for critical care

---

???
- by the forecast date, we were again in a similar position where **the stay-at-home order was expiring**, and the province was planning to enter **step 1** of its latest reopening plan, so we were once again thinking about whether

---

# Is it safe to loosen restrictions?

???
- it would be safe to loosen restrictions

---

???
- although ICU occupancy was **still high**, we were **less worried** about a hospital crisis at this point because

---

???
- we had **vaccines**!
- the provinces' mass vaccination effort took off in **early 2021**
- we actually **prioritized** getting **first doses** in as many arms as possible, initially planning to delay second doses by **up to 16 weeks**
- in practice, the delay ended up being about **8-12 weeks**
- though this the delay posed a risk because...

---

???
- the Delta variant had **arrived** in the province

---

???
- Alpha had become **dominant** in mid-march
- and while surveillence for **Delta** variant was a bit patchier, we believe Delta began to make up a majority of the share of "non-Alpha" strains in early May
- so although we had **mass vaccination taking off**, we had a new variant that was 50% more transmissible than _Alpha_, so posed to **take it over** as the most dominant strain 
- but most problematically,

First-dose vaccine-efficacy against Delta: 30%

???
-----
- Delta seemed to decrease vaccine efficiacy, **especially first dose efficacy** to about 30% against symptomatic disease
- with such a low first-dose efficacy, it wasn't clear whether the the **first-dose strategy** would be **undermined** by a strain that could still be **spread by partially vaccinated people**
- one dose of the vaccine would certainly reduce the amount of severe disease to an extent, but if it wasn't **blocking transmission** sufficiently, the virus could potentially **find holes** in the population and take off again

---

Trends in infection reports, ICU occupancy, and vaccination favored reopening, while Delta threatened it.

???
- so trends in...

---

???
- here is the **set of forecasts** we produced in early june, with the extended model featuring two-dose vaccination and increasing frequency of the delta variant
- we had to consider scenarios for the effect of **step 1 reopening** on **transmission** (colour of each line)
- but we also had to consider scenarios for the **vaccination effort**, as it seemed poised to **continue expanding** based on announced supply increases at the time (line type)
- for reference, the **180k doses/day** scenario is most representative of what actually happened
- we saw that although step 1 reopening would be relatively safe if we kept transmission from increasing too much, **we could still see a fourth wave**, even with significantly **more vaccination**, which was a problem, because we were still coming down from that **massive ICU wave**, and we didn't want to challenge our hospital system any more, or risk the possiblity of **another stay-at-home order**

---

???
- and here's what happened
- this is not too surprising because step 1 was centred on **outdoor activities**, so although we felt that the low transmission scenarios were the most likely, we wanted to show that we **weren't impervious to a Delta wave** if we pursued a more aggressive reopening, as that's what we **saw in the UK** a few weeks before we made this forecast

Safe reopening was **possible** but not **guaranteed**.

???
- so the message was that...
- to **emphasize the effect** of including both vaccination and the Delta variant in our model, we can consider **counterfactual forecasts** where we omit one or both of these mechanisms, as we did with the february forecast, where we omited the explicit variant effect
- i'll just show **one counterfactural** forecast now in the interest of time

---

???
- here is the exact same forecast **without vaccination** but **with the Delta variant**

The **stay-at-home order** followed by **expedited vaccination** stifled Delta, even with the **first-dose strategy**.

???
- vaccination helped... **even with Delta in the picture**

---

# Extended model

???
- so the success of these two forecasts really hinged on extending the base model to include vaccines and variants
- so let's dig into the extended model now

---

# Vaccination

???
- let's start with the vaccination model

---

background-image: url("data:image/png;base64,#figs/flowchart1.png")
background-position: center
background-size: contain

???
- **recall** the base model 
- this is the **epidemiology** we're working with
- for vaccination, we're going to add a **layer for each vaccine state**

---

background-image: url("data:image/png;base64,#figs/flowchart2.png")
background-position: center
background-size: contain

???
- each vaccine stratum has **its own copy** of the epidemiological compartments
- there are **five vaccine strata**...

### Vaccine effects

???
- what does the vaccine do in the model?

Reduction of **susceptibility**

???
- most important effect for forecasting infection reports

Decreased **disease severity**

---

# Variants of Concern

???
- so that's the vaccination model
- the extended model also includes the...

---

# Variant frequency

Want to account for **takeover** of a more fit, invading **variant**

???
- to start, we have to model the variant's frequency over time

???
- as an example, here is the variant model we used for the february forecast
- basically a **logistic model**, but it's not a simple regression

Curve from **epi-evolutionary theory**: [Day & Gandon 2007](https://onlinelibrary.wiley.com/doi/10.1111/j.1461-0248.2007.01091.x)

???
- derived from evo theory of the competition between two strains
- i won't show the equation but i will mention that

Depends on:

- the variant proportion **upon introduction**

- a **selective advantage** coefficient, of the variant against the dominant strain

???
- selective advantage coefficient controls the rate of growth of this curve
- we don't usually know the initial variant prevalence, `$p(0)$`, with a lot of certainty
- instead, we usually **peg this curve** to a point when we believe the estimate of the variant proportion is particularly reliable
- so for instance, here, we pegged the variant model to a **point prevalence study** conducted by Public Health Ontario in late january, where they checked every single sample positive for SARS-CoV-2 from a single day for Alpha

Variant is assumed to **increase transmission** and **decrease vaccine efficacy**.

???
- so we adjust transmission and vaccine efficacy in the model based on the current frequency of the variant

---

# Forecast from 20 Sep 2021 in Ontario

---

???
- here is our latest forecast
- here we wanted to explore the **potential effect of schools** (and other workplaces) reopening, which, as of last week could still not be gleaned from the data
- (we're approaching the period where we should reliably be able to calibrate the effect)
- so the scenarios are set up to simulate school reopening
- first row is **status quo**---just fit the latest transmission rate (in sept 2021) and propagate the effect forward
- still a rather large uncertainty associated with this fit
- second row in red is if transmission were to **increase** by **15%** in the first week of sept (coincident with schools reopening)
- third row is if transmission were to **decrease** by **15%** at the same point in time

The **effect of schools** is still unknown, so our projections have a **high degree of uncertainty**.

(read more [here](https://mac-theobio.github.io/forecasts/outputs/McMasterOntarioForecastsBlog2021-09-20))

???
- though we're starting to see the effect now and we seem to be in good shape...
- blog post detailing this forecast is up for the curious

---

## Summary

The COVID-19 pandemic **continues to evolve** and short-term forecasting is still needed to inform public health planning.

How do we ensure COVID-19 models continue to provide **useful and informative forecasts**?

Look **beyond infection reports**, monitor the **global situation**, and update models according to **changes in the epidemiological landscape**.

---

# Thank you for listening!

---

[https://github.com/mac-theobio/McMasterPandemic](https://github.com/mac-theobio/McMasterPandemic)

???
- i'll also mention that the model and forecasting software are implemented as a **publicly-available R package**, which you can find on **Github**

---

# Forecast updates

[https://mac-theobio.github.io/covid-19/](https://mac-theobio.github.io/covid-19/)

???
- blog posts detailing our latest forecasts

# Questions?