Reports from Israel suggest one dose of Pfizer vaccine could be less effective than expected

  • timer3 min
  • today2021-01-29 21:05
  • person
  • linkbmj.com
  1. Elisabeth Mahase
  2. The BMJ

Concerns have been raised over how much protection a single dose of the Pfizer BioNTech covid-19 vaccine provides, following reports from Israel that it is much lower than expected.

Israel, which, like the UK, is currently in its third national lockdown, has so far vaccinated more than 75% of its older people with at least one dose. Early reports from the vaccine rollout have suggested that the first dose led to a 33% reduction in cases of coronavirus1 compared with efficacy of at least 52% reported in clinical trials.2

A preliminary report from the Clalit Research Institute compared the infection data of 200 000 people aged 60 and over who were not vaccinated with the infection data of 200 000 people of the same age group who received one vaccine dose and were monitored for at least 11 days from the date of vaccination. On day 14 there was a “significant decrease of about 33% in the rate of positive tests for the coronavirus” among those who had been vaccinated. This decrease remained the same between days 15 and 17.

The report has raised concerns, as published results have suggested that the efficacy of the Pfizer vaccine was 52.4% between the first and second dose (spaced 21 days apart), and data assessed by Public Health England indicated it could be as much as 89% protective from day 15 to 21.

The Clalit Research Institute stressed, however, that its results included only people aged 60 and over—whereas Pfizer trials also included younger people—and that the findings have not yet been peer reviewed. Additionally, the Clalit study identified those infected according to laboratory tests of those who chose to be tested, while Pfizer’s studies only referred to the appearance of symptomatic disease.

In the UK, the vaccine policy prioritises getting as many at risk people vaccinated with one dose over ensuring people get two doses within the time specified in clinical trials. While there are data to suggest the chosen 12 week interval between the two doses is effective for the Oxford AstraZeneca vaccine, there are no data to support this interval for the Pfizer vaccine.3

As such, a leading statistician has written to UK health secretary Matt Hancock urging him to investigate the effects of the decision to extend the gap between the first and second dose of the Pfizer BioNTech vaccine.4

On the reports from Israel, UK chief scientific adviser Patrick Vallance, in an interview with Sky News, said, “We know that when you get into real world practice, things are seldom quite as good as clinical trials. I don’t know exactly what data Israel are looking at, but we need to look at this carefully.”

Meanwhile, professor of pharmacoepidemiology at the London School of Hygiene and Tropical Medicine Stephen Evans said, “The UK will soon have its own data showing efficacy after the first dose for the different vaccines currently in use and any policy changes should await more robust data. If, for example, efficacy after one dose was 33% but efficacy after two doses was 60%, the UK policy would still be justified. Similar arguments apply in relation to efficacy against serious disease.”

This article is made freely available for use in accordance with BMJ’s website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.https://bmj.com/coronavirus/usage

  1. Clalit study: decreased infection in the corona due to the vaccine. 13 January 2021. www.clalit.co.il/he/your_health/family/Pages/pfizer_covid_vac_effect.aspx.

Peter Doshi: Pfizer and Moderna’s “95% effective” vaccines—let’s be cautious and first see the full data – The BMJ

  • imer6 min
  • today2021-01-22 12:39
  • create2020-11-26 14:24
  • person
  • linkblogs.bmj.com

Only full transparency and rigorous scrutiny of the data will allow for informed decision making, argues Peter Doshi

In the United States, all eyes are on Pfizer and Moderna. The topline efficacy results from their experimental covid-19 vaccine trials are astounding at first glance. Pfizer says it recorded 170 covid-19 cases (in 44,000 volunteers), with a remarkable split: 162 in the placebo group versus 8 in the vaccine group. Meanwhile Moderna says 95 of 30,000 volunteers in its ongoing trial got covid-19: 90 on placebo versus 5 receiving the vaccine, leading both companies to claim around 95% efficacy.

Let’s put this in perspective. First, a relative risk reduction is being reported, not absolute risk reduction, which appears to be less than 1%. Second, these results refer to the trials’ primary endpoint of covid-19 of essentially any severity, and importantly not the vaccine’s ability to save lives, nor the ability to prevent infection, nor the efficacy in important subgroups (e.g. frail elderly). Those still remain unknown. Third, these results reflect a time point relatively soon after vaccination, and we know nothing about vaccine performance at 3, 6, or 12 months, so cannot compare these efficacy numbers against other vaccines like influenza vaccines (which are judged over a season). Fourth, children, adolescents, and immunocompromised individuals were largely excluded from the trials, so we still lack any data on these important populations.

I previously argued that the trials are studying the wrong endpoint, and for an urgent need to correct course and study more important endpoints like prevention of severe disease and transmission in high risk people. Yet, despite the existence of regulatory mechanisms for ensuring vaccine access while keeping the authorization bar high (which would allow placebo-controlled trials to continue long enough to answer the important question), it’s hard to avoid the impression that sponsors are claiming victory and wrapping up their trials (Pfizer has already sent trial participants a letter discussing “crossing over” from placebo to vaccine), and the FDA will now be under enormous pressure to rapidly authorize the vaccines.

But as conversation shifts to vaccine distribution, let’s not lose sight of the evidence. Independent scrutiny of the underlying trial data will increase trust and credibility of the results. There also might be important limitations to the trial findings we need to be aware of.

Most crucially, we need data-driven assurances that the studies were not inadvertently unblinded, by which I mean investigators or volunteers could make reasonable guesses as to which group they were in. Blinding is most important when measuring subjective endpoints like symptomatic covid-19, and differences in post-injection side-effects between vaccine and placebo might have allowed for educated guessing. Past placebo-controlled trials of influenza vaccine were not able to fully maintain blinding of vaccine status, and the recent “half dose” mishap in the Oxford covid-19 vaccine trial was apparently only noticed because of milder-than-expected side-effects. (And that is just one of many concerns with the Oxford trial.)

In contrast to a normal saline placebo, early phase trials suggested that systemic and local adverse events are common in those receiving vaccine. In one Pfizer trial, for example, more than half of the vaccinated participants experienced headache, muscle pain and chills—but the early phase trials were small, with large margins of error around the data. Few details from the large phase 3 studies have been released thus far. Moderna’s press release states that 9% experienced grade 3 myalgia and 10% grade 3 fatigue; Pfizer’s statement reported 3.8% experienced grade 3 fatigue and 2% grade 3 headache. Grade 3 adverse events are considered severe, defined as preventing daily activity. Mild and moderate severity reactions are bound to be far more common.

One way the trial’s raw data could facilitate an informed judgment as to whether any potential unblinding might have affected the results is by analyzing how often people with symptoms of covid-19 were referred for confirmatory SARS-CoV-2 testing. Without a referral for testing, a suspected covid-19 case could not become a confirmed covid-19 case, and thus is a crucial step in order to be counted as a primary event: lab-confirmed, symptomatic covid-19. Because some of the adverse reactions to the vaccine are themselves also symptoms of covid-19 (e.g. fever, muscle pain), one might expect a far larger proportion of people receiving vaccine to have been swabbed and tested for SARS-CoV-2 than those receiving placebo.

This assumes all people with symptoms would be tested, as one might expect would be the case. However the trial protocols for Moderna and Pfizer’s studies contain explicit language instructing investigators to use their clinical judgment to decide whether to refer people for testing. Moderna puts it this way:

It is important to note that some of the symptoms of COVID-19 overlap with solicited systemic ARs that are expected after vaccination with mRNA-1273 (eg, myalgia, headache, fever, and chills). During the first 7 days after vaccination, when these solicited ARs are common, Investigators should use their clinical judgement to decide if an NP swab should be collected.

This amounts to asking investigators to make guesses as to which intervention group patients were in. But when the disease and the vaccine side-effects overlap, how is a clinician to judge the cause without a test? And why were they asked, anyway?

Importantly, the instructions only refer to the first seven days following vaccination, leaving unclear what role clinician judgment could play in the key days afterward, when cases of covid-19 could begin counting towards the primary endpoint. (For Pfizer, 7 days after the 2nd dose. For Moderna, 14 days.)

In a proper trial, all cases of covid-19 should have been recorded, no matter which arm of the trial the case occurred in. (In epidemiology terms, there should be no ascertainment bias, or differential measurement error). It’s even become common sense in the Covid era: “test, test, test.” But if referrals for testing were not provided to all individuals with symptoms of covid-19—for example because an assumption was made that the symptoms were due to side-effects of the vaccine—cases could go uncounted.

Data on pain and fever reducing medicines also deserve scrutiny. Symptoms resulting from a SARS-CoV-2 infection (e.g. fever or body aches) can be suppressed by pain and fever reducing medicines. If people in the vaccine arm took such medicines prophylactically, more often, or for a longer duration of time than those in the placebo arm, this could have led to greater suppression of covid-19 symptoms following SARS-CoV-2 infection in the vaccine arm, translating into a reduced likelihood of being suspected for covid-19, reduced likelihood of testing, and therefore reduced likelihood of meeting the primary endpoint. But in such a scenario, the effect was driven by the medicines, not the vaccine.

Neither Moderna nor Pfizer have released any samples of written materials provided to patients, so it is unclear what, if any, instructions patients were given regarding the use of medicines to treat side effects following vaccination, but the informed consent form for Johnson and Johnson’s vaccine trial provides such a recommendation:

“Following administration of Ad26.COV2.S, fever, muscle aches and headache appear to be more common in younger adults and can be severe. For this reason, we recommend you take a fever reducer or pain reliever if symptoms appear after receiving the vaccination, or upon your study doctor’s recommendation.”

There may be much more complexity to the “95% effective” announcement than meets the eye—or perhaps not. Only full transparency and rigorous scrutiny of the data will allow for informed decision making. The data must be made public.

Spanish translation of this article

German translation of this article

Peter Doshi, associate editor, The BMJ.

Competing interests: I have been pursuing the public release of vaccine trial protocols, and have co-signed open letters calling for independence and transparency in covid-19 vaccine related decision making.