The design
Putting politics aside, it is interesting to consider certain statistical aspects of the design of the Russian trial of the vaccine against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The main reason for this is that Russia was the first country to "approve" such a vaccine, to the surprise of the whole world. The approval was announced by President Vladimir Putin himself, who assured that the vaccine in question (named Sputnik V)” has gone through all the necessary tests". The details of the trial can now be found in clinical.trials.gov as well as in a peer reviewed Lancet article.
The trial in question can be divided into two parts with respect to formulation and into two additional parts with respect to phase. In this short blog, we will concentrate on the Phase 2 part for one of the formulations with 20 subjects. This sample size was estimated based on data from previous clinical trials of MERS vaccines. An expectation of 99% efficiency, prompted inclusion of 16 participants in each study. Considering the possibility of early dropout of volunteers, it was decided that 20 volunteers should be enough.
Efficacy: The primary outcome measure for efficacy (immunogenicity) was change from baseline in antigen-specific antibody levels i.e. from day 0 to day 42 after vaccination in Phase 2.
Safety: Safety is measured by the number of participants with adverse events from day 0 to day 42 after vaccination in phase 2.
The concerns
The authors of the article state that the vaccine is safe and that it induced a strong immune response in all participants. A number of statistical methods are mentioned that have been used in the analyses and the data itself can be made available upon request. With this as a starting point, many questions remain unanswered and therefore, we argue that it is too early to conclude that the vaccine is working. This is especially true if "works" means that "the vaccine is safe and protects from infection by the virus in a general diverse population with respect to age, sex, ethnicity etc.". Here are some of the concerns:
As it turned out, the approval was based on Phase 1/2 data, while a large phase 3 study is expected to start soon. This is very unusual as Phase 1 and 2 trials usually only provide limited safety and proof of concept indications, while the definitive demonstrations of safety and efficacy are based on one or more Phase 3 trials.
Beyond the results of the trial, to be able to say that the vaccine "works", the primary efficacy endpoint (immunogeneticy) needs to translate into Phase 3 endpoints similar to those used by Moderna in their Phase 3 trial: "prevention of symptomatic COVID-19 disease", "prevention of severe COVID-19 disease, as defined by the need for hospitalization", "prevention of infection by SARS-CoV-2, the virus that causes COVID-19, regardless of symptoms".
Safety was assessed based on the total sample size of 76 subjects obtained by pooling all data from phases 1 and 2 as well as the two formulations. This number is still too low since adverse events occurring in e.g. 1 in 1000 would most likely not be detected at this stage but would still have catastrophic consequences when vaccinating hundreds of millions of subjects as can be seen from a simple calculation based on a Binomial distribution.
Ideally, the Gold standard for a trial is a randomized controlled clinical trial, while the current trial lacks control group and randomization. While this can be acceptable as an early trial, it cannot be the basis of approval.
Patients with the highest level of unmet need of a vaccine against SARS-CoV-2 are elderly men and women with multiple co-morbidities. The participants in this trial are mostly young fit young men with an average age of 26.4, which is way younger than those in the most prioritized group.
The participants in the trial are 100% Caucasians. However, ethnicity has been discussed as an issue related to COVID-19, whereas it is thought that subjects of African origin may be 5 times more prone to have complications.
An image of a of the coronavirus 2 (SARS-CoV-2)
In summary
Although the Sputnik V trial seems to be fit for purpose as a Phase 1/2 trial, it seems way too early to assume that it "has gone through all the necessary tests". More "real world evidence", from larger and longer trials using other endpoints and populations, is needed before claiming that the vaccine is safe and efficacious. Hopefully, this is what the Phase 3 trials of all currently competing vaccine candidates will deliver, to the benefit of the entire world.
Ziad Taib
ziad@atlasbiostat.com