top of page

“I have felt there was something more congenial and profitable to do than to compile statistics.” Joseph Lister

According to the WHO, “Washing your hands with soap and water or using alcohol-based hand rub kills viruses that may be on your hands”. Today, the vast majority of people believe this to be true and practice handwashing to avoid infection by the Corona virus.


Of course, it has not always been like that. The idea that washing your hands reduces the risk of infection, was first (*) proposed by Joseph Lister, in his seminal paper in the Lancet in 1867. Lister suggested that surgeons should wash their hands as well as instruments prior to performing surgery on patients, but his proposal was contested and met with skepticism during a long period of time. His opponents claimed that the whole thing was non-scientific lacking theoretical ground and demanded scientific arguments and statistics. In reality, however, professional pride and resistance to new ideas, could have played a certain role. And to be fair, around 1860, the existence of invisible, disease causing germs was still an idea advocated by a handful scientists like the French chemist Louis Pasteur. Moreover, Lister himself did not believe that statistics could add anything to the matter and had the attitude that “anything can be proven by statistics”. His reluctance to publish any “data” is explained by himself in the following quote from an article by Thomas Chlich. “The truth is, that life is short, and when every day begins, one has to consider what is the occupation which is most likely to be valuable. … I have felt there was something more congenial and profitable to do than to compile statistics”.

Eventually, Lister did publish some data in a paper from 1870: out of 40 subjects who were treated under the new cleaner conditions, 34 patients survived (85%) surgery while only 19 out of 35 patients treated using the old (control) group survived (54%). Today this type of data can be presented as a contingency table and analyzed in a variety of simple ways. The most obvious, albeit not the best, way is perhaps to compare the death rates in the two “arms” of the study directly. The corresponding absolute risk reduction is thus 0.46 - 0.15 = 0.31 giving a number needed to treat NNT of 3.22. Another approach is testing for association between survival and hand washing by performing Pearson’s chi square test or Fisher’s exact test. Applied to Lister’s data, these tests yield the p-values 0.003561 and 0.005018 respectively. Logistic regression provides a model based third alternative method where interesting covariates could be accounted for in the analysis. Of course, none of these was available in 1867 since Karl Pearson’s paper on the chi-2 test first appeared in the year 1900 and Fisher’s exact test article is from the year 1922.

By the time of the eruption of the Spanish flu in the united states, the idea of handwashing as a way to fight against the pandemic was more accepted. In November 1918, ads appeared in newspapers explaining to people exactly why washing their hands was important. Today, almost 100 years later, the use of statistical methods in what is called evidence-based medicine is universally accepted and Biostatistics has developed into an important branch of statistics.

 

(*) More about Joseph Lister can be found in a paper by Töhler from 2014. Interestingly, Steven Dugan reports in a blog about flu pandemics and handwashing that in 1846, the Hungarian doctor Ignaz Semmelweis had suspected that germs played a part in causing diseases, but that he had no proof. He instructed his staff at the Vienna General Hospital to wash their hands and instruments with soap and chlorinated water before delivering babies. As a result, the death rate of the mothers dropped dramatically, from 20% to 2%. Still, the medical community could not believe that handwashing so dramatically slowed the spread of germs and lowered death rates. Most doctors were offended by his claim.


Updated: Sep 10, 2020

The design

Putting politics aside, it is interesting to consider certain statistical aspects of the design of the Russian trial of the vaccine against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The main reason for this is that Russia was the first country to "approve" such a vaccine, to the surprise of the whole world. The approval was announced by President Vladimir Putin himself, who assured that the vaccine in question (named Sputnik V)” has gone through all the necessary tests". The details of the trial can now be found in clinical.trials.gov as well as in a peer reviewed Lancet article.


The trial in question can be divided into two parts with respect to formulation and into two additional parts with respect to phase. In this short blog, we will concentrate on the Phase 2 part for one of the formulations with 20 subjects. This sample size was estimated based on data from previous clinical trials of MERS vaccines. An expectation of 99% efficiency, prompted inclusion of 16 participants in each study. Considering the possibility of early dropout of volunteers, it was decided that 20 volunteers should be enough.

  • Efficacy: The primary outcome measure for efficacy (immunogenicity) was change from baseline in antigen-specific antibody levels i.e. from day 0 to day 42 after vaccination in Phase 2.

  • Safety: Safety is measured by the number of participants with adverse events from day 0 to day 42 after vaccination in phase 2.


The concerns

The authors of the article state that the vaccine is safe and that it induced a strong immune response in all participants. A number of statistical methods are mentioned that have been used in the analyses and the data itself can be made available upon request. With this as a starting point, many questions remain unanswered and therefore, we argue that it is too early to conclude that the vaccine is working. This is especially true if "works" means that "the vaccine is safe and protects from infection by the virus in a general diverse population with respect to age, sex, ethnicity etc.". Here are some of the concerns:

  • As it turned out, the approval was based on Phase 1/2 data, while a large phase 3 study is expected to start soon. This is very unusual as Phase 1 and 2 trials usually only provide limited safety and proof of concept indications, while the definitive demonstrations of safety and efficacy are based on one or more Phase 3 trials.


  • Beyond the results of the trial, to be able to say that the vaccine "works", the primary efficacy endpoint (immunogeneticy) needs to translate into Phase 3 endpoints similar to those used by Moderna in their Phase 3 trial: "prevention of symptomatic COVID-19 disease", "prevention of severe COVID-19 disease, as defined by the need for hospitalization", "prevention of infection by SARS-CoV-2, the virus that causes COVID-19, regardless of symptoms".


  • Safety was assessed based on the total sample size of 76 subjects obtained by pooling all data from phases 1 and 2 as well as the two formulations. This number is still too low since adverse events occurring in e.g. 1 in 1000 would most likely not be detected at this stage but would still have catastrophic consequences when vaccinating hundreds of millions of subjects as can be seen from a simple calculation based on a Binomial distribution.


  • Ideally, the Gold standard for a trial is a randomized controlled clinical trial, while the current trial lacks control group and randomization. While this can be acceptable as an early trial, it cannot be the basis of approval.


  • Patients with the highest level of unmet need of a vaccine against SARS-CoV-2 are elderly men and women with multiple co-morbidities. The participants in this trial are mostly young fit young men with an average age of 26.4, which is way younger than those in the most prioritized group.


  • The participants in the trial are 100% Caucasians. However, ethnicity has been discussed as an issue related to COVID-19, whereas it is thought that subjects of African origin may be 5 times more prone to have complications.


An image of a of the coronavirus 2 (SARS-CoV-2)



In summary

Although the Sputnik V trial seems to be fit for purpose as a Phase 1/2 trial, it seems way too early to assume that it "has gone through all the necessary tests". More "real world evidence", from larger and longer trials using other endpoints and populations, is needed before claiming that the vaccine is safe and efficacious. Hopefully, this is what the Phase 3 trials of all currently competing vaccine candidates will deliver, to the benefit of the entire world.


Ziad Taib

ziad@atlasbiostat.com

bottom of page