If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Department of Clinical Research, University Hospital Basel, University of Basel, Basel, SwitzerlandMeta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USAMeta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany
Corresponding author: Dr John P.A. Ioannidis, Departments of Medicine, Health Research and Policy, Biomedical Data Science, and Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University, 1265 Welch Rd, Medical School Office Building Room X306, Stanford, California 94305, USA.
Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USAMeta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, GermanyDepartment of Medicine, Stanford University School of Medicine, Stanford, California, USADepartment of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, USADepartment of Biomedical Data Science, Stanford University School of Medicine, Stanford, California, USADepartment of Statistics, Stanford University School of Humanities and Sciences, Stanford, California, USA
The COVID-19 crisis led to a flurry of clinical trials activity. The COVID-evidence database shows 2814 COVID-19 randomized trials registered as of February 16, 2021. Most were small (only 18% have a planned sample size > 500) and the rare completed ones have not provided published results promptly (only 283 trial publications as of February 2021). Small randomized trials and observational, nonrandomized analyses have not had a successful track record and have generated misleading expectations. Different large trials on the same intervention have generally been far more efficient in producing timely and consistent evidence. The rapid generation of evidence and accelerated dissemination of results have led to new challenges for systematic reviews and meta-analyses (eg, rapid, living, and scoping reviews). Pressure to regulatory agencies has also mounted with massive emergency authorizations, but some of them have had to be revoked. Pandemic circumstances have disrupted the way trials are conducted; therefore, new methods have been developed and adopted more widely to facilitate recruitment, consent, and overall trial conduct. On the basis of the COVID-19 experience and its challenges, planning of several large, efficient trials, and wider use of adaptive designs might change the future of clinical research. Pragmatism, integration in clinical care, efficient administration, promotion of collaborative structures, and enhanced integration of existing data and facilities might be several of the legacies of COVID-19 on future randomized trials.
Résumé
La crise de la COVID-19 a entraîné une vague d'essais cliniques. La base de données COVID-evidence répertorie 2 814 essais randomisés sur la COVID-19 consignés en date du 16 février 2021. Il s'agit pour la plupart d'essais de petite envergure (la taille prévue de l’échantillon dépassait 500 pour seulement 18 % d'entre eux) et les rares essais achevés n'ont pas rapidement mené à la publication de résultats (seulement 283 essais publiés en février 2021). Les petits essais randomisés et les analyses d'observation non randomisées n'ont pas fait leurs preuves et ont généré des attentes trompeuses. Les divers essais de plus grande envergure sur la même intervention ont généralement été beaucoup plus efficaces pour produire des données probantes constantes dans les meilleurs délais. La production rapide de données probantes et la diffusion accélérée des résultats ont entraîné de nouveaux défis pour les revues systématiques et les méta-analyses (p. ex., revues rapides, revues avec mises à jour et revues de la portée). La pression sur les organismes de réglementation s'est également accentuée en raison de très nombreuses autorisations d'urgence, mais certaines d'entre elles ont dû être révoquées. Les circonstances de la pandémie ont bouleversé la façon dont les essais sont menés; par conséquent, de nouvelles méthodes ont été élaborées, puis adoptées plus largement afin de faciliter le recrutement et le consentement, ainsi que la conduite générale des essais. Selon l'expérience apportée par la COVID-19 et ses défis, la planification de plusieurs essais efficients de grande envergure ainsi qu'une utilisation plus large des plans adaptatifs pourraient changer l'avenir de la recherche clinique. Le pragmatisme, l'intégration aux soins cliniques, l'administration efficiente, la promotion des structures de collaboration et une plus grande intégration des données et des installations existantes pourraient compter parmi les répercussions de la COVID-19 sur les futurs essais randomisés.
The COVID-19 pandemic has required an ultra-urgent response to generate evidence to handle a major public health crisis. Effective treatments, vaccines, and other measures were important to develop, test, and implement as quickly as possible. An unprecedent number of randomized controlled trials (RCTs) to assess therapeutics for COVID-19 were initiated, and to a lesser extent for preventive measures, in particular, vaccines.
For many other influential decisions (eg, nonpharmaceutical interventions), evidence sadly depended almost exclusively on precarious modelling and observational data.
In this report, we review the challenges of and what we have learned from the COVID-19 clinical research agenda in the first 14 months of the pandemic and how we can put this knowledge to good use moving forward. The lessons learned are potentially important not only for COVID-19, but, perhaps most importantly for the future of RCTs in general.
The COVID-19 Agenda of Randomized Trials
In March 2020, the COVID-evidence database (www.covid-evidence.org) was launched with the aim to gain insight on the COVID-19 clinical research agenda.
Janiaud P, Axfors C, Saccilotto R, Hemkens L. COVID-evidence: a living database of trials on interventions for COVID-19. Available at: https://osf.io/gehfx. Accessed March 1, 2021.
Using a multimethod approach combining peer-reviewed search strategies of study registries and publication databases, continuous automated extraction of search results, automated classifications combined with manual screening and data extraction, and quality control through expert review, it provides information about worldwide planned, ongoing, and completed RCTs on any intervention to treat or prevent SARS-CoV-2-infection. In the first 100 days of the pandemic, more than 500 RCTs had already been registered on ClinicalTrials.gov and the World Health Organization International Clinical Registry Platform. Most trials were to assess interventions for treating patients with COVID-19; conversely only 11% focused on interventions to prevent COVID-19 infection, and none assessed social distancing or lockdown measures. RCTs were mostly small and often investigated the same interventions.
Although these clinical research efforts were much needed, the excessive duplication of efforts and lack of collaboration in the early COVID-19 clinical research agenda was putting it at risk of creating research waste
characterized by unnecessary duplication of trials, poor study designs, and insufficient reporting of results.
The cumulative number of RCTs registered has steadily increased (Fig. 1A), with a total of 2814 RCTs included in COVID-evidence as of February 16, 2021. Of note, the monthly number of RCTs registered has been slowly decreasing after a peak in the number of registrations in April 2020 (n = 592; Fig. 1A). Although the clinical research agenda was initially largely dominated by trials conducted in Asia (mostly China), by March 2020, the number of trials registered in the rest of the world quickly increased after the spread of the pandemic (Fig. 1B). Most trials have continued to be small (Fig. 2) with only 18% of the RCTs planned to include > 500 participants. Treatment with hydroxychloroquine was explored by a total of 304 RCTs of 2814 RCTs (in the first 100 days, every 1 in 6 RCTs investigated this intervention). The results from the Randomised Evaluation of COVID-19 Therapy (RECOVERY) showed no clinical benefit
Overall, as of March 2, 2021, there were 6 drug treatments (remdesivir, COVID-19 convalescent plasma, bamlanivimab, baricitinib in combination with remdesivir, casirivimab and imdevimab, and bamlanivimab and etesevimab) with Emergency Use Authorizations (EUAs) by the US Food and Drug Administration (FDA)
Figure 1Number of registrations over time. (A) Cumulative registration (blue) and number of registrations per month (red). (B) Cumulative number of registrations per month according to continent. Sixteen trials were registered before January 1, 2020 and adapted their protocol to include patients with COVID-19.
Figure 2Planned sample size for the 2814 registered randomized controlled trials (RCTs; trials registered as of February 16, 2021). For 97 registered RCTs the planned sample size was not available. The dotted lines represent the intersection of the cumulative frequency of 82% of the RCTs that included < 500 participants.
Drug development is a long and costly endeavour, antinomic to the urgent need of finding therapeutics for an acute pandemic. Therefore, most of the RCTs focused on testing the repurposing of already existing drugs that had been already approved or had been under investigation for other indications (Table 1).
However, a review that focused on the reporting of clinical results for 19 potential COVID-19 drugs showed that 40% of the completed trials that assessed those drugs before the pandemic, did not report their results on ClinicalTrials.gov or in the scientific literature.
For hydroxychloroquine, 37% of the trials were unreported.
Table 1Overview of RCTs assessing vaccines and well-known interventions (most explored in RECOVERY; on the basis of 2814 RCTs registered as of February 16, 2021)
Missing planned sample size for 25 RCTs for hydroxychloroquine; 5 for vaccine; 2 for ritonavir; 2 for lopinavir; 7 for azithromycin; 1 for tocilizumab; 1 for dexamethasone; and 4 for chloroquine. Of note, lopinavir and ritonavir were either studied together or separately. Chloroquine and remdesivir were not assessed in RECOVERY.
Since the beginning of the pandemic, the most pressing clinical question has been, “How do we prevent deaths?” However, it has been previously shown that only 15.8% of registered RCTs that assessed a treatment intended to use mortality as a primary end point.
; the most common one being a 7-point ordinal scale (1, death; 2, hospitalized with invasive mechanical ventilation or extracorporeal membrane oxygenation; 3, hospitalized, with noninvasive ventilation or high-flow oxygen; 4, hospitalized and requiring low-flow supplemental oxygen; 5, hospitalized not requiring supplemental oxygen but requiring ongoing medical care (COVID-19-related or otherwise); 6, hospitalized not requiring supplemental oxygen no longer requiring ongoing medical care; and 7, not hospitalized). Although ordinal scales have the advantages to capture data on multiple clinical states ranging from cure to death, and to gain in statistical power,
Another predominant issue is the representativeness of the patients included in the COVID-19 clinical research agenda. An early review that assessed 12 published RCTs showed that the median age of the included patients was 56.3 years and most exclusions were on the basis of comorbidities significantly more prevalent in patients 65 years old and older, highlighting the under-representation of elderly individuals
Population-level COVID-19 mortality risk for non-elderly individuals overall and for non-elderly individuals without underlying diseases in pandemic epicenters.
Of note, the extraordinary role of RECOVERY is reflected also by its representativeness with approximately 20% of the patients aged 80 years and older.
and it is unlikely to be different in the COVID-19 era. The issue is more severe for COVID-19 trials and the under-representation might be worse for elderly people who are also frail. Frail nursing home residents accounted for a large proportion of deaths in most European countries and the United States,
but were not represented properly in either therapeutic or vaccine trials. Other reviews have highlighted the lack of inclusion of pregnant and lactating women, and children.
Effectiveness of adding a mask recommendation to other public health measures to prevent SARS-CoV-2 infection in Danish mask wearers: a randomized controlled trial.
The search for a vaccine was another cornerstone of the strategy against COVID-19. As of mid-February 2021, there were 175 vaccine RCTs registered planning to include 899,808 participants (Table 1). Vaccine trials represented 6.2% of the clinical research agenda compared with 10.8% for hydroxychloroquine trials but included twice as many participants (899,808 vs 436,403 participants, respectively). Nineteen of the registered RCTs had a publication. Despite the unprecedented speed of the clinical research agenda, it resulted in only very few authorized vaccines: Pfizer-BioNTech, Janssen, and Moderna vaccines obtained EUAs by the FDA
but with the duplication of many small studies during the pandemic the phenomenon has been aggravated.
On the basis of the information in clinical trial registries, as of mid-February 2021, 168 of 2814 registered RCTs (6%) were listed as being completed and 41 (1%) as terminated early. Of note, only 14 RCTs had results posted in the registry. Delays in the research agenda have been observed, affecting its potential to deliver much needed evidence.
Because registries might be not consistently updated, it is important to acknowledge that some more trials have been completed.
The COVID-evidence database also contains 283 publications of RCTs reporting results; 171 of which were linked to a registered RCT. In-depth evaluation was conducted for 50% of them (141/283), randomly selected: 73% (103/141) seemed to have completed the planned protocol, 19% (27/141) were terminated early, and 8% (11/141) were ongoing and presenting preliminary or arm-specific results. Of the 27 trials terminated early, the most common reasons, among others, were enrollment difficulties with a decrease in number of cases (n = 13), futility (n = 5), and emergence of new evidence (n = 5).
Major Lessons Learned
The COVID-19 experience has informed some of the longstanding debates about the optimal design and conduct of an RCT agenda. The key lessons learned are summarized in Box 1.
Observational studies, although essential when pertinent, have generated misleading results later contradicted by subsequent randomized evidence.
•
Large trials can rapidly generate evidence on outcomes that matter.
•
Adaptive platform trials embedded in routine clinical care have efficiently and largely contributed to evidence generation and created synergies for collaborations.
•
Uncertainties and lack of harmonization in decision-making and dissemination of research results might contribute to skepticism.
•
The lockdown and social distancing measures have accelerated the implementation of innovative and remote solutions to conduct clinical trials.
•
Regulatory agencies and ethics and monitoring committees have tried to streamline their processes.
Traditionally, most medical decisions in the past have been informed from results of mostly small trials. Large trials have been uncommon. Although large trials were deemed desirable, and calls for large, simple trials were made decades ago,
their conduct has been increasingly perceived to be difficult, requiring a long time to recruit and achieve sufficient follow-up. The choice of small trials has also generated high dependence on surrogate and composite outcomes rather than on simple, clinical end points that truly matter. During the COVID-19 pandemic, however, our cumulative experience shows that the paradigm of large, definitive trials can work and can provide rapid evidence on outcomes that matter.
Conversely, it is questionable whether much was learned from the many hundreds of small trials that were initiated. Most of these small trials were underpowered to generate any results in a time frame that would be deemed useful for dealing with the pandemic, and many might never generate any worthwhile results or might even be abandoned because of futility. Small trials seem to have disadvantages at multiple fronts, including those previously perceived as potential advantages. Not only is their outcome selection and pragmatism questioned, but also the timing for accumulating robust evidence from multiple small trials can be actually slower than the timing of a large trial. Small trials and their meta-analyses have led to misleading inferences in the past in fields that had the benefit of conducting also large trials (eg, cardiovascular medicine). Interventions like magnesium in patients with myocardial infarction offer cautionary tales.
ISIS-4: a randomised factorial trial assessing early oral captopril, oral mononitrate, and intravenous magnesium sulphate in 58,050 patients with suspected acute myocardial infarction. ISIS-4 (Fourth International Study of Infarct Survival) Collaborative Group.
The COVID-19 experience reinforced the need for timely, large trials.
Observational data vs randomized trials
In the past decade, there has been a major push to promote the use of observational data for obtaining large-scale, so-called real-world evidence in a timely fashion for comparative effectiveness questions. This trend has been questioned as to its validity by some methodologists and some empirical data,
but has been heavily endorsed by many proponents. The track record of nonrandomized clinical trials and of observational data set analyses of effectiveness during the pandemic has been rather disappointing with regard to providing reliable evidence. This is reminiscent of past disappointments with high-profile observational data refuted by mega-trials in cardiovascular medicine and other fields (eg, hormone replacement or vitamin E).
For example, in August 2020, the FDA issued an EUA for convalescent plasma for the treatment of hospitalized patients with COVID-19 in a situation without sound randomized trial evidence
Interestingly, within 5 months, large-scale randomized evidence contradicted not only the beneficial results of this analysis but also many of the perceived limitations made in the preprint against conducting RCTs in the pandemic situation to assess plasma. The large RECOVERY trial published results for the convalescent plasma arm in January 2021, showing the clear feasibility to randomize 10,406 patients and showing no benefit on mortality.
Similarly for hydroxychloroquine, numerous very large observational studies contributed to high expectations but were contradicted by subsequent randomized evidence. The lowest point was the rapid publication and retraction of data from a large-scale observational analysis
Mehra MR, Desai SS, Ruschitzka F, Patel AN. RETRACTED: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis [e-pub ahead of print] [erratum in: Lancet https://doi:org/10.1016/S0140-6736(20)31249-6] [retraction in: Lancet https://doi:org/10.1016/S0140-6736(20)31324-6]. Lancet. doi:10.1016/S0140-6736(20)31180-6, Accessed February 26, 2021.
Faking large-scale observational data is “easier” than faking an equivalent-size RCT, as illustrated by the former example and the fact that large-scale clinical trials have typically much stronger oversight, by regulators and independent monitoring committees.
Of note, some very small randomized trials also suggested potential benefits (eg, for convalescent plasma). It is unknown whether this might reflect chance findings, especially with surrogate outcomes and superimposed selective reporting or publication. Sadly, there is some evidence from the pre-COVID-19 era that a large share of small RCTs, perhaps most, from some jurisdictions might be “zombie trials” because their data suggest fraud and/or are implausible.
With an increasing number of RCTs done in jurisdictions without strong clinical research tradition and with weak oversight, these problems might become more prominent.
However, nonrandomized evidence is likely to continue to be extremely important for mapping adverse events for licensed interventions when these are applied widely in the community, outside of a trial setting, and when long-term follow-up is afforded. This is particularly relevant for urgently authorized vaccines. A challenge with emerging observational data on harms is that the evidence base is often fluid, changing, and open to different interpretation by regulatory agencies, as in the case of adverse events from the Astra-Zeneca
However, placebo-controlled trials might become again possible or even necessary in the future, if new, relevant questions arise (eg, the speculated need for a third dose of vaccination).
Replication and meta-analysis
Although large RCTs seem to be the solution in the COVID-19 pandemic, we have to be cautious to avoid claiming that they are always correct. Large RCTs can have their own shortfalls. For example, they might include multiple sites, some of which are inexperienced and thus get unreliable results affecting the final result for the entire trial.
Moreover, they might not be able to examine sources of potential heterogeneity (eg, different dose or timing of the intervention) that would be provided when the same sample size comes from several trials that are combined in a meta-analysis.
It is thus useful to have more than just 1 large trial. For COVID-19, this was the case in many circumstances. The World Health Organization Solidarity and RECOVERY adaptive trials, for example, both assessed hydroxychloroquine and consistently showed no clinical benefit on 28-day mortality (rate ratio, 1.19; 95% confidence interval, 0.89-1.59 and rate ratio, 1.11; 95% confidence interval, 0.98- 1.26, respectively).
COVID-19 has also led to rethinking the role, conduct, and dissemination of systematic reviews and meta-analyses. With the rapid evolution of the emerging evidence, many traditional systematic reviews and meta-analyses are at risk of being outdated before being published. Rapid reviews, scoping reviews, and living reviews are becoming increasingly popular;
Boutron I, Chaimani A, Meerpohl JJ, et al. Interventions for preventing and treating COVID-19: living systematic reviews and network meta-analyses. Available at: https://zenodo.org/record/4018607#.YDj_r5P0k1I. Accessed February 26, 2021.
very large numbers of such evidence syntheses were undertaken and published very quickly.
In addition, many systematic reviews have been conducted on observational studies often concluding that RCTs are needed to confirm the observed results. Such as, for example, the systematic reviews on physical distancing, face masks, and eye protection to prevent COVID-19, which provided evidence graded as “moderate certainty” for social distancing and “low certainty” for the others.
Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis.
More robust randomized evidence is still lacking and is needed to confirm the observed associations.
Adaptive designs
The COVID-19 clinical research agenda was triggered by the urgent need to determine effective therapies; nevertheless, most potential therapies were evaluated in small 2-arm trials rendering the process extremely inefficient. Adaptive platform trials give the opportunity to simultaneously assess multiple interventions all the while allowing to drop or include new interventions as new evidence emerges.
Such designs aim to maximize flexibility in the trial without compromising its integrity and validity. By analyzing accumulating data through prespecified interim analyses, prompt adaptations can be made such as, for example, stopping early a treatment arm for superiority or futility and changing the randomization allocation ratio, in favour of the most promising treatment arms. The RECOVERY platform trial is probably the most illustrative example of a successful adaptive design
The FDA issued an EUA on March 28, 2020, which was then revoked on June 15, 2020. The EMA recommended that it should only be used in clinical trials or national emergency use programs and issued reports on safety concerns.
Lopinavir-ritonavir
June 29, 2020
October 5, 2020
Azithromycin
December 14, 2020
December 14, 2020 (preliminary results)
February 2, 2021
Dexamethasone
June 16, 2020
June 22, 2020 (preliminary results)
July 17, 2020 (preliminary results) February 25, 2021 (final report)
The EMA endorsed the use of dexamethasone on September 18, 2020. The UK authorized the use of dexamethasone on June 19, 2020.
Convalescent plasma
January 15, 2021
The FDA issued an EUA on August 23, 2020 for hospitalized COVID-19 patients. The EUA issued a revision on February 4, 2021, excluding low titer.
Tocilizumab
February 11, 2020
NCT04292899 (severe COVID-19)
Remdesivir
April 29, 2020
May 27, 2020
The FDA issued an EUA on May 1, 2020 for severe COVID-19, expanded on August 28, 2020 to all hospitalized patients. On October 22, 2020, remdesivir was approved for adults and pediatric patients (12 years of age or older and weighing at least 40 kg) for the treatment of COVID-19 requiring hospitalization. The EMA granted a conditional marketing authorization on June 25, 2020.
NCT04292730 (moderate COVID-19)
Remdesivir
June 1, 2020
August 21, 2020
NCT04368728
Pfizer-BioNTech vaccine: BNT162b1 and BNT162b2
Phase 3: November 9, 2020 (interim analyses - BNT162b2) November 18, 2020 (final efficacy analysis- BNT162b2)
Phase 1/2: July 1, 2020 (interim report - BNT162b1) August 28, 2020 (preliminary results - BNT162b2)
Phase 1/2: August 12, 2020 (interim report, BNT162b1 and BNT162b2) October 14, 2020 (safety results, BNT162b1 and BNT162b2) Phase 3: December 10, 2020 (safety and efficacy, BNT162b2)
The FDA issued an EUA on December 11, 2020. The EMA granted conditional marketing authorization for BNT162b2 on December 21, 2020.
NCT04470427
Moderna vaccine
Phase 1: May 18, 2020 (interim analysis) Phase 3: November 16, 2020 (primary end point)
Phase 3: December 30, 2020
The FDA issued an EUA on December 18, 2020. The EMA granted conditional marketing authorization on January 6, 2021.
November 23, 2020 (primary end point pool analysis phase 2/3 and phase 3)
Phase 2/3: February 4, 2021 (UK efficacy results) Phase 2/3 and 3: February 1, 2021 (efficacy booster dose)
Phase 1/2: July 20, 2020 (preliminary report on safety) December 17, 2020 (exploratory analysis on the immune response) Phase 2/3 and 3: November 19, 2020 (UK preliminary results) December 8, 2020 (interim pooled analysis of 4 studies) February 19, 2021 (efficacy booster dose)
The EMA granted conditional marketing authorization on January 29, 2021. In March 2021, several European countries decided to pause the vaccination because of events involving blood clots. The EMA led an investigation and maintained its position that the benefits outweigh the risk of side effects.
Phase 1/2a: September 25, 2020 (interim analysis) Phase 3: January 29, 2021 (interim analysis)
Phase 1/2a: September 25, 2020 (interim analysis)
Phase 1/2a: January 13, 2021 (interim analysis) March 11, 2020 (results from a unique centre) Phase 3: April 21, 2021 (primary analysis)
The FDA issued an EUA on February 27, 2021. The EMA granted conditional marketing authorization on March 11, 2021. In April 2021, the FDA recommended pausing the vaccination because of rare blood clots events. After review of the cases, the FDA lifted the recommended pause.
NCT04436471 NCT04437875 NCT04530396
Sputnik vaccine (Gamaleya Research Institute of Epidemiology and Microbiology, Health Ministry of the Russian Federation, Moscow, Russia)
November 11, 2020 (interim phase 3 efficacy) December 14, 2020 (efficacy analysis)
September 4, 2020 (phase 1/2) February 2, 2021 (interim phase 3 efficacy)
Approved by the Russian Federation on August 11, 2020.
EMA, European Medicines Agency; EUA, Emergency Use Authorization; FDA, US Food and Drug Administration; RECOVERY, Randomised Evaluation of COVID-19 Therapy; UK, United Kingdom.
Platform trial assessing multiple intervention arms with a factorial design (ie, a patient can receive more than 1 experimental treatment).
•
Adaptive design allowing to add or drop intervention arms as evidence emerges.
•
Simplified online randomization easing the work of investigational sites.
•
Pragmatic features with broad eligibility criteria (ie, hospitalized, confirmed, or suspected SARS-CoV-2 infection, and no specific contraindication to their participation).
•
Data on primary end point, all-cause mortality at 28 days, are collected through a unique online follow-up form filled out at death, discharge, or at day 28, whichever comes first.
•
Linkage to national health registries to make use of routinely collected data.
•
As of February 2021, more than 36,000 patients have been recruited in more than 180 investigational sites (of which 2 are from outside of the United Kingdom).
Other trials were originally designed for other conditions. For example, the Randomised, Embedded, Multi-factorial, Adaptive Platform Trial for Community-Acquired Pneumonia (REMAP-CAP) trial was designed
REMAP-CAP. A Randomised, Embedded, Multi-factorial, Adaptive Platform Trial for Community-Acquired Pneumonia. Available at: https://www.remapcap.org. Accessed February 26, 2021.
as an adaptive platform trial to assess interventions for community-acquired pneumonia. As early as March 2020, they adapted their protocol to include patients with COVID-19 and to assess COVID-19-relevant interventions.
In the 2814 registered RCTs in COVID-evidence, 15 were registered before January 2020 but then included patients with COVID-19.
Dissemination, transparency, data-sharing
COVID-19 is also characterized by an unprecedented volume of research being quickly disseminated in the scientific literature and preprint servers with over 200,000 items published by early December 2020.
and medRxiv saw its median number of daily submission increasing by > 8-fold from before the pandemic (median, 6; interquartile range, 4-8) compared with during the pandemic (median, 51; interquartile range, 23-83) reaching a total of 7695 posted articles between July 2019 and June 2020.
The publication process was drastically shorter for COVID-related articles ranging from 6 to 20 days from submission to publication vs > 100 days for non-COVID-19 related articles.
Concerns have been raised regarding the quality of the publications being disseminated with most not presenting original data (ie, expert opinion pieces) and when original data were presented > 80% showed intermediate to high risk of bias and included small numbers of patients.
Many journals have experienced a massive increase in submissions putting heightened pressure on editors and peer reviewers. Many suboptimal reports might pass through thinned-out review filters
and/or because they fit favoured narratives. In addition, dissemination of misinformation has been exacerbated by media attention and mob behaviour in social media platforms
The rapidly evolving and fragmented landscape of the COVID-19 pandemic requires good research practice, reproducibility, and transparency to ensure informed decision-making and maintain public trust.
Data sharing allows reanalysis, synthesis, and building on existing evidence, increasing the reproducibility and transparency of the scientific method. Although those advantages of data sharing are well known and have been endorsed by the International Committee of Medical Journal Editors, the gap between the intent of sharing data and the actual availability of the data remains wide
Regulatory authorities throughout the world have been under immense pressure to make decisions as the evidence appeared, sometimes regardless of the source of the evidence and its credibility. For example, press releases of trial results have had a substantial effect with the FDA revoking the EUA for hydroxychloroquine just 10 days after or dexamethasone being recommended by the United Kingdom authorities just 3 days after the press releases from RECOVERY (Table 2).
Up to April 2020, the FDA had issued 174 EUAs; most were for diagnostic tests for various disease outbreaks and only 3 had been issued in the past for therapeutics, which were all for the treatment of the 2009 influenza A (H1N1). Currently there are 6 EUAs for therapeutics and 3 EUAs for vaccines for COVID-19
but no COVID-19 therapeutics have yet received a full or accelerated approval from the FDA. EUAs are crucial in a public health crisis but they should not compromise the integrity of the scientific method.
Doing trials under pandemic disruption
Conducting non-COVID-19 trials during the pandemic has been challenging:
and hospitals overwhelmed because of COVID-19 cases might have redirected their resources away from clinical research. As early as March 2020, the FDA had issued guidance for conducting trials during the pandemic
ensuring patient safety all the while minimizing the effect on the conduct of trials not related to COVID-19. Yet, from January to March 2020, an average of 1147 non-COVID-19 trials were stopped (ie, recruiting status changed to active, not recruiting, suspended, completed, terminated, or withdrawn) per month vs 638 trials per month in 2019.
Although the non-COVID-19 clinical research agenda was disrupted, this experience has also emphasised the need to implement innovative solutions such as online recruitment and informed consent,
streamlining interactions between stakeholders (funders, site staff, trial manager, and steering and data monitoring committees [DMCs]) including remote monitoring of clinical trials and shorter regulatory processes (detailed in the section: Doing Trials Differently in the Future), remote delivery of interventions using mail drop-off of pills or devices,
and collection of outcomes data using electronic health records. Some of these lessons and gained skills might be useful to implement also in the post-pandemic world and might enhance the efficiency of the clinical research enterprise at large.
Doing Trials Differently in the Future
The COVID-19 situation highlighted a number of excellent approaches to overcome research challenges in the pandemic. Many of them represent promising solutions for the most common perceived limitations of RCTs in general.
Planning
Putting research in context is critical to increase research value and return of investment. Instead of separately planning small trials, joining forces and contributing to large trials is key. This requires that such large trials are identifiable and open for collaboration. Because large collaborative studies acquire great prestige, visibility, and citations, they might offer a stronger incentive for investigators to join them rather than perform their own underpowered study.
Platforms such as the COVID-evidence project aim to show the trial research agenda on a website and actively bring together research groups and trial teams to collaborate.
Janiaud P, Axfors C, Saccilotto R, Hemkens L. COVID-evidence: a living database of trials on interventions for COVID-19. Available at: https://osf.io/gehfx. Accessed March 1, 2021.
Such trials need to be expandable (ie, designed in a way that centres can easily be added), to lower the bar for collaboration and contribution. The RECOVERY trial had such expandability and started with > 130 hospitals recruiting 1000 patients in 2 weeks, and 176 United Kingdom hospitals had then recruited 10,000 patients in 2 months.
Approximately 1 year later, in February 2021, it even expanded to a different continent with centres in Nepal and Indonesia, then having recruited more than 36,000 patients globally.
A critical issue is the avoidance of creating an environment in which trials compete for the same patients (ie, 2 trials conducted in the same location with similar eligibility criteria). Trials often use the exclusion criterion “enrollment in another clinical trial.” However, this needs to be carefully considered when planning a clinical trial and allowing patients to participate in a second treatment protocol might be desirable when no other treatments are available.
Centres with limited research expertise might not be able to perform rigourous research under acute circumstances when there is limited or no sufficient time and resources for training and for auditing centre performance. Careful planning of included centres should be done upstream because inclusion of inexperienced, poorly performing centres might affect the overall trial results, as has been shown in some examples in the pre-COVID era.
The COVID-19 trial research agenda has clearly shown that RCTs can be launched very quickly. The setup of novel data infrastructures is time-consuming but can often be avoided. Using available infrastructures, either from existing clinical trials, cohorts, registries, or from routine care can be extremely helpful to rapidly generate evidence. The REMAP-CAP trial used its existing trial processes and infrastructures
and has been expanded for COVID-19 patients. The RECOVERY trial used nationwide electronic health record data for outcome assessments and combined this with short-term active data collection.
Protocol for a partially nested randomised controlled trial to evaluate the effectiveness of the scleroderma patient-centered intervention network COVID-19 home-isolation activities together (SPIN-CHAT) program to reduce anxiety among at-risk scleroderma patients.
Future pandemic research preparedness might include the setup of very large, nationwide cohorts in which multiple interventions could be tested as in Trials Within Cohorts (TWiCs).
Several trials showed how clinical research can be embedded in routine clinical care, becoming even the standard of care. The RECOVERY trial with its 3 inclusion criteria and streamlined recruitment processes avoided artificial research settings by embedding the trial in routine care and ensuring inclusivity of the recruitment. One of 6 patients hospitalized for COVID-19 in the United Kingdom took part in RECOVERY.
This resembled real-world treatment choices and not only boosted trial feasibility—by design, it also maximized the applicability of the results. Similar highly pragmatic trials that are embedded in clinical routine might systematically evaluate treatment strategies, ideally for all relevant health care choices also beyond COVID-19.
Streamlined administration and collaborations
Funders, institutional review boards (ie, ethics committees), DMCs, and regulators have shown remarkable adaptation and flexibility when confronted with the urgency of the pandemic. Many countries have implemented a fast-track procedure for the authorization of clinical trials including ethical reviews.
In Switzerland, the Swiss Association of Research Ethics Committees lists all approved and submitted but not yet approved trials, promoting transparency of their processes.
DMCs play a crucial role in the successful conduct of adaptive platform trials by reviewing interim analyses and safety data, and making recommendation on the fate of trials (or treatment-specific arms). Trials are often underpowered to assess safety; better coordination is needed between the DMCs of trials addressing similar clinical questions, allowing them to share emerging evidence to better assess the risk and benefit of interventions.
The lack of coordination and collaboration raises important ethical issues to the detriment of patients but also to the medical staff already overwhelmed.
have shown their ability to set priorities and foster nation- and worldwide collaborations.
The pandemic has shed light on the need and feasibility to streamline the excessively complex administrative procedures that have burdened clinical research. Stakeholders need close interoperability allowing for data and resources flow between stakeholders nationally and internationally (ie, regulatory authorities, funders, clinical sites, and monitoring committees).
Overall, COVID-19 has clearly challenged the traditional myths that conducting clinical trials must be difficult and time-consuming, resulting in trials that are too small or take too long to be completed. This crisis has clearly magnified those shortcomings but has also shown how to make clinical research more useful—by doing trials with elegant designs reduced to essential elements and prepared to be combined with others, allowing them to be performed with speed and flexibility in large collaborations.
Funding Sources
The Meta-Research Innovation Center at Stanford (METRICS), Stanford University, is supported by a grant from the Laura and John Arnold Foundation. The work of John Ioannidis is supported by an unrestricted gift from Sue and Bob O'Donnell. COVID-evidence is supported by the Swiss National Science Foundation (31CA30_196190). The funders had no role in the study design; in the collection, analysis, or interpretation of data; in the writing of the report; or in the decision to submit the report for publication.
Disclosures
The authors have no conflicts of interest to disclose.
References
Janiaud P
Axfors C
Van't Hooft J
et al.
The worldwide clinical trial research response to the COVID-19 pandemic - the first 100 days.
Janiaud P, Axfors C, Saccilotto R, Hemkens L. COVID-evidence: a living database of trials on interventions for COVID-19. Available at: https://osf.io/gehfx. Accessed March 1, 2021.
Population-level COVID-19 mortality risk for non-elderly individuals overall and for non-elderly individuals without underlying diseases in pandemic epicenters.
Effectiveness of adding a mask recommendation to other public health measures to prevent SARS-CoV-2 infection in Danish mask wearers: a randomized controlled trial.
ISIS-4: a randomised factorial trial assessing early oral captopril, oral mononitrate, and intravenous magnesium sulphate in 58,050 patients with suspected acute myocardial infarction. ISIS-4 (Fourth International Study of Infarct Survival) Collaborative Group.
Mehra MR, Desai SS, Ruschitzka F, Patel AN. RETRACTED: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis [e-pub ahead of print] [erratum in: Lancet https://doi:org/10.1016/S0140-6736(20)31249-6] [retraction in: Lancet https://doi:org/10.1016/S0140-6736(20)31324-6]. Lancet. doi:10.1016/S0140-6736(20)31180-6, Accessed February 26, 2021.
Boutron I, Chaimani A, Meerpohl JJ, et al. Interventions for preventing and treating COVID-19: living systematic reviews and network meta-analyses. Available at: https://zenodo.org/record/4018607#.YDj_r5P0k1I. Accessed February 26, 2021.
Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis.
REMAP-CAP. A Randomised, Embedded, Multi-factorial, Adaptive Platform Trial for Community-Acquired Pneumonia. Available at: https://www.remapcap.org. Accessed February 26, 2021.
Protocol for a partially nested randomised controlled trial to evaluate the effectiveness of the scleroderma patient-centered intervention network COVID-19 home-isolation activities together (SPIN-CHAT) program to reduce anxiety among at-risk scleroderma patients.
00285-3&id=gr1.jpg){kind=link}
00285-3&id=gr2.jpg){kind=link}