Early stopping of randomized clinical trials for overt efficacy is problematic

doi:10.1016/j.jclinepi.2007.07.016

Journal of Clinical Epidemiology

Volume 61, Issue 3, March 2008, Pages 241-246

https://doi.org/10.1016/j.jclinepi.2007.07.016 Get rights and content

Abstract

Objective

To illustrate controversial issues associated with stopping randomized controlled trials (RCTs) early for apparent benefit.

Study Design and Setting

The article presents our review of prior relevant work and our research group's reflections on early stopping.

Results

Compelling evidence suggests that trials stopped early for benefit systematically overestimate treatment effects, sometimes by a large amount. Unresolved controversies in trials stopped early for benefit include ethical and statistical problems in the interpretation of results.

Conclusions

The best strategy to minimize the problems associated with early stopping of RCTs for benefit is not to stop early. As an alternative, we suggest a threefold approach: a low P-value as the threshold for stopping at the time of interim analyses, not to look before a sufficiently large number of events has accrued and continuation of enrollment and follow-up for a further period.

Section snippets

Four reasons for early termination

There are four major reasons for stopping a randomized clinical trial (RCT) early. First, the trial may show serious adverse effects and may be stopped for unacceptable safety. This was the case, for instance, in an RCT that investigated the effect of hydrocortisone treatment on survival without bronchopulmonary dysplasia in preterm infants [1]. This study was discontinued early because of gastrointestinal perforations in the hydrocortisone group [1].

Second, investigators may stop the trial if

Motivations for early stopping

There are two potentially justifiable reasons for early termination of a trial showing apparent large benefit. First, one might make an argument that it is unethical to continue to randomize patients in the face of such a result. This consideration may be compelling for investigators, patients, and their advocates, and DMCs. Second, scarce research resources might be better invested in addressing other questions if one believes an apparent large benefit has adequately addressed a research

Acknowledgment

We wish to thank the UK MRC for supporting our ongoing work on the impact of early stopping of trials. Matthias Briel is supported by a grant from the Swiss National Science Foundation.

References (35)

O. Peltoniemi et al.
Pretreatment cortisol values may predict responses to hydrocortisone administration for the prevention of bronchopulmonary dysplasia in high-risk infants
J Pediatr
(2005)
D. Moher et al.
The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials
Lancet
(2001)
D. Bassler et al.
Systematic reviewers neglect bias that results from trials stopped early for benefit
J Clin Epidemiol
(2007)
D. Moher et al.
Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses
Lancet
(1999)
K.F. Schulz et al.
Multiplicity in randomised trials. II: Subgroup and interim analyses
Lancet
(2005)
S. Pocock et al.
Trials stopped early: too good to be true?
Lancet
(1999)
T.A. Trikalinos et al.
Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time
J Clin Epidemiol
(2004)
R.C. Bailey et al.
Male circumcision for HIV prevention in young men in Kisumu, Kenya: a randomised controlled trial
Lancet
(2007)
R.H. Gray et al.
Male circumcision for HIV prevention in men in Rakai, Uganda: a randomised trial
Lancet
(2007)
P. Thistle et al.
A randomized, double-blind, placebo-controlled trial of combined nevirapine and zidovudine compared with nevirapine alone in the prevention of perinatal transmission of HIV in Zimbabwe
Clin Infect Dis
(2007)

A. Laupacis et al.

How should results from completed studies influence ongoing clinical trials? The CAFA study experience

Ann Intern Med

(1991)

M.R. Smith et al.

Celecoxib versus placebo for men with prostate cancer and a rising serum prostate-specific antigen after radical prostatectomy and/or radiation therapy

J Clin Oncol

(2006)

E.G. Papanikolaou et al.

In vitro fertilization with single blastocyst-stage versus single cleavage-stage embryos

N Engl J Med

(2006)

Draft guidance for clinical trial sponsors on the establishment and operation of clinical trial data monitoring...

M.R. Sydes et al.

Reported use of data monitoring committees in the main published reports of randomized controlled trials: a cross-sectional study

Clin Trials

(2004)

DAMOCLES Study Group

NHS health technology assessment programme: a proposed charter for clinical trial data monitoring committees: helping them to do their job well

Lancet

(2005)

A.S. Slutsky et al.

Data safety and monitoring boards

N Engl J Med

(2004)

Cited by (106)

Intermediate versus standard-dose prophylactic anticoagulation and statin therapy versus placebo in critically-ill patients with COVID-19: Rationale and design of the INSPIRATION/INSPIRATION-S studies
2020, Thrombosis Research
Citation Excerpt :
The DSMB members have full access to the study data, with help from an independent statistician, as well as all the reports by the Clinical Events Committee. To provide the most informative results, without risking the possibility of an overinflated estimate of the effect size, the steering committee decided not to set pre-specified criteria for early termination for efficacy [41]. Similarly, the steering committee specified not to stop the trial prematurely for futility, unless further recruitment seemed unfeasible due a change in the disease wave or logistical limitations at enrolling centers, in which case the DSMB will make a determination of futility based on conditional probability.
Microvascular and macrovascular thrombotic events are among the hallmarks of coronavirus disease 2019 (COVID-19). Furthermore, the exuberant immune response is considered an important driver of pulmonary and extrapulmonary manifestations of COVID-19. The optimal management strategy to prevent thrombosis in critically-ill patients with COVID-19 remains unknown.
The Intermediate versus Standard-dose Prophylactic anticoagulation In cRitically-ill pATIents with COVID-19: An opeN label randomized controlled trial (INSPIRATION) and INSPIRATION-statin (INSPIRATION-S) studies test two independent hypotheses within a randomized controlled trial with 2 × 2 factorial design. Hospitalized critically-ill patients with reverse transcription polymerase chain reaction confirmed COVID-19 will be randomized to intermediate-dose versus standard dose prophylactic anticoagulation. The 600 patients undergoing this randomization will be screened and if meeting the eligibility criteria, will undergo an additional double-blind stratified randomization to atorvastatin 20 mg daily versus matching placebo. The primary endpoint, for both hypotheses will be tested for superiority and includes a composite of adjudicated acute arterial thrombosis, venous thromboembolism (VTE), use of extracorporeal membrane oxygenation, or all-cause death within 30 days from enrollment. Key secondary endpoints include all-cause mortality, adjudicated VTE, and ventilator-free days. Key safety endpoints include major bleeding according to the Bleeding Academic Research Consortium definition and severe thrombocytopenia (platelet count <20,000/fL) for the anticoagulation hypothesis. In a prespecified secondary analysis for non-inferiority, the study will test for the non-inferiority of intermediate intensity versus standard dose anticoagulation for major bleeding, considering a non-inferiority margin of 1.8 based on odds ratio. Key safety endpoints for the statin hypothesis include rise in liver enzymes >3 times upper normal limit and clinically-diagnosed myopathy. The primary analyses will be performed in the modified intention-to-treat population. Results will be tested in exploratory analyses across key subgroups and in the intention-to-treat and per-protocol cohorts.
INSPIRATION and INSPIRATON-S studies will help address clinically-relevant questions for antithrombotic therapy and thromboinflammatory therapy in critically-ill patients with COVID-19.
Photo induced antibacterial activity of CeO<inf>2</inf>/GO against wound pathogens
2020, Arabian Journal of Chemistry
Citation Excerpt :
Due to its distinguished property graphene oxide had been explored in various biological field such as drug delivery, anticancer, sensing and antibacterial application (Wu et al., 2014; Matharu et al., 2020). GO has more reactive groups that can interact strongly with cells and cell surfaces, showing higher affinity for bacteria and can produces an oxidative stress on its interaction that results in cell apoptosis intern high antibacterial property (Quinteros et al., 2016; Gao et al., 2017; Bassler et al., 2008; Hussein et al., 2019; Khan et al., 2015). GO individually shows photocatalytic activity.
Infections are the main cause for delayed wound healing. Graphene oxide (GO) and cerium oxide nanoparticles are used for many biomedical applications because of their biocompatibility and therapeutic property. Cerium Oxide (CeO₂)/Graphene Oxide (GO) nanocomposite was synthesized in this work. It shows good antimicrobial property against wound pathogens. The aggregation of CeO₂ particles on GO results in the generation of reactive oxygen species (ROS) and inhibits microbial growth. The synthesized nanocomposites were analysed by X-ray diffraction, Transmission electron microscopy, Raman Spectroscopy, UV–vis diffuse reflectance spectroscopy and Fourier transformation-IR spectroscopy. The antimicrobial activity of CeO₂/GO nanocomposite was tested against Escherichia coli (E. coli), Pseudomonas aeruginosa (P. aeruginosa) Staphylococcus aureus (S. aureus) and Salmonella typhi (S. typhi). Well diffusion method, growth curve studies, bio film inhibition studies and colony count studies were done in both presence and absence of visible light. The mechanistic investigation was done by the determination of reactive oxygen species (ROS), Lactate dehydrogenase leakage (LDH) assay and DNA fragmentation assay.
Calculation of the fragility index of randomized controlled trials in epilepsy published in twelve major journals
2020, Epilepsy Research
Fragility index is the minimum number of participants in a trial whose status has to be changed from an ‘event’ to a ‘non-event’ for a dichotomous primary outcome to turn the P-value (calculated by Fischer’s exact test) statistically non-significant (P > 0.05). This study was performed to evaluate the fragility index of randomized controlled trials (RCTs) in epilepsy published in 12 major journals.
We identified the relevant RCTs published in six major epilepsy-related and six Neurology journals from January 2015 to September 2019 and determined the fragility index of those RCTs which reported statistically significant results of dichotomous primary outcomes. We also calculated the Spearman correlation coefficients between the fragility index and the sample size, the event rate, and the reported P-value.
A total of 1395 RCTs were screened and finally ten were eligible for the analysis. The median (IQR) fragility index was only 1.5 (11). There was no significant correlation between the fragility index and the sample size (r = 0.620, P = 0.056), the event rate (r = < 0.0001, P = 1.0), and the reported P-value (r = − 0.315, P = 0.447).
The median fragility index of the included RCTs on epilepsy was very low. The addition of only two alternate events to an arm of the average trial would have rejected the statistical significance. Fragility index should be used while reporting the results of dichotomous primary outcomes of RCTs.
The new US and European guidelines in hypertension: A multi-dimensional analysis
2019, Contemporary Clinical Trials
The Systolic Blood Pressure Intervention Trial (SPRINT) compared the clinical outcomes between target systolic blood pressure (SBP) levels between 140 and 120 mmHg or lower. Both,the 2017 ACC/AHA and the 2018 ESC/ESH guidelines in hypertension are derived from the SPRINT trial and advise initiation and/or intensification of treatment at lower blood pressure thresholds. The ACC/AHA guidance supersedes the 2014 Eight Joint National Committee guideline (JNC-8) which advised initiation of treatment when the BP was 140/90 mmHg or higher; in adults 60 years or over, the target was 150/90 mmHg. Compared to JNC-8, the new guidelines lower the SBP target by 10 mmHg in patients under age of 60 years, and by 20 mmHg in the elderly. We performed a qualitative multi-dimensional analysis in order to answer two key questions: will the new guidelines deliver the stated benefits? and, will translation to the clinic be simple, risk-free, and affordable? A major investment by national healthcare administrations will be necessary for the initiation and support of this program but this decision can only be justified by a valid expectation of clinical benefit. At this time, a definitive answer is not available and a “wait and see” attitude appears appropriate and reasonable. In the interim, efforts are best directed to the immediate problem of untreated hypertension worldwide.
Psychedelic Therapy: A Primer for Primary Care Clinicians - The Strengths, Weaknesses, Opportunities, and Threats of Psychedelic Therapeutics
2024, American Journal of Therapeutics
Does prefusion F protein-based respiratory syncytial virus immunization in pregnancy safely promote transplacental transfer of neutralizing antibodies?
2024, Journal of Perinatology

View all citing articles on Scopus

View full text

Review ArticleEarly stopping of randomized clinical trials for overt efficacy is problematic

Abstract

Objective

Study Design and Setting

Results

Conclusions

Section snippets

Four reasons for early termination

Motivations for early stopping

Acknowledgment

J Pediatr

Lancet

J Clin Epidemiol

Lancet

Lancet

Lancet

J Clin Epidemiol

Lancet

Lancet

A randomized, double-blind, placebo-controlled trial of combined nevirapine and zidovudine compared with nevirapine alone in the prevention of perinatal transmission of HIV in Zimbabwe

Clin Infect Dis

How should results from completed studies influence ongoing clinical trials? The CAFA study experience

Ann Intern Med

Celecoxib versus placebo for men with prostate cancer and a rising serum prostate-specific antigen after radical prostatectomy and/or radiation therapy

J Clin Oncol

In vitro fertilization with single blastocyst-stage versus single cleavage-stage embryos

N Engl J Med

Reported use of data monitoring committees in the main published reports of randomized controlled trials: a cross-sectional study

Clin Trials

NHS health technology assessment programme: a proposed charter for clinical trial data monitoring committees: helping them to do their job well

Lancet

Data safety and monitoring boards

N Engl J Med

Review Article
Early stopping of randomized clinical trials for overt efficacy is problematic