Original Articles
Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements

https://doi.org/10.1016/j.jclinepi.2003.09.014Get rights and content

Abstract

Objective

Randomized controlled trials (RCTs) with dichotomous outcomes may be analyzed with or without adjustment for baseline characteristics (covariates). We studied type I error, power, and potential reduction in sample size with several covariate adjustment strategies.

Study Design and Setting

Logistic regression analysis was applied to simulated data sets (n = 360) with different treatment effects, covariate effects, outcome incidences, and covariate prevalences. Treatment effects were estimated with or without adjustment for a single dichotomous covariate. Strategies included always adjusting for the covariate (“prespecified”), or only when the covariate was predictive or imbalanced.

Results

We found that the type I error was generally at the nominal level. The power was highest with prespecified adjustment. The potential reduction in sample size was higher with stronger covariate effects (from 3 to 46%, at 50% outcome incidence and covariate prevalence) and independent of the treatment effect. At lower outcome incidences and/or covariate prevalences, the reduction was lower.

Conclusion

We conclude that adjustment for a predictive baseline characteristic may lead to a potentially important increase in power of analyses of treatment effect. Adjusted analysis should, hence, be considered more often for RCTs with dichotomous outcomes.

Introduction

Randomized controlled trials (RCTs) have emerged as the principal research tool to inform and influence clinical practice [1], [2]. Clinicians rely increasingly on efficient, well-designed RCTs to distinguish between worthwhile, useless, or harmful interventions [2]. Efforts have been made to improve quality in reporting RCTs [3], but the way to analyze RCTs properly is still under discussion [4], [5], [6], [7].

In particular, the treatment effect in a RCT can be analyzed and shown either as an average effect, as an adjusted effect, or both [8]. Adjusted effect estimates attempt to take the heterogeneity of patients in RCTs into account. The heterogeneity of patients is related to their prognostic baseline characteristics [4], which may be used to obtain adjusted treatment effects. Common methods of adjustment are baseline imbalance adjustment [5], [9], subgroup analysis [5], [10], [11], [12], stratified randomization plus adjustment [13], and covariate adjustment (poststratification) [14], [15], [16], [17], [18].

Covariate adjustment provides more individualized effect estimates, especially in nonlinear models such as logistic regression and Cox proportional hazards regression [8], [13], [14], [15], [16], [17], [18], [19], [20]. Furthermore, adjusted effect estimates take into account chance differences in baseline characteristics between treatment arms [14], [15] and improve the power, that is, the ability to identify treatment effects when they really exist [15], [16], [17], [18], [19], [20], [21], [22].

The use of covariate adjustment in the current literature is not consistent [5], [11], probably because the strategies have not been fully developed and tested [23], [24]. A key aspect of the adjustment strategies is the way of selection of the covariate to be adjusted for [25]. Moreover, the effects on power and type I error after adjustment have not been studied thoroughly.

We used various strategies for covariate adjustment in simulated logistic regression models with one dichotomous covariate. Our aim was to identify the pros and the cons of each covariate adjustment strategy, with a focus on changes in statistical power. We expressed any increase in statistical power as the decrease in sample size that gives the same power as an unadjusted analysis.

Section snippets

Treatment effects and adjustment strategies

Logistic regression models were used to analyze the effects of treatment on a dichotomous outcome (e.g., 30-day mortality). A dichotomous baseline characteristic was entered as covariate to achieve adjustment of the treatment effect. The formula is: log odds (outcome) = β0 + β1Treatment + β2Covariate. The logistic regression coefficients and their standard errors (SE) were estimated with standard maximum likelihood procedures. Statistical significance was based on the Wald statistic

Results

Table 3 shows the main results of our simulations. When there was no treatment effect (OR = 1), SEs from adjusted analysis were larger in direct relation to the strength of the covariate effect. The type I error was rather similar for most adjustment strategies and for all covariate effects. The results were mainly slightly below 5%, especially when the covariate effect was strong and the significant imbalance strategy was used (type I error 3.8%). When there were no covariate effects, covariate

Discussion

Covariate adjustment increased the power of statistical analyses of a treatment effect in the context of a randomized trial, without inflation of type I error. The increase in power was translated into moderate potential reductions in sample size, indicating that adjusted analyses might give the same power as an unadjusted analysis but with a smaller sample size. We found that prespecified covariate and significant predictor covariate adjustment strategies were the most statistically efficient,

Acknowledgements

The authors wish to thank Yvonne Vergouwe, MSc, PhD, for her helpful comments on a previous version of this manuscript and Jennifer Kealy, MPH, for her suggestions on the English language. Adrián Hernández was supported by the Netherlands Organization for Scientific Research (ZON/MW 908-02-117), and Ewout Steyerberg was supported by a fellowship from the Royal Netherlands Academy of Arts and Sciences (KNAW).

References (31)

Cited by (190)

View all citing articles on Scopus
View full text