Background

Random allocation to intervention groups remains the only method of ensuring that the groups being compared are on an equivalent footing at study outset, thus eliminating selection and confounding biases. This technique has allowed RCTs to play a key role in advancing medical science.

The success of randomisation depends on 2 interrelated processes.^{1, 2} The first entails generating a sequence by which the participants in a trial are allocated to intervention groups. To ensure the unpredictability of that allocation sequence, investigators should generate it by a random process. The second process, allocation concealment, shields those involved in a trial from knowing upcoming assignments. Without this protection, investigators and patients have been known to change who gets the next assignment, making the comparison groups less equivalent.^3–6

For example, suppose that an investigator creates an adequate allocation sequence using a random number table. However, the investigator then affixes the list of that sequence to a bulletin board, with no allocation concealment. Those responsible for admitting participants could ascertain the upcoming treatment allocations and then route participants with better prognoses to the experimental group and those with poorer prognoses to the control group, or vice versa. Bias would result. Inadequate allocation concealment also exists, for example, when assignment to groups depends on whether a participant's hospital number is odd or even or on translucent envelopes that allow discernment of assignments when held up to a light bulb.

Allocation concealment should not be confused with blinding. Allocation concealment concentrates on preventing selection and confounding biases, safeguards the assignment sequence before and until allocation, and can always be successfully implemented.^{1, 2} Blinding concentrates on preventing study personnel and participants from determining the group to which participants have been assigned (which leads to ascertainment bias), safeguards the sequence after allocation, and cannot always be implemented.^1–7

Reporting of methods

Investigators must not only minimise bias but must also communicate those efforts to the reader. Readers should not have to assume or guess the methods used. Yet assessments of the reporting quality of published trials have consistently found major flaws.^{3, 8–14} Only 9% of trials in the specialist journals and 15% in the general journals reported both an adequate method of generating random sequences and an adequate method of allocation concealment.^{3, 8, 15} Of trials reported as double blind, only 45% described similarity of the treatment and control regimens, and only 26% provided information on the protection of the allocation schedule.¹⁶ Most reports simply provide no information on methods.

With so little relevant information, many of us resort to inappropriate markers of trial quality. 2 noteworthy examples are described here. First, many designate a trial as high quality if it is “double blind”, as if double blinding is the sine qua non of an RCT. Although double blinding can reflect good methods, it is not the sole criterion of quality. As I shall discuss later, adequate allocation concealment actually appears to be the more important indicator. Moreover, many trials cannot be double blinded. Those trials must be judged on other merits and not on an inapplicable standard based on double blinding.

Second, some assume that a good-quality trial contains groups of equal size, while a poor-quality trial contains groups of unequal size. That standard applies only when the investigators use a restricted randomisation generation scheme that aims for equality. A simple randomisation method will seldom yield equal sample sizes. In fact, equal numbers in treatment groups may mean that some process other than randomisation was used, for example, allocation of every second patient to the intervention group or the use of odd and even birth dates or chart numbers as a way to assign participants to study groups.

Although RCT reporting remains weak, it is improving. Methodologists, editors, and clinicians addressed the prevailing flaws in reporting by publishing the Consolidated Standards of Reporting Trials (CONSORT) statement.¹⁷ Currently, more than 70 journals have adopted the standards, including such high-profile general medical journals as JAMA, the Lancet, BMJ, and Annals of Internal Medicine. Yet, even with improvement, readers of RCTs should be wary of the information provided in many current trial reports.

Empirical evidence of bias

Recent studies have shown that poor-quality RCTs and poorly reported RCTs yield biased results. For example, in a study of 250 controlled trials from 33 meta-analyses in pregnancy and childbirth, investigators found that alleged RCTs with inadequate and unclear allocation concealment yielded larger estimates of treatment effects (41% and 33%, respectively, on average) than trials in which authors reported adequate concealment.⁵ Investigators found similar results for trials in digestive diseases, circulatory diseases, mental health, and stroke.¹⁸ Those trials that used inadequate or unclear allocation concealment yielded 37% larger estimates of effect, on average, than those that used adequate concealment.

These exaggerated estimates of treatment effects reveal meaningful levels of bias. If a study is designed to detect a decrease in mortality of 25% or 50% from a particular treatment, biases of 30% to 40% would overwhelm estimates of the treatment effect. The elimination of bias is crucial in trials designed to detect moderate effects.

Double blinding also appears to reduce bias. Trials that were not double blinded yielded larger estimates of treatment effects than did trials in which authors reported double blinding (odds ratios exaggerated, on average, by 17%).⁵ Another recent analysis has also noted the importance of double blinding.¹⁹ However, although double blinding appears to prevent bias, its effect appears weaker than that of allocation concealment. Indeed, Moher and colleagues found little effect from double blinding.¹⁸

Conclusions

As users of RCT results, we must understand the potential for humans to interject bias. By including assessments of allocation concealment and double blinding, abstracts in this journal will help readers to discern those trials that have made superior efforts to minimise bias. Judging the quality of allocation concealment and blinding reflects current empirical research and reflects the commitment of the editors of this journal to apply the principles of evidence-based medicine to the practice of reporting.