Regression to the mean and other artefacts

When you select a sample from a population based on high scores on a given measurement scale, say high depression scores on the Hamilton Depression Rating Scale (HAMD), and when you examine depression change over time based on scores on the very same scale, then you will necessarily find that, on average, and irrespective of treatment received, there is a clear reduction in depression scores. But this is not necessarily a true treatment effect. 

The reduction in depression scores is also due to random variation in repeated measurements (ie, random measurement error and random symptom fluctuation), which, in the case of high scores at baseline, by consequence tends to result in significantly lower scores over time.4 5 This methodological artefact is known as regression to the mean. Simply put, regression to the mean is a consequence of the observation that, on average, extreme scores do not persist over time,6 and it can mislead researchers to conclude that an active treatment (or an inert placebo) is effective when in fact it is due to regression to the mean7 or other factors, including in particular spontaneous remission, assessors’ treatment expectancies, or receipt of additional treatments.8 For example, according to the most recent meta-analysis of antidepressant trials by Cipriani et al,9 rescue medications (typically benzodiazepines and other sedative hypnotics) were allowed in 36% of all placebo arms, and all patients receive thorough clinical management in addition to placebo pills (ie, close monitoring and regular contact with a doctor). According to a meta-analysis, just two additional visits to the doctor during the first 6 weeks of treatment may represent 34%–44% of the placebo response and account for 1.75 points improvement on the HAMD.10 This is about the same magnitude as the average drug-placebo difference in selective serotonin reuptake inhibitor (SSRI) trials.11 The origin of this particular effect of additional visits is most likely psychological, which also includes the tendency to react according to the investigators’ expectations when patients are aware of being under study. This phenomenon is also known as the Hawthorne effect.12

Research has further shown that regression to the mean is particularly pronounced when outcome measurements are examined exclusively on a subsample that was selected based on a high baseline score on the same measurement scale.4 This is exactly what happens in basically all antidepressant trials and also in many psychotherapy trials for acute major depression. Unfortunately, we cannot know whether an active treatment, or an inert placebo, is effective when we do not have a trial arm with untreated participants. This led various statisticians to question whether the placebo effect is genuine. For instance, more than 30 years ago, McDonald et al 13 demonstrated that ‘Regression accounts for an important share of the improvement observed with placebo treatment.’ More recently, Senn6 stated, ‘It is regression to the mean that is a very plausible explanation for the placebo effect, since entry into clinical trials is usually only by virtue of an extreme baseline value. This does not matter as long as you compare the treated group to the placebo group, since both groups will regress to the mean. It does mean, however, that you have to be very careful before claiming that any improvement in the placebo group is due to the healing hands of the physician or psychological expectancy.’

An example

For illustration readers are referred to a recently published randomised clinical trial,14 in which military veterans with established treatment-resistant major depression were assigned to either repetitive transcranial magnetic stimulation or a sham treatment (ie, inert placebo). At baseline, 49% of participants had comorbid post-traumatic stress disorder, 61% were considered suicidal and 54% showed substance abuse, so these patients clearly were highly impaired before trial entry (significantly more impaired than the participants enrolled in most antidepressant trials). Nevertheless, remission rates after a treatment period of 5–12 days based on the HAMD were 41% for active treatment versus 37% for sham treatment (p=0.67). Such an outstandingly high remission rate in people with treatment-resistant depression is truly miraculous and most likely a methodological artefact due to regression to the mean, since severely impaired patients with treatment-resistant depression impossibly can truly remit at such a high rate in less than 2 weeks. Unfortunately, neither the authors, nor the reviewers or the editor, apparently saw the elephant in the room. Instead, the authors argue in their paper that this outstandingly high remission rate was due to close clinical surveillance, regular interaction with clinic staff and rigorous monitoring of medication. This interpretation is unlikely and implausible. First, if medication had such a profound influence, then the participants should not have been regarded as treatment resistant, that is, unresponsive to at least two previous drug treatments (besides that no approved drug has ever achieved such a high remission rate in merely 5–12 days). And, second, if clinical surveillance and interaction with staff was that powerful (remember that we are talking of an average remission rate of about 37% in less than 2 weeks in the placebo group), then one might wonder why such impressive improvements are rarely seen in real-world inpatient practice.

Remission rates in treated and untreated trial participants

Given that only trials that compare a placebo group with a group of untreated participants can establish whether the placebo effect is genuine, we will now resort to this literature. Two comprehensive meta-analyses have shown that response rates (or continuous symptom reduction scores) in placebo recipients do not significantly differ from response rates in untreated people with depression.15 16 However, it is worth noting that even the most recent analysis included a total of 324 patients only, thus absence of a significant difference could also be due to a lack of statistical power. Yet, another meta-analysis confirmed that remission rates in untreated people with mostly moderate depression are surprisingly high: 23% remit within 3 months, 32% within 6 months and 53% within 12 months.17 So now let us compare these remission rates in untreated people with those estimated for patients undergoing acute antidepressant therapy. According to a comprehensive meta-analysis of placebo-controlled antidepressant trials of 6–12 weeks’ duration, the placebo remission rate at the end of acute treatment is about 26% and that of SSRI is about 35%.11 However, note that this remission rate for SSRI is exaggerated, because industry-sponsored efficacy trials carefully preselect the participants who are most likely to respond favourably to drug treatment.18 In the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial,19 which enrolled the largest and most representative sample of people with major depression to date, the mean HAMD score at baseline was 21.8 (SD=5.2), indicating that the vast majority of participants entered the trial with moderate depression. The remission rate with HAMD after a median treatment duration with citalopram of 12 weeks was 28% (no placebo group was included). Given that the outcome of citalopram is comparable to that of other newer generation antidepressants,9 it is assumed that this rate would likewise apply to other drugs. Therefore, these findings indicate that, in representative real-world samples, less than 30% of patients might achieve remission within roughly 12 weeks with antidepressant treatment. Since antidepressants and psychotherapy are equally effective in the acute treatment of major depression,1 2 the short-term remission rate for antidepressants also broadly applies to psychotherapy (however, note that in the long term psychotherapy appears to be superior to pharmacotherapy).20

One of the few studies that purportedly showed a genuine placebo effect in people with major depression is a randomised three-arm 8-week trial by Leuchter et al.21 This work reported a remission rate of 23% in the antidepressant group, 31% in the placebo group and of 0% in the untreated group (ie, supportive care only). However, this particular trial is biased to an extent that the results are not interpretable. First, it was severely underpowered and thus at risk of producing biased estimates,22 as there were only n=20 people in the untreated group, n=29 in the placebo group and n=39 in the antidepressant group. Second, the dropout rate was 40% (8 of 20) in the untreated group, 26% (10 of 39) in the antidepressant group, but only 10% (3 of 29) in the placebo group. Accordingly, the mean time in the trial was 5.7 weeks in the untreated group, 7.6 weeks in the antidepressant group, but 8.3 weeks in the placebo group, which is a significant difference. Third, and worst of all, the authors used last observation carried forward to impute missing values, which results in a considerable underestimation of symptom change in the untreated group relative to both placebo and antidepressant groups, because many untreated participants left the study early.23 24

Summary and conclusions

The short-term remission rate for antidepressants in real-world patients with mostly moderate depression (28%)19 is just marginally better than that observed in placebo recipients in efficacy trials (26%)11 and it is also just slightly better than that seen in untreated people (23%).17 Although the remission rate in antidepressant groups is significantly higher than in placebo groups, this small difference is clinically probably not relevant.11 25 Moreover, direct comparisons do not indicate that the placebo response differs significantly from spontaneous improvements seen in untreated people with depression,15 16 but this could also be due to a lack of statistical power. Nevertheless, it follows that the placebo effect in antidepressant trials is largely (though not entirely) a methodological artefact,8 and that the symptom reduction seen in placebo recipients is mostly due to both regression to the mean6 and spontaneous remission.16 Moreover, being in the placebo group in a drug trial does not indicate that participants merely swallow a placebo pill. In most depression trials, participants in the placebo group also receive thorough clinical management (which may trigger a Hawthorne effect)12 and they build a therapeutic relationship with their doctors, which is among the most effective treatment effects in depression and clearly separable from a genuine placebo effect (ie, mere treatment expectancy).26 In order to reduce bias due to regression to the mean and to arrive at efficacy estimates that are more likely to reflect true treatment effects, researchers should use hard (objective) real-world outcomes on which the sample was initially not selected for, such as, for instance, employment rates or receipt of disability benefits.