Figures
Abstract
Background
Major depressive disorder afflicts an estimated 17% of individuals during their lifetimes at tremendous suffering and costs. Cognitive therapy may be an effective treatment option for major depressive disorder, but the effects have only had limited assessment in systematic reviews.
Methods/Principal Findings
Cochrane systematic review methodology, with meta-analyses and trial sequential analyses of randomized trials, are comparing the effects of cognitive therapy versus ‘treatment as usual’ for major depressive disorder. To be included the participants had to be older than 17 years with a primary diagnosis of major depressive disorder. Altogether, we included eight trials randomizing a total of 719 participants. All eight trials had high risk of bias. Four trials reported data on the 17-item Hamilton Rating Scale for Depression and four trials reported data on the Beck Depression Inventory. Meta-analysis on the data from the Hamilton Rating Scale for Depression showed that cognitive therapy compared with ‘treatment as usual’ significantly reduced depressive symptoms (mean difference −2.15 (95% confidence interval −3.70 to −0.60; P<0.007, no heterogeneity)). However, meta-analysis with both fixed-effect and random-effects model on the data from the Beck Depression Inventory (mean difference with both models −1.57 (95% CL −4.30 to 1.16; P = 0.26, I2 = 0) could not confirm the Hamilton Rating Scale for Depression results. Furthermore, trial sequential analysis on both the data from Hamilton Rating Scale for Depression and Becks Depression Inventory showed that insufficient data have been obtained.
Discussion
Cognitive therapy might not be an effective treatment for major depressive disorder compared with ‘treatment as usual’. The possible treatment effect measured on the Hamilton Rating Scale for Depression is relatively small. More randomized trials with low risk of bias, increased sample sizes, and broader more clinically relevant outcomes are needed.
Citation: Jakobsen JC, Lindschou Hansen J, Storebø OJ, Simonsen E, Gluud C (2011) The Effects of Cognitive Therapy Versus ‘Treatment as Usual’ in Patients with Major Depressive Disorder. PLoS ONE 6(8): e22890. https://doi.org/10.1371/journal.pone.0022890
Editor: Josef Priller, Charité-Universitätsmedizin Berlin, Germany
Received: February 23, 2011; Accepted: June 30, 2011; Published: August 4, 2011
Copyright: © 2011 Jakobsen et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: These authors have no support or funding to report.
Competing interests: The authors have declared that no competing interests exist.
Introduction
According to the WHO, major depressive disorder is the second largest healthcare problem worldwide in terms of disability caused by illness [1]. It afflicts an estimated 17% of individuals during their lifetimes at tremendous cost to the individual and society [2], [3], and roughly a third of all depressive disorders take a chronic course [4], [5]. Compared to other medical disorders, major depressive disorder causes the most significant deterioration in individual life quality [6]. Approximately 15% of depressive patients will commit suicide over a 10 to 20 year period [7].
Antidepressant medication remains the mainstay in the treatment of depression [8]. However, meta-analyses have shown that newer antidepressants presumably only obtain beneficial effect in severely depressed patients, and this effect seems to be clinically small [9], [10]. Antidepressants might, however, decrease the risk of relapse [11]. The therapeutic benefits of antidepressants seem to be limited and this raises the question if there are other effective treatments for this serious illness?
Aaron T. Beck originally developed cognitive therapy for depression [12]. Beck believed that critical life events could accentuate hidden negative beliefs, which could generate negative automatic thoughts. These negative thoughts could lead to symptoms of depression, which then could reinforce more negative automatic thoughts. The main goal of the ‘cognitive model of depression’ is to correct these negative beliefs and thoughts in order to treat the depressive symptoms [12]. A Cochrane review shows that cognitive therapy has a preventive effect against recurrent depression, and that this effect clearly surpasses the preventive effects of antidepressant medication [13]. Furthermore, cognitive therapy appears to be an effective treatment for major depressive disorder [14], but we were unable to find any meta-analysis with Cochrane methodology [15] examining the effect of cognitive therapy versus ‘treatment as usual’ for major depressive disorder.
Methods
We conducted our systematic review of randomized clinical trials involving meta-analysis [15] and trial sequential analysis [16], [17] to answer the question: what are the beneficial and harmful effects of cognitive therapy versus ‘treatment as usual’ in the treatment of major depressive disorder? We used assessment of bias risk to reduce systematic errors [15], and trial sequential analysis to reduce the risk of random errors [16], [17].
For details regarding the methodology please consult our protocol published on our website (www.ctu.dk) in February 2010 before we began systematic literature searches in all relevant databases, data-extraction, and analysis [18].
In short, we included all randomized clinical trials comparing the effect of cognitive therapy alone or in combination with any co-intervention versus ‘treatment as usual’ alone or in combination with a similar co-intervention. These co-interventions had to be administered equally in both intervention groups. The trials were included irrespective of language, publication status, publication year, and publication type - based on searches in The Cochrane Library's CENTRAL, MEDLINE via PubMed, EMBASE, Psychlit, PsycInfo, and Science Citation Index Expanded. The timeframe for the search was all trials published before February 2010.
To be included, participants had to be older than 17 years, with a primary diagnosis of major depressive disorder. Trials were only included if the diagnosis of depression was based on one of the standardized criteria, such as ICD 10 [19], DSM III [20], DSM III-R [21], or DSM IV [22]. Comorbidity with other psychiatric diagnoses was not an exclusion criterion. The following types of trials were excluded:
- Trials focusing on depressed participants with comorbid serious somatic illness, e.g., myocardial infarction, multiple sclerosis, cerebral stroke, cancer, etc.
- Trials focusing on ‘late life’ depression or depression in the elderly, most often participants over 65 years.
- Trials focusing on pregnancy related depression, e.g., postpartum depression, postnatal depression, etc.
- Drug or alcohol dependence related depression.
These exclusions were conducted because we expect participants in such trials to respond differently to standardized psychotherapy than other depressed patients, and these types of depressed patients are traditionally examined in separate trials [23]–[26].
Interventions
Cognitive therapy.
Cognitive therapy and cognitive-behavioral therapy are collective terms for a range of different forms of interventions and it is difficult to find a simple definition, which adequately describes this psychotherapeutic method. However, we selected the following criteria from Beck et al., 1979 as being necessary for the intervention to be classified as ‘cognitive therapy’ [12]:
- That the intervention seeks to link thoughts, feeling and behavior, and relates these to the depressive symptoms.
- That the intervention seeks to record and correct irrational thoughts or behavioral patterns, and relates this to the depressive symptoms.
- That the intervention seeks to teach the patient alternative methods of thinking or behaving, and to be able to relate this to the depressive symptoms.
- That the intervention is undertaken face-to-face either individually or in a group.
We accepted any co-intervention to cognitive therapy as long as this co-intervention was similar and administered equally to the experimental group (cognitive therapy) and the control group (‘treatment as usual’).
Furthermore, the trials had to present a treatment manual and had to document adherence to the treatment manual in order for the intervention to be classified as ‘adequately defined’. All other trials using the term ‘cognitive therapy’ or ‘cognitive-behavioral therapy’ were included, but the intervention was classified as ‘not adequately defined’.
‘Treatment as usual’.
For ‘treatment as usual’ control interventions we accepted any non-specific treatments described as: ‘treatment as usual’, ‘standard care’, or ‘clinical management’, etc. To be included the ‘treatment as usual’ intervention had to include some kind of non-specific supportive treatment.
We accepted any co-intervention to ‘treatment as usual’ as long as this co-intervention was similar and administered equally to the experimental group (cognitive therapy) and the control group (‘treatment as usual’).
Trial selection
Three of the review authors (JJ, JLH, and OJS) independently selected relevant trials. If a trial was selected by three or two of the three, it was included. If a trial was identified only by one of the three, it was discussed whether the trial should be included. Excluded trials were entered on a list, stating the reason for exclusion.
Data extraction
Data were extracted for trial design, bias risk, and outcomes independently by two authors (JJ and JLH). Disagreements were resolved by discussion or through arbitration (CG). We used the instructions in The Cochrane Handbook for Systematic Reviews of Interventions [15] in our evaluation of the methodology and hence bias risk of the trials. We assessed the bias risk in respect to generation of the allocation sequence; allocation concealment; blinding; intention-to-treat analysis; dropouts; reporting of outcome measures; economic bias; and academic bias. These components enable classification of the included trials into trials with ‘low risk of bias’ or with ‘high risk of bias’. The trials were overall classified as ‘high risk of bias’ if one or more of the above components was ‘uncertain’ or ‘high risk of bias’ [15], [27]–[29]. This classification is important because trials with ‘high risk of bias’ may overestimate positive intervention effects and underestimate negative intervention effects [15], [27], [28], [30], and we wanted to relate the validity of our results to the risk of bias in the included trials.
Primary outcome measures
Depressive symptoms.
Our primary outcome was the mean value on the Hamilton Rating Scale for Depression (HDRS) [31], Beck Depression Inventory (BDI) [32], or Montgomery-Asberg Depression Rating Scale (MADRS) [33] at follow-up. We included data based on the total number of randomized patients (intention-to-treat analysis) if these data were reported. We planned to estimate the therapeutic follow-up responses at two time points:
- At cessation of treatment: The trials original primary choice of completion date was used. This was the most important outcome measure time point in this review.
- At maximum follow-up.
Adverse events.
We classified adverse events as serious or non-serious. Serious adverse events were defined as medical events that are life threatening; result in death; disability or significant loss of function; that cause hospital admission or prolonged hospitalization; a hereditary anomaly; or fetal injury [34]. All other adverse events (that is, events that have not necessarily had a causal relationship with the treatment, but that resulted in a change in or cessation of the treatment) were considered as non-serious events.
Secondary outcome measures
Participants without remission.
The proportion of participants not having achieved remission was our first secondary outcome. We included data based on the total number of randomized participants (intention-to-treat analysis) - if possible. If the results were not based on the total number of participants, we performed an intention-to-treat analysis assuming that the participants not included in the results did not achieve remission [15]. We pragmatically defined remission as HDRS of less than 8, BDI less than 10, or MADRS less than 10 [31]–[33].
Statistical methods
This meta-analysis was undertaken according to the recommendations stated in The Cochrane Handbook for Systematic Reviews of Interventions [15]. In analyzing continuous outcomes with both fixed-effect and with random-effects models, we used the mean difference (MD) with a 95% confidence interval. We used RevMan version 5.0 [35]. We did not use ‘standardized mean difference’ so each outcome measure was analyzed separately. We did not adjust the outcome variables at follow-up according to the baseline values [15].
We used the risk odds ratio (OR) with a 95% confidence interval to estimate intervention effects on dichotomous outcomes with both fixed-effect and with random-effects models [35].
We performed ‘test of interaction’ [36] for all subgroup analyses [18].
For primary outcome measures, we also conducted trial sequential analyses. In order to calculate the required information size and the cumulative Z-curve’s eventual breach of relevant trial sequential monitoring boundaries [16], [17], the required information size of the trial sequential analysis was based on a type I error of 5%, a beta of 20% (power of 80%), the variance of all the trials (as no trial had low risk of bias), and a minimal relevant difference on two points on the HDRS or four points on the BDI.
Results
Search results
Our primary literature search identified 4536 publications. According to our protocol [18] we excluded 4137 publications on the basis of the title or abstract, and further 339 citable units were excluded on the basis of the full publication. These exclusions were done either because the publications did not relate to cognitive therapy and major depressive disorder, or because they were not randomized trials comparing cognitive therapy versus ‘treatment as usual’. Further 42 publications [37]–[78] were excluded because the trial participants or the interventions did not meet our inclusion criteria.
Included trials
We included 18 publications [79]–[96] on eight trials [14], [79]–[82], [84], [85], [97] randomizing a total of 719 participants (see Figure S1).
Only five of the trials used an intervention that we classified as ‘adequately defined’ (see above) [14], [79], [81], [82], [97]. We classified the therapists’ level of experience and/or education in three trials as ‘high’ [14], [79], [81], in two trials as ‘intermediate’ [84], [85], and in the last three as ‘unclear’ [80], [82], [97].
Two trials used cognitive group therapy as experimental intervention [80], [82], and one trial used both group and individual therapy [97]. The remaining five trials used only individual therapy [14], [79], [81], [84], [85].
The duration and the extent of the cognitive therapy also varied in the different trials from eight weekly group sessions [80] to 20 weekly individual sessions [14]. The specific content of the ‘treatment as usual’ interventions were generally not standardized or not reported, and the duration and extent of the ‘treatment as usual’ interventions varied greatly between the different trials. Four of the trials allowed antidepressant medication as a part of the ‘treatment as usual’ intervention, but it was not reported to what extent the participants in the four trials received antidepressants. We have described both the experimental and the control interventions from the included trials in Table 1.
One of the included trials used antidepressants as add-on therapy in both the experimental group (cognitive therapy) and the control group (clinical management) [80].
Miranda et al. examined the effect of cognitive therapy versus community care [97]. The results showed that cognitive therapy significantly reduced depressive symptoms at cessation of treatment compared with community care. However, the authors did not report means and SD and did not report data on remission rates at cessation of treatment. We have written to the authors requesting the necessary data, but have received no answer. Therefore, we have not been able to include the results from this trial in the following meta-analyses. However, the authors did report rates of remission at six and 12 months follow-up. The results at six months showed no significant difference regarding remission. The results at 12 months showed that cognitive therapy significantly increased the probability of remission compared with community care (P = 0.01).
DeRubeis et al. examined the effect of cognitive therapy versus clinical management plus a placebo pill [81]. The results did not differ significantly regarding HDRS score midway through the intervention period. The authors did not report means and SD, and did not include assessment at cessation of treatment for these outcome measures. We have written to the authors requesting the necessary data, but have received no answer. Therefore, we have not been able to include the results from this trial in the following analysis.
Wiles et al. included participants from primary care who had not responded to at least six weeks of antidepressant medication [85]. The participants were randomized to cognitive therapy versus ‘usual care’. Those who received cognitive therapy had a mean on BDI 9 points lower than those in the ‘usual care’ group, suggesting a beneficial effect of cognitive therapy compared with ‘usual care’. The authors did not report means and SD in the publication. Through e-mail correspondence the authors kindly reported that they were unable to provide the necessary data, so we have not been able to include the results from this trial in the following analysis.
Table 1 summarizes the characteristics of the eight included trials.
Bias risk
We assessed all eight trials [14], [79]–[82], [84], [85], [97] as having ‘high risk of bias’ due to unclear or inadequate components as described in Table 2.
Primary outcome measures
Depressive symptoms at cessation of treatment.
Four trials assessed HDRS as a continuous outcome measure at cessation of treatment [14], [79], [82], [84]. Four trials assessed BDI at cessation of treatment [14], [79], [80], [82].
HDRS.
Meta-analysis with the fixed-effect model on the HDRS data from the four trials [14], [79], [82], [84] show that cognitive therapy at the end of therapy significantly reduced depressive symptoms compared with ‘treatment as usual’ (Figure 1) (mean difference −2.15 HDRS; 95% CI −3.70 to −0.60; P<0.007, I2 = 0). The I2 statistic describes the percentage of variation across trials that are due to heterogeneity rather than chance. Meta-analysis with the random-effects model gave an identical result.
BDI.
Meta-analysis with the fixed-effect model on the data from the four trials [14], [79], [80], [82] using BDI at cessation of treatment was in agreement with the results from HDRS (mean difference −6.03 BDI; 95% CI −8.33 to −3.72; P = 0.00001, I2 = 89%). Meta-analysis with the random-effects model showed that cognitive therapy compared with ‘treatment as usual’ did not significantly reduce depressive symptoms on BDI (mean difference −4.85; 95% CI −12.08 to 2.39; P<0.19, I2 = 89%) (Figure 2). Due to the substantial heterogeneity on the BDI results we performed a sensitivity analysis. We excluded the results from Embling et al. trial and found thereafter no heterogeneity [80]. The possible explanations why the results from Embling et al. differed from the rest of the included trials [14], [79], [82] are discussed below. Meta-analysis with the fixed-effect model on the three remaining trials [14], [79], [82] showed that cognitive therapy compared with ‘treatment as usual’ did not significantly reduce depressive symptoms on the BDI (mean difference −1.57 (95% CL −4.30 to 1.16; P = 0.26, I2 = 0). Meta-analysis with the random-effects model gave an identical result.
Trial sequential analysis on the HDRS data and the BDI data showed that ‘insufficient data’ have been obtained to decide if cognitive therapy is superior to ‘treatment as usual’ (Figures 3 & 4).
The required information size of 742 participants is calculated based on an intervention effect compared with ‘treatment as usual’ of 2 points on the HDRS, a variance of 94.5 on the mean difference, a risk of type I error of 5% and a power of 80%. With these presumptions, the cumulated Z-curve (blue curve) do not cross the trial sequential monitoring boundaries (red inner sloping lines) implying that there is no firm evidence for a beneficial effect of cognitive therapy compared with ‘treatment as usual’.
The required information size of 462 participants is calculated based on an intervention effect compared with ‘treatment as usual’ of 4 points on the BDI, a variance of 235.4 on the mean difference, a risk of type I error of 5% and a power of 80%. With these presumptions, the cumulated Z-curve (blue curve) do not cross the trial sequential monitoring boundaries (red inner sloping lines) implying that there is no firm evidence for a beneficial effect of cognitive therapy compared with ‘treatment as usual’.
Follow-up.
Verduyn et al. included maximal follow-up assessment at 12 months after the beginning of treatment on HDRS and BDI [82]. They found no significant difference between the different intervention groups on either of outcome measures.
Miranda et al. reported rates of remission at six and 12 months follow-up [97]. The results are described under ‘Included trials’.
None of the remaining trials included assessment data after the cessation of treatment.
Adverse events.
DeRubeis et al. reported that two participants dropped out due to adverse events, but the particulars about the events were not reported [81]. Both participants were from the control group receiving placebo. None of the remaining trials reported on adverse events.
Quality of life.
Wiles et al. assessed quality of life as outcome measure [85]. They found no significant difference between the two intervention groups at cessation of treatment. Means, SD, or choice of outcome measure for quality of life was not reported. None of the remaining trials used any assessment of quality of life.
Secondary outcome measures
Participants without remission.
Two trials [14], [84] reported the proportion of participants without remission at cessation of treatment as a dichotomous outcome measure. We had planned to define remission as HDRS less than 8, BDI less than 10, or MADRS less than 10. However, this was not possible, so we adopted the slightly different definitions used by the two trials. Elkin et al. defined remission in two different ways: as HDRS<7 and BDI<10 [14]. Scott et al. defined remission as HDRS<7 [84].
Meta-analysis on the data from the two trials reporting on HDRS [14], [84] showed that cognitive therapy compared with ‘treatment as usual’ did not significantly decrease the risk of ‘no remission’ (odds ratio of ‘no remission’ in favor of cognitive therapy of 0.71 (95% CI, 0.38 to 1.32; P = 0.28, I2 = 56%) (Figure 5). The BDI data from Elkin et al. also showed that cognitive therapy compared with ‘treatment as usual’ did not significantly decrease the risk of ‘no remission’ (P = 0.33) [14].
Subgroup analyses
In subgroup analyses stratified according to the type of therapy (group compared to individual therapy) and to the therapists' level of education and experience (‘high’ compared to ‘intermediate’ and ‘unclear’), ‘test of interaction’ [36] on the HDRS data showed no difference in treatment effect between these subgroups (setting P = 0.83; education and experience P = 0.69). Furthermore, we found no heterogeneity in our meta-analysis result on the HDRS data, This indicates that these factors do not seem to influence the effect of cognitive therapy measured on the HDRS.
We had also planned a subgroup-analysis according to risk of bias [18]. However, as all trials were classified as ‘high risk of bias’ it was not possible to conduct this analysis.
Discussion
The results of our systematic review with meta-analysis (fixed-effect model and random-effects model) indicate that cognitive therapy is likely to significantly reduce depressive symptoms on HDRS compared with ‘treatment as usual’. The result of our meta-analysis after a sensitivity analysis on the BDI data (fixed effect-model and random-effects model) did, however, not show a significant reduction on the BDI. Trial sequential analysis on both on the HDRS data and BDI data showed that insufficient data have been obtained. Finally, cognitive therapy compared with ‘treatment as usual’ did not significantly decrease the risk of ‘no remission’. BDI is a ‘self report’ questionnaire and HDRS is an observer dependant interview. This enables a more objective and blinded assessment of the degree of depressive symptoms with HDRS, but only three trials were assessed as having adequate blinding. We believe the neutral effects on BDI combined with the small effects on HDRS suggest that cognitive therapy may not have dramatic effects.
Trial sequential analysis is a statistical analysis that is adjusting the type I error level to decrease the risk of random errors due to sparse data and multiple testing on accumulating data. Therefore, is a more robust analysis than traditional cumulative meta-analysis [16], [98]. Our analysis demonstrates that we lack firm evidence on the intervention effect of cognitive therapy versus ‘treatment as usual’ for major depressive disorder. The trial sequential analysis result also indicates that in order to detect or reject an intervention effect with a minimal relevant difference of two points on HDRS, an information size of 742 participants may be needed.
The heterogeneity on the results on the BDI data is generated by the results from one trial [80]. The results from the Embling et al. trial showed that cognitive therapy compared with the control intervention, decreased the BDI score with a much greater effect-size than the rest of the trials. Embling et al. was the only of the included trials using antidepressants as add-on therapy as part of both the experimental and control interventions [80]. Although head-to-head comparisons are needed in order to thoroughly examine differences between intervention groups, this finding suggests that adding antidepressants to cognitive therapy might have a greater effect compared to cognitive therapy alone. The Embling et al. trial did only report results on the BDI which is a self report questionnaire hindering a blinded assessment of the depressive symptoms. Furthermore, the trial had only two out of the eight bias risk components classified as ‘low risk of bias’ increasing the risk of biased results. These considerations may support the validity of our post-hoc sensitivity analysis excluding the results from this trial in our meta-analysis.
Strengths
The present review has a number of strengths. Our protocol [18] was published before we began the systematic literature searches in all relevant databases, data extraction, and data analyses. Data was extracted by two independent authors minimizing the risk of inaccurate data-extraction, and we assessed the risk of bias in all trials according to The Cochrane Handbook for Systematic Reviews of Interventions [15]. We meta-analyzed data both with fixed-effect and random-effects models. Furthermore, we performed trial sequential analysis to control for random errors [16], [98].
Limitations
Our systematic review has a number of limitations. Only one of the included trials was assessed as being free of ‘selective outcome measure reporting bias’ [15]. There is therefore a risk of within-study selective outcome reporting in seven of the eight included trials. All eight trials had an overall assessment as ‘high risk of bias’ - so our results may be questionable. Moreover, for the positive findings trial sequential analysis showed that we could not exclude the risk of random errors [16], [98]. Due to the limited number of included trials we did not perform a funnel plot or other analysis to explore the risk of publication bias [99]. Other meta-analyses have shown that publication bias significantly has influenced the results from former publications [9]. It is a further limitation that we are not able to assess the risk of publication bias.
Cognitive therapy is generally considered to be one of most evidence-based forms of psychotherapy and we expected to find more randomized trials. However, our literature search did only identify eight trials with a limited number of participants. Only four of the eight trials reported mean and SD for HDRS, and only four of the eight trials reported means and SD for BDI. Our results show that cognitive therapy compared with ‘treatment as usual’ did not significantly decrease the risk of ‘no remission’, but only two out of the eight included trials reported relevant data on remission at end of treatment, while one reported remission rates at six and 12 months follow-up. We might find different results if we had more relevant randomized trials or if we made our inclusion criteria broader (e.g., including trials comparing cognitive therapy with antidepressants).
Only two of the trials included assessments after the cessation of treatment. Therefore it is not clear whether cognitive therapy has an effect on depressive symptoms in the longer term.
Only one of the trials reported measures of quality of life. Outcome measures of quality of life are generally not standardized and thoroughly validated [100]. The use of standardized outcome measures for quality of life in research has been limited by difficulties in administering and scoring quality of life [100], but quality of life can be used as a valid outcome measure [29], [100]. The effect of cognitive therapy on quality of life compared with ‘treatment as usual’ is therefore unclear.
Only one of the included trials reported on some adverse events and none of the trials included records of suicide inclination, suicide attempts, or suicides. Typically adverse events are not reported as thoroughly as beneficial outcome measures [101]. Some psychological interventions might have harmful effects. E.g., psychological debriefing for preventing post-traumatic stress disorder in some clinical trials has showed to have a harmful effect [102]. Possible harmful effects of this kind of therapy are therefore not thoroughly examined.
A number of subgroups of depressed patients (e.g., inpatients) were not assessed in the eight trials we identified and included. These subgroups may react differently to psychotherapy and our results cannot be generalized to other than the patient groups included in the eight trials. Moreover, the extent and form of the ‘treatment as usual’ intervention varied greatly and the specific content of the ‘treatment as usual’ interventions were generally not standardized or reported (Table 1). E.g., four of the trials allowed antidepressants as a part of the ‘treatment-as-usual’ intervention but the extent of the antidepressants medication were not reported or controlled for. Due to the unclear content of the control interventions it is possible that the participants in the control groups actually received some kind of psychotherapeutic intervention - possibly including cognitive therapeutic interventions. Furthermore, the duration and extent of the cognitive therapy interventions did also vary in the different trials (Table 1). Although head-to-head comparisons are needed in order to thoroughly examine a difference in effect between two interventions, we found no heterogeneity on our results on either HDRS or BDI (after sensitivity analysis) indicating that there might be no difference in effect between the different interventions. Moreover, only five of the included trials presented a treatment manual and documented adherence to the treatment manual for the cognitive experimental intervention. The possible difference between cognitive therapy and ‘treatment as usual’ could be due to this manualization rather than to the specific cognitive technics. These aspects are further limitations and make our results less generally applicable.
As mentioned, only five of the included trials used an intervention that we classified as ‘adequately defined’, i.e., using and documenting the use of a therapeutic manual. And although we did not find any heterogeneity on the HDRS data it is imperative in clinical trials that the interventions are adequately defined and described [103]. Factors like personal style, communication skills, and personality of the therapist evidently will influence the way psychotherapy is delivered [104]. It is difficult to describe and control for these subjective factors, and this makes it even more important to relate the therapy to a treatment manual. Otherwise it is unclear what kind of intervention the participants were receiving, and it is difficult to apply any result in clinical practice.
Implications
Our meta-analysis show that the possible benefit from this relatively extensive treatment compared with ‘treatment as usual’ was only a few points on the HDRS. From a clinical point of view it could be argued that this possible benefit is not clinically relevant - especially if you relate this mean difference to the extent and length of the intervention. Furthermore, the NICE guidelines [105] recommend that a mean difference on 3 on the HDRS are needed in order for a intervention to be considered significantly clinically effective [105]. We found a mean difference on 2.15 on the HDRS. Other meta-analyses have used this definition to judge if an intervention should be considered clinically effective [9].
In our protocol [18] we chose HDRS, BDI, and MADRS as our primary outcome measures because we expected that most trials would only use these assessment measures, and HDRS has in many years been the gold standard to quantify depressive symptoms in clinical trial [106]. Severity of depression as measured by the total HDRS score has failed to predict suicide attempts [107], and some publications have questioned the usefulness of the HDRS and concluded that the scale is psychometrically and conceptually flawed [106]. MADRS and BDI probably correspond to HDRS [108], [109]. We do not know if these scales are able to assess any potential beneficial effects of cognitive therapy. From the patient's point of view, a score on HDRS, BDI, or MADRS is not necessarily a measure of the degree of suffering, and other assessment methods could demonstrate a more or less substantial effect of any given intervention for depression. The HDRS during 40 years has been the gold standard to quantify depressive symptoms in clinical trials [106]. There is a need for trials assessing and reporting more clinically relevant outcome measures. We believe such assessment methods should be reporting on adverse events and suicidal tendencies, or assessment methods that correspond to clinically relevant outcomes seen from the patient's point of view.
Future research should focus on comparing the effect of cognitive therapy versus ‘treatment as usual’ for major depressive disorder. First and foremost such trials should be conducted with longer follow-up, low risk of bias (systematic errors) and low risk of random errors (play of chance) [110]. Such trials should also report on adverse events, suicide inclination, suicide attempts, and numbers of suicides and the specific content of the ‘treatment as usual’-interventions should be reported. There seems to be a need for a new gold standard assessment method other than HRDS to assess depressive symptoms.
Conclusions
Cognitive therapy might not be an effective intervention for major depressive disorder compared with ‘treatment as usual’. The possible treatment effect measured on the HDRS is relatively small. More randomized trials with low risk of bias, low risk of random errors, and longer follow-up are needed. Future trials should assess the effect of cognitive therapy on adverse events, suicidal tendencies, quality of life, and other clinically relevant outcomes.
Author Contributions
Analyzed the data: JJ JLH CG. Contributed reagents/materials/analysis tools: JJ JLH OJS. Wrote the paper: JJ CG. Disagreements were resolved by discussion or through arbitration by CG. ES contributed with psychiatric expertise. All authors contributed to and have approved the manuscript.
References
- 1. Levav I, Rutz W (2002) The WHO world health report 2001. New understanding- new hope. Israel Journal of Psychiatry & Related Sciences 39: 50–56.
- 2. Greenberg P, Stiglin LE, Finkelstein SN, Berndt ER (1993) The economic burden of depression in 1990. J Clin Psychiatry 405–418.
- 3. Kessler RC, McGnagle KA, Zhao S, Nelson CB, Hughes M, et al. (1994) Lifetime and 12- month prevalence of DSM – III- R psychiatric disorders in the united states: Results from the Natinal Comorbidity Survey. Arch Gen Psychiatry 51: 8–19.
- 4. Spijker J, de GR, Bijl RV, Beekman AT, Ormel J, et al. (2002) Duration of major depressive episodes in the general population: results from The Netherlands Mental Health Survey and Incidence Study (NEMESIS). Br J Psychiatry 181: 208–213.
- 5. Arnow BA, Constantino MJ (2003) Effectiveness of psychotherapy and combination treatment for chronic depression. Journal of Clinical Psychology 893–905.
- 6.
Bech P (1999) Stress & livskvalitet (Stress & quality of life).
- 7. Fawcett J (1993) The morbitity and mortality of clinical depression. International Clinical Psychopharmacology 217–220.
- 8. Cipriani A, Santilli C, Furukawa TA, Signoretti A, Nakagawa A, et al. (2009) Escitalopram versus other antidepressive agents for depression. Cochrane Database Syst Rev CD006532.
- 9. Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, et al. (2008) Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Medicine 5:
- 10. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358: 252–260. 358/3/252 [pii];doi: https://doi.org/10.1056/NEJMsa065779.
- 11. Geddes JR, Carney SM, Davies C, Furukawa TA, Kupfer DJ, et al. (2003) Relapse prevention with antidepressant drug treatment in depressive disorders: a systematic review. Lancet 361: 653–661. S0140-6736(03)12599-8 [pii];doi: https://doi.org/10.1016/S0140-6736(03)12599-8.
- 12. Beck AT, Rush AJ, Shaw BF, Emery G (1979) Cognitive therapy of depression. Aust N Z J Psychiatry 36: 275–278. 1015b [pii].
- 13.
Vittengl JR, Clark LA, Dunn TW, Jarrett RB (2007) Reducing Relapse and Recurrence in Unipolar Depression: A Comparative Meta-Analysis of Cognitive-Behavioral Therapys Effects.
- 14. Elkin I, Shea MT, Watkins JT, Imber SD, Sotsky SM, et al. (1989) National Institute of Mental Health Treatment of Depression Collaborative Research Program. General effectiveness of treatments. Arch Gen Psychiatry 46: 971–982.
- 15.
Higgins J, Green S (2008) Cochrane Handbook for Systematic Reviews of interventios, Version 5.0.0.
- 16. Wetterslev J, Thorlund K, Brok J, Gluud C (2008) Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. J Clin Epidemiol 61: 64–75. S0895-4356(07)00147-3 [pii];doi: https://doi.org/10.1016/j.jclinepi.2007.03.013.
- 17. Brok J, Thorlund K, Gluud C, Wetterslev J (2008) Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analysis. J Clin Epidemiol 763–769.
- 18.
Jakobsen J, Lindschou Hansen J, Storebø O, Simonsen E, Gluud C (2010) The effect of cognitive therapy versus treatment as usual in patients with major depressive disorder. A systematic review of randomised clinical trials with meta-analysis and trial sequential analysis. Protocol published at www.ctu.dk February 2010.
- 19.
World Health Organization (1992) International Statistical Classification of Diseases an Related Health Problems (10th Revision) ICD 10.
- 20.
American Psychiatric Association (1980) Diagnostic and Statistical Manual of Mental Disorders (DSM III).
- 21.
American Psychiatric Association (1987) Diagnostic and Statistical Manual of Mental Disorders (DSM III-R).
- 22.
American Psychiatric Association (1994) Diagnostic and Statistical Manual of Mental Disorders.
- 23. Sofuoglu M, Sugarman DE, Carroll KM (2010) Cognitive function as an emerging treatment target for marijuana addiction. Exp Clin Psychopharmacol 18: 109–119. 2010-06751-001 [pii];doi: https://doi.org/10.1037/a0019295.
- 24. Davidson KW, Rieckmann N, Clemow L, Schwartz JE, Shimbo D, et al. (2010) Enhanced depression care for patients with acute coronary syndrome and persistent depressive symptoms: coronary psychosocial evaluation studies randomized controlled trial. Arch Intern Med 170: 600–608. 170/7/600 [pii];doi: https://doi.org/10.1001/archinternmed.2010.29.
- 25. Wilkins CH, Mathews J, Sheline YI (2009) Late life depression with cognitive impairment: evaluation and treatment. Clin Interv Aging 4: 51–57.
- 26. Howard M, Battle CL, Pearlstein T, Rosene-Montella K (2006) A psychiatric mother-baby day hospital for pregnant and postpartum women. Arch Womens Ment Health 9: 213–218.
- 27. Gluud LL (2006) Bias in clinical intervention research. Am J Epidemiol 493–501.
- 28. Wood L, Egger M, Gluud LL, Schulz KF, Juni , et al. (2008) Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 336: 601–605. bmj.39465.451748.AD [pii];doi: https://doi.org/10.1136/bmj.39465.451748.AD.
- 29. Gluud C (2006) The culture of designing hepato-biliary randomised trials. J Hepatol 44: 607–615. S0168-8278(05)00823-8 [pii];doi: https://doi.org/10.1016/j.jhep.2005.12.006.
- 30. Kjaergaard L, Villumsen J, Gluud C (2001) Reported methodologic quality and discrepancies between large and small randomized trials in meta-analysis. ANN INTERN MED 135: 982–989.
- 31. Hamilton M (1960) A rating scale for depression. J NEUROL NEUROSURG PSYCHIATRY 23: 56–61.
- 32. Bech AT (1961) An inventory for measuring depression. Archives of General Psychiatry 561–571.
- 33. Montgomery SA, Asberg M (1979) A new depression scale designed to be sensitive to change. Br J Psychiatry 134: 382–389.
- 34.
ICH-GCP (1997) Code of Federal Regulations & Guidelines Vol. 1. International Committee on Harmonization Philadelphia, US: Barnett International/PAREXEL, 1997.
- 35.
The Nordic Cochrane Centre TCC (2008) Review Manager (RevMan) [Computer program]. Version 5.0. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration.
- 36. Altman DG, Bland JM (2003) Interaction revisited: the difference between two estimates. BMJ 326: 219.
- 37. Bowers WA (1990) Treatment of depressed in-patients: Cognitive therapy plus medication, relaxation plus medication, and medication alone. British Journal of Psychiatry 156: 73–78.
- 38. Jarrett RB, Schaffer M, McIntire D, Witt-Browder A, Kraft D, et al. (1999) Treatment of atypical depression with cognitive therapy or phenelzine: A double-blind, placebo-controlled trial. Archives of General Psychiatry 56: 431–437.
- 39. Gordon VC, Matwychuk AK, Sachs EG, Canedy BH (1988) A 3-year follow-up of a cognitive-behavioral therapy intervention… to reduce depression in women. Archives of Psychiatric Nursing 2: 218–226.
- 40. Scott J, Teasdale JD, Paykel ES, Johnson AL, Abbott R, et al. (2000) Effects of cognitive therapy on psychological symptoms and social functioning in residual depression. British Journal of Psychiatry 177: 440–446.
- 41. Hautzinger M, Welz S (2004) Cognitive behavioral therapy for depressed elderly outpatients - A controlled randomized trial. [German]. Verhaltenstherapie & Psychosoziale Praxis 36: 789–798.
- 42. Hautzinger M, de Jong-Meyer R, Treiber R, Rudolf GA (1996) Process analyses and prediction of treatment outcome of psychological and pharmacological therapy of nonendogenous unipolar depression. [German]. Zeitschrift fur Klinische Psychologie 25: 146–154.
- 43. Rohan KJ, Roecklein KA, Lacy TJ, Vacek PM (2009) Winter depression recurrence one year after cognitive-behavioral therapy, light therapy, or combination treatment. Behavior Therapy 40: 225–238.
- 44. Maynard CK (1993) Comparison of effectiveness of group interventions for depression in women. ARCH PSYCHIATR NURS 7: 277–283.
- 45. Rohan KJ, Roecklein KA, Tierney Lindsey K, Johnson LG, Lippy RD, et al. (2007) A randomized controlled trial of cognitive-behavioral therapy, light therapy, and their combination for seasonal affective disorder. [References]. Journal of Consulting and Clinical Psychology 75: 489–500.
- 46. Vittengl JR, Clark LA, Jarrett RB (2004) Self-Directed Affiliation and Autonomy Across Acute and Continuation Phase Cognitive Therapy for Recurrent Depression. [References]. Journal of Personality Assessment 83: 235–247.
- 47. Wilson PH, Goldin JC, Charbonneau-Powis M (1983) Comparative efficacy of behavioral and cognitive treatments of depression. Cognitive Therapy and Research 7: 111–124.
- 48. Sado M, Knapp M, Yamauchi K, Fujisawa D, So M, et al. (2009) Cost-effectiveness of combination therapy versus antidepressant therapy for management of depression in Japan. [References]. Australian and New Zealand Journal of Psychiatry 43: 539–547.
- 49. Covi L, Lipman RS (1987) Cognitive behavioral group psychotherapy combined with imipramine in major depression. Psychopharmacology Bulletin 23(1): 173–6.
- 50. Pace TM, Dixon DN (1993) Changes in depressive self-schemata and depressive symptoms following cognitive therapy. Journal of Counseling Psychology 40: 288–294.
- 51. Paykel ES, Scott J, Teasdale JD, Johnson AL, Garland A, et al. (1999) Prevention of relapse in residual depression by cognitive therapy: A controlled trial. Archives of General Psychiatry 56: 829–835.
- 52. Mirabel-Sarron C, Blanchet A (1992) Analysis of the conversations of patients with depression. [French]. Psychologie Francaise 37: 277–289.
- 53. Jacobson NS, Fruzzetti AE, Dobson K, Whisman M, Hops H (1993) Couple therapy as a treatment for depression: II. The effects of relationship quality and therapy on depressive relapse. [References]. Journal of Consulting and Clinical Psychology 61: 516–519.
- 54. Horowitz JL, Garber J, Ciesla JA, Young JF, Mufson L (2007) Prevention of depressive symptoms in adolescents: A randomized trial of cognitive-behavioral and interpersonal prevention programs. [References]. Journal of Consulting and Clinical Psychology 75: 693–706.
- 55. Barnhofer T, Crane C, Hargus E, Amarasinghe M, Winder R, et al. (2009) Mindfulness-based cognitive therapy as a treatment for chronic depression: A preliminary study. [References]. Behaviour Research and Therapy 47: 366–373.
- 56. Gonzalez GS, Fernandez RC, Perez RJ, Amigo I (2006) Depression secondary prevention in primary care. Psicothema 18: 471–477.
- 57. King M, Sibbald B, Ward E, Bower P, Lloyd M, et al. (2000) Randomised controlled trial of non-directive counselling, cognitive-behaviour therapy and usual general practitioner care in the management of depression as well as mixed anxiety and depression in primary care. Health Technology Assessment 4(19). ()(pp i+iii–iv+1–73), 2000 Date of Publication: 2000 i+iii-73.
- 58. Comas-Diaz L (1981) Effects of cognitive and behavioral group treatment on the depressive symptomatology of Puerto Rican women. Journal of Consulting and Clinical Psychology 49: 627–632.
- 59. Harpin RE, Liberman RP, Marks I (1982) Cognitive-behavior therapy for chronically depressed patients. A controlled pilot study. Journal of Nervous and Mental Disease 170(5). ()(pp 295–301), 1982 Date of Publication: 1982 295–301.
- 60. Wong DF (2008) Cognitive behavioral treatment groups for people with chronic depression in Hong Kong: a randomized wait-list control design. Depression and Anxiety 25: 142–148.
- 61. Teasdale JD, Fennell MJ, Hibbert GA, Amies PL (1984) Cognitive therapy for major depressive disorder in primary care. British Journal of Psychiatry 144: 400–406.
- 62. Usaf SO, Kavanagh DJ (1990) Mechanisms of improvement in treatment for depression: Test of a self-efficacy and performance model. Journal of Cognitive Psychotherapy 4: 51–70.
- 63. Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, et al. (2005) Computer-Assisted Cognitive Therapy for Depression: Maintaining Efficacy While Reducing Therapist Time. [References]. AM J PSYCHIATRY 162: 1158–1164.
- 64. Wong FKD (2009) A six-month follow-up study of cognitive-behavioural treatment groups for Chinese people with depression in Hong Kong. Behaviour Change 26: 130–140.
- 65. Simons AD, Levine JL, Lustman PJ, Murphy GE (1984) Patient attrition in a comparative outcome study of depression: A follow-up report. J Affect Disord 6: 163–173.
- 66. Murphy GE, Simons AD, Wetzel RD, Lustman PJ (1984) Cognitive therapy and pharmacotherapy. Singly and together in the treatment of depression. Archives of General Psychiatry 41(1). ()(pp 33–41), 1984 Date of Publication: 1984 33–41.
- 67. Scott C, Tacchi MJ, Jones R, Scott J (1997) Acute and one-year outcome of a randomised controlled trial of brief cognitive therapy for major depressive disorder in primary care. British Journal of Psychiatry 171: 131–134.
- 68. Shamsaei F, Rahimi A, Zarabian MK, Sedehi M (2008) Efficacy of pharmacotherapy and cognitive therapy, alone and in combination in major depressive disorder. HONG KONG J PSYCHIATRY 18: 76–80.
- 69. Dozois DJA, Bieling PJ, Patelis-Siotis I, Hoar L, Chudzik S, et al. (2009) Changes in self-schema structure in cognitive therapy for major depressive disorder: A randomized clinical trial. [References]. Journal of Consulting and Clinical Psychology 77: 1078–1088.
- 70. Hollon SD, DeRubeis RJ, Evans MD, Wiemer MJ (1992) Cognitive therapy and pharmacotherapy for depression: Singly and in combination. Archives of General Psychiatry 49: 774–781.
- 71. Blackburn IM, Bishop S, Glen AIM (1981) The efficacy of cognitive therapy in depression: A treatment trial using cognitive therapy and pharmacotherapy, each alone and in combination. British Journal of Psychiatry 139(3). ()(pp 181–189), 1981 Date of Publication: 1981 181–189.
- 72. Scott MJ, Stradling SG (1990) Group cognitive therapy for depression produces clinically significant reliable change in community-based settings. Behavioural Psychotherapy 18: 1–19.
- 73. Ross M, Scott M (1985) An evaluation of the effectiveness of individual and group cognitive therapy in the treatment of depressed patients in an inner city health centre. Journal of the Royal College of General Practitioners 35(274): 239–42.
- 74. Blackburn IM, Bishop S (1983) Changes in cognition with pharmacotherapy and cognitive therapy. British Journal of Psychiatry 143: 609–617.
- 75. Miller IW, Norman WH, Keitner GI, Bishop SB (1989) Cognitive-behavioral treatment of depressed inpatients. Behavior Therapy 20: 25–47.
- 76. Whisman MA, Miller IW, Norman WH, Keitner GI (1991) Cognitive therapy with depressed inpatients: Specific effects on dysfunctional cognitions. [References]. Journal of Consulting and Clinical Psychology 59: 282–288.
- 77. Miller IW, Norman WH, Keitner GI (1990) Treatment response of high cognitive dysfunction depressed inpatients. Comprehensive Psychiatry 31: 62–71.
- 78. Miller IW, Norman WH, Keitner GI (1989) Cognitive-behavioral treatment of depressed inpatients: Six- and twelve-month follow-up. AM J PSYCHIATRY 146: 1274–1279.
- 79. Dimidjian S, Hollon SD, Dobson KS, Schmaling KB, Kohlenberg RJ, et al. (2006) Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression. [References]. Journal of Consulting and Clinical Psychology 74: 658–670.
- 80. Embling S (2002) The effectiveness of cognitive behavioural therapy in depression. Nursing standard: 1987) 17: 33–41.
- 81. DeRubeis RJ, Hollon SD, Amsterdam JD, Shelton RC, Young PR, et al. (2005) Cognitive Therapy vs Medications in the Treatment of Moderate to Severe Depression. Archives of General Psychiatry 62: 409–416.
- 82. Verduyn C, Barrowclough C, Roberts J, Tarrier N, Harrington R (2003) Maternal depression and child behaviour problems: Randomised placebo-controlled trial of a cognitive-behavioural group intervention. British Journal of Psychiatry 183: 342–348.
- 83. Elkin I, Shea MT, Watkins JT, Imber SD (1989) National Institute of Mental Health Treatment of Depression Collaborative Research Program: General effectiveness of treatments. Arch Gen Psychiatry 46: 971–982.
- 84. Scott AI, Freeman CP (1992) Edinburgh primary care depression study: treatment outcome, patient satisfaction, and cost after 16 weeks. BMJ 304: 883–887.
- 85. Wiles NJ, Hollinghurst S, Mason V, Musa M, Burt V, et al. (2008) A randomized controlled trial of cognitive behavioural therapy as an adjunct to pharmacotherapy in primary care based patients with treatment resistant depression: A pilot study. [References]. Behavioural and Cognitive Psychotherapy 36: 21–33.
- 86. Imber SD, Pilkonis PA, Sotsky SM, Elkin I, Watkins JT, et al. (1990) Mode-specific effects among three treatments for depression. Journal of Consulting and Clinical Psychology 58: 352–359.
- 87. Sotsky SM, Glass DR, Shea MT, Pilkonis PA (1991) Patient predictors of response to psychotherapy and pharmacotherapy: Findings in the NIMH Treatment of Depression Collaborative Research Program. Am J Psychiatry 148: 997–1008.
- 88. Shea MT, Pilkonis PA, Beckham E, Collins JF (1990) Personality disorders and treatment outcome in the NIMH Treatment of Depression Collaborative Research Program. The American Journal of Psychiatry 147: 711–718.
- 89. Shea MT, Elkin I, Imber SD, Sotsky SM (1992) Course of depressive symptoms over follow-up: Findings from the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Archives of General Psychiatry 49: 782–787.
- 90. Barber JP, Muenz LR (1996) The role of avoidance and obsessiveness in matching patients to cognitive and interpersonal psychotherapy: Empirical findings from the Treatment for Depression Collaborative Research Program. [References]. Journal of Consulting and Clinical Psychology 64: 951–958.
- 91. Stewart JW, Garfinkel R, Nunes EV, Donovan S, Klein DF (1998) Atypical features and treatment response in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Journal of clinical psychopharmacology 18: 429–434.
- 92. Dobson KS, Hollon SD, Dimidjian S, Schmaling KB, Kohlenberg RJ, et al. (2008) Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the prevention of relapse and recurrence in major depression. [References]. Journal of Consulting and Clinical Psychology 76: 468–477.
- 93. Miranda J, Azocar F, Organista KC, Dwyer E, Areane P (2003) Treatment of depression among impoverished primary care patients from ethnic minority groups. PSYCHIATR SERV 54: 219–225.
- 94. Miranda J, Green BL, Krupnick JL, Chung J, Siddique J, et al. (2006) One-year outcomes of a randomized clinical trial treating depression in low-income minority women. [References]. Journal of Consulting and Clinical Psychology 74: 99–111.
- 95. Elkin I, Parloff MB, Hadley SW, Autry JH (1985) NIMH treatment of Depression Collaborative Research Program: Background and research plan. Arch Gen Psychiatry 42: 305–316.
- 96. Revicki DA, Siddique J, Frank L, Chung JY, Green BL, et al. (2005) Cost-effectiveness of Evidence-Based Pharmacotherapy or Cognitive Behavior Therapy Compared With Community Referral for Major Depression in Predominantly Low-Income Minority Women. Archives of General Psychiatry 62: 868–875.
- 97. Miranda J, Chung JY, Green BL, Krupnick J, Siddique J, et al. (2003) Treating Depression in Predominantly Low-Income Young Minority Women: A Randomized Controlled Trial. JAMA: Journal of the American Medical Association 290: 57–65.
- 98. Brok J, Thorlund K, Gluud C, Wetterslev J (2008) Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analysis. Journal of Clinical Epidemiology 763–769.
- 99.
Higgins J, Green S (2008) Cochrane Handbook for Systematic Reviews of Interventions, Version 5.0.0.
- 100. Higginson IJ, Carr AJ (2001) Measuring quality of life: Using quality of life measures in the clinical setting. BMJ 322: 1297–1300.
- 101. Hopewell S, Wolfenden L, Clarke M (2008) Reporting of adverse events in systematic reviews can be improved: survey results. J Clin Epidemiol 61: 597–602. S0895-4356(07)00367-8 [pii];10.1016/j.jclinepi.2007.10.005 [doi].
- 102.
Rose S, Bisson J, Churchill R, Wessely S (2002) Psychological debriefing for preventing post traumatic stress disorder (PTSD).
- 103. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P (2008) Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med 148: 295–309. 148/4/295 [pii].
- 104. Walwyn R, Roberts C (2010) Therapist variation within randomised trials of psychotherapy: implications for precision, internal and external validity. Stat Methods Med Res 19: 291–315. 0962280209105017 [pii];doi: https://doi.org/10.1177/0962280209105017.
- 105. Kendrick T, Peveler R (2010) Guidelines for the management of depression: NICE work? Br J Psychiatry 197: 345–347. 197/5/345 [pii];doi: https://doi.org/10.1192/bjp.bp.109.074575.
- 106. Bagby RM, Ryder AG, Schuller DR, Marshall MB (2004) The Hamilton Depression Rating Scale: has the gold standard become a lead weight? Am J Psychiatry 161: 2163–2177. 161/12/2163 [pii];doi: https://doi.org/10.1176/appi.ajp.161.12.2163.
- 107. Chakraborty R, Chatterjee A (2007) Predictors of Suicide Attempt Among those with Depression in an Indian Sample: A Brief Report. The Internet Journal of Mental Health 4:
- 108. Fitzgibbon ML, Cella DF, Sweeney JA (1988) Redundancy in measures of depression. J Clin Psychol 44: 372–374.
- 109. Heo M, Murphy CF, Meyers BS (2007) Relationship between the Hamilton Depression Rating Scale and the Montgomery-Asberg Depression Rating Scale in depressed elderly: a meta-analysis. Am J Geriatr Psychiatry 15: 899–905. 15/10/899 [pii];doi: https://doi.org/10.1097/JGP.0b013e318098614e.
- 110. Keus F, Wetterslev J, Gluud C, van Laarhoven CJ (2010) Evidence at a glance: error matrix approach for overviewing available evidence. BMC Med Res Methodol 10: 90. 1471-2288-10-90 [pii];10.1186/1471-2288-10-90 [doi].