Non-adherence to guideline development standards

The IOM’s standards for developing trustworthy guidelines include, among other criteria, the following: (1) the process by which guidelines are developed should be detailed explicitly and be publicly accessible; (2) CPG developers should use systematic reviews that meet standards set by the IOM; (3) the guideline development group should be independent from industry, and at a minimum, neither the chair nor the majority of members should have financial conflicts of interest (FCOI); (4) the guideline development group should be interdisciplinary, balanced and be comprised of a variety of methodological experts and stakeholders; (5) for each recommendation, there should be a rating of the strength of, and level of confidence in, the recommendations; and (6) the guideline should be sent out for external review before release.8

Recently, a guideline on the recognition and management of mixed depression was published in CNS Spectrums.19 This guideline was described as the ‘first ever’ for patients with Major Depressive Disorder (MDD) with mixed features20 (mixed features is a course specifier that may be applied to MDD; it refers to the presence of subthreshold hypomanic or manic symptoms). On the surface, given the U.S. Preventive Services Task Force call for routine depression screening, this might appear to be a timely set of diagnostic and treatment recommendations that could enhance collaborative decision-making. However, this guideline fails to meet any of the IOM standards for trustworthy guidelines and the two-and-a-half-page disclosure of conflicts of interest by the majority of the authors raises questions about the potential for bias.

For example, the guideline was written by ‘a group of experts’ convened by an unidentified entity and there was no description of how the 20 experts were selected. It was published in CNS Spectrums ‘in association with the Neuroscience Education Institute’, a company, founded by the lead author of the guideline, that provides for-profit continuing medical education for psychiatric disorders. The lead author of the guideline is also the editor-in-chief of the journal which published the guideline. There is no description of the process used to develop the recommendations; there is no report of a systematic review of the evidence being performed or a report on how evidence was evaluated for quality (it was not graded and there was no rating of the strength of recommendations). Representatives of stakeholder groups such as patients, public health entities or any other disciplines or specialties beyond psychiatry were not identified. There does not appear to have been any external review of the guideline prior to publication.

The guideline also runs the risk of widening diagnostic boundaries and unnecessarily exposing people to treatment. The Diagnostic and Statistical of Mental Disorders-5 (DSM-5) criteria for making the diagnosis of ‘mixed features’ are extended with additional ‘non-DSM-5 criteria’ (p. 205).19 However, the expert panel did not provide adequate empirical support for their significant expansion of these criteria. In fact, many of the symptoms that the panel suggests are indicative of ‘mixed depression’ (eg, insomnia, irritability, rumination) are classic symptoms of unipolar depression. Furthermore, clinicians are strongly encouraged to look for evidence of subthreshold symptoms (‘Ask every patient. Every time’ (in bold on the second page of the guideline)). It is also emphasised that ‘mixed depression’ is significantly underdiagnosed in clinical practice and is ‘especially common in children and adolescents’ (p. 212).19

The first antipsychotics that are recommended, lurasidone (Latuda) and asenapine (Saphris), are not available as generic and cost >$1000 a month and >$500 a month, respectively.21 22 If the evidence clearly shows significant benefit of these patented medications and if the guideline development group met IOM’s standards for trustworthy guidelines, then this would be an evidence-based, not simply industry-friendly, recommendation. However, the devil is in the details.

Significant financial conflicts of interest among members of the guideline development group

Recognising that ‘the management of COIs in guidelines is often unsatisfactory’, the G-I-N Network Board of Trustees developed nine core principles to guide the disclosure of interests and management of COIs (p. 548).23 Their first principle is that ‘Guideline developers should make all possible efforts to not include members with direct financial or relevant indirect COIs’ (p. 550),23 and they emphasise that when COIs cannot be avoided ‘the challenge lies in judicious management’ (p. 552).23 Similarly, the National Institute for Health and Care Excellence (NICE) has a robust policy for disclosing and managing FCOI.24 As previously noted, in 2011 the IOM developed a set of specific recommendations for enhancing trustworthy guidelines, emphasising the critical importance of having independent guideline developers.

In contrast to these principles and standards, at least 13 out of the 20 authors (including the lead author) had ties to the drug companies that manufacture antipsychotic medications recommended in this guideline, and there was no information about the management of FCOI. Five of the authors served on speakers’ bureaus, including the lead author who serves on the speakers’ bureau for Sunovion, the manufacturer of lurasidone. While any financial arrangement is considered to be a conflict of interest, participation on speakers’ bureaus—an extension of the marketing departments of manufacturers—is widely recognised to constitute a significant conflict of interest.25 Pharmaceutical companies refer to individuals who serve on speakers’ bureaus as ‘key opinion leaders’ because they are seen as essential to the marketing of new disorders as well as the drugs to treat them.26

Poorly established evidence base to support treatment recommendations

Seven studies were cited as evidence for the recommendation of antipsychotics for mixed depression.27–33 Four of these were post hoc analyses, and thus because of multiple testing there is a significant increased risk of type I (false positive) errors,34 thereby giving a misleading sense of efficacy. Also, post hoc analyses should only be used to generate hypotheses for future research; they should not be used as evidence of the efficacy of an intervention.35

The fifth study cited in support of their recommendation was a clinical trial for a different condition, not depression.32 Thus, there were only two randomised controlled trials (RCTs) on which the recommendation to treat a depressed patient with lifelong antipsychotic medication was based.30 33 One RCT assessed lurasidone and the other assessed ziprasidone. The lurasidone study was funded by the manufacturer (Sunovion); all authors of that study also had commercial ties to the company, and six authors, including the person to whom correspondence is to be directed, are listed as employees of the company. They also thank a non-author for ‘providing editorial and medical writing assistance, which was funded by Sunovion’ (p. 406).30 The ziprasidone study was also funded by the manufacturer (Pfizer); six of the authors had industry ties, and it was disclosed that ‘Pfizer researchers were involved in helping to design the study’ (p. 1).33

Regarding the two on-patent drugs recommended as front-line interventions, asenapine and lurasidone, we searched clinical trials.gov using the terms asenapine/Saphris *and* mixed depression and lurasidone/Latuda *and* mixed depression. There were no completed trials for asenapine for mixed depression. There were three trials listed for lurasidone, two for efficacy and one assessing treatment-emergent adverse events (TEAEs). One efficacy study was published (noted above), and the other was withdrawn. The TEAE study was unpublished, but it was reported on clinicaltrials.gov that although no serious adverse events were noted, >66% for participants in the drug arm reported TEAEs. However, the study had <50 participants, making it impossible to obtain clinically relevant information about side effects (clinicaltrials.gov accessed 21 June 2017). It is noteworthy that there was no mention in this guideline of the fact that in 2011 the U.S. Food and Drug Administration issued a warning that serious allergic reactions were reported with the use of asenapine.36

Can practice guidelines be a public health risk?

There are other examples of guidelines that are unduly influenced by industry and that could result in negative health consequences. One of the more famous examples is the controversial 2006 National Kidney Foundation (NKF) guideline on anaemia in chronic kidney disease.37 The NKF partnered with Amgen, the makers of erythropoietin alfa and darbepoetin-alfa, two of the drugs recommended to increase haemoglobin levels. The guideline recommended a target haemoglobin in chronic kidney disease of >11.0 g/dL, a level of correction that increases the likelihood of cardiovascular events.38 Because of this and other examples, Kung and colleagues reviewed 114 guidelines published in the NGC against the IOM standards.39 They found that the overall median number of IOM standards satisfied was only 8 of 18 (44.4%), a rate that has not changed in almost 15 years.4

The ‘first ever’ guideline on the recognition and management of mixed depression highlights ongoing problems in guideline development, namely, that poorly developed, untrustworthy guidelines continue to be produced and conflicts of interest are still a major problem in guideline development. Indeed, if this guideline is followed by clinicians, it could result in harm to a significant number of patients. One of the recommended first-line drugs, asenapine, is associated with serious allergic reactions.36 A table in the guideline shows several alternatives, for example, aripiprazole, with more favourable safety profiles. Why were these safer options not recommended more strongly? Our hypothesis is that it was due to the significant COIs of the guideline panel. The guideline meets none of the IOM criteria, and the methods used to develop the guideline were not reported.

A review of this guideline also illustrates the difference between reporting, managing and eliminating COIs. Although the authors of this guideline disclose their conflicts (spanning almost three pages of the manuscript), they clearly did not manage their COI. Management could have been accomplished by allowing, per IOM and as other groups do, conflicted panel members to serve as subject matter experts but without writing or voting privileges. Also, as IOM emphasises, conflicted panel members should be the exception not the rule—the best option is to eliminate COIs by not allowing members to have financial or intellectual COIs.

What can be done to ensure guidelines are trustworthy?

At the very least, a guideline is not a guideline if it fails to adhere to the standards put forth by public health institutions and by organisations that develop and disseminate guidelines such as IOM, NICE, National Health and Medical Research Council (NHMRC), G-I-N and AHRQ. Although there are certainly high-quality guidelines that routinely follow these standards and that enhance evidence-based collaborative care, because of the harms associated with poor quality and unreliable guidelines, more safeguards are needed. Thus, we believe that all guidelines should be assessed prior to publication to determine whether they meet the standards required by institutions and organisations such as IOM and G-I-N. Furthermore, editors of journals should only publish guidelines that meet these standards. Also, although the AGREE II instrument is often used as a quality assessment tool, it requires a substantial amount of time and familiarity with the instrument which the busy clinician often does not have. Organisations outside the USA, such as NICE and NHMRC, have a formal process for quality control; they seek external peer and expert review to identify any potential risks associated with FCOI or other issues. We believe that it is time for the USA to consider adopting a national guideline assessment process. In terms of group composition, panels need to be multidisciplinary to prevent ‘group-think’ and guild interests from unduly influencing the development process, and methodologists should be members of each guideline panel or at least serve as consultants.8 15

Consolidating guideline development efforts would also likely improve guidelines. We do not need multiple guidelines (often recommending different things) on the same topic developed by multiple groups. Guideline development needs to be prioritised for those diseases with significant practice variability and for which a valid evidence base can guide recommendations. Guideline development should be commissioned to panels with demonstrated technical and clinical expertise. To achieve this, guideline development should be centralised under the guidance of AHRQ in the USA or similar group in other countries (eg, G-I-N). Until this happens, we will continue to see the proliferation of industry-friendly opinion pieces, not genuine guidelines, developed by conflicted experts.