Comparative effectiveness of biological medicines in rheumatoid arthritis: systematic review and network meta-analysis including aggregate results from reanalysed individual patient data
BMJ 2020; 370 doi: https://doi.org/10.1136/bmj.m2288 (Published 07 July 2020) Cite this as: BMJ 2020;370:m2288Linked Opinion
Lessons from a network meta-analysis of biologics in rheumatoid arthritis
- Kirsten Janke, researcher1,
- Katharina Biester, senior researcher1,
- Dietmar Krause, rheumatologist3,
- Bernd Richter, coordinating editor4,
- Christoph Schürmann, statistician2,
- Katharina Hirsch, statistician2,
- Helmut Hörn, researcher1,
- Michaela Florina Kerekes, data manager1,
- Petra Kohlepp, data manager1,
- Beate Wieseler, head of department1
- 1Drug Assessment Department, Institute for Quality and Efficiency in Health Care, Im Mediapark 8, Cologne 50670, Germany
- 2Medical Biometry Department, Institute for Quality and Efficiency in Health Care, Cologne, Germany
- 3Rheumatology Practice Gladbeck, Gladbeck, Germany
- 4Cochrane Metabolic and Endocrine Disorders Group, Institute of General Practice, Medical Faculty of the Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany
- Correspondence to: B Wieseler beate.wieseler{at}iqwig.de
- Accepted 30 April 2020
Abstract
Objective To assess the comparative effectiveness of biological medicines in rheumatoid arthritis in sufficiently similar patient populations, based on the current definitions of key outcomes.
Design Systematic review and network meta-analysis including aggregate results from reanalysed individual patient data.
Data sources Clinical study reports and aggregate results from reanalyses of individual patient data on key outcomes for rheumatoid arthritis provided by study sponsors for studies conducted up to 2017, and several databases and registries from inception up to February 2017.
Eligibility criteria for selecting studies Randomised controlled trials investigating patient relevant outcomes in adults with rheumatoid arthritis treated with biological medicines in combination with methotrexate after methotrexate failure for at least 24 weeks.
Results 45 eligible trials were identified. Combining data from clinical study reports and aggregate results from reanalyses of individual patient data allowed extensive analyses yielding sufficiently similar populations and homogeneous study results for network meta-analyses, including up to 35 studies on eight biological medicines combined with methotrexate. These analyses showed few statistically significant differences between the combination treatments. For example, anakinra showed less benefit than almost all the other seven biological medicines regarding clinical remission or low disease activity (clinical disease activity index ≤2.8 or ≤10, respectively) and certolizumab pegol showed more harm than the other seven biological medicines regarding serious adverse events or infections. Some outcomes had very wide 95% confidence intervals, potentially implying unidentified differences between the eight biological medicines, but wide 95% confidence intervals were less prominent for low disease activity, serious adverse events, and infections. Owing to a lack of head-to-head trials, results were mainly based on indirect comparisons with a limited number of studies, and recently approved Janus kinase inhibitors could not be included.
Conclusions For patients with rheumatoid arthritis after methotrexate failure, only minor differences in benefits and harms were seen between biological medicines in combination with methotrexate. However, the analysis was hampered by a lack of long term direct comparisons. The substantial information gain achieved by the reanalysis of individual patient data calls for the routine availability of individual patient data.
Introduction
Rheumatoid arthritis is one of the most common chronic inflammatory diseases. Joint inflammation can lead to pain, stiffness, swelling, structural damage, and impaired function.123 Ultimately, rheumatoid arthritis can cause progressive disability and premature death.4567
Treatment for the rheumatoid arthritis primarily aims to reduce disease activity to a level where patients are free of signs and symptoms of inflammatory activity (that is, clinical remission).8 Low disease activity can also be an acceptable treatment goal, especially for patients with previous treatment failure.9 The most recent and stringent definitions for clinical remission and low disease activity have been suggested by the American College of Rheumatology (ACR) and the European League Against Rheumatism (EULAR).10
Current options for rheumatoid arthritis include disease modifying antirheumatic drugs (DMARDs), which can be combined with non-steroidal anti-inflammatory drugs (NSAIDs) and corticosteroids.3911 DMARDs are divided into conventional synthetic DMARDs (eg, methotrexate), targeted synthetic DMARDs (Janus kinase inhibitors), and biological DMARDs (biological medicines or bDMARD).12 Although a wide range of biological medicines is available, their comparative effectiveness is still under discussion.
This systematic review of randomised controlled trials aimed to compare the benefits and harms of biological medicines in combination with methotrexate in patients with rheumatoid arthritis after methotrexate failure. Given the many biological medicines available, we used a network meta-analysis to compare several different treatments. An important prerequisite for such an analysis is the similarity of the studies included. We obtained similar patient populations by using aggregate results from systematic reanalyses of individual patient data performed by study sponsors for those studies with heterogeneous populations. We also obtained aggregate results according to current definitions of clinical remission and low disease activity by reanalyses of individual patient data by study sponsors.
The systematic review formed part of a health technology assessment of biological medicines in rheumatoid arthritis conducted by the German health technology assessment agency, the Institute for Quality and Efficiency in Health Care (IQWiG). The full report (in German) and protocol are available on the IQWiG website.13 IQWiG’s methodological approach for systematic reviews is described in its methods paper.14
Methods
Search strategy and study selection
Our systematic review was an update and extension of a previous systematic review published in 2013.15 Information retrieval was based on the previous review. We conducted a bibliographic search for relevant primary studies and secondary publications (systematic reviews and health technology assessment reports). A list of the databases and search strategies is presented in supplement 1. The last search was conducted in February 2017.
The following additional sources were searched to identify further studies: reference lists of four current systematic reviews,16171819 clinical trial registries and study results databases, as well as the websites of the European Medicines Agency (EMA) and US Food and Drug Administration (FDA). We also asked marketing authorisation holders of biological medicines approved by the EMA and included in our systematic review to provide unpublished studies and additional unpublished data from published and unpublished studies. The biological medicines included and companies contacted are listed in supplement table 1.
As a prerequisite for the use of unpublished data, IQWiG asked the marketing authorisation holders to sign an agreement requiring submission of a list of all sponsored published and unpublished studies investigating the biological medicine concerned, and of CONSORT compliant documents (in general, the complete clinical study reports) on all relevant studies selected by IQWiG. This procedure was required to avoid bias due to selective provision of data. For all investigator initiated trials selected by IQWiG, sponsors were also asked to supply complete clinical study reports, if available.
To meet both the methodological requirements of a network meta-analysis and the current recommendations of rheumatoid arthritis societies, we aimed to include results from sufficiently similar patient populations and to use the most current definitions of rheumatoid arthritis outcomes. For this purpose, aggregate results of two types of reanalyses of individual patient data from the studies included were requested from the study sponsors. Firstly, if only a subpopulation of the overall study population was relevant for the current systematic review, aggregate results from a reanalysis of characteristics of this subpopulation (eg, of the patients without pre-treatment with biological medicines) and results on all outcomes were requested. Secondly, for all studies or study subpopulations included, aggregate results from reanalyses of clinical remission (clinical disease activity index (CDAI) ≤2.8) and low disease activity (CDAI ≤10) were requested, if not already available in the study documentation.
We included published and unpublished studies with the following characteristics: randomised clinical trials of adults (aged ≥18) with rheumatoid arthritis and a minimum study duration of 24 weeks comparing biological medicines with each other or with a potential common comparator for inclusion in a network meta-analysis. Studies were included if the biological medicines (approved until November 2016) were administered according to regulatory approval (supplement table 1). For the analyses presented in this publication, we selected studies in patients treated with biological medicines in combination with methotrexate after methotrexate failure.
We included studies investigating at least one of the following predefined patient relevant outcomes: clinical remission, low disease activity, pain, physical function, health related quality of life, fatigue, serious adverse events, discontinuation due to adverse events, infections, serious infections, and mortality. The availability of a full text document (eg, journal article or clinical study report) was required. No restrictions applied to the date of publication or to language. Non-English or non-German citations had to include an English title or abstract indicating potential relevance for the review; in this case, the full text was obtained and translated.
Two reviewers independently screened titles and abstracts of the retrieved citations to identify potentially eligible publications. The full texts of these articles were independently evaluated by the same two reviewers. Primary publications on rheumatoid arthritis were identified and the full set of inclusion and exclusion criteria was then applied to identify eligible studies. All documents retrieved from non-bibliographic sources were also screened for eligibility or relevant information on studies by two reviewers. Disagreements were resolved by consensus.
Data extraction
The individual steps of the data extraction and risk-of-bias assessment procedures were always conducted by one person and checked by another; disagreements were resolved by consensus. The clinical study report was the primary source for data extraction; if no report was available, we used journal articles, registry entries, and the corresponding results. Documents presenting reanalyses of study data provided by study sponsors were additional sources. Study data were extracted by standardised tables routinely used by IQWiG and specifically adapted for this systematic review.
The following information was extracted from each included study: study characteristics (citation, study design and duration, sample size, location, number of centres, time of study conduct, primary and secondary outcomes); intervention characteristics; characteristics of the planned population; baseline characteristics of patients included (general and disease specific characteristics, including previous and concomitant drug treatment); patient relevant outcomes and risk-of-bias items. After a period of public consultation on the preliminary protocol and report in March 2017 and July 2018, allowing four weeks to provide comments on the German language report, the original protocol was amended. One major change affected the definition of low disease activity used in the review (from disease activity score (DAS) 28 <3.2 to CDAI ≤10; the DAS 28 includes inflammatory parameters that could be influenced more by some biological medicines (eg, tocilizumab920) than by others, and could hamper a fair comparison of the effects of biological medicines on patient relevant outcomes).
The information extracted for this systematic review and the network meta-analysis is included in supplement 2. The complete datasets extracted from the individual studies are available in the full German language report.13
Quality assessment and risk of bias
The risk-of-bias assessment followed IQWiG’s methods 14 (see supplement tables 5-14 for assessment items). Risk of bias was assessed for all studies providing direct comparisons of relevant biological medicines. For any pairwise contrast contributing to an indirect comparison within the network, risk of bias was only assessed if two criteria were fulfilled: that the effect for the indirect comparison was statistically significant and the pairwise contrast consisted of only one study. This approach was applied because, according to IQWiG’s methods, a statistically significant effect for an indirect comparison in this constellation is relevant only if the single study shows a low risk of bias. If more than one study with a high risk of bias contributes to the respective contrast, the effect is considered to be relevant, independently of the risk of bias.
The certainty of the conclusions from a network meta-analysis was determined by the number of studies informing the pairwise contrasts, inclusion of direct comparisons, homogeneity of the studies and consistency of direct and indirect comparisons, and the risk of bias of the studies contributing to an effect (supplement 2).
Data synthesis and statistical analysis
The prerequisite for conducting the network meta-analysis was an adequate structural quality of the study pool—that is, availability of a study pool meeting the assumptions of similarity, homogeneity, and consistency. In general, an indirect comparison allows estimation of an effect between two treatments A and B in the absence of direct comparative trials if other trials comparing A or B with a common comparator C (intermediate comparator) are available. The validity of such an indirect estimate relies on the assumption that the evidence obtained from the second group of trials is valid for the comparison of interest. This validity requires the assumption of similar study characteristics, such as the study design applied or the population investigated. If this assumption is not met, possible effect modifiers could affect the indirect estimate. A consequence of this assumption is that both the direct and the indirect estimations of the effect of treatments A versus B produce comparable results. While the conceptual assumption is called “similarity” or “transitivity,” the statistical consequence is called “consistency.”
Homogeneity is also a consequence of the similarity assumption, but can also generally be seen as a requirement in conventional meta-analyses.21222324 For the check-of-similarity assumptions, the following factors of the studies or study populations were considered: age, sex, disease severity, disease duration and pre-treatment of study population, intervention, concomitant medication, study duration, year of study conduct, and the outcome measures considered. Homogeneity was assumed if no substantial heterogeneity occurred in the study pool for a given contrast comprising two or more studies. Consistency was assumed if the estimates from an indirect comparison were confirmed by the estimates from a direct comparison in a closed loop of the network. Study results were not pooled and analysed via the network meta-analysis if any of the assumptions were rejected. If the check of homogeneity or consistency was not possible due to the network structure, the network meta-analysis was still conducted, but the certainty of the conclusion was downgraded.
We assessed heterogeneity using a test of interaction. Inconsistency was tested locally within each loop by the node splitting procedure.25 Details on the sensitivity analyses conducted in the case of heterogeneity or inconsistency (significance at the 0.05 level) are available in supplement 2.
Risk ratios and mean differences were the effect measures used for binary and continuous outcomes, respectively. Risk ratios and their standard errors were normalised by taking logarithms of data from contingency tables. In the case of zero events in either treatment group, a factor of 0.5 was added to each cell. Estimates of mean differences and their standard errors were extracted from the original scales used in the primary studies. In case of statistically significant effects, the final network meta-analysis results of a continuous outcome analysis were supported by an additional analysis based on standardised effects by Hedges’ g to assess the relevance of the effect. A relevant difference was assumed if the effect of the network meta-analysis based on standardised data was outside the (−0.2 to 0.2) interval.14 All network meta-analyses were conducted within a frequentist setting using random effects models.26 We used the software package netmeta in R (supplement 2 includes the programming code). Each hypothesis of no difference between any two treatments was tested locally at the 0.05 level. We did not adjust for multiple testing. The resulting estimates from the network meta-analysis were presented with 95% confidence intervals.
Two types of sensitivity analyses were performed for all analyses with the following goals. The first type of analyses identified sufficiently homogeneous and consistent study pools. The second type of analyses checked whether the results were robust with regard to the similarity assumptions (that is, whether results changed after removing studies including patients with different clinical characteristics that were accepted during the check of similarity; see supplement 2 for details).
We aimed to estimate differences between biological medicines quantitatively. We refrained from ranking the network meta-analysis results, because our focus was on estimating actual treatment contrasts. In addition, each ranking method responds to a different question of interest and various methodological problems can hamper interpretation.2728 For example, ranking methods have been shown to be sensitive to network composition,29 do not reflect a treatment’s effect size,30 and cannot always be accompanied by confidence or credible intervals.28 In contrast to the presentation of effect estimates and confidence or credible intervals, so far there is no established standard ranking should be used and, if so, which method is to be preferred.
Patient and public involvement
Patients and the general public were involved in the full health technology assessment according to IQWiG’s methods, which are described in its methods paper.14 After the publication of the preliminary protocol and preliminary report, patients and the general public were asked to comment on the protocol and report. The comments and changes resulting from the comments are discussed in detail in separate German language documents available on the IQWiG website.13 A major change from the preliminary to the final protocol was the adaption of the definitions for clinical remission and inclusion of the corresponding definitions for low disease activity according to the current guidelines of the leading rheumatoid arthritis societies.
Results
Description of studies
Of 118 studies meeting the inclusion criteria for the health technology assessment (fig 1), 45 investigated biological medicines/methotrexate in the treatment of patients with rheumatoid arthritis after methotrexate failure (supplement table 2). Most studies used the 1987 ACR criteria for rheumatoid arthritis, while some also used the ACR/EULAR 2010 criteria. Supplement tables 3 and 4 show the characteristics of these studies; these data were used to check the similarity assumption.
Evidence search and study selection. *Records excluded in screening of title, abstract, or full text (reasons for exclusion in title/abstract screening: violation of minimum inclusion criteria (that is, wrong indication or intervention), not studying people, or no secondary publication (n=709); reasons for exclusion in full text screening: wrong population (n=28), intervention (n=113), comparator (n=26) or study design (n=694); no full publication (n=96); wrong study duration (n=39) or language (n=5); and no patient relevant outcome (n=0)). †Infliximab/methotrexate versus placebo/methotrexate. ‡One study each for following comparisons: adalimumab/methotrexate versus etanercept/methotrexate, tocilizumab/methotrexate versus etanercept/methotrexate. §One study each for following comparisons: adalimumab/methotrexate versus placebo/methotrexate, tocilizumab/methotrexate versus placebo/methotrexate. ¶Etanercept/methotrexate versus other synthetic disease modifying antirheumatic drugs. HTA=health technology assessment; MTX=methotrexate
We saw insufficient similarity in three studies (fig 1), and similarity could not be assessed for two studies because of a lack of data on the relevant subpopulation. For 14 of the remaining 40 studies, new information on subpopulations of the complete study populations was necessary to meet the similarity requirement for the network meta-analysis. The main reasons were inclusion of patients pre-treated with biological medicines or inclusion of patients with varying concomitant drug treatment. New information for these subpopulations were available from the clinical study reports (two studies), and from reanalysis of individual patient data of the original study by the study sponsor (12 studies). Two of the 40 studies meeting the similarity assumption were not linked to the network owing to the lack of a common comparator. Data from 38 sufficiently similar studies were thus available for the network meta-analysis.
Study characteristics
In the 38 studies, the mean age in most of the study arms ranged between 50 and 60 years; 75-90% of patients were women. Disease duration was mostly between six and 12 years. Most studies included patient populations with severe rheumatoid arthritis and poor prognostic factors (eg, moderate to high disease activity or high swollen joint counts). The studies were initiated between 1997 and 2012.
During the assessment of similarity, uncertainties remained concerning disease severity, which was either unknown or moderate (four studies), as well as an unknown or considerable proportion of patients (5-20%) pre-treated with biological medicines (three studies). These uncertainties were investigated in two separate sensitivity analyses. For one study with missing information on disease severity and two studies on pre-treatment with biological medicines, we did an additional sensitivity analysis to investigate uncertainties caused by a lack of information. Supplement 2 provides details on the conduct and results of the sensitivity analyses (methods section and supplement tables 16-19).
The 38 studies investigated eight different biological medicines in combination with methotrexate. Only two studies reported a direct comparison of biological medicines (abatacept/methotrexate v adalimumab/methotrexate, and adalimumab/methotrexate v certolizumab pegol/methotrexate). All other studies compared a biological medicine/methotrexate with placebo/methotrexate: abatacept (n=6), adalimumab (n=8), anakinra (n=3), certolizumab pegol (n=4), etanercept (n=3), golimumab (n=3), infliximab (n=2), and tocilizumab (n=7).
Most randomised clinical trials (34 of 38) were double blinded and lasted six to 12 months; in contrast, both direct comparisons and one placebo controlled study lasted two and three years, respectively (supplement table 3).
Network characteristics
Of 38 studies included in the network meta-analysis, analyses based on the prioritised CDAI definitions of clinical remission (CDAI ≤2.8) or low disease activity (CDAI ≤10) were available for four studies, and were provided after reanalysis of individual patient data by the study sponsors for 30 studies. The following presentation of data are based on these reanalyses, where applicable. The remaining four studies without relevant data were thus excluded from the network meta-analysis for these outcomes.
The number of studies with available data per outcome ranged between 19 and 35 (supplement table 15). For all outcomes, most studies reported data after 24 or 30 weeks. For some outcomes, data were also (or only) available after 52 weeks. For all outcomes, data after 24 and 30 weeks were considered if available, because at these points fewer treatment adjustments or study discontinuations had occurred than later on in the studies (in the event of lack of efficacy, 12 of 38 studies recommended treatment switching and 26 of 38 studies recommended study discontinuation). If only data after 52 weeks were available, these were included in the network meta-analysis.
Because only two direct comparisons were available, the assessment of consistency was only feasible for the closed loops between abatacept/methotrexate, adalimumab/methotrexate, certolizumab pegol/methotrexate, and placebo/methotrexate. Supplement tables 16 and 17 show the results for the checks of the homogeneity and consistency assumptions. Overall, the networks were mainly built from placebo controlled studies (fig 2 and supplement figures 1 and 2). All statistically significant and clinically relevant effects (defined as standardised effects by Hedges’ g outside the (−0.2 to 0.2) interval for continuous data) from this network meta-analysis did not include data from direct comparisons. Therefore, the certainty of the conclusions on the advantages or disadvantages of one biological medicine compared with another was low for all outcomes.
Network plots of treatment comparisons for clinical remission, low disease activity, serious adverse events, infections, pain, and physical function in patients with rheumatoid arthritis. Network plots include the number of trials and the number of included patients for each comparison. CDAI=clinical disease activity index; HAQ-DI=health assessment questionnaire-disability index; MTX=methotrexate; n=number of included patients; VAS=visual analogue scale
For each study outcome, we did homogeneity and consistency analyses as well as sensitivity analyses investigating uncertainties for similarity assumptions to build the final network (supplement 2). Complete results from the network meta-analysis for each outcome are presented in supplement tables 20 to 23. Table 1 and table 2 summarise the statistically significant and clinically relevant effects observed in the network meta-analysis.
Statistically significant results representing added benefit of biological medicines/methotrexate combinations for rheumatoid arthritis, from network meta-analyses comparing combinations
Statistically significant results representing increased harm of biological medicine/methotrexate combinations for rheumatoid arthritis, from network meta-analyses comparing combinations
Comparative effects of biological medicines
Results for clinical remission (CDAI ≤2.8) were available from 34 studies covering eight biological medicines. The final network after investigating assumptions for network meta-analysis (fig 2) included eight biological medicines (34 studies with 10 869 patients). Statistically significant effect estimates from the network meta-analysis are presented in table 1. Statistically significant advantages were found for adalimumab, certolizumab pegol, and golimumab versus anakinra (table 1).
Results for low disease activity (CDAI ≤10) were available from 33 studies covering eight biological medicines. The final network after investigating assumptions for network meta-analysis (fig 2) included eight biological medicines (27 studies with 8198 patients). Statistically significant advantages were found for abatacept, adalimumab, infliximab, and tocilizumab versus anakinra (table 1).
Results for pain (measured via a visual analogue scale) were available from 30 studies covering seven biological medicines. The final network after investigating assumptions for network meta-analysis (fig 2) covered all biological medicines apart from certolizumab pegol/methotrexate and golimumab/methotrexate (12 studies with 3931 patients). Statistically significant and clinically relevant advantages (upper limit of 95% confidence interval of standardised treatment effect <−0.2 Hedges’ g) were found for abatacept and tocilizumab versus anakinra (table 1).
Results for physical function (measured via the Health Assessment Questionnaire31) were available from 33 studies covering eight biological medicines. The final network after investigating assumptions for network meta-analysis (fig 2) included eight biological medicines (29 studies with 9247 patients). No statistically significant and clinically relevant differences were seen between the biological medicines in the network (supplement table 21).
Results for serious adverse events were available from 34 studies covering eight biological medicines. The final network after investigating assumptions for network meta-analysis (fig 2) included eight biological medicines (31 studies with 9934 patients). A statistically significant disadvantage was found for certolizumab pegol versus abatacept, adalimumab, anakinra, etanercept, and infliximab (table 2).
Results for infections were available from 35 studies covering eight biological medicines. The final network after investigating assumptions for network meta-analysis (fig 2) included eight biological medicines (31 studies with 10 036 patients). A statistically significant disadvantage was found for certolizumab pegol versus abatacept, anakinra, etanercept, golimumab, and tocilizumab (table 2).
Other outcomes
The networks of further outcomes are presented in supplement figures 1 and 2. A statistically significant and clinically relevant advantage was found for golimumab versus anakinra with regard to health related quality of life (table 1). Statistically significant disadvantages were also found for anakinra and certolizumab pegol versus several other biological medicines with regard to discontinuation due to adverse events and serious infections, respectively; tocilizumab versus abatacept with regard to discontinuation due to adverse events; and golimumab and tocilizumab versus infliximab with regard to serious infections (table 2). We found no statistically significant differences for fatigue and mortality (supplement tables 22 and 23).
Uncertainty of effects
The network meta-analysis identified only a limited number of statistically significant and clinically relevant effects (table 1 and table 2). However, about 30% of the non-significant effects (74 of 238 effects) had wide 95% confidence intervals of risk ratios (including 0.5 and 2.0) or of mean differences (including 10% of the scale range; supplement tables 20 to 23). These wide confidence intervals underline the uncertainty of the results and could have been caused by the fact that the network meta-analysis included only a few direct comparisons and that several indirect comparisons were based on a limited number of studies.
On the other hand, wide 95% confidence intervals were observed in only five of 28 (18%) comparisons for low disease activity, in three of 28 (11%) for serious adverse events, and in none for infections. The results for these key outcomes were thus less uncertain.
Discussion
Principal findings
Overall, our network meta-analysis showed only few statistically significant differences in benefits and harms between the biological medicines included. Anakinra/methotrexate showed less benefit than any other biological medicine (except for etanercept/methotrexate) with regard to clinical remission or low disease activity, and certolizumab pegol/methotrexate showed more harm than any other biological medicine with regard to serious adverse events or infections. Furthermore, golimumab/methotrexate and tocilizumab/methotrexate showed more infections than infliximab/methotrexate.
Confidence intervals were very wide for some outcomes with low event numbers (eg, remission and mortality), potentially implying unidentified differences between biological medicines. However, wide confidence intervals were less prominent in the key outcomes of low disease activity, serious adverse events, and infections, which had higher event numbers. In our opinion, these results suggest that overall differences between biological medicines are indeed only minor.
In the past few decades, the treatment goal for rheumatoid arthritis has changed substantially, from slowing down progression to low disease activity and ultimately to clinical remission. Furthermore, the definition of clinical remission is now stricter. The remission criterion of the DAS 28 allowing for a higher number of swollen joints was replaced by more stringent criteria, of which the CDAI remission criterion seems to be the most useful tool for clinical decision making. In addition, compared with the DAS 28 definition or other response criteria such as the ACR50, the new definitions for low disease activity (such as CDAI ≤10) are also stricter, allowing for less disease burden than the older definitions. We therefore used the updated definitions in our network meta-analysis and asked study sponsors to reanalyse the study data accordingly.
In an era of treat-to-target strategies and novel therapeutic drugs such as biological medicines, clinical remission of rheumatoid arthritis remains the primary treatment goal, even after methotrexate failure. Except for anakinra, we saw no proof of differences in clinical remission between the other biological medicines. This result is reflected by the current rheumatoid arthritis guidelines of the ACR (2015) and the EULAR (2019). Except for anakinra, the lack of proven added benefit of one biological medicine combined with methotrexate over another means that all combinations are similarly suitable treatment choices. The choice of a specific biological medicine should therefore be based on relevant patient characteristics, including comorbidities (if possible) and patients’ preferences. In addition, the cost of treatment should be considered, and becomes even more important as an increasing number of biosimilars becomes available.9
This network meta-analysis found higher rates of serious adverse events and infections for certolizumab pegol/methotrexate than for all other biological medicines. In contrast, a recently published report from the British Society for Rheumatology Biologics Register for Rheumatoid Arthritis found a lower rate of serious infections for certolizumab pegol than for all other biological medicines (hazard ratio 0.75, 95% confidence interval 0.58 to 0.97).32 Both methods generating these results (that is, indirect comparisons via network meta-analyses on the one hand and adjusted analyses from observational data on the other) inherently decrease the certainty of conclusions. In the event of contradicting results, the open question can only be solved by data from a more reliable study design (that is, a randomised clinical trial). Given the large patient numbers required, preferably a large and simple (registry based) randomised trial3334 might be the solution. In the case of serious adverse events and infections in patients receiving certolizumab pegol, this study is even more important, because certolizumab pegol is one of the few biological medicines approved for pregnant women.
Strengths and limitations of this study
Ensuring the similarity and homogeneity of the studies for a network meta-analysis often is hampered by the fact that journal publications, which still represent the main data source for many systematic reviews, provide insufficient information on patient and study characteristics.3536 This problem was negligible for our network meta-analysis because we had access to clinical study reports for 40 of the 45 studies considered. These comprehensive documents, which include study protocols and complete data on all items collected in a study, are particularly helpful when a detailed description of studies and patients is required.
In this network meta-analysis, for the first time reanalyses of many studies were able to make relevant patient subpopulations from studies with broader inclusion criteria available for a network meta-analysis. We were therefore able to include 12 (of a total of 40) studies that would otherwise have been excluded from our study pool owing to a lack of similarity. This finding underlines the value of individual patient data for secondary analyses of studies and supports the call for broad availability of anonymised individual patient data to maximise clinical research findings.3738
To our knowledge, this network meta-analysis is the first to focus on similar, up-to-date outcome data on clinical remission and low disease activity (based on the CDAI). Even more recent analyses used the DAS 28 and ACR response criteria,173940 even though they are no longer recommended.9 The changes in definitions of clinical remission and low disease activity also resulted in different outcome measures in the studies included in our network meta-analysis. While newer studies used the definitions from ACR and EULAR,10 data analysed according to these definitions were not available in older studies, which would have prevented the inclusion of these important outcomes in our network meta-analysis for a large proportion of studies. The switch to the CDAI was only possible because study sponsors reanalysed the respective low disease activity and clinical remission outcomes from the individual patient data (low disease activity: 26 of 27 studies; remission: 30 of 34 studies).
Overall, the reanalysis based on individual patient data from primary studies is one of the major strengths of our network meta-analysis. A further strength is the identification and inclusion of studies that fulfilled the predefined criteria for similarity, as described and discussed below.
This network meta-analysis also had several limitations. For instance, one biological medicine and two Janus kinase inhibitors approved recently (sarilumab, tofacitinib, and baricitinib) were not included. However, the individual assessment of these new drugs by IQWiG also only found minor differences compared with established biological medicines in patients after methotrexate failure.4142434445 An update of the present network meta-analysis should nevertheless include these drugs.
Treatment strategies have changed over the past years. Nowadays, the diagnosis and treatment of rheumatoid arthritis start at a much earlier stage. In addition, the treat-to-target paradigm has resulted in the earlier escalation of treatment. Furthermore, our findings probably do not apply to patients with moderate rheumatoid arthritis, because the network meta-analysis largely included patients with high disease activity.
Comparison with other studies
Several network meta-analyses of biological medicines in rheumatoid arthritis are available, including two recent Cochrane reviews.173940 A major difference between these network meta-analyses and our analysis seems to be the approach for testing the assumptions for the network meta-analyses, specifically similarity and homogeneity. For instance, while we extensively analysed patient characteristics and treatments to identify similar studies before inclusion in the network meta-analysis, Singh et al17 did not report an assessment of similarity and included more heterogeneous study pools. Furthermore, they did not report tests for homogeneity in the pairwise meta-analyses included in the networks. Although they performed sensitivity analyses for several factors, overall it remains unclear to what extent the assumptions for a network meta-analysis were met in their analyses. Hazlewood et al3940 reported more detailed data on study and patient characteristics and reported tests for homogeneity in pairwise meta-analyses. However, like Singh et al, the authors did not report measures or quantitative results to assess or test the network meta-analysis assumptions (such as P values for heterogeneity tests or measures to estimate inconsistency). Thus, it also remains unclear to what extent the assumptions for network meta-analysis were met in their analyses.
Unanswered questions and future research
The PRISMA extension statement for reporting systematics reviews containing network meta-analyses includes a clear requirement (in the checklist) only with regard to the check of consistency.21 The information on the check of the similarity and homogeneity assumptions is less precise and, while the elaboration document of the statement suggests reporting information on study characteristics and heterogeneity tests, it does not seem to require that these assumptions are met as a prerequisite for a network meta-analysis. Since the network meta-analysis concept relies on the similarity and homogeneity assumptions,46 further discussion is needed on reporting requirements and on approaches to ensure that these assumptions are met.
Owing to a very low number of direct comparisons in our networks, we were not able to assess the consistency of direct and indirect effect estimates for most of the contrasts. Therefore, we could only derive conclusions of low certainty. This finding underlines the fact that a network meta-analysis cannot compensate a lack of direct comparisons; a sufficient number of direct comparisons is required to fully exploit the potential of this type of analysis.
The comprehensive information retrieval conducted for this network meta-analysis identified a large number of studies. However, the proportion of direct comparisons and long term studies was very low. Given the fact that several biological medicines have been available for about 20 years and that rheumatoid arthritis affects many patients, the dominance of short term placebo controlled studies indicates a failure of study planning in this treatment indication. This problem was already described several years ago and has obviously not been solved.47
Conclusions and policy implications
For patients with rheumatoid arthritis after methotrexate failure, our network meta-analysis using rigorous methods showed overall minor differences in benefits and harms between biological medicines in combination with methotrexate. However, the analysis was hampered by a lack of long term direct comparisons. The substantial information gain achieved by the reanalysis of individual patient data calls for the routine availability of individual patient data.
What is already known on this topic
Previous meta-analyses have shown that most biological medicines combined with methotrexate are more effective at controlling disease activity in rheumatoid arthritis than treatment with methotrexate alone
However, the comparative benefits and harms of the biological medicines are still under discussion
Previous network meta-analyses on this topic have included heterogeneous patient populations and older definitions of key outcomes such as clinical remission and low disease activity, making the results of these analyses inconclusive
What this study adds
Inclusion of data from clinical study reports and reanalyses from individual patient data has enabled the inclusion of similar patient populations and unknown results on current definitions of clinical remission and low disease activity and other endpoints in this network meta-analysis
This systematic review and network meta-analysis indicates minor differences between biological medicines for the treatment of rheumatoid arthritis, when combined with methotrexate after methotrexate failure
This information could be important for clinical practice, considering the changes in treatment strategies for rheumatoid arthritis and the increasing number of biosimilar biological medicines available
Acknowledgments
We thank Corinna Kiefer for supporting statistical analyses and Natalie McGauran for editorial support; and the study sponsors (supplement table 1) for providing reanalyses of individual patient data for the studies included in the network meta-analysis.
Footnotes
Contributors: BW, KB, KJ, CS, DK, and BR conceived and designed the study. KJ, KB, CS, KH, DK, BR, and BW analysed and interpreted the data. KJ, KB, BW, and CS drafted the first version of the manuscript. KJ, KB, and HH screened studies for inclusion and KJ, HH, PK, and MFK extracted data. CS and KH checked data extraction and performed statistical analyses. All authors approved the final draft of the manuscript. BW is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding: This work was supported by the Institute for Quality and Efficiency in Health Care. No external financial support was received. Eight of the 10 authors are IQWiG employees and (as the remaining two authors) were involved in the study design and in the collection, analysis, and interpretation of data as well as in the writing of the report and the decision to submit the article for publication.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Institute for Quality and Efficiency in Health Care for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not required.
Data sharing: Additional data are available in the full (German language) report on the IQWiG website.13
The manuscript’s guarantor (BW) affirms that the manuscript is an honest, accurate, and transparent account of the review being reported; that no important aspects have been omitted; and that any discrepancies from the original protocol have been explained.
Dissemination to participants and related patient and public communities: The results of our study have been made available to the patient representatives who are part of the decision making body for which the original health technology assessment report was prepared and to patients and the public by publication of the health technology assessment report on IQWiG’s website. An easily understandable summary will be made available to patients and the public on IQWiG’s patient information website (English version: https://www.informedhealth.org/).
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.