You are here

  • Home
  • Online First
  • Retrieval barriers in individual participant data reviews with network meta-analysis

Article Text

Article menu

other Versions

Original research
Retrieval barriers in individual participant data reviews with network meta-analysis
  1. Areti Angeliki Veroniki1,2,
  2. Lesley A Stewart3,
  3. Susan P C Le1,
  4. Mike Clarke4,
  5. Andrea C Tricco1,5,
  6. Sharon E Straus1,6
  1. 1Knowledge Translation Program, St Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada
  2. 2Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
  3. 3Centre for Reviews and Dissemination, University of York, York, UK
  4. 4Queen’s University Belfast, Belfast, UK
  5. 5Epidemiology Division & Institute of Health Policy, Management, and Evaluation, Dalla Lana School of Public Health University of Toronto, Toronto, Ontario, Canada
  6. 6Department of Geriatric Medicine, University of Toronto, Toronto, Ontario, Canada
  1. Correspondence to Dr Areti Angeliki Veroniki, Knowledge Translation Program, St Michael's Hospital, Unity Health Toronto, Toronto, ON M5B 1T8, Canada; areti-angeliki.veroniki{at}unityhealth.to

Abstract

Objectives Individual participant data (IPD) from randomised controlled trials (RCTs) can be used in network meta-analysis (NMA) to underpin patient care and are the best analyses to support the development of guidelines about the use of healthcare interventions for a specific condition. However, barriers to IPD retrieval pose a major threat. The aim of this study was to present barriers we encountered during retrieval of IPD from RCTs in two published systematic reviews with IPD-NMA.

Methods We evaluated retrieval of IPD from RCTs for IPD-NMA in Alzheimer’s dementia and type 1 diabetes. We requested IPD from authors, industry sponsors and data repositories, and recorded IPD retrieval, reasons for IPD unavailability, and retrieval challenges.

Results In total, we identified 108 RCTs: 78 industry sponsored, 11 publicly sponsored and 19 with no funding information. After failing to obtain IPD from any trial authors, we requested it from industry sponsors. Seven of the 17 industry sponsors shared IPD for 12 950 participants (59%) through proprietary-specific data sharing platforms from 26 RCTs (33%). We found that lack of RCT identifiers (eg, National Clinical Trial number) and unclear data ownership were major challenges in IPD retrieval. Incomplete information in retrieved datasets was another important problem that led to exclusion of RCTs from the NMA. There were also practical challenges in obtaining IPD from or analysing it within platforms, and additional costs were incurred in accessing IPD this way.

Conclusions We found no clear evidence of retrieval bias (where IPD availability was linked to trial findings) in either IPD-NMA, but because retrieval bias could impact NMA findings, subsequent decision-making and guideline development, this should be considered when assessing risk of bias in IPD syntheses.

  • methods
  • systematic reviews as topic

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

https://doi.org/10.1136/bmjebm-2022-112024

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • Well-conducted individual participant data (IPD) meta-analyses are considered a ‘gold standard’ in evidence synthesis and can be used to inform patient care. Analysis of IPD can facilitate more tailored decision making and the development of clinical guidelines.

    • Failure to retrieve IPD could bias IPD-network meta-analysis (NMA) findings and decision making, if availability is linked to trial results, and can occur for an entire study dataset or a part of it (eg, missing information on a treatment group or a specific outcome or a type of participant).

    WHAT THIS STUDY ADDS

    • IPD availability in Alzheimer’s dementia and type 1 diabetes remained low; despite efforts for obtaining IPD through contacting original authors, IPD were available only from study sponsors or data sharing platforms.

    • Lack of randomised controlled trial (RCT) identifiers (eg, National Clinical Trial number) in published papers and unclear data ownership, particularly when an RCT involves multiple sponsors, are considerable challenges in IPD retrieval.

    • In our examples, we found no clear evidence of IPD retrieval bias.

    • There were practical and analytical challenges in analysing IPD from data sharing platforms, including additional costs.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • IPD were predominantly available for trials published after 2005, the year the International Committee of Medical Journal Editors mandated the registration of clinical trials. Researchers planning to undertake an IPD-NMA should consider the feasibility of accessing IPD from older trials and whether restricting to newer trials would cause any detriment to, or possibly improve their IPD-NMA.

    • Access to IPD can facilitate tailored decision making but retrieval bias could impact IPD-NMA findings, associated decision making and guideline development and should be considered when assessing risk of bias in IPD syntheses.

    Introduction

    Synthesis of individual participant data (IPD) from randomised controlled trials (RCTs) can provide best evidence for decision making, especially if brought together in a network meta-analysis (NMA). The use of IPD in NMA can strengthen evidence used for decision making, and is considered a ‘gold standard’ approach to evidence synthesis.1 2 Retrieval of IPD usually requires contacting RCT authors, but if authors cannot be located, as an alternative to contacting them, trial sponsors or data repositories might be contacted. NMAs bring together evidence from multiple intervention comparisons for a specific condition, allowing for intervention ranking according to their efficacy and are increasingly common3 4 with some very large examples.5

    Empirical evidence suggests that approximately 50% of RCTs in IPD meta-analyses are publicly sponsored (50%) and 10% are non-sponsored,6 and that the extent to which IPD can be retrieved may depend on RCT characteristics, such as funding source, study size, study quality and treatment effect.7 There is some evidence that IPD are more likely to be retrieved from large, multicentre, industry-sponsored RCTs compared with small, single-centre, publicly sponsored RCTs.8 9

    Reluctance to share IPD is a barrier,10 which can result in evidence syntheses having to be abandoned due to lack of access to IPD.11 Only 25% of the IPD meta-analyses published up to the year 2015 retrieved 100% of the eligible IPD, and 43% retrieved data on no more than 80% of participants.2 6 IPD retrieval has been found to be greatest in systematic reviews with meta-analyses that have funding support, asses non-pharmacological interventions, include fewer studies and have researchers who were coauthors of at least one eligible RCT.12

    A scoping review showed that 67% of IPD-NMAs published until the year of 2014 retrieved IPD through the establishment of a collaborative group that included researchers who contributed their study data.13 IPD-NMAs are more frequently industry sponsored compared with NMAs with aggregate data only or a combination of aggregate data and IPD. For example, using previous scoping reviews of IPD-NMAs and NMAs irrespective type of data, industry-sponsored studies were identified in 17 (46%) of the 37 IPD-NMAs and in 151 (13%) of the total 1144 NMAs with aggregate data only or in combination with IPD.13 14

    Well-conducted IPD-NMAs are considered the optimal evidence synthesis since they can produce more reliable results through high-quality statistical analysis.1 15 16 However, results from IPD-NMAs may be subject to retrieval bias (also known as availability bias). If IPD availability is linked to study results or characteristics, then IPD-NMA findings may be biased and not be representative of the whole evidence base.17 For example, if IPD were provided only for those RCTs that showed benefit of newer interventions, but no IPD were provided for those that favoured standard care, older interventions or those showing that the two groups were equivocal, then the IPD-NMA may overestimate the effectiveness of the newer interventions. In a network of interventions, retrieval bias from a single study can impact both direct and indirect intervention comparisons, since information flows via the network paths of evidence. Contribution of each study and direct comparison to the network estimate should be considered when retrieval bias is apparent.18 Greater understanding of these limitations and their potential impact on IPD-NMA findings (and thereby clinical decision making) is required and we have evaluated retrieval of IPD from industry-sponsored RCTs eligible for inclusion in two IPD-NMAs.

    Methods

    We sought IPD from authors and industry-sponsors of the RCTs included in two recent systematic reviews with IPD-NMAs. Two authors (ACT and LS) contacted original authors and another two authors (AAV and LS) contacted industry-sponsors of the included RCTs through email to request IPD. We contacted all industry sponsors reported in each RCT publication, some of whom indicated that their data were available through data sharing platforms, such as Vivli (https://vivli.org/), Clinical Study Data Request (CSDR)19 and Yale University Open Data Access (YODA).20 These consortium platforms provide an independent system for accessing anonymised IPD from multiple sponsors. At the time of our initial contact with the sponsors (January 2017),7 13 pharmaceutical companies were part of the CSDR consortium and YODA was a partnership between Yale University and the Medtronic, Johnson & Johnson, and SI-BONE companies. The of the Vivli platform was not contacted at that time since it was launched in 2018.21

    For the IPD requisition process, we followed Dillman’s methods,22 23 where we sent (1) an email to the corresponding author followed by the next-in-order author, as presented in each eligible RCT in case of non-working emails, to obtain IPD, (2) four email reminders to the authors (ie, at 2, 6, 10 and 14 weeks intervals after the first email), (3) a reminder letter in week 7 and (4) a telephone reminder in week 15. We offered coauthorship on our systematic review in case authors provided their anonymised IPD and met the International Committee of Medical Journal Editors (ICMJE) authorship criteria for the underlying systematic review.24 We recorded funding as reported in each RCT, and confirmed it with the original author. We contacted industry funders after navigating the data sharing process through their websites, online portals, emails or phone inquiries. We sent two follow-up reminders to the funders in case of no initial response.

    For each eligible study in each IPD-NMA, we extracted the year of publication, treatment effect (ie, mean difference (MD)) along with corresponding standard errors per treatment comparison for each primary outcome, study size and funding type (industry/mixed-sponsored, publicly sponsored, non-sponsored RCTs). For industry-sponsored RCTs, we also captured IPD availability. If IPD were unavailable, we recorded the sponsors’ reasoning for this (if provided), and summarised our challenges with retrieving IPD. We also summarised challenges once IPD were obtained from the sponsor or IPD repository using communication trackers and task logs.

    We captured the overall IPD retrieval rate and explored the distribution of IPD availability per year and per absolute z-score of each study-specific treatment effect. We used the absolute z-score values across all treatment effects, irrespective of treatment comparison, to capture the magnitude of the treatment effect in relation to its precision. We generated forest plots of the average treatment effects and their 95% confidence intervals according to IPD availability for each treatment comparison, separately. We used the random effects model and the metagen function of the meta package in R25 to present the adjusted IPD (using all requested (and provided) covariates) against the aggregate data as reported in the publications. More details about the provided covariates and the IPD analyses are presented in the relevant systematic reviews.9 26 We assessed IPD acquisition over publication years, using the Cox and Stuart trend test and the trend library in R.27 We summarised the total number of RCTs with and without IPD for each treatment comparison. We compared the study-specific MD and associated SE between RCTs with available IPD and RCTs without IPD, as well as IPD retrieval for trials published before and after 2005.28 We compared the study size across different funding RCT types in box plots and assessed whether study size was associated with IPD availability.

    Results

    We explored IPD retrieval in two clinical fields: Alzheimer’s dementia and type 1 diabetes (see network plots in online supplemental appendix 1). We requested IPD from the authors of 125 RCT publications (96 for Alzheimer’s dementia and 29 for type 1 diabetes mellitus, including unique RCTs and indentified companion reports), but none of the authors shared their IPD. We ended pursuit for a given study if its author did not respond after a fourth reminder. The time taken for authors to respond to our enquiry was between 0 and 117 days. Of the 125 RCT publications, 17 were companion reports and authors of both the main and companion reports were contacted to increase chances of IPD retrieval. Hence, we identified 108 unique RCTs; of these, 78 were funded by at least one industry sponsor, 11 were publicly sponsored and 19 did not report funding information. We then contacted industry sponsors only, as we were unable to locate contact information for public-sponsors (eg, government grant and university research funders). Seventeen industry sponsors funded these 78 RCTs. Five sponsors had partnered with three data sharing platforms (CSDR, Vivli and YODA). Seven (41%) industry sponsors agreed to share their IPD through proprietary sponsor-specific platforms (e, five sponsors through CSDR and YODA, and two sponsors through their specific platforms; one of the sponsors switched to the Vivli platform while we were navigating IPD). Overall, the 7 sponsors shared IPD for 26 of the 78 RCTs (33%) through proprietary-specific platforms.

    Supplemental material

    Alzheimer’s dementia IPD-NMA

    Overall 80 trials (21 138 participants) were eligible for inclusion in the Alzheimer’s dementia NMA. Of these, 55 were industry sponsored (40 single industry sponsor, 15 multiple industry sponsors), 9 were publicly sponsored and 16 did not report funding information. We requested data from the 15 industry sponsors of the 55 RCTs. Six sponsors shared IPD for 14 RCTs (8007 participants; figure 1), facilitated by three data sharing platforms (CSDR for 4 RCTs, YODA for 5 RCTs, Vivli for 2 RCTs (the latter 2 RCTs were initially provided through the AbbVie platform)) and 1 industry sponsor (Lundbeck) for three RCTs. Of the remaining nine sponsors, five did not respond to any of our multiple emails, so we decided to no longer pursue and four declined to provide data.

    Figure 1
    Figure 1

    Flow diagram for individual participant data (IPD) inclusion (A) refers to the Alzheimer’s dementia systematic review and (B) to the type 1 diabetes systematic review.

    Type 1 diabetes IPD-NMA

    Overall, 28 RCTs (7428 participants) were eligible for inclusion in the type 1 diabetes mellitus NMA, of which 23 were industry sponsored, 2 were publicly sponsored and 3 did not report funding information. Of the 23 industry-sponsored RCTs, 22 were funded by a single industry sponsor and one was jointly funded by two industry sponsors. One sponsor shared IPD for 12 RCTs (4943 participants; figure 1).

    Challenges with IPD and impact on analysis

    In total, we retrieved IPD for 26 RCTs through the following data repositories: CSDR (EISAI (1 RCT), GlaxoSmithKline (1 RCT), Novartis (2 RCTs)),19 YODA (Janssen (5 RCTs)),20 Lundbeck (3 RCTs), Novo Nordisk (12 RCTs) and Vivli (AbbVie (2 RCTs)). We failed to retrieve IPD from 52 industry-sponsored RCTs (67%): 41 RCTs in Alzheimer’s dementia and 11 RCTs in type 1 diabetes (online supplemental appendix 2). The reasons for not providing IPD varied. The most frequent reason was difficulty with study-identification (21 RCTs, 40%) because sponsors could not locate eligible studies without an RCT identifier (eg, National Clinical Trial (NCT) (https://www.clinicaltrials.gov/); figure 2). Locating IPD was particularly challenging for RCTs published before the ICMJE requirement for prospective clinical trial registrations in 2005, since many of these did not have an NCT number or the International Standard Randomised Controlled Trial Number (https://www.isrctn.com/). For IPD ownership for RCTs with multiple sponsors, the sponsors were unable to accurately confirm which sponsor had full ownership of the IPD (11 RCTs, 21%). Other reasons for not sharing IPD included: age of RCT and IPD being no longer available (seven RCTs, 13%), publication language of RCT (one RCT, 2%) and lack of response from a study’s principal investigator who held provision rights (one RCT, 2%). There was a lack of further clarification for non-sharing IPD for nine RCTs (17%), and lack of a response from sponsors in three RCTs (6%).

    Figure 2
    Figure 2

    Challenges encountered with IPD retrieval process through sponsors and data repositories for 52 industry-sponsored RCTs and once IPD were obtained in 26 industry-sponsored RCTs. Notes *Several challenges may apply to multiple randomised controlled trials (RCTs). In total, we included 78 industry-sponsored RCTs (23 in type 1 diabetes and 55 in Alzheimer’s dementia): 26 RCTs with retrieved IPD and 52 with no access to IPD. Of the 26 RCTs with IPD, we were able to include IPD for 24 RCTs in the IPD-NMA, and the remaining were included as aggregate data from the study publications. ∧ Free access to all the individual participant data (IPD) through the proprietary-specific remote-access platforms was provided for a limited time. However, a cost was associated with accessing IPD after this period for 14 RCTs. †Incomplete data included: data for placebo only or no information about a reported outcome (eg, only baseline values were available) or coded date of follow-up and first/last dates were unclear. This does not entail missing covariates or outcome/covariate participant values. At least one covariate was missing from provided IPD in 10 RCTs. ‡All platforms prohibited installation of new R packages; some R packages were not available, while others were older versions.

    We encountered several challenges after IPD were identified through the data repositories (figure 2). First, data sharing agreements were a prerequisite for all sponsors and data repositories providing access to IPD. Initial communication clarifying data sharing arrangements between all parties took between 0 and 24 days, and final approval of data sharing agreement ranged from 154 to 474 days after submitting the research proposal. A timeline for IPD retrieval across the sponsor process is presented in figure 3.7 Apart from the regular project costs associated with IPD acquisition, including administration, legal work, library staff and research staff, there were additional costs associated with accessing data through repositories. Some sponsors and data repositories required a license to access certain coding dictionaries. For example, some costs were associated with the WHO Drug Dictionary license to obtain access to the history of medications used for each participant. Two sponsors required this license as a prerequisite for IPD provision; the approximate cost for this was just under US$9000 per sponsor. The sponsors had initially used the dictionary to code their data; as such, we would have required the WHO Drug Dictionary to read and understand the IPD coding for the relevant studies. Second, some sponsors and data repositories permitted limited-time access to the IPD through their remote-access platforms (ranging from one to two years, on average). When this time frame passed, a cost was associated with retaining access to IPD in some cases (eg, US$25 per day for the AbbVie studies through Vivli). In other cases, we were able to extend access without a charge and data sharing agreement renewal. Third, shared IPD did not include full information collected in the RCT and reported in the associated publication (eg, only the placebo arm data were available, only baseline data were available with final outcome data missing, or date of follow-up was coded, making it impossible to determine first and last time-points). For example, we were able to include only 12 (6906 participants) of the 14 RCTs that provided data for our Alzheimer’s dementia IPD-NMA due to incomplete outcome data.9 Furthermore, at least one covariate that had been collected in the RCTs was missing from provided IPD in 10 RCTs (ie, six of 14 RCTs in Alzheimer’s dementia and four of 12 RCTs in type 1 diabetes). In such cases, we used aggregate data in our NMAs and performed two-stage IPD-NMA models to avoid excluding studies not providing or with incomplete IPD. Fourth, IPD were only available through proprietary sponsor-specific platforms per sponsor or data repository, which did not allow combination of all retrieved data in a single space to enable development of single model (eg, for a one-stage NMA). Fifth, relevant software was not available in all platforms or, older versions were provided, or we were not allowed to instal any new software or routines within these platforms (eg, the R package mice29 was missing or an older version of the lme430 R package was provided). Sixth, some sponsors (ie, AbbVie and Janssen through YODA) switched platforms while we were navigating the data, which caused additional delays in completing our IPD-NMA.

    Figure 3
    Figure 3

    Sponsor process timeline for individual participant data (IPD) retrieval. AbbVie switched to the Vivli platform while we were navigating IPD. Each sponsor shared available IPD for requested studies at a single timepoint. CSDR, Clinical Study Data Request; DSA, data sharing agreement; IPD, individual participant data; yodA, Yale University Open Data Access.

    Characteristics of retrieved IPD

    There was no clear trend in the success of retrieving IPD by year of publication (p=0.68; figure 4A). In the Alzheimer’s dementia dataset, IPD were only available for trials published after 2005, when ICMJE mandated the registration of clinical trials (online supplemental appendix 3). Availability was irrespective of the ratio of treatment effect over its precision (ie, z-score) or statistical significance (online supplemental appendix 4). We visually explored the possibility of retrieval bias across treatment effects, and found no clear evidence of important differences in comparisons with and without IPD in the two diseases (online supplemental appendix 5).

    Figure 4
    Figure 4

    (A) Stacked bar plot over year of publication and per individual participant data (IPD) availability; (B) box plots of study size per IPD availability.

    Funding source did not play an important role in the treatment effect statistical significance, but industry-sponsored RCTs enrolled a higher number of participants compared with publicly sponsored RCTs or those with unclear funding (online supplemental appendices 6 and 7). Sponsors of larger studies were more likely to share their IPD compared with smaller studies (figure 4B, online supplemental appendix 8). In Alzheimer’s dementia, 16 RCTs included more than 250 participants, 11 of which shared their IPD and were published after 2005. Similarly, in type 1 diabetes, 14 studies included more than 250 participants, 11 of which shared their IPD, with 7/11 RCTs published after 2005. Larger studies with a small treatment effect were more likely to provide IPD (online supplemental appendix 9).

    There was a tendency to obtain IPD in treatment comparisons informed by at least two RCTs (online supplemental appendix 10). Also, of the 13 treatment comparisons for which we retrieved IPD, 11 (85%) treatment comparisons were predominantly informed (>50%) by RCTs published after 2005.

    Discussion

    Access to IPD allowed us to overcome potential reporting bias in our two IPD-NMAs.9 26 In particular, we were able to include four RCTs for which data or entire outcomes were missing from the publications, and to assess for potential effect modifiers that were not reported in the original publications (eg, comorbidities, history of medications) where we investigated treatment-by-covariate interactions at the patient level. However, a large number of RCT and high proportion of IPD were not retrieved, although availability did not appear to be related to the individual study findings.

    Unavailable IPD can occur for part of a study dataset (eg, missing information on entire treatment group or a specific outcome) or for an entire study and can lead to the exclusion of the RCT from the analysis. Retrieval bias is potentially a major threat for any IPD-NMA, and could undermine its findings and use in decision making. Hence, we recommend that retrieval bias should be explored in future IPD-NMAs, particularly when IPD-NMA results are used to inform guidelines which will impact healthcare and research.16 Similarly, systematically missing covariates in retrieved IPD (with some covariates available in some RCTs but missing from the others, even if planned to be collected in the RCT protocol)31 should be considered in IPD-NMA. Retrieval bias can impact the network geometry, and hence network estimates, treatment ranking and validity of assumptions in NMA. The possibility of reporting bias in RCT publications included in NMAs should also be explored and there is an important role for trial registries in reducing the risk of reporting bias.32

    Despite efforts to obtain IPD through contacting original authors, study sponsors or data sharing platforms, data availability for RCTs in Alzheimer’s dementia and type 1 diabetes was low. A limitation for both our IPD-NMAs is that we waited up to 117 days for authors to reply to our request for IPD, and up to 474 days for a data sharing agreement to be finalised after submitting the research proposal. This waiting time frame for a systematic review with IPD-NMA highlights a need to consider that slow communication from sponsors and data repositories after the initial inquiry can mean that it takes 1 or 2 years in IPD-NMAs to retrieve and access IPD. Another limitation in our efforts to obtain IPD was that we were restricted to contacting only industry sponsors, due to the challenge of locating up-to-date contact information for public-sponsors (eg, government grant and university research funders). Hence, our findings mainly represent IPD retrieval from industry-sponsored RCTs, which may be different in publicly sponsored studies. Also, small-study effects was evident in the RCTs for Alzheimer’s dementia, and this may have affected IPD availability.9

    Locating the relevant RCTs and data ownership were major challenges in IPD retrieval. Importantly, for our Alzheimer’s dementia NMA, IPD were only available for trials published after 2005, the year of the ICMJE mandate, which increased the likelihood of an RCT being registered in a publicly accessible database. In total, we retrieved IPD for five RCTs (only for the type 1 diabetes NMA) published before 2005 and 21 RCTs published after 2005 (for both IPD-NMAs). Other factors may also play an important role in IPD retrieval, such as study size. Researchers planning to undertake an IPD-NMA should consider the feasibility of accessing IPD from older trials, especially when they are small in size, and whether restricting to the most recent trials would harm or benefit their IPD-NMA (eg, in applicability to current practice or in reducing the chance of retrieval (or availability) bias). Any decision to restrict the IPD-NMA to more recent trials should be made a priori, at the protocol stage, for transparency.

    Our study highlights the problems of data repositories for sharing trial data for evidence syntheses. Accessing IPD via a repository is time-consuming and can take more than a year after an initial inquiry with a sponsor or data repository.7 More challenges can arise if IPD from different RCTs are not all available through the same repository/platform, and if they are available at different time points. Additional costs may be incurred in retaining access to IPD beyond an initial ‘free of charge’ window, in order to enable access to all trials across all platforms at the same time. Obtaining access to data coding dictionaries and IPD may also incur a cost. The use of different proprietary-specific platforms, multiple costs of doing so, inability to analyse all data in the same space (and model), and lack of software in data repositories should be considered when planning an IPD-NMA. Researchers should weigh the benefits and challenges of retrieving and using IPD at the project design/protocol stage, before embarking on an IPD meta-analysis, including NMA.

    Data availability statement

    All data relevant to the study are included in the article or uploaded as supplementary information.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Not applicable.

    Acknowledgments

    We thank the following sponsors for sharing the data with us for our two IPD-NMAs: “This publication is based on research using data from data contributors, AbbVie, Inc, that has been made available through Vivli, Inc. Vivli has not contributed to or approved, and is not in any way responsible for, the contents of this publication.” “This study, carried out under YODA Project #2017-1671, used data obtained from the Yale University Open Data Access Project, which has an agreement with JANSSEN RESEARCH & DEVELOPMENT, L.L.C. The interpretation and reporting of research using this data are solely the responsibility of the authors and does not necessarily represent the official views of the Yale University Open Data Access Project or JANSSEN RESEARCH & DEVELOPMENT, L.L.C.” This publication used data obtained from Eisai, GlaxoSmithKline, and Novartis carried under www.ClinicalStudyDataRequest.com. This publication used data obtained from Novo Nordisk through their online platform, as well as from Lundbeck.

    References

    1. ↵
      1. Tierney J,
      2. Stewart L,
      3. Clarke M
      . Cochrane Handbook for Systematic Reviews of Interventions. In: Higgins J, Thomas J, Chandler J, et al., edseds. Cochrane Handbook for systematic reviews of interventions. 2nd Edition. Chichester, UK: John Wiley & Sons, 2019.
    2. ↵
      1. Simmonds M,
      2. Stewart G,
      3. Stewart L
      . A decade of individual participant data meta-analyses: a review of current practice. Contemp Clin Trials 2015;45:76–83.doi:10.1016/j.cct.2015.06.012pmid:http://www.ncbi.nlm.nih.gov/pubmed/26091948
      OpenUrlCrossRefPubMed
    3. ↵
      1. Lee AW
      . Use of network meta-analysis in systematic reviews: a survey of authors. Syst Rev 2016;5:8.doi:10.1186/s13643-015-0174-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/26791268
      OpenUrlPubMed
    4. ↵
      1. Zarin W,
      2. Veroniki AA,
      3. Nincic V, et al
      . Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review. BMC Med 2017;15:3.doi:10.1186/s12916-016-0764-6pmid:http://www.ncbi.nlm.nih.gov/pubmed/28052774
      OpenUrlCrossRefPubMed
    5. ↵
      1. Weibel S,
      2. Schaefer MS,
      3. Raj D, et al
      . Drugs for preventing postoperative nausea and vomiting in adults after general anaesthesia: an abridged Cochrane network meta-analysis. Anaesthesia 2021;76:962–73.doi:10.1111/anae.15295pmid:http://www.ncbi.nlm.nih.gov/pubmed/33170514
      OpenUrlCrossRefPubMed
    6. ↵
      1. Nevitt SJ,
      2. Marson AG,
      3. Davie B, et al
      . Exploring changes over time and characteristics associated with data retrieval across individual participant data meta-analyses: systematic review. BMJ 2017;357:j1390.doi:10.1136/bmj.j1390pmid:http://www.ncbi.nlm.nih.gov/pubmed/28381561
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Veroniki AA,
      2. Ashoor HM,
      3. Le SPC, et al
      . Retrieval of individual patient data depended on study characteristics: a randomized controlled trial. J Clin Epidemiol 2019;113:176–88.doi:10.1016/j.jclinepi.2019.05.031pmid:http://www.ncbi.nlm.nih.gov/pubmed/31153977
      OpenUrlCrossRefPubMed
    8. ↵
      1. Nevitt SJ,
      2. Sudell M,
      3. Weston J, et al
      . Antiepileptic drug monotherapy for epilepsy: a network meta-analysis of individual participant data. Cochrane Database Syst Rev 2017;12:Cd011412.doi:10.1002/14651858.CD011412.pub3pmid:http://www.ncbi.nlm.nih.gov/pubmed/29243813
      OpenUrlPubMed
    9. ↵
      1. Veroniki AA,
      2. Ashoor HM,
      3. Rios P, et al
      . Comparative safety and efficacy of cognitive enhancers for Alzheimer's dementia: a systematic review with individual patient data network meta-analysis. BMJ Open 2022;12:e053012.doi:10.1136/bmjopen-2021-053012pmid:http://www.ncbi.nlm.nih.gov/pubmed/35473731
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Villain B,
      2. Dechartres A,
      3. Boyer P, et al
      . Feasibility of individual patient data meta-analyses in orthopaedic surgery. BMC Med 2015;13:131.doi:10.1186/s12916-015-0376-6pmid:http://www.ncbi.nlm.nih.gov/pubmed/26040278
      OpenUrlPubMed
    11. ↵
      1. Jaspers GJ,
      2. Degraeuwe PLJ
      . A failed attempt to conduct an individual patient data meta-analysis. Syst Rev 2014;3:97.doi:10.1186/2046-4053-3-97pmid:http://www.ncbi.nlm.nih.gov/pubmed/25189273
      OpenUrlPubMed
    12. ↵
      1. Wu IXY,
      2. Xiao F,
      3. Wang H, et al
      . Trials number, funding support, and intervention type associated with IPDMA data retrieval: a cross-sectional study. J Clin Epidemiol 2021;130:59–68.doi:10.1016/j.jclinepi.2020.10.011pmid:http://www.ncbi.nlm.nih.gov/pubmed/33098991
      OpenUrlCrossRefPubMed
    13. ↵
      1. Veroniki AA,
      2. Straus SE,
      3. Soobiah C, et al
      . A scoping review of indirect comparison methods and applications using individual patient data. BMC Med Res Methodol 2016;16:47.doi:10.1186/s12874-016-0146-ypmid:http://www.ncbi.nlm.nih.gov/pubmed/27116943
      OpenUrlPubMed
    14. ↵
      1. Veroniki AA,
      2. Tsokani S,
      3. Zevgiti S, et al
      . Do reporting guidelines have an impact? Empirical assessment of changes in reporting before and after the PRISMA extension statement for network meta-analysis. Syst Rev 2021;10:246.doi:10.1186/s13643-021-01780-9pmid:http://www.ncbi.nlm.nih.gov/pubmed/34507621
      OpenUrlCrossRefPubMed
    15. ↵
      1. Riley RD,
      2. Debray TPA,
      3. Fisher D, et al
      . Individual participant data meta-analysis to examine interactions between treatment effect and participant-level covariates: statistical recommendations for conduct and planning. Stat Med 2020;39:2115–37.doi:10.1002/sim.8516pmid:http://www.ncbi.nlm.nih.gov/pubmed/32350891
      OpenUrlCrossRefPubMed
    16. ↵
      1. Tierney JF,
      2. Vale C,
      3. Riley R, et al
      . Individual participant data (IPD) meta-analyses of randomised controlled trials: guidance on their use. PLoS Med 2015;12:e1001855.doi:10.1371/journal.pmed.1001855pmid:http://www.ncbi.nlm.nih.gov/pubmed/26196287
      OpenUrlCrossRefPubMed
    17. ↵
      1. Stewart LA,
      2. Tierney JF
      . To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Eval Health Prof 2002;25:76–97.doi:10.1177/0163278702025001006pmid:http://www.ncbi.nlm.nih.gov/pubmed/11868447
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Papakonstantinou T,
      2. Nikolakopoulou A,
      3. Rücker G, et al
      . Estimating the contribution of studies in network meta-analysis: paths, flows and streams. F1000Res 2018;7:610.doi:10.12688/f1000research.14770.1pmid:http://www.ncbi.nlm.nih.gov/pubmed/30338058
      OpenUrlPubMed
    19. ↵
      1. So D,
      2. Knoppers BM
      . Ethics approval in applications for open-access clinical trial data: an analysis of researcher statements to clinicalstudydatarequest.com. PLoS One 2017;12:e0184491.doi:10.1371/journal.pone.0184491pmid:http://www.ncbi.nlm.nih.gov/pubmed/28886178
      OpenUrlPubMed
    20. ↵
      1. Krumholz HM,
      2. Waldstreicher J
      . The Yale Open Data Access (YODA) Project--A Mechanism for Data Sharing. N Engl J Med 2016;375:403–5.doi:10.1056/NEJMp1607342pmid:http://www.ncbi.nlm.nih.gov/pubmed/27518657
      OpenUrlCrossRefPubMed
    21. ↵
      1. Li R,
      2. Scott J,
      3. Rockhold F, et al
      . Moving data sharing forward: the launch of the Vivli platform. NAM Perspect 2018;8.doi:10.31478/201811b
    22. ↵
      1. Dillman DA
      . Mail and Internet Surveys: The Tailored Design Method. In: Update with new Internet, visual, and Mixed-Mode guide. 2nd ed. New York: Wiley, 2007.
    23. ↵
      1. Dillman DA
      . Internet, phone, mail, and Mixed-Mode surveys: the tailored design method. 4th ed. Wiley, 2014.
    24. ↵
      1. Nikolakopoulou A,
      2. Higgins JPT,
      3. Papakonstantinou T, et al
      . Cinema: an approach for assessing confidence in the results of a network meta-analysis. PLoS Med 2020;17:e1003082.doi:10.1371/journal.pmed.1003082pmid:http://www.ncbi.nlm.nih.gov/pubmed/32243458
      OpenUrlCrossRefPubMed
    25. ↵
      1. Balduzzi S,
      2. Rücker G,
      3. Schwarzer G
      . How to perform a meta-analysis with R: a practical tutorial. Evid Based Ment Health 2019;22:153–60.doi:10.1136/ebmental-2019-300117pmid:http://www.ncbi.nlm.nih.gov/pubmed/31563865
      OpenUrlAbstract/FREE Full Text
    26. ↵
      1. Veroniki AA,
      2. Seitidis G,
      3. Stewart L, et al
      . Comparative efficacy and complications of long-acting and intermediate-acting insulin regimens for adults with type 1 diabetes: an individual patient data network meta-analysis. BMJ Open 2022;12:e058034.doi:10.1136/bmjopen-2021-058034pmid:http://www.ncbi.nlm.nih.gov/pubmed/36332950
      OpenUrlAbstract/FREE Full Text
    27. ↵
      1. R Development Core Team
      . R: a language and environment for statistical computing. R foundation for statistical computing, Vienna, Austria. ISBN 3-900051-07-0, 2008. http://www.R-project.org
    28. ↵
      1. De Angelis CD,
      2. Drazen JM,
      3. Frizelle FA, et al
      . Is this clinical trial fully registered? A statement from the International Committee of medical Journal editors. Ann Intern Med 2005;143:146–8.doi:10.7326/0003-4819-143-2-200507190-00016pmid:http://www.ncbi.nlm.nih.gov/pubmed/16027458
      OpenUrlPubMedWeb of Science
    29. ↵
      1. van Buuren S,
      2. Groothuis-Oudshoorn K
      . mice : Multivariate Imputation by Chained Equations in R. J Stat Softw 2011;45:1–68.doi:10.18637/jss.v045.i03
      OpenUrlCrossRef
    30. ↵
      1. Bates D,
      2. Mächler M,
      3. Bolker B, et al
      . Fitting Linear Mixed-Effects Models Using lme4. J Stat Softw 2015;67:1–48.doi:10.18637/jss.v067.i01
      OpenUrlCrossRefPubMed
    31. ↵
      1. Fibrinogen Studies Collaboration,
      2. Jackson D,
      3. White I, et al
      . Systematically missing confounders in individual participant data meta-analysis of observational cohort studies. Stat Med 2009;28:1218–37.doi:10.1002/sim.3540pmid:http://www.ncbi.nlm.nih.gov/pubmed/19222087
      OpenUrlCrossRefPubMedWeb of Science
    32. ↵
      1. Wong EK,
      2. Lachance CC,
      3. Page MJ, et al
      . Selective reporting bias in randomised controlled trials from two network meta-analyses: comparison of clinical trial registrations and their respective publications. BMJ Open 2019;9:e031138.doi:10.1136/bmjopen-2019-031138pmid:http://www.ncbi.nlm.nih.gov/pubmed/31492792
      OpenUrlAbstract/FREE Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @AVeroniki

    • Contributors AAV, ACT and SES conceived and designed the study. AAV abstracted data, contacted sponsors, analysed data, interpreted results, and wrote a draft manuscript. AAV is the guarantor of the study. SPCL contacted authors, sponsors and data sharing repositories, and edited the manuscript. LAS, MC, ACT, and SES provided input into the design, interpreted results, and edited the manuscript. All authors read and approved the final manuscript.

    • Funding SES is funded by a Tier 1 Canada Research Chair in Knowledge Translation. ACT is funded by a Tier 2 Canada Research Chair in Knowledge Synthesis.

    • Competing interests AAV is on the editorial board for the journal but was not involved with the peer review process or decision to publish.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.