Background In November 2017, the Food and Drug Administration (FDA) approved a version of a second-generation antipsychotic, aripiprazole, embedded with a sensor (Abilify MyCite).
Objective To systematically review the evidence supporting the FDA’s approval of digital aripiprazole and how that evidence was disseminated in the scientific literature and news reports.
Study selection Prospective, double-blind, randomised controlled trials (RCTs), non-randomised and non-comparative studies were included if they focused on the use of digital aripiprazole. All scientific publications citing the trials were included if written in English. For the news reports, all languages were included if an English translation was available, and all records that were published after FDA approval were included.
Findings In the primary evidence search, no RCT comparing digital aripiprazole with a non-digital formulation, other active comparators or placebo was found. Only three non-comparative uncontrolled cohorts were found. No study provided data on remission, quality of life or any efficacy outcome. Fourteen scientific papers were identified that cited the trials and 70 news stories met the inclusion criteria. Almost 80% (11/14) of the scientific papers and three-fourths (52/70) of the news stories conveyed an unsupported impression of benefit.
Conclusions Regulatory approval for this first-ever digital drug was based on weak evidence, and there was no evidence of better adherence with the digital version of aripiprazole compared with the non-digital version. The possibilities afforded by this technology make room for a new type of evergreening (ie, patenting of older drugs with a sensor as a ‘new invention’). Both the scientific literature and news reports conveyed an unsupported impression of benefit.
Trial registration number CRD42018089515.
- public health
- mental health
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Global spending on prescription medicines in 2018 was US$1.2 trillion and is predicted to exceed US$1.5 trillion by 2023,1 and avoidable healthcare costs due to non-adherence are estimated to be billions per year.2 3 There have been a number of recent technological advances (eg, electronic adherence monitors or EAMs—devices that record when medications have been opened) to more accurately measure adherence and more quickly intervene when needed.4 5 The latest technology is the use of a digital medicine system in which a drug is combined with an ingestible sensor that can transmit a signal when the drug–device combination is exposed to gastric acid in the stomach to allow real-time information about medication ingestion. It is hoped that this tracking will increase medication adherence and in turn result in improved health outcomes and be cost-saving (ie, result in decreased healthcare costs).
In November 2017, the Food and Drug Administration (FDA) approved a version of a second-generation antipsychotic, aripiprazole, embedded with a sensor (Abilify MyCite). Because this is the first time that a regulatory body has approved such a drug–device combination, this approval sets a precedent for how technology-enhanced products will be evaluated before marketing. Thus, a critical review of the evidence supporting this approval is needed. It is also important to examine how the clinical trial evidence for approval was represented in these news stories and reports.6 Indeed, recent research has documented the presence of ‘spin’ (distorted interpretation of trial evidence giving a greater impression of benefit than is warranted by the data) in both the scientific literature7–9 as well as in media reports.10–12 Imbalanced and distorted healthcare reporting can generate false hope and undermine the ability of healthcare consumers to make informed choices about their healthcare.10 13 14 A UK study on 556 media reports on drug treatments found that most articles focusing on the benefits of medicines were about new drug treatments, and few of these reports made any mention of the risks or side effects of the medicines.15
Given the money spent globally on aripiprazole (over US$7 billion the year before the patent expired in 201516), the documented need for an evidence-based approach to inform prescribing for chronic conditions17 and the medicolegal issues created by the use of digital drugs, regulatory approval should be based on strong clinical trial evidence and the scientific literature and news reports should accurately represent the clinical data. Thus, we designed a study with three main aims: (1) to review the clinical trial data that were used to support the FDA’s approval of Abilify MyCite; (2) to examine the presence of spin (defined more specifically below) in the published scientific literature; and (3) to examine the presence of spin in the news reports/stories.
We developed and followed a standard protocol and registered the protocol in the international prospective register of systematic reviews (PROSPERO) prior to conducting this study.
Review of clinical trials
We reviewed the data from randomised controlled trials (RCTs) that were submitted to the FDA for approval of digital aripiprazole. If these data were unavailable, we searched for non-comparative evidence of the chip’s effectiveness (including in non-randomised studies).
We included studies of patients with schizophrenia or manic and mixed episodes associated with bipolar I disorder or depression (as an add-on treatment), as aripiprazole has regulatory approval for these conditions. Prospective, double-blind RCTs were eligible if they focused on the use of digital aripiprazole compared with either aripiprazole (non-digital formulation) and/or any other active comparator (any other drug approved in the USA for the same indication) and/or placebo. In addition, non-randomised and non-comparative studies were included if they focused on the use of digital aripiprazole. For studies of acute episodes to symptomatic remission and relapses or new episodes for continuation or maintenance studies, we assessed rates of remission as a primary outcome. Secondary outcomes were quality of life, score on a symptom scale (eg, Positive and Negative Syndrome Scale (PANSS) for schizophrenia, Montgomery–Åsberg Depression Rating Scale (MADRS) for depression and so on), adherence, total withdrawals, withdrawals due to adverse events, serious adverse events, total adverse events and effectiveness of the digital ingestion tracking system (indicating that the chip is working). We did not limit included studies by date or language.
To identify eligible studies, we searched PubMed/Medline, Cochrane Library and Embase, including conference abstracts. On PubMed, the keywords used were ‘(abilify OR Aripiprazole) AND (digital OR sensor OR sensors OR biosens* OR wearable* OR ingestible* OR pharmaco-vigilance OR pharmacovigilance OR wireless)’ (search strategies in other databases are described in the online supplementary appendix). We also requested copies of unpublished studies from the US FDA and from the pharmaceutical companies Otsuka and Proteus (see online supplementary appendix 1 for a detailed description of the request sent via email and the email responses). Additionally, we searched the databases ClinicalTrials.gov and Current Controlled Trials for relevant trials. We contacted the authors of abstracts by email for further information and the study references. If no response was obtained to a first request, they were contacted a second time.
Two researchers (IAC and FN) independently, at both the initial stage and for duplicates, screened studies for inclusion, resolving disagreements by consensus or in consultation with a third reviewer (LC). Two review authors (IAC and FN) independently extracted the data using a data extraction sheet based on the Cochrane Handbook for Systematic Reviews of Interventions guidelines,18 extracting information on study characteristics, funding, financial conflicts of interest (FCOI) during the last 3 years, use of a ghost writer and the characteristics of the journal. We planned to assess the risk of bias for RCTs using the Cochrane Collaboration’s tool for assessing risk of bias19 and observational studies with a tool from the Joanna Briggs Institute.20 We planned to perform meta-analyses if trials were sufficiently similar to be combined.
Selection of citations in the scientific literature to assess for spin
To identify subsequent reporting of the initial research findings, one author (FN) searched the Web of Science to identify all publications written in English citing the trials.
Selection of news stories and press releases to assess for spin
With the assistance of a medical librarian, we searched the database NexisUni for all news stories and press releases about the approval of AbilifyMyCite (search dates: 1 January 2015 to 23 January 2018; publication type: industry trade press, web-based publications, newswires and press releases, newspapers, blogs, business opportunities, newsletters, magazines and journals, legal news, news transcripts, scientific materials, undefined, news, patent filings). The following search terms were used: (abilify OR Aripiprazole) AND (digital OR sensor OR sensors OR biosens* OR wearable* OR ingestible* OR pharmaco-vigilance OR pharmacovigilance OR wireless). All languages were included if an English translation was available, and we included all records that were published after the device was approved by the FDA (13 November 2017).
Classification of spin in the scientific literature and news stories
Congruent with previous research, we defined spin as ‘a specific way of reporting, intentional or not, to highlight that the beneficial effect of the experimental treatment [digital aripiprazole] in terms of efficacy or safety is greater than that shown by the results’ (ie, overstate efficacy and/or understate harm; see Yavchitz et al, 2016) (see online supplementary appendix 2 for a detailed description of how Yavchitz et al 21 22 and Haneef et al 23 24 defined and operationalised spin and how we used Haneef et al 23 24 to develop our specific questions). Two review authors independently extracted the data from the studies included. Disagreements were resolved by consensus or in consultation with a third reviewer (LC). Following Yavchitz et al 21 22 and Haneef et al 23 24 we used the questions noted below to assess the classification of spin in the scientific literature and in the news reports, modifying question 5 slightly to assess whether the authors of the scientific literature were supported by the manufacturers of the sensor (Proteus) or of aripiprazole (Otsuka).
Did the news article acknowledge that there is no evidence that monitoring is associated with increased adherence?
Did the news article acknowledge that there is scarce safety data?
Did the news article acknowledge that the digital version was not tested against an active non-digital comparator?
Did the news article give the impression of clinical benefit to patients for which there are currently no data (eg, consistent use of words/phrases such as ‘big advance’, ‘innovation in healthcare’ or unsupported claims of positive health outcomes)?
Was an expert cited and was the expert independent (ie, was not a company representative/employee)? (In our review of the scientific literature, we asked if the authors were supported by the manufacturers of the sensor (Proteus) or of aripiprazole (Otsuka).)
We used descriptive statistics for most of the data. For the features described above, numbers and percentages for qualitative outcomes were reported. Data are shared on the Open Science Framework (https://osf.io/auc9e/).
Primary evidence search (clinical trials and observational studies)
We identified 130 articles in the databases of published trials and 20 through contacts with the FDA, manufacturers and clinical trial registries. According to Otsuka and the FDA, there were no unpublished studies that were submitted to the FDA. A representative from Proteus stated that the company was not involved in any of the trials for Abilify MyCite. The medical reviews that describe all of the clinical trials supporting the FDA’s approval are available at https://www.accessdata.fda.gov/drugsatfda_docs/nda/2017/207202Orig1s000TOC.cfm. In summary, the FDA decision was based on two review cycles. In the first cycle, we found three open-label, single-arm studies in psychiatric patients (studies 316-13-204, 316-13-215, 316-14-220). In the second cycle (resubmission), we found an additional study in psychiatric patients (DC-001576). This study was not included in our review since it was not a clinical study but rather a 2-day ‘simulated’ study in which the participants did not ingest the drug but simply placed the tablets in a container to simulate ingestion.
After removing duplicates, 107 articles remained, 95 of which were excluded at the title and abstract stage. We obtained and screened the full text of the remaining 12 articles and excluded 9, mainly narrative reviews. The three studies identified in the FDA report were included and no additional study was found (figure 1).
We found no prospective, double-blind RCTs comparing digital aripiprazole with the non-digital formulation, other active comparators or placebo. The three included studies (one was unpublished) were non-comparative uncontrolled cohorts25 26 (see table 1 for study characteristics), involving 67 patients with schizophrenia, 49 psychiatric patients (22 with bipolar 1 disorder, 12 with major depressive disorder and 15 with schizophrenia) and 58 psychiatric patients (35 with bipolar disorder and 23 with major depressive disorder). Effects on adherence, total withdrawals, withdrawals due to adverse events, serious adverse events, total adverse events and effectiveness of the digital ingestion tracking system (indicating that the chip is working) are presented in table 1. Neither study provided data on remission, relapse or new episodes, nor on quality of life as measured on a symptom scale. An assessment of Clinical Global Impression was planned in study 316-13-204 but not reported in the FDA summary.
Examination of spin in citing articles in the scientific literature
Of the 14 papers in the scientific literature that cited the two included studies, 71% (10/14) did not acknowledge the lack of efficacy data from clinical trials, 93% (13/14) failed to report on both the scarcity of safety data and the fact that no comparator was used in clinical trials, and 79% (10/14) gave an unsupported impression of benefit. Regarding FCOI, 57% (8/14) of the papers had at least one author with a financial tie to either Otsuka or Proteus, and in 43% (6/14) the authors were employees of either Otsuka or Proteus (see online supplementary table 2 for an overview).
Examination of spin in the news stories/reports
We identified 861 records through the systematic search and two press releases through a search of the companies’ websites; 70 met the inclusion criteria (see figure 2). Of these, 57% (40/70) did not acknowledge the lack of efficacy data from RCTs, 93% (65/70) did not report on the scarcity of safety data, and no story reported on the absence of a non-digital comparator in clinical trials. Three-fourths (52/70) conveyed an impression of benefit without mentioning the lack of research to support that impression. Most of the news stories (77%, 54/70) cited an expert, and of those 39% (21/54) cited experts who had financial ties to either Otsuka or Proteus (see online supplementary table 2 for an overview).
Our review of the clinical trial data submitted to the FDA for the approval of Abilify MyCite reveals that the data submitted were limited to trials that simply assessed whether patients could use the product as intended. There was no evidence of superiority or non-inferiority27 compared with non-digital versions of aripiprazole, other active comparators or placebo, and scarce data on safety. In fact, in the FDA’s clinical review letter, it was noted that the one simulated trial (ie, one in which the participants did not ingest the drug but simply placed the tablets in a container to simulate ingestion), performed for resubmission, ‘provides no additional data regarding adherence or data regarding data transmission times. Considering these limitations, the most accurate statement regarding Abilify MyCite’s capabilities is that Abilify MyCite successfully tracks ingestion of aripiprazole with embedded sensor’ (emphasis added) (p11).28 As acknowledged in the FDA letter, the lack of a single comparative trial means there is no way to know whether digital aripiprazole improves treatment adherence, quality of life, psychiatric symptoms or remission. However, the risk management plan specifically asks for an ‘open label, longitudinal post marketing trial’ without requiring any comparison group. Given the weakness of the design and the lack of control group, no reliable postmarketing information on effectiveness will be added.
This finding adds to the concern previously noted for non-digital versions of this drug, namely the extension of its use to disorders for which there is very limited evidence of efficacy and overall utility.29 30 For example, the FDA approved non-digital aripiprazole’s use in in the maintenance of bipolar disorder based on a single trial, which had many design and methodological flaws.17 The researchers expressed strong concerns that ‘the uncritical acceptance of this trial may be diverting patients away from more effective treatments’.17
Patients with serious psychiatric illnesses often suffer from paranoia. An ingestible drug with a sensor brings surveillance to a new level, and the potential negative effects on this patient population merit careful consideration.31 The potential harm of the surveillance aspect of this technology to the therapeutic alliance and to patients has not been adequately assessed. Thus, it is reasonable to ask if there was a financial rather than a scientific impetus for choosing aripiprazole as the first-ever digital drug. The sales and patent status of aripiprazole are noteworthy. In 2014 aripiprazole was the best-selling drug in the USA, costing on average over US$800 for a month’s supply and generating over US$7.5 billion in sales from October 2013 through September 2014.16 However, after the patent expired in the USA, sales revenues dropped to US$600 million by 2015,32 which is when Otsuka and Proteus first submitted an application for market approval for the digital version.
Drug manufacturers have developed a number of strategies to extend market monopoly after a blockbuster drug (defined as over US$1 billion in yearly revenue) goes off-patent. These are known as ‘evergreening’ strategies, with highly questionable benefit to patients.33–36 Evergreening involves the patenting of a slight modification (eg, subtle changes to the medicine’s structure) of an existing drug as a new invention. For the manufacturer, the result is that their product is considered as a new chemical entity that qualifies for market exclusivity (ie, no generic version is available). The possibilities afforded by sensor-based technology make room for a new dimension of practice, an evergreening 2.0.37 That is, ‘digital evergreening’ may develop as a means whereby manufacturers can gain market exclusivity for a generically available medicine (such as aripiprazole) by combining it with a monitoring technology. The recent United Nations Development Programme’s guidance on patent applications for pharmaceutical products does not address the issue of such technology.38 The present study raises concerns about digital evergreening and approving a ‘novel’ drug/device that may not be either as efficacious or as safe but will cost significantly more; while the generic oral version of aripiprazole costs approximately US$20 per month, Abilify MyCite costs almost US$1700 for a month’s supply.39 The increased cost has significant implications for medication adherence. In 2015, it was estimated that almost 10% of adults in the USA were non-adherent due to cost of the medicine.40
Additionally, this study adds to the growing body of literature documenting the extent and effects of spin in the scientific literature and in the media reports.7 22 Indeed, the extent of spin found in the present study is likely controlled by industry; not only were authors of the published studies Otsuka employees, but the following disclosure was also made: ‘Editorial assistance was provided by the medical communications company C4 MedSolutions LLC (Yardley, PA, USA), a CHC Group company’, with Otsuka funding. Both studies were published in the psychiatric journal Neuropsychiatric Disease and Treatment, whose editor is described as an ‘independent pharma consultant [who] advises and consults worldwide to several pharmaceutical and venture capital organizations’. The unsubstantiated positive characterisation of this digital medication is a public health issue because this is the scientific literature that will guide prescribing practices and policy initiatives, and will likely be used to inform the development of new treatment guidelines.41
Also, previous research has consistently found that journalists typically fail to address the quality of the evidence and the assessment of harm over benefit,10 42 and our study shows that after two decades this pernicious problem persists. The fact that over one-third of the experts cited in the news stories had financial ties to the manufacturers of this new product highlights the importance of decoupling commercial messages from science by identifying independent medical experts to whom science journalists and health reporters can turn.43
As Haneef et al 23 have noted, despite advances in operationalising definitions of spin, any assessment of the misrepresentation of clinical trial data or results will inevitably involve some degree of subjectivity. Therefore, although two researchers evaluated all of the assessments of spin independently, this process cannot eliminate an element of subjectivity. Our study’s findings cannot provide information about how readers of both the scientific literature and the news reports were—or if they were—misled by the spin that we identified. Finally, the fact that we only studied one drug–device combination limits the generalisability of our results.
Our case study reveals that the approval of this digital drug for marketing in the USA was granted on very limited data. Both the scientific literature and the popular news reports conveyed an unsupported impression of benefit. As a result, the general public and healthcare professionals may be making medical decisions based on industry-friendly, but not necessarily scientifically accurate, information about the efficacy and safety of this new product. Also, if patients are incentivised to take the digital version (eg, by being offered lower copayments or by being offered outpatient treatment—rather than forced inpatient treatment), the line between incentivising and coercion will be blurred.31 We recommend that other regulatory bodies (eg, the European Medicines Agency) take note of the findings in the current study as well as the medicolegal issues that emerge with the use of digital drugs.
What is already known on this topic
In November 2017, the US Food and Drug Administration (FDA) approved digital aripiprazole, a version of a second-generation antipsychotic embedded with a sensor.
This first approval of a drug–device combination sets a precedent for how technology-enhanced products will be evaluated before marketing.
Evergreening is a strategy used by industry to effectively extend patent protection by making small changes to existing products, changes that have almost no added benefit to the patient.
What are the new findings
Our review of the evidence submitted to the FDA for approval of a digital version of aripiprazole shows that approval was based on weak evidence—no prospective, double-blind, randomised controlled trials comparing digital aripiprazole with non-digital formulations of aripiprazole or other active comparators or placebo were found.
This case example illustrates the ways in which sensor-based technology can facilitate a new type of evergreening (ie, patenting of older, off-patent drugs with a sensor as a new invention to regain or maintain market exclusivity).
There was clear evidence of ghost writing in the dissemination of the trial data in the scientific literature.
This finding adds to the concern about high-tech medical companies operating in a climate of ‘stealth-research’ (Ioannidis, 2015; Cristea et al, in press) where their products are sheltered from the scrutiny of other scientists.
How might it impact on clinical practice in the foreseeable future?
Our results suggest that the general public and healthcare professionals may be making medical decisions based on industry-friendly, but not necessarily scientifically accurate, information about the efficacy and safety of this new product.
The authors thank Rebecca Troeger, Justin M Karter and Jacqueline Hogan for their assistance in gathering and collating the news stories and for their helpful comments and feedback.
Contributors LC conceived of the study. LC and FN developed the design. LC, FN, IAC, BM and AFS contributed to data analysis and interpretation and review of the results. All authors discussed the results and contributed to writing the manuscript. LC took the lead in writing the initial draft, and FN wrote the draft of the primary evidence search and results. All authors contributed to writing subsequent revisions of the manuscript.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data are shared on the Open Science Framework (https://osf.io/auc9e/).
Correction notice This article has been corrected since it was published Online First. Second affiliation for Ioana Alina Cristea has been corrected.
Patient consent for publication Not required.