Intended for healthcare professionals

Feature

Complications: tracking down the data on oseltamivir

BMJ 2009; 339 doi: https://doi.org/10.1136/bmj.b5387 (Published 08 December 2009) Cite this as: BMJ 2009;339:b5387

This article has a correction. Please see:

  1. Deborah Cohen, BMJ
  1. dcohen{at}bmj.com

    A Cochrane group’s attempt to reproduce an analysis underpinning the use of oseltamivir in pandemic influenza hit a brick wall. Deborah Cohen retraces its steps

    It started this July with an inquiry from a Japanese paediatrician, Keiji Hayashi, to the Cochrane Collaboration about its 2008 review of the treatment of influenza with oseltamivir.

    “You described that oseltamivir 150 mg daily prevented lower respiratory tract complications,” commented Dr Hayashi. “However, we have found that this conclusion is based on the other review by Kaiser and colleagues1 and not on your own data analysis.”

    Hayashi suspected that Kaiser’s conclusion about complications was mainly determined by data from eight unpublished randomised controlled trials (box 1).

    Box 1: Hayashi’s criticism of previous review

    “We have some questions on the conclusion in your Oseltamivir review especially about the prevention of complication (sic). You described that Oseltamivir 150 mg daily prevented lower respiratory tract complications (OR 0.32, 95% CI 0.18 to 0.57) in abstract.

    However, we have found that this conclusion is based on the other review (Kaiser, 2003) and not on your own data analysis. The authors of the review were four employees of F. Hoffman-La Roche Ltd, one paid consultant to F. Hoffman-La Roche Ltd and Kaiser. We cannot find any raw data about this conclusion from your review.

    Kaiser’s review included 10 RCTs; two RCTs (Nicholson, 2000 and T Treanor, 2000) were published as articles in the peer-reviewed medical journal (JAMA and Lancet), but other 8 RCTs were proceedings of congress (5 RCTs), abstracts of the congress (one RCT) and meeting (one RCT) and data on file, Hoffmann-La Roche, Inc, Nutley, NJ (one RCT).

    [There was no significant difference in lower respiratory tract complication rates] between Oseltamivir and placebo, and their Odds Ratios (ORs) were 1.81. But ORs of other 8 RCTs were 4.37.

    We strongly suppose that the reviewer’s conclusion about the complications was mainly determined by these 8 RCTs, we should appraise the 8 trials rigidly. Without this process it is difficult to conclude that Oseltamivir can prevent lower respiratory tract complications.”

    Hayashi asked the Cochrane group to analyse the data. But the Cochrane group’s attempt to reconstruct the Kaiser analysis, and a joint investigation between the BMJ and Channel 4 News, exposed a complex interplay between politics, public health planning, availability of trial data, publishing, and drug regulation. It also led the Cochrane Collaboration to question both its own methodology and the scientific rationale for using oseltamivir to prevent secondary complications in healthy adults—a key strategy in the Department of Health’s pandemic preparedness plan. The investigation also led to a declaration by Roche that it would make all study summaries of oseltamivir, including key data, available from 7 December (for Roche’s point by point response to questions put by BMJ and Channel 4, see bmj.com2).

    Oseltamivir (brand name Tamiflu) is a multi-billion dollar drug that Roche claimed reduced hospital admissions by 61% (Tamiflu media briefing, 7 Sept 2009); secondary complications (including bronchitis, pneumonia, and sinusitis) by 67% in otherwise healthy individuals3; and lower respiratory tract infections requiring antibiotics by 55% (Tamiflu media briefing, 7 Sept 2009). These statements, Roche said, were based on the conclusions of the Kaiser paper.

    At the start of September, Bill Burns, CEO of Roche, told a global audience via CNBC:

    “What Tamiflu can do is actually reduce the hospitalizations by more than 60% which is really important if we are in the midst of a major pandemic and it also shortens the duration and severity of the flu and the side effects are no more than one experiences with placebo so a lot of the side effects we do get are from influenza itself. So I do think there is a real role for this product and that is why the WHO have now been recommending this product to governments for a number of years.”

    Roche quoted the conclusions of the Kaiser review for the use of oseltamivir in pandemic influenza and in seasonal influenza repeatedly. Along with oseltamivir’s effects on duration of symptoms and infectivity, its apparent effect in reducing hospital admissions and complications was key to the Department of Health’s stockpiling of over 30 million doses of oseltamivir.4 As the Department of Health document pointed out, without any evidence of the effectiveness of oseltamivir in a pandemic, policy was based on the data from seasonal influenza.

    Writing in the Journal of the Royal Society of Medicine in 2006, Sir Liam Donaldson said that the UK had taken the precaution of building up a stockpile of oseltamivir for use in a pandemic.

    “Anti-viral drugs hinder the replication of the influenza virus. In the context of seasonal flu, they can shorten illness by a day and importantly, halve hospitalizations. They are already offered to those at risk of more serious illness following infection with seasonal flu,” he said.5

    The Kaiser paper was also cited by Professor Fred Hayden when he recommended the stockpiling of oseltamivir. Hayden was a co-author of the Kaiser paper as well as a co-author on one of the trials it included.6 He currently advises WHO and the Department of Health and coordinates influenza activities at the Wellcome Trust. Previously a paid consultant to Roche (an interest he declared on his published papers), he gave up this position over four years ago.

    “Because of their proven therapeutic effects in reducing influenza lower respiratory complications, these agents [oseltamivir and zanamivir] are the preferred ones for treatment and would be the logical choice for stockpiling,” Hayden wrote in 2004, citing the Kaiser paper.7

    Although Treanor6 had shown an effect on complications in healthy people, the more dramatic effect shown in the Kaiser paper concerned Dr Hayashi. Being from Japan—which up until recently consumed 80% of the world’s supplies of oseltamivir—he had a particular interest in the drug. He was also aware of the possibility of rare but severe side effects in children as reported by Dr Rukuro Hama, former chairman of the Japan Institute of Pharmacovigilance.8

    To Hayashi the risk-benefit profile of the drug was crucial. “Even if there are the rare fatal side effects of Tamiflu, if it prevents more complications and death, it is very valuable,” he told the BMJ.

    So he set about trying to understand how the information on complications had been analysed, but found it difficult. Much of the data were unpublished—hence, to his mind, unverifiable. The methods and data had not been peer reviewed.

    “It is difficult to evaluate the data included from Kaiser 2003,” he said. “Because there [are] no individual RCT data, I could not compare the individual data [from the] eight RCTs [that] were not available.”

    He was also concerned that four authors of the review were employees of F Hoffman-La Roche Ltd; one—Hayden—was a paid consultant to the company at the time of the Kaiser analysis; and only one, Professor Kaiser, had no financial relationship to Roche.

    Two weeks later, on 28 July, but unconnected to Hayashi’s comment, the UK’s NHS Research and Development and the Australian National Health and Medical Research Council commissioned an updated review of the use of oseltamivir for the treatment of influenza in healthy adults from the Cochrane Collaboration—an independent group of academics.

    In 2005, the Cochrane group had produced a substantial update of their review on oseltamivir. The group concluded that the drug shortened the duration of influenza by just under a day, reduced viral transmission, and reduced the likelihood of secondary complications, such as lower respiratory tract infections. A version was published in the Lancet in 2006.9 A further update in 2008 had made no major change to the conclusions.

    Under Cochrane rules, Jefferson and his co-authors were obliged to respond to Hayashi’s comments within six months. They believed that the only way to do this with any authority would be to analyse the data included in the 2003 Kaiser analysis themselves.

    So Jefferson set about trying to get the data necessary to answer Hayashi’s comments by emailing the lead authors of the key randomised controlled trials included in the previous review—treatment trials in healthy adults—and of the Kaiser analysis (box 2).

    Box 2: Papers included in the 2005 Cochrane review9 about the treatment of influenza in healthy adults

    • Treanor JJ, Hayden FG, Vrooman PS, Barbarash R, Bettis R, Riff D, et al. Efficacy and safety of the oral neuraminidase inhibitor oseltamivir in treating acute influenza: a randomized controlled trial. JAMA 2000;283:1016-24.6

    • Nicholson KG, Aoki FY, Osterhaus AD, Trottier S, Carewicz O, Mercier CH, et al. Efficacy and safety of oseltamivir in treatment of acute influenza: a randomised controlled trial. Lancet 2000;355:1845-50.10

    • Li L, Cai B, Wang M, Zhu Y. A double-blind, randomized, placebo-controlled multicenter study of oseltamivir phosphate for treatment of influenza infection in China. Chin Med J 2003;116:44-8.11

    • Kashiwagi S, Kudoh S, Watanabe A, Yoshimura I. [Clinical efficacy and safety of the selective oral neuraminidase inhibitor oseltamivir in treating acute influenza—placebo-controlled double-blind multicenter phase III trial] Kansenshogaku Zasshi 2000;74:1044-61.12

    • Kaiser L, Wat C, Mills T, Mahoney P, Ward P, Hayden F. Impact of oseltamivir treatment on influenza-related lower respiratory tract complications and hospitalizations. Arch Intern Med 2003;163:1667-72.1

    On 10 August he emailed the corresponding author of the Kaiser analysis, Professor Hayden, sending him Hayashi’s comments and asking for “raw data.” On 14 August, Hayden replied: “I have searched but cannot find the original files related to this 2003 publication. Before and again after my 2+ years at WHO in Geneva, I was obliged to move offices at the university several times and downsize. The files appear to have been discarded,” he said. “If original data or unpublished study reports are required, they will likely need to come from Roche, the sponsor of these studies,” he added.

    Jefferson got a similar response from the study’s lead author, Professor Laurent Kaiser, who suggested he contact Roche directly.

    Not long after Jefferson had received the commission and replies from Hayden, he was contacted by a Channel 4 News science reporter, Tom Clarke, who was interested in taking a closer look at vaccines and oseltamivir, fuelled in part by the publication of a Cochrane review of the treatment and prophylaxis of influenza in children published in the BMJ.13 Jefferson told Clarke that he was about to update his review and explained that he would have to go to Roche to get the data, as Hayden and Kaiser had suggested. Clarke wondered if Channel 4 News would be able to get the data, so producer Philip Carter, offered to contact the Roche press office to help out. Carter explained to Pam Dann, a Roche press officer, that Kaiser and Hayden had suggested that Jefferson contact Roche, as they couldn’t locate the relevant files. Carter gave Roche Jefferson’s contact details as the key person on the Cochrane review on the 10 September—an approach which Roche subsequently told the BMJ is in conflict with accepted standard practice, since it emanated from a media organisation and “not from within the scientific community.” It was the last contact Carter had with Roche.

    The following day, Dr Michelle Rashford, medical director of Roche, contacted Jefferson to say she had passed his request to the clinical development team. Later that month, with no news from Roche, Jefferson contacted the lead and corresponding authors of the other randomised controlled trials included in the 2005 Cochrane review to see if they could help obtain the data. The responses were varied in their detail, but all those who replied all directed Jefferson back to Roche.

    When Rashford replied to Jefferson’s request on 1 October, she said they needed Jefferson to sign a confidentiality agreement before they extracted the necessary data.

    The draft agreement sent to Jefferson specifically stated that the intended purpose of disclosing the data to Jefferson was that of updating the Cochrane review on neuraminidase inhibitors. It stipulated that if Jefferson signed it, he could not mention the existence or terms of the agreement—meaning that he would not be able to divulge the fact that he had entered into a confidentiality agreement.

    It also stated, in Rashford’s words: “that you will not be able to publish the data in full, but my understanding is that this is not your intent?” (see web appendix of linked analysis article doi:10.1136/bmj.b5164 for a copy of the agreement).

    Jefferson wrote back asking for clarification. “Updating a Cochrane review means publishing the data that our conclusions are based on. We cannot make statements unsupported by facts. So does signature of the agreement mean that we can publish the data on healthy adults and draw conclusions?” he said.

    Jefferson told the BMJ that he believed that to enter into the agreement would contravene the basic principles for which the Cochrane reviews are known: honesty, transparency, and reproducibility.

    But Jefferson didn’t hear from Roche after sending this query, so he emailed again on 7 October explaining that he needed to answer Hayashi’s comment and that the deadline by which he’d asked for the data to be produced was that day.

    But then, in a change of tack, on 8 October Hanna Rind, a PR executive from Roche, replied:

    “Following discussions with our medical teams both in the UK and Basel, unfortunately we are unable to send you the data requested as a similar meta-analysis is currently commencing with which there are concerns your request may conflict. We have been approached by an independent expert influenza group and as part of their meta-analysis we have provided access to Roche’s study reports. I can confirm that these reports have also previously been provided to both the FDA and European Health Authorities.”

    In later correspondence to the BMJ, Roche explained that they “offered to provide the data under a confidentiality agreement, as is commonplace within the scientific community to ensure the responsible use of data.” They also added that they had given access to Roche’s oseltamivir clinical database to the UK’s Medical Research Council epidemiology research unit following such a confidentiality agreement. “Dr Jefferson was unwilling to enter into the agreement with Roche,” they told the BMJ.

    After being turned down, Jefferson queried with Roche the need for exclusivity and how the Cochrane’s review would conflict with another group’s. He explained that two or more separate research groups looking at a topic should lead to a strengthening of the conclusions about the effects of the interventions.

    “Given that we are seeking the data in order to update our existing Cochrane review, for which a protocol setting out the methods was published, I trust that you will agree that we are well placed to be able to do this updating in a transparent and unbiased way,” he wrote.

    On 21 October, six weeks after the original request from Jefferson, Dr David Reddy, Roche’s Global Pandemic Taskforce Leader and Dr James Smith, Roche’s International Medical Leader, sent Jefferson excerpts from the final study reports for each of the 10 studies that contributed to the Kaiser et al’s analysis, including the specific data tables concerning lower respiratory tract infections and antibiotic use.

    “Due to the fact that secondary complications are relatively infrequent during normal seasonal influenza outbreaks, each individual trial is unlikely to have enrolled sufficient patients to assess, in a robust manner, the impact of oseltamivir treatment on secondary complications, antibiotic use and hospitalizations,” they wrote.

    Using these data the Cochrane group attempted to reconstruct the unpublished datasets included in Kaiser, but they failed. They found the data incomplete, as Jefferson pointed out to Roche on 24 October.

    “We could not reconstruct the denominators of the Treanor abstract which we called 2000b (cited by Kaiser). The abstract appears to be made of several of these studies with no indication of which ones. Randomisation schedules, blinding, selection criteria are not in the excerpts and rarely are the population comorbidities (if any) described, so we cannot be absolutely sure we are dealing with healthy adults,” he wrote.

    On 28 October, Dr Smith told Jefferson that they were “working on the questions you raised and will send additional information next week.”

    However, the Cochrane team had hit its deadline for completing the review, and Jefferson told them that any further information would have to be included in the next update.

    So the Cochrane review published in this week’s BMJ14 based its conclusions about influenza complications on the two published trials included in the Kaiser review and two other trials.11 12

    “Paucity of good data has undermined previous findings for oseltamivir’s prevention of complications from influenza. Independent randomised trials to resolve these uncertainties are needed,” they concluded.

    When the BMJ queried with Roche the lack of access to the complete datasets in the Kaiser analysis with Roche they said that: “Roche sought to provide Dr Jefferson with the key clinical data in the form of the data tables concerning influenza-related complications and hospitalizations, to allow him to address his questions directly.”

    Roche also emphasised that the study reports in the Kaiser analysis had been submitted to health authorities—including the US Food and Drugs Administration (FDA) and European Medicines Agency(EMEA)—for their review.

    However, when the BMJ and Channel 4 News looked at the FDA and the EMEA recommendations on the effect of oseltamivir on secondary complications, two different pictures emerged.

    Although the EMEA did not respond to questions from the BMJ and Channel 4 News, its summary of product characteristics in 2009 concluded that oseltamivir reduced the risk of complications. It stated that for adults and adolescents: “The proportion of subjects who developed specified lower respiratory tract complications (mainly bronchitis) treated with antibiotics was reduced from 12.7 % (135/1063) in the placebo group to 8.6 % (116/1350) in the oseltamivir treated population (p = 0.0012)” (www.emea.europa.eu/humandocs/PDFs/EPAR/tamiflu/emea-combined-h402en.pdf).

    However, the FDA gave the opposite view. In a 2008 review of the information contained on the product label, it said: “Serious bacterial infections may begin with influenza-like symptoms or may coexist with or occur as complications during the course of influenza. TAMIFLU has not been shown to prevent such complications.”15

    When the BMJ and Channel 4 News asked the FDA about this difference in advice a spokesperson said: “The clinical trials in a variety of different populations (healthy adults and children, nursing home patients, adults and children with underlying cardiac/respiratory conditions) failed to demonstrate any significant difference in rates of hospitalization, complications, or mortality in patients receiving either Tamiflu or placebo, probably because these are relatively rare events. The clinical trials, although relatively large, were not powered to detect these clinical endpoints.”

    The fact that the trials were limited is a position that Professors Hayden and Kaiser now seem to acknowledge— despite the conclusions in the abstract of their paper claiming that “oseltamivir treatment of influenza illness reduces LRTCs [lower respiratory tract complications], antibiotic use and hospitalizations in both healthy and “at-risk” adults. In an email to the BMJ and Channel 4 they said:

    “While significant effects on lower respiratory events and all-cause hospitalizations was found for all subjects combined, the differences between placebo and oseltamivir groups did not reach significance in the subgroups analyzed. In summary, our analysis was limited in scope and in the strength of its conclusions by the small numbers of serious outcomes, specifically influenza-associated pneumonia and hospitalizations,” they wrote.

    However, in its response to the BMJ Roche said: “We would emphasise that the benefits of oseltamivir have been shown in randomised controlled clinical trials, not simply through observational studies.”

    When they sent Jefferson the data, Roche also sent a list of “real world” analyses—observational data—looking at the effectiveness of oseltamivir in patients with a clinical diagnosis of influenza, compared with a group of patients also with influenza but given no antiviral drug.

    The BMJ asked Professor Nick Freemantle and Mel Calvert to analyse these observational data.16 They concluded that there was limited evidence of clinical benefit from using oseltamivir in healthy adults to prevent serious complications.

    Freemantle told the BMJ, “We reviewed the nine observational studies that were sent to us by the manufacturer, Roche. We looked at the results very carefully. We were assuming because of where these studies came from that they are likely to be presenting a relatively optimistic case for oseltamivir—in fact the manufacturer practically told us that. But when we look at the studies we find in absolute terms the treatment effects of oseltamivir are very small in patients with no other existing health conditions, with no existing comorbidities or health challenges. In absolute terms, the benefits are vanishingly small. To put it in perspective, if we treated at least 100 possibly as many as 1000 people, we might prevent one case of pneumonia developing in otherwise healthy people with symptoms of influenza.”

    The BMJ put this point to Roche. They replied that although Freemantle’s point about small benefits may apply in seasonal influenza, where complications are relatively infrequent, the benefits “can change with increasing pathogenicity of either avian or pandemic viruses.”

    In a further letter to the BMJ from Roche 17 and in an email sent by Hayden, other observational studies conducted during the current pandemic rather than in seasonal influenza were cited as evidence to support the use of oseltamivir. One of these studies was also cited by Hayden at a recent press conference to promote the view that more liberal use of antivirals was needed to reduce admissions to intensive care units.18

    The BMJ asked Freemantle to comment on these observational studies in pandemic influenza. He said: “I expected to see differences in mortality and major outcomes that weren’t transient—I didn’t see those. There’s really no convincing evidence at all that anything very worthwhile is going on with oseltamivir in otherwise healthy people.”

    What does the evidence say about the policy?

    Freemantle saw very little evidence to support the widespread use of oseltamivir in the otherwise healthy population who are developing signs of influenza-like illness.

    It’s a view shared by WHO, who suggest that patients not considered to be at higher risk of developing severe or complicated illness and who have uncomplicated illness due to confirmed or strongly suspected influenza virus infection need not be treated with antivirals.

    Roche has told the BMJ it agrees with WHO’s position, but it said that “one third of patients with very serious illness admitted to intensive care units were previously healthy persons.” But despite this, governments have made oseltamivir widely available to otherwise healthy adults. Given limited financial resources, was this the right decision? Professor Freemantle doesn’t think so.

    “We have remarkably few resources in this country to spend on pharmaceuticals on health and it’s surprising to see such widespread use of oseltamivir.” By 5 October the UK government had spent £500m on drugs for swine flu, with future commitments taking the total to over £1bn. Confidentiality clauses in contracts with manufacturers don’t allow further breakdown (www.publications.parliament.uk/pa/ld200809/ldhansrd/text/91005w0012.htm).

    All in the public domain?

    What should have been a relatively straightforward exercise to review the evidence for the use of oseltamivir—as commissioned by the NHS and Australian governments—had turned into a series of negotiations with the manufacturers. It begged the question: why were so many of the trials still unpublished and not easily accessible?

    When the BMJ expressed concern to Roche that eight of 10 treatment trials were unpublished and therefore unverifiable by the general medical community, Roche said that the additional studies “provided little new information and would therefore be unlikely to be accepted for publication by most reputable journals.”

    They also added that now it is standard practice for Roche to publish all its clinical trial data, but this was not standard policy within Roche or elsewhere within the industry seven to 10 years ago. “At the time, it was considered that the studies that were published (2 abstracts and 2 full manuscripts) reflected accurately the benefits of the drug,” they said.

    So why wasn’t the biggest trial included in the Kaiser paper—bigger than both the published studies of Nicholson and colleagues and Treanor and colleagues put together, with the highest rate of influenza infectivity (73.5%)—published in full? And who conducted the trial?

    This randomised controlled trial of 1447 adults and adolescents aged 13-80 was conducted between December 1998 and February 1999 in the United States.19 In the Kaiser analysis and in company documents sent to Jefferson, its abstract credits Professor Treanor as the sole author and is given the study number M76001.

    In an email exchange between Hayden and James Smith of Roche that was sent to the BMJ, Hayden commented that “it was unfortunate that this study has not been published subsequently.”

    So the BMJ went directly to Professor Treanor to ask why he had not published the study. He told the BMJ that as far as he could remember, the trial published in JAMA was the only large study of oseltamivir he had ever participated in.

    “My recollection (which is dim) of this was that it was probably a poster, and from the title, was probably a rehash of data from the original trial, but going into more detail about the rates of minor complications, like bronchitis,” he said.

    From his records the only other trial he participated in was a study to look more closely at the impact of the time between onset of symptoms and onset of therapy on the therapeutic response, but he only enrolled 16 patients in that study.

    “I am pretty sure that data were ultimately published by Fred Aoki (Aoki FY, Macleod MD, Paggiaro P, et al. Early administration of oral oseltamivir increases the benefits of influenza treatment. J. Antimicrob. Chemother. 2003;51:123-129.). I don’t have anything in my files of old studies that’s identified as M76001, but I didn’t always include the sponsor’s study number in the files that I keep,” he said.

    “I’m not an author on that paper [Aoki et al] because other than enrolling a few subjects we didn’t really have much to do with it,” he added.

    Nevertheless, his name was attached to the trial in the Kaiser study and on further analysis the paper published by Professor Aoki did not correspond to trial M76001.

    Channel 4 News put it to Roche that Professor Treanor said that he didn’t actually participate in study M76001 and doesn’t remember presenting it a meeting in 2000.

    Dr David Reddy, Roche’s Global Pandemic Taskforce leader, said: “It’s not infrequent that you may have somebody who authors but they don’t actually present it at a conference, it depends upon their availability.”

    He added: “I think what you’re doing is you’re confusing perhaps a conference publication discussion versus a peer reviewed publication. And the standards of a peer reviewed publication are that the material is far more in depth when the people put this together.”

    Who did what?

    The Treanor abstract wasn’t the only trial with questions over involvement of people credited to the trial. During the review further questions arose about who did what on the published papers.

    In the clinical study reports sent by Roche to Jefferson—and given to the BMJ—names appended to the trials were not credited on the published papers. Causing yet more confusion about the roles of people involved in the trial, a series of files showing Roche submissions to NICE in May 2000—leaked to the BMJ and Channel 4 News—did not show names that appeared on the published trials in JAMA and the Lancet (see web appendix). However, Professor Nicholson and Professor Treanor meet the criteria for authorship for their Lancet and JAMA papers according to the International Committee of Medical Journal Editor guidelines—namely, they both made substantial contributions to the study, were involved in drafting or reviewing the manuscript, and approved the submission.

    We asked the lead authors and Roche for the contributions of those listed on the various documents. Roche explained that employees from the company’s regulatory department made submissions to NICE and other regulatory authorities and were therefore named on the documents. Others were responsible for compiling pharmacology reports. But it remains unclear who ultimately takes responsibility for the trials and the data.

    That might explain why different names appeared on regulatory documents and published papers, but it does raise questions about responsibility. Ike Iheanacho, editor of Drug and Therapeutics Bulletin, suggests: “If you have someone who is lead author but is replaced in that position when the work is presented to another audience, then who is actually responsible for the work? If you have variations between who appears to be doing the work at different times it’s much more difficult to be confident that at any one stage you are talking to the people who really know the data and are prepared to answer for it.”

    The confusion about who did what was further highlighted when two former employees of a medical communications agency approached the BMJ. They had previously worked for Adis International—a communications company now known at the UK site by its parent company name, Wolters Kluwer. They told the BMJ that their company held the Roche account. Oseltamivir was just one of the drugs they worked on.

    Documents given to the BMJ and Channel 4 News showed that one of the medical writers was expected to “ghost write” and “edit original articles/editorials/letters for several drugs,” including oseltamivir. The job description also included devising abstracts, posters, and slides for conferences.

    “Tamiflu was a big account for Adis at the time. Ghost-written manuscripts such as these included the Nicholson 2000 (Lancet) oseltamivir treatment papers,” one of the former employees told the BMJ.

    While Roche has admitted that “medical writers were used to draft some of the above papers” and Nicholson said that Roche did employ a medical writer to draft the manuscript, they both argued that at the time of submission—before the 2003 Good Publication Practice Guidelines, produced with the help of the drug industry and recently updated 20—it was standard practice for unnamed medical writers to be used.

    Treanor denies that his JAMA paper was ghostwritten. “The manuscript in question was written by myself. I can state unequivocally that no ghost writer was involved, and I do not recall ever having contact with anyone from Adis regarding this manuscript or its content,” he said.

    The former employees at Adis said medical writers were under pressure regarding the content of the articles. They said that they liaised directly with Roche’s marketing department: “We were under pressure to get messages out. The Tamiflu accounts had a list of key messages that you had to get in. It was run by the marketing department and you were answerable to them. In the introduction for Tamiflu, I had to say what a big problem influenza is. I’d also have to come to the conclusion that Tamiflu was the answer,” they said.

    When asked about this by the BMJ, Nicholson said: “I understand that all Roche clinical trials are reviewed internally by a multidisciplinary team that includes a representative from marketing. However, the content of the Lancet paper was reviewed, revised, and finally approved by the authors who had complete independence. The publication contains no inappropriate messages—it is unquestionable that influenza is an important public health problem.”

    Although Roche admits that using “key messages” was common at the time, it points out that “the term ‘key messages’ is no longer used in the context of publications as it has been frequently misrepresented.” Roche denied that it had ever obliged or put pressure on writers to include in papers any particular wording or component of the content.

    “Content of any paper involving a medical writer would be developed in conjunction with the authors, and would be reviewed and approved by them before submission, and under their control at all times. We have records verifying that the authors had access to data and provided review and comments on the manuscripts,” Roche said.

    When Channel 4 News and the BMJ asked Treanor about his access to both summary and primary data both now and at the time, he said: “I did not perform an independent analysis of the primary data, which was not required or requested by JAMA at the time of submission, and I do not have access to the primary data, which I also never requested.”

    When asked a similar question, Nicholson said he did not recall seeing the primary data. He said that the statistical analysis had been conducted by Roche and he analysed the summary data.

    “I do not recall seeing individual patient data; however I could have done upon request. I certainly didn’t analyse the dataset (this was done by another co-author, Kinnersley, a Roche statistician, whose affiliation is clearly stated on the paper) and so used summary data tables generated by the statistician. Upon my request, Roche provided additional data for me to check during the preparation of the manuscript. I have confirmed with Roche that I have always had access to the primary data set upon request,” he said.

    At the time of submission to JAMA, all authors had to sign an agreement that stipulated: “I attest that, if requested by the editors, I will provide the data or will cooperate fully in obtaining and providing the data on which the manuscript is based for examination by the editors or their assignees,” confirmed Dr Cathy DeAngelis, editor in chief of the JAMA group of journals—although, she admits, it is unlikely that the data were ever requested.

    Treanor pointed out that he was unsure if anything an author signs on submission to a journal about the primary data is meant to last in perpetuity. “I didn’t save any of that kind of journal correspondence so who knows what I might have signed. If JAMA were to produce the agreement and ask for the primary data (which they haven’t) I would guess that Roche would provide it, but I suspect that there would be arguments about whether the agreement still applies so many years after submission, and would probably limit the data they provided to the minimum required to support the paper,” he said.

    Not only had Roche performed the statistics on the Nicholson and Treanor papers, according to Kaiser: “the statistical analysis and data base management were performed by the Roche team,” although both Hayden and Kaiser later said in a joint reply that they “oversaw” the statistical performance at Roche.

    Nevertheless, before contacting the BMJ in response to questions about the paper, Hayden contacted Roche to clarify the position.

    “The first issue relates to access to the data. Since neither I nor Laurent have remaining hardcopies of the original datasets, I would ask for written confirmation from you that Roche will provide these to us if requested by us, the editors of Archives of Internal Medicine, or their assignees,” he wrote. In the reply, Roche confirmed that it had the original datasets and was willing to provide data in response to legitimate requests.

    Roche commits to making data available

    As a result of the investigation by the BMJ and Channel 4 News Roche has now published (7 December 2009) on roche-trials.com the study summaries (including key data) relating to the Kaiser manuscript to “ensure transparency of process” and results and to “maintain public confidence.”

    The corresponding full study reports “will also be made available on a password-protected site within the coming days to physicians and scientists undertaking legitimate analyses,” Roche said.

    When Peter Doshi and Tom Jefferson of the Cochrane group checked out the information posted on the website of the 10 randomised clinical trials included in Kaiser on 7 December, they were unconvinced by what they saw.

    “The webpages contain tables of data on complications. These are the same tables that appeared in the report excerpts Roche provided to Jefferson in October. This means any claims of Kaiser 2003 regarding a reduction in risk of serious complications from influenza in healthy adults still cannot be verified,” Peter Doshi told the BMJ.

    “For example, in the largest of the 10 RCTs (M76001, which was never published), the new reports still leave unclear what proportion of the >1400 patients were otherwise healthy adults. Major questions also remain about the methods: Roche has provided insufficient detail of inclusion criteria and no exclusion criteria for these trials which could lead to inaccurate conclusions regarding the trials’ generalisability to clinical practice.”

    Notes

    Cite this as:BMJ 2009;339:b5387

    References