Does Tamiflu really work?

The British Medical Journal (BMJ) was dominated in 2009 by a cluster of articles on oseltamivir (Tamiflu). Between them the articles conclude that the evidence that oseltamivir reduces complications in otherwise healthy people with pandemic influenza is now uncertain and that we need a radical change in the rules on access to trial data.

The use of meta-analysis is governed by the Cochrane review protocol. Cochrane Reviews investigate the effects of interventions for prevention, treatment and rehabilitation in a healthcare setting. They are designed to facilitate the choices that doctors, patients, policy makers and others face in health care. Most Cochrane Reviews are based on randomized controlled trials, but other types of evidence may also be taken into account, if appropriate.

If the data collected in a review are of sufficient quality and similar enough, they are summarized statistically in a meta-analysis, which generally provides a better overall estimate of a clinical effect than the results from individual studies. Reviews aim to be relatively easy to understand for non-experts (although a certain amount of technical detail is always necessary). To achieve this, Cochrane Review Groups like to work with “consumers”, for example patients, who also contribute by pointing out issues that are important for people receiving certain interventions. Additionally, the Cochrane Library contains glossaries to explain technical terms.

Briefly, in updating their Cochrane review, published in late 2009, Tom Jefferson and colleagues failed to verify claims, based on an analysis of 10 drug company trials, that oseltamivir reduced the risk of complications in healthy adults with influenza. These claims have formed a key part of decisions to stockpile the drug and make it widely available.

Only after questions were put by the BMJ and Channel 4 News has the manufacturer Roche committed to making “full study reports” available on a password protected site. Some questions remain about who did what in the Roche trials, how patients were recruited, and why some neuropsychiatric adverse events were not reported. A response from Roche was published in the BMJ letters pages and their full point by point response is published online.

Should the BMJ be publishing the Cochrane review given that a more complete analysis of the evidence may be possible in the next few months? Yes, because Cochrane reviews are by their nature interim rather than definitive. They exist in the present tense, always to be superseded by the next update. They are based on the best information available to the reviewers at the time they complete their review. The Cochrane reviewers have told the BMJ that they will update their review to incorporate eight unpublished Roche trials when they are provided with individual patient data.

Where does this leave oseltamivir, on which governments around the world have spent billions of pounds? The papers in last year’s journal relate only to its use in healthy adults with influenza. But they say nothing about its use in patients judged to be at high risk of complications: pregnant women, children under 5, and those with underlying medical conditions; and uncertainty over its role in reducing complications in healthy adults still leaves it as a useful drug for reducing the duration of symptoms. However, as Peter Doshi points out on this outcome it has yet to be compared in head-to-head trials with nonsteroidal inflammatory drugs or paracetamol. And given the drug’s known side effects, the risk-benefit profile shifts considerably if we are talking only in terms of symptom relief.

We don’t know yet whether this episode will turn out to be a decisive battle or merely a skirmish in the fight for greater transparency in drug evaluation. But it is a legitimate scientific concern that data used to support important health policy strategies are held only by a commercial organization and have not been subject to full external scrutiny and review. It can’t be right that the public should have to rely on detective work by academics and journalists to patch together the evidence for such a widely prescribed drug. Individual patient data from all trials of drugs should be readily available for scientific scrutiny.