Outcome Switching

Make my Trials!

A hot topic that is now being discussed in the scientific research is 'outcome switching'.  In layman’s language outcome switching means the authors of a study did not report something they set about to find they set out initially but included additional outcome without disclosing the result of the original findings, with no explanation for the change. The field is so new that there is no Wikipedia entry on this, yet!

The most discussed case in outcome switching is that of a clinical trial named ‘Study 329’. That study was sponsored by GlaxoSmithKline (GSK) on antidepressant paroxetine with tradename Paxil. The result that was published in 2001, claimed the demonstrated that the drug was well tolerated and effective as an antidepressant for kids. This way in which the result of this study has been published demonstrates what outcome switching is about. 

Study 329 of GSK set out to monitor the efficacy of Paxil as an antidepressant on eight specific parameters. On all these parameters the research showed that it had no significant impact; the drug was no better than the placebo sugar pill. The researchers then came up with additional 19 new measures. Just 4 of the 19 new parameters showed result. In the paper that was published the researchers presented the results on these four only without discussing the other components as if the study was set out to measure the impact of these four parameters only. So even though the pre-decided parameters showed negative outcomes these were not discussed in the publication and few additional parameters that suited the study got reported.

The above is not a rare incident. The Economist reported about a study published in BMC Medicine in 2015 which found that 31% of the clinical trials did not stick to their original parameters. The problem is beginning to receive academic attention. University of Oxford, has launched the COMPareProject. The  project aims to systematically check every trial published in the top five medical journals - the New England Journal of Medicine, the Journal of the American Medical Association, The Lancet, the Annals of Internal Medicine and British Medical Journal (BMJ). The finding so far are revealing. COMPare team has so far studied 67 trials (information in their site as on 3^rd April). They found 9 trials conducted as per original protocol questions. In other cases, they found that 300 outcomes were not reported and 357 new outcomes were added.

GSK’s study 329 was initiated in 1992 and got completed in 1998. 

Fortunately, there has been changes in the regulatory processes after this. All trials have to be registered before they begin and the specified outcomes have to be published on website clinicaltrial.gov or similar national sites.

The website, retractionwatch carries an interview with Ben Goldcare the project leader of COMPare. He explains that not all changes in the outcomes of clinical trials are for nefarious reasons. What is of concern according to him is that when every time outcomes are switched, that creates a culture of permissiveness that lets other people do the same to tweak the trial’s conclusions.

The motivation for outcome switching can be many. Could it be mere survivorship bias? Survivorship bias is the logical error of concentrating on the people or things that "survived" some process and inadvertently overlooking those that did not because of their lack of visibility. This can lead to false conclusions in several different ways. But in clinical trials this can be very costly. But the evidence above weighs otherwise.

But the price may have been paid by patients. GSK's Paxil has been prescribed to millions of children and young adults. By early 2000s its sale was nearly US$ 2 billion a year!

You may read more about it in the Vox article or in the aptly titled article in The Economist - For My Next Trick.