Missing data – the silent problem that’s finally causing a noise

Systematic reviews of research give more reliable evidence of the benefits and risks of treatments, yet they are often hampered by missing data. Dr Rob joins others in calling on drug companies to publish all their data and reassures that this project will too.

Researchers carrying out a recent systematic review on Tamiflu weren’t able to get all the data they required for the research. Creative commons image by kanonn.

I recently wrote about the fact that some of my colleagues had come down with a cold, and that I was feeling rather lucky because I was fine. Well, last Friday I developed a headache, sore throat, runny nose, and spent the weekend at home feeling miserable.

I didn’t feel the need to go and see my GP and get some medicine, but if I had they would have used clinical guidelines about what to prescribe me. These guidelines are typically based on systematic reviews of the research evidence; a process that combines the results of separate studies into one big data analysis. Doing this gives us greater statistical confidence about the real benefits or risks of a drug than we can get by only looking at the results of individual studies. A single study might have produced a fluke (or chance) result and by repeating it we can be more confident about the true effects of a treatment.

Systematic reviews are performed by academics who painstakingly search databases of published research from around the world. Sometimes during this process it becomes apparent that studies have been carried out on the topic they are interested in, but that the results haven’t ever been fully published. This is concerning because this missing data might well change a conclusion as to whether the benefits outweigh the harms of a drug and only by having access to all the data can we understand the truth.

Exactly this situation has arisen with Tamiflu, one of the main drugs used to treat cases of flu in people at greater risk of serious complications. We know that Tamiflu reduces the length of time someone with flu has symptoms for by about 21 hours. However, we are much less certain about how well it prevents more serious complications of flu such as pneumonia or death. We aren’t confident about this because the researchers performing the most recent systematic review of Tamiflu weren’t able to get access to all the data from studies they know have been carried out in humans and looking at these questions.

A recent book on this issue by Ben Goldacre*, and a campaign led by The British Medical Journal have both made a strong case that the drug company involved should make this data available. I completely support this campaign, but change hasn’t happened as yet, and so doctors are currently left in a position of uncertainty about some of the most important potential benefits of the drug.  

There won’t be missing data issues for schools taking part in Decipher my Data. Most studies only tend to make data available at the end of the project, but we’re doing it live, giving your school the chance to look at the results as they appear. It means that we have a responsibility to be extremely cautious about how we interpret the results, because mistakes and errors occur in data that will need to be resolved and cleaned up along the way. But we’ll learn all about the strengths and weaknesses of the data together, and the conclusions we can draw from them, which I think is a very exciting way to carry out research.

*I should declare a conflict of interest here as Ben is a friend and we have talked about this issue many times.