The unbearable pointlessness of research

In Sweden we have just endured a minor uproar over Superstar Surgeon, Paolo Macchiarinis fall from grace. Macchiarini has been recently convicted of falsifying test results and continuing to perform his new operations despite controversy. Six of the eight patients who received his synthetic trachea transplant died since undergoing the procedure. Alarmingly highly regarded medical journal The Lancet, published a paper by Macchiarini and colleagues in 2011.

This confirms what many researchers fear – much of what we publish is at best nonsense and lethal at its worst.

Not that any of my writing has had an enormous impact. It is possible that my friends wash their jeans slightly less often with a slightly positive impact on energy and water consumption. But I do – on some level – recognise the feeling of meaninglessness.

Writing a scientific article is an exercise in slowly numbing your darlings. Baring your ideas to anonymous reviewer, receiving ruthless feedback, reading up on reviewers’ favourite theories, citing said ‘anonymous’ reviewers in a flattering light, rewriting, resubmitting. After the third or fourth re-submission your once poignant truth is watered down to limp, nearly unrecognisable shadow.

Being a peer reviewer is not much better. Spurred on by a niggling feeling of one’s duty to contribute to the scientific project, you begrudgingly accept to review an article based on an abstract vaguely in your area. After skimming the article, you realise that to review it thoroughly you will need to read five more papers and see the authors data-set. You try your best but you are never sure if you understood exactly what the authors intend and if your feedback makes their findings clearer or easier to replicate.

Is there any evidence that our hallowed peer-reviewing process actually makes articles better?  I haven’t seen any proof – regardless I continue the farce. Publications are the academy’s currency. The more articles you have, the easier to find a job, secure a promotion, write successful grant applications…

As a result a huge volume of our nonsense makes its way into the pages of scientific journals. Since Sokal‘s famous mumbo-jumbo acceptance to Social Text in 1994, there has been an increase in the acceptance of hoax articles, even in credible publishers like Sage and Elsevier. One researcher, John Bohannon, received more than a 50% acceptance rate for 304 versions of the same nonsensical research in 2013. This against the background of the more sinister proliferation of ‘instant article software‘ where software helps your research and put together articles. This, ladies and gentlemen, is what forms modern knowledge.

We have known about the pointlessness since at least 2005 when John P. A. Ioannidis published his eye-opening Why Most Published Research Findings Are False. And it is now more pertinent than ever with the recent Open Science Collaboration, who replicated 100 experiments from papers published in 2008 in three high-ranking psychology journals. Less than one half of the studies gave significantly similar results to those originally reported. What we publish is rubbish.

Yes, discrediting is the easiest game in town.  I actually don’t think that these experiments undermine the entire scientific project. They show, rather, that self-critique has grown bolder within the academy, which can only mean an improvement in the quality of research. And an improvement in our collective body of knowledge. Because that is the point of research.

http://www.nature.com/news/robust-research-institutions-must-do-their-part-for-reproducibility-1.18259

This post was inspired by a seminar by Roland Paulsen who is presenting again at Lund Konsthall 6pm April 21st. See you there?