Modern medical research is based on the principle that all evidence must be empirical, i.e. that new knowledge only can be developed with the use of observations and experiments. Randomised trials and epidemiological studies, in the way we now design and analyse them, did not take place until after world war II. Medical publications consisted earlier mainly of descriptive case reports and subjective hypotheses or expert opinions. An important problem with such publications is, however, that the chance that the presented information is useful for future patients cannot be easily assessed.
It follows from the principle of empirical evidence that research reports must describe the limitations of what has been observed and that the uncertainty of the findings must be honestly presented to the reader. It is not sufficient to just describe observations and outcomes that support the author’s conclusions. Circumstances that raise questions about the validity of the conclusions must also be disclosed. The author needs to be impartial vis-à-vis the findings and objectively quantify their uncertainty. Statistics, known as the science of uncertainty (1), provides a scientific methodology for this.
The trends of the modern society have, however, a major tendency towards alarmism and exaggeration. This is a phenomenon that is related to the flow of information and disinformation on the internet and its effects on news medias, which as a consequence, now are fighting for their survival. The current relationship between the internet and alarmism is similar to the relationship between the portable transistor radio and the teen culture during the 1960s.
The effects of the internet have many different consequences, and the publication of scientific findings is clearly affected. Scientists face a working environment that is characterised as “publish-or-perish”. An impartiality requirement for assessing the uncertainty of scientific findings is not always credible. Statistics risks being used as a tool for exaggerating empirical support in order to improve the chances of getting manuscripts published, the key to professional fame, financial support, and career advancement.
The trust in science is of course threatened (2), and a severe reproducibility crisis has also been widely discussed in medical science (3). Several important research findings have not been possible to reproduce, and this represents a serious waste of research resources with consequences for us all (4).
To improve the situation we both need to develop a more effective system for peer review and to find a way to help readers become more critical in their interpretations of published results.
1. Evans MJ, Rosenthal JS. Probability and Statistics: The Science of Uncertainty. W.H.Freeman Co Ltd, United States, 2010.
2. The Guardian 20 June 2017. Exaggerations threaten public trust in science says leading statistician.Retrieved from https://www.theguardian.com/science/2017/jun/28/ exaggerations-threaten-public-trust-in-science-leading-statistician-david-spiegelhalter.
3. Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature 2012;483, 531.
4. Harris R. Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hopes, and Wastes Billions. Basic Books, United States, 2017.