Invited Speaker: Dr Veronica Vieland: Reproducibility, Replication and Scientific Evidence
Dr. Veronica J. Vieland, PhD
Battelle Chair in Quantitative and Computational Biology Director;
Vice President for Computational Research, Battelle Center for Mathematical Medicine;
Professor of Pediatrics and Statistics,
The Research Institute at Nationwide Children’s Hospital & The Ohio State University
Click here for event poster.
Recently the Open Science Collaboration (OSC) published an impressively comprehensive evaluation of replicability of results in psychology reporting a high rate of non-replication in a carefully conducted set of studies. This paper received quite a bit of attention both in scientific circles and in the media, and has contributed to the currently popular sport of science bashing. Unfortunately, however, the paper – and the methodological framework in which it is embedded – is marred by deep conceptual problems. OSC perpetuates a common error: the conflation of statistical significance (embodied in the ubiquitous p-value) with scientific evidence. This narrows the sense of scientific reproducibility down to success in repeatedly obtaining small p-values. I’ll discuss what we can and cannot learn from studies such as OSC, and argue that their results teach us absolutely nothing about whether the evidence has gone up or down between the original and replication studies. Accumulating evidence to corroborate or correct previous findings is a core scientific activity, but replicating p-values has very little to do with this activity.