I had more fun than you might expect last week writing a press release about p-values, a perpetually misunderstood and mishandled statistical measure of significance that has infested biomedical research like ants at a picnic.
First of all, Stanford’s John Ioannidis, MD, DSc. — a professional gadfly to experimentalists, forever telling the scientific community what they are doing wrong — came out with a paper in today's JAMA, showing that more and more biomedical papers are reporting p-values. As I explain in a press release (which also offers a basic introduction to p-values), because p-values are so often misapplied, their increased use is actually bad news. More p values doesn’t indicate an improvement in the way biomedical research is conducted or the way data are analyzed, the researchers found.
P-values have been a hot topic for several years. See Regina Nuzzo’s classic article in Nature from two years ago. Or check out Christie Aschwanden’s hilarious video (and accompanying article) at FiveThirtyEight, of scientists here at Stanford at a METRICS conference, struggling to explain p-values in simple language. Faye Flam, of the Philadelphia Inquirer, wrote about the problem back in 2012
And just when our office was getting ready to send my press release out to news media, the American Statistical Association released a six-point statement formally condemning the way p-values are used. Monya Baker wrote about it for Nature, and Trevor Butterworth wrote it up for STATS.
ASA prefaced their statement by saying:
Let’s be clear. Nothing in the ASA statement is new. Statisticians and others have been sounding the alarm about these matters for decades, to little avail. We hoped that a statement from the world’s largest professional association of statisticians would open a fresh discussion and draw renewed and vigorous attention to changing the practice of science with regards to the use of statistical inference.
Along with the statement were a series of commentaries by experts. One of them, by Boston University epidemiologist Kenneth Rothman, DMD, DrPH, included this comment: "It is a safe bet that people have suffered or died because scientists (and editors, regulators, journalists and others) have used significance tests to interpret results, and have consequently failed to identify the most beneficial courses of action."
Maybe, with all this fresh attention, we’ll start seeing better statistical methods in the coming decade, with results that can actually be replicated.
Previously: On communicating science and uncertainty: A podcast with John Ioannidis, A conversation with John Ioannidis, “the superhero poised to save” medical research
Photo, which originally appeared in STANFORD Magazine, by Robyn Tworney
