It was an hours-delayed flight and a $10 food voucher that did it. Annoyed, Anders Huitfeldt, PhD, a postdoctoral scholar at the Meta-Research Innovation Center at Stanford (METRICS), decided to spend his voucher on the most impractical item he could find in the airport. After some searching, he found it -- a minuscule spoon of high-end caviar.
As he recalled to me:
It was the smallest amount you could possibly have. And it was wonderful. I kind of got addicted.
The experience got Huitfeldt thinking, but not just about fancy caviar. As a researcher at METRICS, he is interested in making scientific research findings more accurate and reproducible. He explains:
I have had a long-standing interest in trying to understand why published research papers often fail to find the truth. It seems that often researchers are confused about what they are actually trying to do.
Huitfeldt explores the problem in the British Medical Journal's Christmas issue -- a lighthearted collection of articles that address important scientific concepts. His piece, "Is caviar a risk factor for being a millionaire", examines how the term 'risk factor' can have at least four distinct meanings in scientific literature. For example, does caviar consumption predict current wealth (is it diagnostic)? Or the likelihood of amassing wealth in the future (prognostic)? Does it actually play a role in the mechanism through which wealth accumulates (perhaps it is stealthily distributed in solid gold containers)? Or does the act of caviar consumption simply increase the probability of wealth (perhaps by bringing the consumer into close proximity of other wealthy movers and shakers with whom profitable deals can be struck)? As Huitfeldt explained:
The outcome of the study varies tremendously depending on what the researchers mean by 'risk factor'. Until they agree, it's not even clear how the question should be addressed. And this uncertainty becomes a serious impediment to processing information correctly to arrive at the scientific truth.
It's a fun read of a serious issue. For a more in-depth look at the many issues affecting research reproducibility, and the ways that Stanford scientists are attempting to address them, check out my Stanford Medicine magazine article "Can you Repeat That?" Or maybe just ask Huitfeldt to discuss further. I hear he likes caviar.
Previously: Clinical research's flaws highlighted by Stanford's John Ioannidis, Reproducible research: A hunt for the truth and On communicating science and uncertainty: A podcast with John Ioannidis
Photo by THOR
