Much has been written about the lack of reproducibility of results claimed by even well-meaning, upright scientists. Notably, a 2005 PLoS paper (by Stanford health-research policy expert John Ioannidis, MD, DSci) with the unforgettable title, “Why Most Published Research Findings Are False”, has been viewed more than a million times.
Who knew that relief could come in the form of hordes of science-naive gamers?
The notion of crowdsourcing difficult scientific problems is no longer breaking news. A few years ago I wrote a story about Stanford biochemist Rhiju Das, PhD, who was using an interactive online videogame called EteRNA he'd co-invented to come up with potential structures for RNA molecules.
RNA is a wiggly wonder. Chemically similar to DNA but infinitely more flexible and mobile, RNA can and does perform all kinds of critical tasks within every living cell. Scientists are discovering more about RNA's once-undreamed of versatility on a steady basis. RNA may even have been around before DNA was, making it the precursor that gave rise to all life on our planet.
But EteRNA gamers need know nothing about RNA, or even about biology. They just need to be puzzle-solvers willing to learn and follow the rules of the game. Competing players' suggested structures for a given variety of RNA molecule are actually tested in Das's laboratory to see whether they, indeed, stably fold into the predicted structures.
More than 150,000 gamers have registered on EteRNA; at any given moment, there are about 40 active players plugging away at a solution. Several broadly similar games devoted to pursuing biological insights through crowdsourcing are also up and running.
Das and EteRNA's co-inventor, Adrien Treuille, PhD, (now at Carnegie Mellon University) think the gaming approach to biology offers some distinct - and to many scientists, perhaps unexpected - advantages over the more-traditional scientific method by which scientists solve problems: form a hypothesis, rigorously test it in your lab under controlled conditions, and keep it all to yourself until you at last submit your methods, data and conclusions to a journal for peer review and, if all goes well, publication.
In this "think piece" article in Trends in Biochemical Sciences, Treuille and Das write:
Despite an elaborate peer review system, issues such as data manipulation, lack of reproducibility, lack of predictive tests, and cherry-picking among numerous unreported data occur frequently and, in some fields, may be pervasive.
There is an inherent hint of bias, the authors note, in the notion of fitting one's data to a hypothesis: It's always tempting to report or emphasize only data that fits your hypothesis or, conversely, look at the data you've produced and then tailor the "hypothesis" accordingly (thereby presenting a "proof" that may never be independently and rigorously tested experimentally).
Das and Treuille argue that the "open laboratory" nature of online games prevents data manipulation, allows rapid tests of reproducibility, and "requires rigorous adherence to the scientific method: a nontrivial prediction or hypothesis must precede each experiment."
Das says, "It only recently hit us that EteRNA, despite being a game, is an unusually rigorous way to do science."
Previously: John Ioannidis discusses the popularity of his paper examining the reliability of scientific research, How a community of online gamers is changing basic biomedical research, Paramecia PacMan: Researchers create video games using living organisms and Mob science: Video game, EteRNA, lets amateurs advance RNA research
Photo by Radly J Phoenix
