Stanford researcher John Ioannidis, MD, DSci, who is well-known for his critiques of much current scientific methodology, has now turned his sights on the National Institutes of Health. He and a colleague, Joshua Nicholson from Virginia Tech, have published an analysis (subscription required) in today's Nature questioning the way the organization funds research proposals.
Ionannidis and Nicholson argue that the peer-review process, in which groups called study sections review and rank research applications submitted by their colleagues, is inherently flawed and encourages "conformity, if not mediocrity."
The result? Only 40 percent of scientists with highly cited papers are listed as the principal investigators on NIH grants. That is, those scientists whose peers value their insights and research most highly in their field are often not receiving federal support for that work.
The article is a pretty rousing condemnation of the status quo. Ioannidis, who is the chief of the Stanford Prevention Research Center, elaborated to me:
Our analysis shows with large-scale evidence that the system is totally broken. The majority of the US authors of the most influential papers in medicine and life sciences in the last decade do not have NIH funding; their funding rate may even be less than the rate of the average applicant. Conversely, study section members are almost always funded (a corollary of their selection process by NIH), but their citation impact is typically modest, nothing exceptional. High-impact innovators and funded study section members are almost completely mutually exclusive groups.
Ioannidis went on to note that the average age of a researcher receiving his or her first independent funding award from the NIH is 44 years (47 for MDs):
An out-of-the-box innovator who waits patiently and complies with orthodoxy until age 47 before becoming independent represents an oxymoron. A truly innovative idea cannot be judged by peers: if it is truly innovative, no peer has any clue about it; if peers already know about it, it is not innovative.
Although the NIH has tried to address concerns about a lack of support for new ideas with specific award categories such as the Pioneer and New Innovator Awards, Ioannidis called the outcome of such efforts "a drop in the bucket:"
We believe that NIH should fund some exceptional scientists based on the objectively measurable citation impact of their previous work. Peer review is the way it is applied now encourages conformity and mediocrity and favors people who know how to network and play the petty games of academia, not those who have brilliant ideas.
Previously: Research shows small studies may overestimate the effects of many medical interventions, Animal studies: necessary, but often flawed, says Stanford's Ioannidis and Outing bias in scientific research
Photo by videocrab
