Tools like blogs and Twitter have fostered a public dialogue that allows anyone to critique, question or comment on scientific papers as soon as they are published. While that rapid communication holds the potential to vet methodologies and results faster, not all academicians are sure how to respond to such feedback. But a recent article (subscription required) in Nature explores the potential for aggregating and quantifying online conversations to improve the evaluation process for scientific studies:
For many researchers, the pace and tone of this online review can be intimidating - and can sometimes feel like an attack. How are authors supposed to respond to critiques coming from all directions? Should they even respond at all? Or should they confine their replies to the conventional, more deliberative realm of conferences and journals? "The speed of communication is ahead of the sheer time needed to think and get in the lab and work," said Felisa Wolfe-Simon, a postdoctoral fellow at the NASA Astrobiology Institute in Mountain View, California, and the lead author on the arsenic paper. Aptly enough, she circulated that comment as a tweet on Twitter, which is used by many scientists to call attention to longer articles and blog posts.
To bring some order to this chaos, it looks as though a new set of cultural norms will be needed, along with an online infrastructure to support them. The idea of open, online peer review is hardly new. Since Internet usage began to swell in the 1990s, enthusiasts have been arguing that online commenting could and should replace the traditional process of pre-publication peer review that journals carry out to decide whether a paper is worth publishing.
The article also explores the strengths and weaknesses of various open review models - and it is well worth taking the time to read.
Previously: Thinking about citations as links and Nature Publishing Group unveils online, open-access journal
