November 10, 2011

Opening Peer Review May Increase Accuracy

Peer review is crucial for evaluation of scientific work, but it could be much more effective if it were not anonymous, according to a study published in the Nov. 9 issue of the online journal PLoS ONE.

The researchers, led by Jeff Leek of Johns Hopkins Bloomberg School of Public Health in Baltimore, developed a theoretical model for the peer review system, as well as an online game that was used to test the model and provide further insight into the social dynamics involved. The results of both the model and the game show that reviewers are "rewarded" for their good reviewing work (i.e. their submission were more likely to be accepted) under non-anonymous open review but not traditional closed review; reviewers and authors are more likely to cooperate under open review; and cooperative peer reviewing behavior, including open review, leads to higher review accuracy.

According to the authors, "new forms of communication have opened the door for changes to the scientific peer review process. Our theoretical and experimental model system for peer review makes it possible to evaluate potential changes to the system".

Together, these results suggest that the current system of closed peer review does not foster optimal reviewer activity.


On the Net: