Happy hour: Peer review accountability

Peer review still remains the best available way for journals to review and accept papers, though this system has some limitations and problems. There have been some exciting changes though in how journals are viewing their reviewing process. Nature is beginning to debate about it, and PLoS now has a new, community peer review system, with PLoS ONE. All very cool trends.

But there was something else that the significant other and I ended up discussing about, and it was what we thought was a rather simple but required step. The present system of peer review is anonymous peer review, where you submit your paper to a journal, and then it is sent off to a couple of experts in your area, who review the paper. But neither you nor any one else in the world (apart from the editors and the reviewers) know who these people are. Though a majority of the reviewers take their job of reviewing papers very seriously (the only two papers I’ve reviewed thus far, I spent hours analyzing. That we can dismiss as early enthusiasm, but still), many do not. They end up doing a rather sloppy job of reviewing papers (usually due to reminders or phone calls from editors). They sometimes pass on their decision without reading all the details of the paper, sometimes basing it on the author reputations, or where the paper’s from (sad, but sometimes true). Or else, they base it on how “cool” the story seems to be (without looking out for appropriate controls in the experiments). These papers slip through the cracks, and get published, even in so called “premier” journals.

So, and idea we had a couple of nights ago was to make the reviewers and editor responsible for the publication of a paper a little more accountable. For this, an obvious solution could be to have anonymous peer review during the process of review, but after a paper has been accepted for publication, the reviewers names should be mentioned in print when the paper is published. In addition, it could be easily possible to include a paragraph at the end of each paper published, where the reviewer(s) can say what they liked about the paper, and what else they would have liked to see. That way, it seems to me that reviewers will strive to do a better job with reviews, since their reputations will (in a way) be at stake. This step should go a long way in ensuring fair but rigorous reviews of papers, with a little more responsibility for the reviewer to do a good job.

(Interestingly, I mentioned this to my advisor, who thought it wasn’t a bad idea. So perhaps it may have some support amongst academics after all (there’s an n of one here, I need two more people supporting this, and this idea becomes practically publishable in itself J)).

Update: More from PLoS One "9. Anonymity
Although reviewers may remain anonymous during the review process, we strongly urge them to relinquish this anonymity to promote open and transparent decision-making.", and "If published, papers will be accompanied by comments from the handling Editorial Board member and will be made available for community-based open peer review involving online annotation, discussion, and rating.". Read about it here. All more than excellent. In addition though, I would like to see the reviewers comments too in the published paper, and their names being made pubic, just like the handling Editorial Board member's comments. Adds incentive to review well.