jump to navigation

peerless August 24, 2007

Posted by ocmpoma in : open access , trackback

So, via Cognitive Daily, I found about BPR3 (developing an icon to indicate peer review), which is in and of itself an interesting concept. For me, though, a couple of posts on peer review were more interesting.

What is peer review? Should the practices at PLoS ONE be considered peer review? Can the editor be trusted? Can the reviewers be trusted?

I’m not sure if these are the right questions. Especially ‘What is peer review?’
‘What is the intent of peer review?’ sounds like a better question to me. If I’m not mistaken, the intent is for people who know what a work is supposed to be about to weigh in and decide if the work is valid within that field. It is supposed to serve as a self-check for the field, whether it be to weed out completely bogus work or simply correct oversights and errors.

If that is indeed the essence, if that’s what peer review is attempting to accomplish, then does it matter so much what the details are? Is there a minimum number of reviewers required to do this? Do they need certain qualifications? If one chases the rabbit of peer review too far down its burrow, pretty soon we’ll all be stuck with nothing be peer-reviewed as we come up with more and more loopholes and chinks in the armor that need to be protected against abuse and misuse.

The process as it currently stands seems to me to be great for a print-based world, where things move (relatively) slowly and information of the scientific-research-paper sort traveles by mail. We no longer live in that world, and I don’t see much reason to maintain its practices. Of course research needs to be carefully considered, its validity and impact weighed.

There’s no way two or three or even three dozen reviewers can do that work as well as the scholarly community as a whole can. With the ability of libraries to afford subscriptions to every journal dwindling and the ability of people to access information electronically rising, the way that information is being distributed is changing. A system where a few reviewers must decide what information to send out to everyone else is set up for the by-mail world. It was necessary, due to the costs and methods of distribution. Such distribution methods are no longer necessary, and the reason we need gatekeepers has changed. Rather then decide what information warrants dissemination, the main purpose now must be to decide what information warrants filtration, expulsion, and what should be spotlit.

Instead of trying to defend keeping control over publication of research in the hands of a limited number of people, we should be trying to find ways to expand the communities’ ability to broaden how review is conducted by authors’ peers and maximize the number of people who have a say in the validity of research.

What is peer review? It’s review of a work with an eye to validity by the authors’ peers. Editing, manuscript review, refereeing, publication, rebuttal, and citation are all forms of review. I see what PLoS is doing as furthering movement to where we need to go — because the more review we get (both pre- and post- publication), the more robust our science will be and the better our progress towards knowledge.

Is it perfect? Hardly. Is there room for error? Of course there is — but nothing is foolproof. Transparency and reproducability matter more in the review process, I think, just as they matter more in the experimentation process. I, for one, fully support PLoS’s innovations and say we need more, not less, of the same.

Tags for this article: ,

[?]

Comments»

no comments yet - be the first?