Peers and Reviewing the Reviewers

Technocrat

Peers and Reviewing the Reviewers

femto

Tue Jan 30 02:54:00 -0800 2007

(in reply to Peers and Reviewing the Reviewers )

manage

Maybe Wikipedia has it right after all? ;-)

If there is no consistent way to predict who is a good peer reviewer, then a random selection of the population might do as good a job? It could be postulated that Wikipedia swings it by a typical reviewer being voluntary and that a person reviewing "for the love of it" has a better chance of doing a good job than a conscript.

Peers and Reviewing the Reviewers

David Leppik

Tue Jan 30 11:48:06 -0800 2007
manage

You read too much into this. They did find correlations, just weak ones, and not ones that they had expected. To quote:
The only significant predictors of quality were working in a university-operated hospital versus other teaching environment and relative youth (under ten years of experience after finishing training). Being on an editorial board and doing formal grant (study section) review were each predictors for only one of our two comparisons. However, the predictive power of all variables was weak.

That's not the same as claiming that a random person off the street would do just as good a job. All of their subjects were experienced reviewers for the journal being studied.

Also, it's important to note how they determined what constitutes a good reviewer. A journal editor ranks each review. If you think that reviewers are too elitist, you should be up in arms against the journal editors.