Tuesday, 27 January 2009

Double-blind refereeing

The habit of double-blind refereeing (reviewing) of academic papers has evolved over the years and is quite common in academic journals. The idea is that when an author submits a paper, the editor makes sure the author's name and address are not evident, then chooses other experts in the field to review it and provide a critique of the strengths and weaknesses of the research and of the way in which the paper is written. It is the basis of scientific quality control, and is also used in reviewing research grant applications, especially when public money us involved. As an editor myself, I am very dependent on the referees' advice when deciding the fate of a paper. But the fate of a paper is my decision, not the referees', and sometimes I don't follow their advice, which can confuse authors and referees.

One interesting thing that frequently happens is that authors refer to their own previous work and thereby reveal their identity to the referees. Sometimes, a conscientious referee will then contact me and ask what to do because they have noticed the authors' identity. My attitude is always the same. It does not matter too much! If authors choose to reveal their identity to referees, they are either not worried about the lack of anonymity, or too naive to realise what they are doing. The key point is that many referees would recognise the writing of someone who was well-known. So the invention of blind refereeing was obviously not to protect well-known people. It was for the benefit of less well known people. It means the judgement of the referee is solely based on the merits of the paper. An unknown research student who submits a paper will have it reviewed by people who cannot guess whether the author is a student or a professor, so junior academics are the ones who benefit the most from blind refereeing.

Of course, referees often think they know who wrote the paper, and they often guess wrong. I have often sent to famous professors comments from encouraging referees who say that with practice and in time, they might be able to get their work up to scratch!

Authors who reveal their identity are either already so well-known that it does not really matter, or they are too naive to realise that blind refereeing requires that they don't reveal their identity. Either way, the conclusion is that the paper can still be refereed by the referee to whom we have sent it.


Anonymous said...

I agree with the conclusion. However, does it really matter that the identity of the author is revealed to the referee of a paper?

Referees in my opinion, are expected to base their critique objectively, based on the merits of the paper and not on personalities.

Will Hughes said...

As you rightly point out, what matters more than anything is that the critique is based on an evaluation of the research. The aim of concealing identities is to have some confidence that that this is what, in fact, takes place. In my experience, it is much quicker to come to a conclusion about a piece of work when I know who has written it. But if I do not know, I have to read it much more carefully, because I cannot make assumptions about the context and background of the person who wrote it. If the author is well-known, and I can discern who it is, then identity is not, in fact, concealed. However, a PhD student who submits a paper for blind review is treated with as much respect and credibility as a famous professor, if the identity is revealed. This is the great strength of this process.

Will Hughes said...

Addendum - http://will-hughes.blogspot.com/2010/06/double-blind-reviewing.html

Will Hughes said...

I just noticed a typo in my own comment of 14 Feb 09. In the penultimate sentence, "concealed" should be "revealed". Whoops.

Search This Blog

Blog Archive

About Me

My photo
Reading, Berkshire, United Kingdom

Total Pageviews