Was blind, but now I see… May 6, 2012
Posted by mareserinitatis in engineering, papers, research.Tags: double blind, papers, peer review, single blind
trackback
I was recently asked to review a paper for a fairly large conference in one of the engineering subfields I’m involved in. This particular conference is one which I’ve not attended, so I had no familiarity with the procedures. As a side note for non-engineers, I discussed before (on my old blog) that many (most?) engineering conferences take full, peer-reviewed papers.
When I received the paper and looked it over, I nearly fell out of my chair. I could see the freakin’ authors!
In most of the conferences where I’ve submitted papers, the peer-review was double blind. One conference in particular was this way because it’s such a small area of research that they wanted to make doubly sure that people are as objective as possible. (In reality, there’s a good chance that you could tell who it was just by what they were doing, but I applaud the effort.) It seems like a very straight-forward thing to do: you submit the paper without any names on it. The session chair knows who it is but picks people to review who will be none the wiser. If the paper is accepted, a revision is submitted with names on it. Easy-peasy.
I have to say that this was very disconcerting for me. I don’t WANT to know whose paper I’m reviewing. I spent the whole time writing this review terrified that knowing who they were, where they were from, how many authors were on the paper, etc. was affecting my perceptions of the paper and destroying my objectivity. I was amazed at all the stupid things I found myself questioning in terms of my reaction. Was I making a mountain out of a molehill? Was I overly impressed by something which shouldn’t have impressed me?
It really isn’t all that hard to keep reviews double-blind when using an automated submission system such as the one used for this conference (and most IEEE conferences), and as a reviewer, I would have been far more comfortable.
I’m curious about other fields, though. Is single-blind review the norm? (When I stumble across these things, I feel like I’ve been living under a rock.)
I also blog at Engineer Blogs, home away from home to some of the best engineering blogs.
In physics single-blind is the norm. I think it is because the reviewer may want to contact the author and ask for clarifications, in some cases, which could entirely alter the recommendation… But of course there are disadvantages as well with single-blind. This is why I support doing away with peer review altogether, and let the number of cites be the one and only arbiter.
I think it’s good to have colleagues review papers before putting them out there, just to clarify things. However, I, in general, agree that cites is a better arbiter with one qualifications – self-citations ought to be distinguished from external citations. I have seen too many people who self-cite excessively just to get numbers up. (In one case, I saw an author who cited an excess of 20 of his own papers in one journal article…)
A lot of people are worried about self-citations… I don’t know, I think it is awfully hard to raise, say, one’s h-index with self-citations… I mean I suppose it is technically possible but I think one would would be noticed, and ridicule would quickly supersede any advantage that one could possibly derive…