Was blind, but now I see… May 6, 2012
Posted by mareserinitatis in engineering, papers, research.Tags: double blind, papers, peer review, single blind
3 comments
I was recently asked to review a paper for a fairly large conference in one of the engineering subfields I’m involved in. This particular conference is one which I’ve not attended, so I had no familiarity with the procedures. As a side note for non-engineers, I discussed before (on my old blog) that many (most?) engineering conferences take full, peer-reviewed papers.
When I received the paper and looked it over, I nearly fell out of my chair. I could see the freakin’ authors!
In most of the conferences where I’ve submitted papers, the peer-review was double blind. One conference in particular was this way because it’s such a small area of research that they wanted to make doubly sure that people are as objective as possible. (In reality, there’s a good chance that you could tell who it was just by what they were doing, but I applaud the effort.) It seems like a very straight-forward thing to do: you submit the paper without any names on it. The session chair knows who it is but picks people to review who will be none the wiser. If the paper is accepted, a revision is submitted with names on it. Easy-peasy.
I have to say that this was very disconcerting for me. I don’t WANT to know whose paper I’m reviewing. I spent the whole time writing this review terrified that knowing who they were, where they were from, how many authors were on the paper, etc. was affecting my perceptions of the paper and destroying my objectivity. I was amazed at all the stupid things I found myself questioning in terms of my reaction. Was I making a mountain out of a molehill? Was I overly impressed by something which shouldn’t have impressed me?
It really isn’t all that hard to keep reviews double-blind when using an automated submission system such as the one used for this conference (and most IEEE conferences), and as a reviewer, I would have been far more comfortable.
I’m curious about other fields, though. Is single-blind review the norm? (When I stumble across these things, I feel like I’ve been living under a rock.)
I also blog at Engineer Blogs, home away from home to some of the best engineering blogs.