Incomplete instructions May 10, 2012Posted by mareserinitatis in papers, research, Uncategorized.
Tags: papers, peer review, research
I know I’ve been talking a lot about reviewing papers, but I figured one more post on the topic couldn’t hurt.
I was very lucky that my MS advisor started us reviewing papers as soon as we took a class from him. In all of his grad level classes, we were usually required to select 2-3 potential published papers for review and then to write up a critical analysis about 4-6 pages in length on one of them. It was a good experience, but I don’t think I would’ve made a very good reviewer my first year or two into grad school.
My real critiquing skills came when I started getting into some of my MS projects and I had to reproduce some of the work already done in papers. The first three I came across, it became very clear that the reviewers hadn’t done the best job: all three were missing critical details that required me to write the authors and ask how they had done certain things. In other words, there wasn’t enough information to replicate the work. That, therefore, became one of the first things that I look at with a paper. It would be nice if, when reviewing, one actually had time to sit down and try to replicate the experiment. Unfortunately, that’s not realistic…although I’ve also had papers with blatant errors that I’ve been trying to reproduce. I hope it’s just an oops that is the result of last minute writing, but I am beginning to think there are a lot of careless authors out there.
I’m not sure why this is the case, other than the fact that maybe people get too far into their experiment and fail to realize that there are many things they do automatically that one cannot take for granted. Even though most of the work I do is in simulations, there are a lot of things that appear superficially minor but can really change your results.
While there are other things one should look at it, I think the quality of most papers I’ve read follows along with the detail presented in laying out the process. If the process is not clearly spelled out, then chances are the other aspects of the paper are going to need some work, too.
So, for those who review papers, do you have things that you really look for in a paper and, if so, why?