Review me, critique me, pan me, print me March 14, 2013
Posted by mareserinitatis in engineering, papers, research.Tags: computers, engineering research, papers, research, simulations
1 comment so far
One of the first things I remember asking my MS advisor was how much detail should I include in a paper for publication. He said to make sure there was enough for someone else to replicate the work. When reviewing papers myself, I also look at this as one of the major criteria for publication.
I have tried very hard to stick with this rule of thumb, though there are things I overlook. Given most of my work is simulation, I sometimes forget that there are certain things which I tend to always do in my work, and not everyone does. Or maybe there’s a setting I never use and so the default stays in place. However, someone else may have a different default for that particular setting. And on and on. Regardless, I do my best.
The past couple weeks, I’ve been working on a new set of simulations. I’m basically taking widgets that other people have designed and seeing if I can use them for a particular, and somewhat unusual, application. I think it’s a rather interesting approach to the problem, but I keep getting mucked up. The reason is that several of the widgets I wanted to use are not described adequately in the papers. I’m not talking about some esoteric setting: some of these papers show widgets that don’t give physical dimensions of any of the parts! I have come across three different papers, all suffering the same problem.
I have decided that these papers are going in the round file. I was, at first, inclined to write to some of the authors of these papers and see if I could get some clarification. However, after encountering the third one, I decided it wasn’t worth the effort and decided to use papers from people who are more careful. I’m lucky in that there are several approaches to making these widgets, so I can be picky. That isn’t always the case, however.
I’m sitting here wondering first why the authors didn’t think to include this information and, second, what were the reviewers doing?! It’s not like these are complicated widgets with a million parts. Is it just my field of research? Am I the only one who replicates other people’s work? As much as I think peer review is awesome, I kind of feel like some people have fallen down on the job. It makes me appreciate those third reviewers that much more.
Writing about writing February 16, 2013
Posted by mareserinitatis in engineering, research, writing.Tags: busy, papers, proposals, waiting, writing
2 comments
I won’t apologize for not updating regularly. I will, however, say I miss it.
I’ve been doing a lot of writing again, and I’m finding that it leaves me with very little to say. I can’t really write about what I’m writing about in any sort of entertaining fashion. Well, I suppose I could complain, but who wants to read that?
In the past month, I’ve had to write a short proposal and put together a presentation, significantly edit a previously written long proposal to submit someplace else, and then write a conference paper. I’m hoping I get at least one of these so I can tell you about it, but for now, I’m still waiting. Heck, I’m still waiting on the proposals I wrote last October.
I suppose I could write about how much I hate waiting, but it would be a very short post. Even shorter than this one.
The art of citations August 15, 2012
Posted by mareserinitatis in engineering, papers, research, science.Tags: art, citations, papers, research
2 comments
I had an art class in Governor’s school that really reminds me of how I feel when people look at my research. Governor’s school, in North Dakota, is a six week program where you get to be immersed in a particular area of interest. Usually this involves some in depth, hands-on experience. I ended up spending six weeks doing research in a biology lab. I came out of the experience knowing I loved research but hated biology, and that ultimately got me interested in a career in science.
Aside from all that, we had enrichment activities in the evenings. My enrichment class was drawing. I can’t remember the specific name of the project, but basically we were supposed to draw part of another image. I chose to draw the Madonna’s face from Rafael’s Madonna de Foligno. I was at a place where I didn’t have access to any good art supplies, so I just did the drawing on lined paper. After I’d finished it up, I was terribly disappointed I’d not had any real drawing paper as it was one of the nicest drawings I’ve ever made. I felt like the lines on the notebook paper really disrupted a beautiful image.
My art teacher was a college student, and even though the term hipster hadn’t yet been coined, that’s what immediately pops to mind when I think about him. Rather than being impressed with my uber-awesome drawing skills, he thought the neatest thing about the drawing was that it was on lined paper. I guess he thought it made it look modern or something like that. I was livid. I’d worked so hard to get the image right, and he only cared about how the paper made it look cool (which it didn’t).
This is how I feel when I get citations.
I really like Google Scholar’s profile option. That being said, I’m almost always let down when I get one. I don’t mean to be picky, but I’ve noticed that certain papers get a lot more citations than others. The problem I have with this is that these aren’t my favorite papers: I think I have other papers that are better quality research.
What seems to happen is that one paper will be cited by someone, and once it’s cited, others will start using it as a reference. Some of this obviously has to do with areas where research is more active, which is understandable. I’m sure some of the papers are cited more simply because there’s more related literature coming out. I have to admit, though, that it’s frustrating when a paper you aren’t all that fond of has far more references than the one you really poured yourself into.
It’s kind of like someone admiring your drawing because it’s on lined paper.
Pick something and go July 20, 2012
Posted by mareserinitatis in engineering, papers, research, work.Tags: overwhelmed, papers, priorities, research
2 comments
I wrote up a list of things I need to be dealing with at work. While it was helpful for me to have a list to reference, it was also rather disheartening. I came up with over 10 things, and all but three were fairly sizable goals, like writing a paper.
I was rather overwhelmed, but happened to think about GMPs recent post on writing in a crunch. Her method was to break things down into bite-size chunks until the project was done. But what do you do when you have half a dozen big projects at the same time? I guess I tried to take a similar approach.
The thing is, I’m not in a huge time crunch to get most of this stuff done, but if I try to tackle several of these things at once, I’m fairly certain that none of them will get done, ever. So I picked off the easy things that I can work on here and there or that have definite deadlines (those first three). Of the 7 remaining items, I prioritized the ones that would be easiest to finish as well as providing the least amount of conflict in terms of computational resources with my current projects. I decided to just focus on the first one until I get to a point where I can’t work any more. Once I reach that point, I’ll shift to the second on the list until I can get back to the first or it gets finished.
I KNOW I can’t multitask well (or even passably, for that matter). The problem is that there are still these six other things that are sitting there, and it makes me uneasy to not even touch them. There’s this little voice that says, “If you don’t work on it now, you might NEVER get to it.” It’s really an irritating voice because it fails to recognize that I can only work on one thing at a time, and I’ll be more productive if I can maintain some decent focus. It also fails to recognize that there is a significant reduction in stress every time I can cross one of those things off my list entirely. And even if I start working on three or four of them, there are some that will have to get left behind as well. There is just no way to work on all of them simultaneously.
I wish I knew where that little voice came from and why it doesn’t listen to reason. Somehow I keep feeling like I could convince it that this is the sane approach. Instead, the best I can do for now is to ignore it.
How do you deal with things when they seem overwhelming?
Incomplete instructions May 10, 2012
Posted by mareserinitatis in papers, research, Uncategorized.Tags: papers, peer review, research
1 comment so far
I know I’ve been talking a lot about reviewing papers, but I figured one more post on the topic couldn’t hurt.
I was very lucky that my MS advisor started us reviewing papers as soon as we took a class from him. In all of his grad level classes, we were usually required to select 2-3 potential published papers for review and then to write up a critical analysis about 4-6 pages in length on one of them. It was a good experience, but I don’t think I would’ve made a very good reviewer my first year or two into grad school.
My real critiquing skills came when I started getting into some of my MS projects and I had to reproduce some of the work already done in papers. The first three I came across, it became very clear that the reviewers hadn’t done the best job: all three were missing critical details that required me to write the authors and ask how they had done certain things. In other words, there wasn’t enough information to replicate the work. That, therefore, became one of the first things that I look at with a paper. It would be nice if, when reviewing, one actually had time to sit down and try to replicate the experiment. Unfortunately, that’s not realistic…although I’ve also had papers with blatant errors that I’ve been trying to reproduce. I hope it’s just an oops that is the result of last minute writing, but I am beginning to think there are a lot of careless authors out there.
I’m not sure why this is the case, other than the fact that maybe people get too far into their experiment and fail to realize that there are many things they do automatically that one cannot take for granted. Even though most of the work I do is in simulations, there are a lot of things that appear superficially minor but can really change your results.
While there are other things one should look at it, I think the quality of most papers I’ve read follows along with the detail presented in laying out the process. If the process is not clearly spelled out, then chances are the other aspects of the paper are going to need some work, too.
So, for those who review papers, do you have things that you really look for in a paper and, if so, why?
Review season May 7, 2012
Posted by mareserinitatis in engineering, papers, research, younger son.Tags: engineering, engineering research, papers, peer review, research, reviewer comments
4 comments
Both Mike and I have been getting requests to review papers, and this has led to a lot of foul language around the house…along with frequent reminders from the younger son that our language is inappropriate.
It’s really hard to restrain yourself, however. As we’re sitting at the dining room table, occasionally one of us will turn our laptop toward the other and ask something like, “What does this look like to you?” or, “What do you think this means?” or, “What the hell were they thinking?”
I have to admit that I appreciate having a second pair of eyes to catch the things that I miss. I’m sure the authors of the papers we’re reviewing probably will not appreciate it. Not only do they have the third reviewer going over their papers, they have two of them. I hope this will result in double the hair pulling and teeth gnashing on their end…because it sure has for us.
Was blind, but now I see… May 6, 2012
Posted by mareserinitatis in engineering, papers, research.Tags: double blind, papers, peer review, single blind
3 comments
I was recently asked to review a paper for a fairly large conference in one of the engineering subfields I’m involved in. This particular conference is one which I’ve not attended, so I had no familiarity with the procedures. As a side note for non-engineers, I discussed before (on my old blog) that many (most?) engineering conferences take full, peer-reviewed papers.
When I received the paper and looked it over, I nearly fell out of my chair. I could see the freakin’ authors!
In most of the conferences where I’ve submitted papers, the peer-review was double blind. One conference in particular was this way because it’s such a small area of research that they wanted to make doubly sure that people are as objective as possible. (In reality, there’s a good chance that you could tell who it was just by what they were doing, but I applaud the effort.) It seems like a very straight-forward thing to do: you submit the paper without any names on it. The session chair knows who it is but picks people to review who will be none the wiser. If the paper is accepted, a revision is submitted with names on it. Easy-peasy.
I have to say that this was very disconcerting for me. I don’t WANT to know whose paper I’m reviewing. I spent the whole time writing this review terrified that knowing who they were, where they were from, how many authors were on the paper, etc. was affecting my perceptions of the paper and destroying my objectivity. I was amazed at all the stupid things I found myself questioning in terms of my reaction. Was I making a mountain out of a molehill? Was I overly impressed by something which shouldn’t have impressed me?
It really isn’t all that hard to keep reviews double-blind when using an automated submission system such as the one used for this conference (and most IEEE conferences), and as a reviewer, I would have been far more comfortable.
I’m curious about other fields, though. Is single-blind review the norm? (When I stumble across these things, I feel like I’ve been living under a rock.)
When copy and paste goes wrong December 30, 2011
Posted by mareserinitatis in engineering.Tags: editing, papers
add a comment
I’ve been waiting to hear about a conference paper I submitted back in October. I finally received word this morning that it had been accepted. The notification letter contained all the standard info until I got to the bottom: the organizers let me know they were looking forward to seeing me at last year’s conference in another country.
I also blog at Engineer Blogs, home away from home to some of the best engineering blogs.
self scrutiny March 25, 2013
Posted by mareserinitatis in engineering, papers, research.Tags: conference, papers, peer review, reviewer comments
add a comment
After spending a considerable amount of time griping about other people’s papers, I feel I would be remiss if I didn’t complain about my own once in a while, as well. I’m currently revising a paper that I and a coauthor submitted to a conference. It was accepted, but there were changes requested. I started to work through some of them, but then realized that some of the comments didn’t make sense.
I sat down with a couple people, including the coauthor, and we started trying to figure out what was going on. After reading through each comment with a fine-tooth comb, we came to the realization that the problem was that we took for granted the method we were using and gave a very succinct explanation. It obviously wasn’t enough: we gathered from the comments that they resulted in a complete misunderstanding of what we were showing.
In other words, I screwed up because I didn’t explain clearly enough what we were doing. This lead to some huge misunderstandings by the reviewers, and some of the more…ummm…cynical? Yes, cynical is a good euphemism. Anyway, this explains some of the more cynical comments we got from reviewers.
The good news is that, with more explanation, I think we’ll have a much better paper when we’re done. However, this has made me realize that I really can’t take for granted what my reviewers may or may not know. It’s best to be as explicit and detailed as possible.