jump to navigation

The Power-Pointification of Papers August 19, 2010

Posted by mareserinitatis in career, papers, writing.
Tags: , ,
trackback

One thing about working with simulations, especially when using commercial software, is that I should have an easy time replicating others’ results, especially when I’m using the same software.

Hypothetically.

The reality is that between half and three-quarters of the papers I have pulled from journals and conference proceedings have had huge glaring mistakes or did not include a lot of essential data. On more than one occasion, I have written to the author of such papers, having tried everything I could think of, only to find out that some of the information in the paper was just plain incorrect. And some of these papers have come from extremely reputable programs.

So what’s the deal? I have come to the conclusion that there are two possible reasons for this sort of thing.

One is that the author(s) of such papers are not organized. I’ve had people tell me that they wrote up papers in a hurry, after they’ve done their projects, and have failed to remember at least a few of the essential details. Hence, there are errors or omissions in their papers.

I understand where they’re coming from. I did similar things myself when I first started grad school. However, after doing this a couple times for class projects, I was introduced to the concept of writing the paper or report first (which was the topic of yesterday’s post). I usually write as much of the paper as I can, including the gory details, and then I start digging into the project. As I am putting together my project, I add anything and everything that I do, turning the report into a sort of lab notebook. I realize there are people who are full of awesome when it comes to the art of keeping a lab notebook. I’ve also noticed that with the type of labs one normally does in school anymore, the emphasis on keeping a notebook has really gone downhill, especially when one is learning about computational techniques. This has been my solution to the problem. Once I have a complete record of what I did and have added my results, I can simply cut things out (e.g. comments such as “Damnit! It didn’t work again!”) to turn my record into a paper.

The other issue that may be a contributing factor is what I call the “powerpointification of papers”. If you ever take an old paper, say something written in the 50s or 60s, you’ll notice that the paper has a different flow than a lot of the ones we see today, especially if you’re dealing with papers covering an analytical approach to a problem. (It’s hit and miss with experimental papers, although I will say that it’s better in general.) Because so much of the paper is dependent on the reader understanding the mathematical logic involved, the author is more apt to write out steps involved in getting from A to B. If they skip steps or use approximations, they will at least mention the method used to get from A to B.

My training says this is the way to do things properly: in a paper, you should include absolutely all the information required to complete an experiment. A presentation, on the other hand, should cover the ‘big picture’.

It seems to me that there is a blurring of the two ways of presenting information. I know that some people have a preference for one or the other. However, it seems like the notion that “people don’t want all the gory details” has seeped into writing papers. The idea of a paper no longer seems to be a detailed description of the procedure so that others can replicate it…it’s becoming an overview of what people have done with the level of detail dependent on how picky your reviewers are.

My thoughts are that anyone who questions whether or not they should leave something out of a paper ought to put it in. If you’re doing emag simulations, for instance, how and where you applied any excitations could be rather critical to replicating your results, particularly in some software packages. No, you don’t have to explain the basics of how the software works, but you should bring up specifics about how you solved your problem.

I realize there are magazines and journals where this isn’t appropriate, such as where strict page limits are involved, but such venues are generally meant to catch the eye and inform the reader of advances in that particular area of research. In that case, leaving things out may be essential. However, it also ideally means that it points to another paper that contains the full set of information required to replicate the findings.

Advertisements

Comments»

1. Fluxor - August 21, 2010

Never mind reproducing results from a paper. Sometimes, I can’t even reproduce results from the colleague sitting next to me with him watching over my shoulders. Our system level sims has probably 200-300 variables (I lost count). But beyond that, sometimes, it’s just some very basic things we forget to check because it’s so basic. Things such as simulation tolerance settings or numerical integration methods.

Like

2. mareserinitatis - August 23, 2010

The stuff I’ve been doing hasn’t been that touchy or complicated. Some of it has been simply to validate software or things like that. But over half the time, the problems have come from the fact that information given in the paper in incorrect, and I’ll find this out when I write the author. Given the correct information, I can run it and get the specified results.

It’s exactly why I’ve been proving things out with simple models…I know it’s going to be a major headache when doing stuff like what you’re mentioning.

Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: