Literature pollution versus continually retreading the same ground

November 23, 2010

On perusing NIGMS Director Berg’s latest post on the relationship between research funding and the number of resulting publications (original, related) I noticed this comment.

Using these kinds of metrics only encourages incremental publications and other forms of literature pollution.

I never understand such remarks and I am looking for my readers to ‘splain to me exactly what the problem is here.

As Scicurious was just remarking thankfully, we live in an era in which the ability to rapidly find publications that present scientific data on a given topic is quite good. Far superior to what was available only 2-3 short decades ago. To some extent the more focused the data in a single paper are, the easier it is to actually find. Why? Because the Abstract that is available in PubMed is limited and the more you squeeze into the paper, the less reflective the Abstract (or Title for that matter) can be. So you run the risk of missing a figure that might be really important to your work if it was only a tangential part of a particular paper.

When I am building a scientific argument or rationale, whether in my own head, for a grant application or for a scientific manuscript, I am looking to sort through specific findings in a synthetic manner. I am focused on Figure 3 from Publication A; Figure 1 from Publication B; Table 2 from Publication C; off-hand remark in Publication D…etc. All leavened by caveats X, Y or Z about the methodologies, controls or limits that attend each paper. That’s how we come to a greater understanding of the natural world.

We most certainly do not increase our understanding in a fundamental way because a single article in Science or Nature, jampacked with poorly described, unvalidated, weakly controlled data from machines that go Ping! knocks it out of the park. We don’t. The knock-your-socks-off pubs are unbelievably infrequent in contrast to the publishing rate for Nature or Science.

Once upon a time I read a lot of literature in the Journal of Experimental Psychology titles and the Journal of the Experimental Analysis of Behavior. Oh. My. God. you wanna talk about some navel inspecting, Bunny Hopping, internally focused academic fields? I still have scars. Anyway, those journals (especially JEAB) would contain papers with unending numbers of microscopically-different experiments on how many angels could dance on the head of a pin how pigeons pecked at keys to get grain under different conditions. Or how rats pressed a lever for a food pellet. Let me tell you, those people thought that this was the only way to publish proper science. To knock it out of the park with pedantry on each and every manuscript that was published.

And they were just as wrong about the way fundamental understanding of a scientific topic advances.

Science advances by data. Step by step, built on the random walks that motivate individual members of the scientific workforce. Pulled together in synthesis by those self-same workers as they move on to their next studies.

So I return to this comment about incremental publication and pollution of the literature to wonder- where is the cost? What does it hurt to publish “incrementally”? This notion of “a complete story” is fiction. And not even a very believable fiction.

On the other side, I do see a great deal of cost involved with excessive disdain for “incremental publication”. The cost of the GlamourMag approach and the cost of the “complete story” fiction is the failure to publish data that are helpful to someone else. Even if it is helpful in a way that you didn’t foresee, but usually because it keeps other people from wasting so much time. If you are working in an area that is so great, impactful and important then it is axiomatic that other someones are going to be working with the same models and approaches to target similar questions. So they are going to fall straight into the same traps that you (or your lab) did. Or they are going to need the same validation and control studies, the same troubleshooting assays, the same scientific footbridge.

If these “incremental” studies or “polluting” data sets do not end up in the literature, then history is bound to repeat itself.

And that, Dear Reader, is a waste of time and Dr. Berg’s carefully husbanded NIH Grant money.

No Responses Yet to “Literature pollution versus continually retreading the same ground”

  1. zoubl Says:

    Eve Marder and colleagues published a commentary in PNAS yesterday that is related to this issue – how trainees or asst professors are pressured to conduct science for the sole purpose of publishing in a glamour mag versus a perfectly respectable journal that doesn’t cater to broad-appeal type manuscripts.

    I’m sorry I don’t have the full link, but look for it in the Nov. 22nd post of “early addition articles”. The title is “Impacting our young”.


  2. drugmonkey Says:

    Interesting editorial indeed. From people who are past presidents of the US based Society for Neuroscience and the Federation of European Neuroscience Societies and the current president of the latter.

    should do it.

    thanks for the tip.


  3. At my institution, the new VP instituted a system by where they compare departments based upon publications and they are only going to count glamour mags now. Fuck!


  4. arrzey Says:

    Okey-dokey. This is NOT my view. I sent the article by Marder et al to various folks at my institution. Here is one response I got back. It is worth considering because this is a person with significant authority/power at my institution. I suspect there is a fairly stiff selection gradient of who reads this blog, and this particular view is not likely to be represented.

    Thank you for the article. I, of course, read it with great interest, but disagree in principle on the analysis of the situtation by the authors. There are several key issues intertwined in the PNAS editorial.

    1. The social (the academy) influence of citation impact. In general, I agree that certain journals hold the premium on prestige, resulting in a rich-gets-richer phenomenon. The strive by junior investigators to publish in the highest impact journals could be detrimental to momentum of their research, because rejection is the norm for 90% of submissions. Nonetheless, all honorific societies (e.g., National Academies) look at impact, particularly when a body of work is published in a general journal (NEJM, Science, Nature…) versus in a more specialized or sub-specialized journal. As a past-president of the American Society for , it is clear that this is what their council members focus on for their decision on whom to elect into the society. This is not meant to be elitist, but it is a fact of our academic social framework of recognizing excellence over mediocrity.

    2. The value of citation impact. Regardless of one’s personal perspective, it is a fact that a publication that is highly cited is one that has made tangible scientific impact and hence should rewarded in our faculty promotion process. Citation impact is not random. If one looks at the institutional publication impact, it is not by mistake that the top institutions (by many measures) make it to the top of the list. See:

    We have always been at the top of the list; hence, there is no reason to think that what we value here isn’t meritorious and shouldn’t be continued.

    For one, I worry that in the spirit of creating a warmer and fuzzier place, we risk lapsing toward mediocrity. I don’t think this is what you meant by sending the article, but it is clear that I have a very visceral perspective on excellence.


  5. DrugMonkey Says:

    People who are on about “mediocre” science do not understand how science moves *forward*.


  6. Pinko Punko Says:

    Former president of whatever is not very smart. How many more Glamour mags are there now with the number of scientists increasing by leaps and bounds? There are less spaces in those magazines, and what does a paper there really mean to a particular author who is not first or last, but one of twenty and understanding how they might function as a scientist in the future. The glamour mag view of reward is very, very bad.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: