Literature pollution versus continually retreading the same ground

On perusing NIGMS Director Berg’s latest post on the relationship between research funding and the number of resulting publications (original, related) I noticed this comment.

Using these kinds of metrics only encourages incremental publications and other forms of literature pollution.

I never understand such remarks and I am looking for my readers to ‘splain to me exactly what the problem is here.

As Scicurious was just remarking thankfully, we live in an era in which the ability to rapidly find publications that present scientific data on a given topic is quite good. Far superior to what was available only 2-3 short decades ago. To some extent the more focused the data in a single paper are, the easier it is to actually find. Why? Because the Abstract that is available in PubMed is limited and the more you squeeze into the paper, the less reflective the Abstract (or Title for that matter) can be. So you run the risk of missing a figure that might be really important to your work if it was only a tangential part of a particular paper.

When I am building a scientific argument or rationale, whether in my own head, for a grant application or for a scientific manuscript, I am looking to sort through specific findings in a synthetic manner. I am focused on Figure 3 from Publication A; Figure 1 from Publication B; Table 2 from Publication C; off-hand remark in Publication D…etc. All leavened by caveats X, Y or Z about the methodologies, controls or limits that attend each paper. That’s how we come to a greater understanding of the natural world.

We most certainly do not increase our understanding in a fundamental way because a single article in Science or Nature, jampacked with poorly described, unvalidated, weakly controlled data from machines that go Ping! knocks it out of the park. We don’t. The knock-your-socks-off pubs are unbelievably infrequent in contrast to the publishing rate for Nature or Science.

Once upon a time I read a lot of literature in the Journal of Experimental Psychology titles and the Journal of the Experimental Analysis of Behavior. Oh. My. God. you wanna talk about some navel inspecting, Bunny Hopping, internally focused academic fields? I still have scars. Anyway, those journals (especially JEAB) would contain papers with unending numbers of microscopically-different experiments on how many angels could dance on the head of a pin how pigeons pecked at keys to get grain under different conditions. Or how rats pressed a lever for a food pellet. Let me tell you, those people thought that this was the only way to publish proper science. To knock it out of the park with pedantry on each and every manuscript that was published.

And they were just as wrong about the way fundamental understanding of a scientific topic advances.

Science advances by data. Step by step, built on the random walks that motivate individual members of the scientific workforce. Pulled together in synthesis by those self-same workers as they move on to their next studies.

So I return to this comment about incremental publication and pollution of the literature to wonder- where is the cost? What does it hurt to publish “incrementally”? This notion of “a complete story” is fiction. And not even a very believable fiction.

On the other side, I do see a great deal of cost involved with excessive disdain for “incremental publication”. The cost of the GlamourMag approach and the cost of the “complete story” fiction is the failure to publish data that are helpful to someone else. Even if it is helpful in a way that you didn’t foresee, but usually because it keeps other people from wasting so much time. If you are working in an area that is so great, impactful and important then it is axiomatic that other someones are going to be working with the same models and approaches to target similar questions. So they are going to fall straight into the same traps that you (or your lab) did. Or they are going to need the same validation and control studies, the same troubleshooting assays, the same scientific footbridge.

If these “incremental” studies or “polluting” data sets do not end up in the literature, then history is bound to repeat itself.

And that, Dear Reader, is a waste of time and Dr. Berg’s carefully husbanded NIH Grant money.