The NIH must dismantle the corrosive competitive culture of science

I like competition, don’t get me wrong. I engaged in inter-school competitive sports from freshman year of high school through my senior year of college. I played intramural sports from late high school through the end of graduate school. I’ve done competitive sports outside of school organizations from high school until…yesterday. Essentially uninterrupted.

Nowadays, I spend a solid plurality of my weekends schlepping one kid or another around to a competitive sporting event.

Just milk? sourceI love what competition does for us on many levels, of course. This should be obvious from the above. Of the many benefits, one thing competition does that is most useful is to make us strive to be better. It makes us practice to improve our play and our game. It makes us get fitter, more accomplished, more capable. It makes us attain performance levels we didn’t know we could reach.

This is true in science as well.

Science is indeed a competitive business, as most of my Readers know full well.

We positively reify the markers of success- getting a particular scientific discovery first. Accomplishing some demonstration or discovery with the greatest panache. Coming to a realization or theory that changes the way everyone else thinks about a topic. Creating a medical therapeutic approach…or the basis for such a thing. The accolades are both arbitrary (prizes, “respect”) and specific (grant funding, jobs of increasing worth, etc). At heart, scientists are trying to learn things about the function of the natural world and so there is an overlying competition to advance knowledge.

In all of this, the pot is sweetened by the competition. The scientist receives part of her respect not merely for accomplishing a certain task but for doing it before, or better than, the next scientist.

AP photo from here

This reality can be fantastic for science. As in sport, the competition makes us work harder, make us work to up our game and motivates our excellence. This speeds the advance of knowledge. One of my favorite formative anecdotes was the late 80s-mid 90s competition between several laboratories to comprehensively identify the role that various medial temporal lobe structures (e.g., the hippocampus) played in memory function. In this case the competition was made more acute (as it often is in science) by disagreement. One lab thought structure X really did Y and another insisted it did Z. Or Y’, perhaps. And every year the Society for Neuroscience annual meeting would have a hilarious slide session in which the labs would bash away at each other. Almost always…pointedly. Sometimes in semi-personal attacks. Then they would scurry back to their labs, publish a paper or three and come back next year ready for more battle with their latest results. Understanding was advanced.

This is where we depart from competitive sports.

The key feature in my anecdote is that the labs would publish. Most if not all of their results. And they would discuss their latest findings at meetings. With. their. competitors. Knowledge was built not just by the major players but by anyone else who cared to chip in as well. Because there was a superseding goal that went beyond the simple question of who crossed a line first or who scored the most points.

That goal was the provision of knowledge to everyone. Because scientific advance requires a collaboration amongst many. This is why we publish papers that include full methodological description. This is why we are expected to be honest about how we did a particular study. This is why we are expected to share the very intellectual property that was necessary for the experiments!

People seemed to understand this, and acted accordingly, during the medial temporal lobe memory warz.

The trouble comes when we start behaving a little too much like sports competition.

Before I get into it, another analogy. Take business. It used to be that competition was about money, yes, but also about providing a service or building a widget. Making something that people wanted and needed. The marker of success was not just driving your competitors out of business…but in being the best to provide rail service from New York to San Francisco. To supply an automobile that people could afford….and that worked. At some point, business became more about scoring the most points or crossing the tape first. The role of arbitrary performance indicators (unimaginable sums of money, unconnected to anything that could be viewed as necessary for the participants) in motivating behavior totally supplanted real indicators. And in many cases the product or service suffered tremendous harm. As did the consumer.

We have reached this point of transition in science. The marker of success is the mere fact of publication* of a paper in Journals of established, but arbitrary, rank. It is no longer about the actual finding or any sense of advancing science or knowledge. Papers are increasingly disconnected from each other and from anything that is of any reasonable importance to know.

So why should the NIH care? No, I don’t mean for the last point here. Yes, the relevance of work funded by the National Institutes of Health does concern me. However, the appropriate valuation across the scales of “basic” to “applied” research are not the topic of today.

The topic of today is the efficiency with which the science that the NIH pays for is advanced.

Sadly, we are in a time of great secrecy within science. Because being first** to some finding is rewarded above and beyond all other things, the very essence of the competition demands not letting anyone else know what you are doing until it is published. The typical manuscript in our most respected journals requires many person-years of work. And much of this work never sees the light of day for various reasons. It is negative. Merely supportive. A blind alley. Or perhaps just of insufficiently amazing interest.

More sordidly, much of this work never sees the light of day because it might help a competitor lab to beat us next time.

This is being done on the NIH dime. Right now, in labs all across the US. Many, many hours and $$$ of work being conducted that will never see the light of day (i.e., be published).

Admittedly there is a lot of work that nobody wants to see. I get this. I am no fan of Open Notebook Science. I want scientists to present their work to me somewhat triaged for interest. But we are well down the road from that level at present.

The “cost” to the NIH is not merely the invisibility of data and findings that they have already paid for. It is also in the future expenditures as another laboratory has to repeat the same experiments, generate the same blind alleys, waste the same time evaluating bad reagents or theories.

Sadly, some labs even lay a false trail by describing their Methods so incompletely that other labs get a wrong impression of what needs to be done.

This can burn years of a trainees time in a lab. No joke and no exaggeration.

And we haven’t even arrived at the discussion of fraud which is also driven by the arbitrary markers of competition.

Time for the NIH to get interested in the way that competition for arbitrary markers in science is wasting their precious taxpayer dollars. Long past time. I’m thinking of writing Sen Grassley’s committee myself! (kidding.)

Solutions? Well, we’re faced in part with a Justice Potter Stewart solution in that we can identify wasteful, GlamourPublication chasing laboratory operations when we see them. We can also take a stab at estimating how many person hours of work are surely being buried in the process but this will start to get a little…forensic. But if it were easy…..

I’m going to suggest going after Glamour idiocy at two places. Empower the Program Officers to demand a better ratio of work payed for to publications resulting, first. Second, the study section. Yep, beef up the analysis of “productivity” by creating a set of bullet point guidelines for how to asses. They have them for the other aspects of grant review, right? The Significance, Innovation, etc criteria? Well, no problem beefing up the assessment of Productivity.

Heck, this should be a formal criterion on all grant review, not just continuation proposals. It dovetails nicely with Michael Eisen’s proposals for lab-based or person-based funding, doesn’t it? How many people have you had working in your lab and how many figures have been published? What is your total lab support, including fellowships, TAships, etc for your trainees? Have you published as much of this work as you possibly can?

Or are you engaging in competition for arbitrary markers and are relegating much of the work to the dark corners of forgotten harddrives?

Additional Reading on Fixing the NIH:
Shut off the PhD tap
We are going to fix the NIH

__
*If you ever catch yourself saying “my Cell paper”, “the Jones lab’s Nature paper” or “her Science paper” in preference or addition to a short description of the topic of the paper….you are part of the problem. And you need to step back from the brink of GlamourDouchery before you fall in for good.

**Two labs could have the essentially same idea about solving a given problem, say the function of a gene. They could beaver away with 5-20 people contributing various science over the course of years. With many millions of dollars of NIH funds expended. If they happen to wrap up their “stories” a mere two months apart, this can be the difference between being accepted into Science or Nature or not. It may even be the case that the second one to be ready is a better demonstration on all features and yet the priority, the mere fact of submitting it for consideration first, rules the day. This is profoundly disturbed.

What is even more disturbed in the system is what happens next. Many aspects of the paper which has been beaten to the punch may not be published at all! That’s right. For the type of lab that is competing on the “get”, i.e., the mere fact of a Nature or Science acceptance, it is “back to work, minions!” time. Time to take the “story” beyond the current state of affairs and hope to win the priority battle for the next story which is big enough for Science or Nature to take it. At the very least replication is lost. More likely, there are a number of differences between the two studies, differences that maybe were of interest to other laboratories. Of interest for different reasons to the same laboratories. Or may later come into focus ten years later because of additional findings. Yet because of the competitive conundrum of science, many of those findings will be lost forever.

ETA: Forgot my disclaimer. I have, in many ways, tried to run to daylight in my scientific choices. This is in part due to what is an intrinsic orientation of mine, in part due to accidents of training history and in part due to explicit decision-making vis a vis the career on my part. I avoid competitive nonsense. I am not in the Glamour Chase. I am not entirely certain whether or not various steps to dismantle the bad effects of Glamour chasing, scooping, priority focused science would be good or bad for me, to be honest.