On contradicting your own stuff

I am struck, today, by the thought that a significant benefit of growing old and comfortable as a laboratory is that you don’t care anymore.

I was just reading a paper, published in a fairly pedestrian journal. It was authored by a postdoc in a fairly well-established laboratory which has published extensively with their…..Bunny Hopper HedgerowDash model.

The paper is actually pretty cool and I’m way down with their findings. It isn’t big stuff but it sheds some light on the workings of the HedgerowDash model. Light, in the nature of methodological variations which might just fundamentally change the outcome of a manipulation, say, amphetamine doping.

Now, as you all know, the Bunny is a crepuscular species. It is most active at dawn and dusk. Your average postdoc or graduate student, however, is most active either in the middle of the day or late at night. (This is established fact.) This may be important.

Now suppose that when the Lab was young, a trainee (or three) generated a set of key initial findings that really put the HedgerowDash model, and therefore the Lab, on the map. Cocaine, amphetamine, nicotine….you name it, any stimulant possible was thrown at the model. Antagonists were deployed. Aaaaaand, the trainee sac’d the Bunnies at the end of the Dash and came up with all kinds of key pathway changes to identify the neuronal circuitry and neuropharmacology that was involved in the HedgerowDash.

Off they went! Whee! and the field followed along and adopted the Dash model in their own Laboratories. And the NIH looked and found that it was Good.

And many, many trainees succeeded….but many, many failures were obtained*. Failures to replicate even the positive control findings with the most basic challenge, amphetamine. Even, years later, in the original Dash Lab. Science is hard, no?

The negative findings never saw the light of day. After all, the Dash Lab had a MODEL. It must work. So any trainees that failed to get the positive control, well, clearly they were doing things wrong. “Go back to GoldenHairedTrainee’s protocol and do that. Precisely”, says the helpful PI. Similar thoughts were extended in the other labs “Did you call the Dash Lab and ask for help? I’ll chat with the PI at the next meeting and see what is up….” says the other PI. Trouble was shot and the graduate student got her data. The postdoc finally got the model to “work”. All were happy.

Meanwhile…..the contrarians.

It doesn’t make any sense, they would observe. This is such a great effect, why can’t we do the experiment in the smart way and get it to work? I run my Bunnies at dawn and dusk….that’s their active period! That’s when they need to run away from the foxes. Don’t you know the Dash Lab papers from back in the day ran their animals at…..NIGHTIME! They bloody well ran studies in sleepy Bunnies, woke them up and hit them with amphetamines. Don’t you think maybe they were a little biased for a low baseline Dash speed? Huh??!!!???? Oh, and did you know it was all single-challenge, between groups analysis? The trainee was sac’ing them for brain measures. These lab bred Bunnies never even saw the Hedgerow course but once. It was totally novel. And they weren’t even hungry for clover. No wonder they ran like hell for cover when they smelled the fox odor.

Perhaps they tried to get their papers published, perhaps a grant Specific Aim or two, on the basis of their objections. “How do we know this is relevant” they would query. And the world would kick them in the teeth because clearly they were incompetent at getting the Dash result that all of the real scientists could produce (finally whew!). Nevermind what makes “sense” for this model, what matters is what works.

Until…finally….in its dotage seniority. The Dash Lab finally fesses up. The PI allows a trainee to publish the warts. And compare the basic findings, done at nighttime in naive bunnies, with what you get during the dawn/dusk period. In Bunnies who have seen the Dash arena before. And maybe they are hungry for clover now. And they’ve had a whiff of fox without seeing the little blighters before.

And it turns out these minor methodological changes actually matter.

Whee! Now we’re cooking with gas, Bunny Hoppers!

As I said, what I’ve been thinking about is the point at which you** have the confidence to contradict yourself in published papers. Do you let out a story which has contradiction right from the get-go? Let the warts out there, even when you don’t yet understand the critical methodological differences that produced a seemingly contradictory outcome?

Or do you publish, even papers which seemingly contradict each other, and let the field start worrying away over the differences and help you figure it out?

__
*In the event any Readers know what paper I am talking about, at this point we are well off into the fiction part of our blogpost. I have no specific knowledge it shook down like this.

**I’m going to leave my apparent views on this (based on my publishing behavior) out for now…