Oh, you aren't alone in your teeth clenching
July 8, 2014
The most replicated finding in drug abuse science
July 8, 2014
Ok, ok, I have no actual data on this. But if I had to pick one thing in substance abuse science that has been most replicated it is this.
If you surgically implant a group of rats with intravenous catheters, hook them up to a pump which can deliver small infusions of saline adulterated with cocaine HCl and make these infusions contingent upon the rat pressing a lever…
Rats will intravenously self-administer (IVSA) cocaine.
This has been replicated ad nauseum.
If you want to pass a fairly low bar to demonstrate you can do a behavioral study with accepted relevance to drug abuse, you conduct a cocaine IVSA study [Wikipedia] in rats. Period.
And yet. There are sooooo many ways to screw it up and fail to replicate the expected finding.
Note that I say “expected finding” because we must include significant quantitative changes along with the qualitative ones.
Off the top of my head, the types of factors that can reduce your “effect” to a null effect, change the outcome to the extent even a statistically significant result isn’t really the effect you are looking for, etc
- Catheter diameter or length
- Cocaine dose available in each infusion
- Rate of infusion/concentration of drug
- Sex of the rats
- Age of rats
- Strain of the rats
- Vendor source (of the same nominal strain)
- Time of day in which rats are run (not just light/dark* either)
- Food restriction status
- Time of last food availability
- Pair vs single housing
- “Enrichment” that is called-for in default guidelines for laboratory animal care and needs special exception under protocol to prevent.
- Experimenter choice of smelly personal care products
- Dirty/clean labcoat (I kid you not)
- Handling of the rats on arrival from vendor
- Fire-alarm
- Cage-change day
- Minor rat illness
- Location of operant box in the room (floor vs ceiling, near door or away)
- Ambient temperature of vivarium or test room
- Schedule- weekends off? seven days a week?
- Schedule- 1 hr? 2hr? 6 hr? access sessions
- Schedule- are reinforcer deliveries contingent upon one lever press? five? does the requirement progressively increase with each successive infusion?
- Animal loss from the study for various reasons
As you might expect, these factors interact with each other in the real world of conducting science. Some factors you can eliminate, some you have to work around and some you just have to accept as contributions to variability. Your choices depend, in many ways, on your scientific goals beyond merely establishing the IVSA of cocaine.
Up to this point I’m in seeming agreement with that anti-replication yahoo, am I not? Jason Mitchell definitely agrees with me that there are a multitude of ways to come up with a null result.
I am not agreeing with his larger point. In fact, quite the contrary.
The point I am making is that we only know this stuff because of attempts to replicate! Many of these attempts were null and/or might be viewed as a failure to replicate some study that existed prior to the discovery that Factor X was actually pretty important.
Replication attempts taught the field more about the model, which allowed investigators of diverse interests to learn more about cocaine abuse and, indeed, drug abuse generally.
The heavy lifting in discovering the variables and outcomes related to rat IVSA of cocaine took place long before I entered graduate school. Consequently, I really can’t speak to whether investigators felt that their integrity was impugned when another study seemed to question their own work. I can’t speak to how many “failure to replicate” studies were discussed at conferences and less formal interactions. But given what I do know about science, I am confident that there was a little bit of everything. Probably some accusations of faking data popped up now and again. Some investigators no doubt were considered generally incompetent and others were revered (sometimes unjustifiably). No doubt. Some failures to replicate were based on ignorance or incompetence…and some were valid findings which altered the way the field looked upon prior results.
Ultimately the result was a good one. The rat IVSA model of cocaine use has proved useful to understand the neurobiology of addiction.
The incremental, halting, back and forth methodological steps along the path of scientific exploration were necessary for lasting advance. Such processes continue to be necessary in many, many other aspects of science.
Replication is not an insult. It is not worthless or a-scientific.
Replication is the very lifeblood of science.
__
*rats are nocturnal. check out how many studies**, including behavioral ones, are run in the light cycle of the animal.
**yes to this very day, although they are certainly less common now
DrugMonkey 