Reproducibility in behavioral assays

February 20, 2019

There’s an interesting issue of Pharmacology, Biochemistry & Behavior that is focused on Reproducibility of animal models for neuropsychiatric diseases.

Reading through the articles I am struck by how this effort is like throwing a bucket of water on a barn blaze. You might think that a “reproducibility” paper would answer a lof of questions. They do. But they also raise more and more and more questions. Take the following from Richetto et al. Effects of light and dark phase testing on the investigation of behavioural paradigms in mice: Relevance for behavioural neuroscience.

This is a pretty duck-soup behavioral pharmacology assay- inject an experimental subject with drugs and see what happens. In this case, inject a mouse with amphetamine and see how much it runs around. (Nice feature – N=6 male and 6 female per group, SABV orthodox, yo!)

Figure 5a from Richetto et al, 2019

Ok, so there’s a light cycle effect. I’ve blogged about that before (2015; 2018). and there was also a light cycle effect on cFos in Nucleus Accumbens and midbrain which did not interact with an effect of amphetamine treatment. So. Whether or not light cycle affects replication in this narrow instance is whether you are interested in quantitative differences or relative differences. The behavioral curves are more or less the same to a first approximation. But what’s missing here? Threshold, for one thing. This is a single dose of amphetamine (2.5 mg/kg, i.p.). What happens at lower or higher doses? At some point you fail to distinguish a drug effect from vehicle….ooops. Where’s the vehicle control. Injection prior to the amphetamine and characterized for only 20 minutes. Where are the mRNA controls coming from? Wait that whole experiment was 30 min after the saline or drug injection. What about later time-points (25th time bin) when the behavioral difference emerged?

Crap, back to the behavioral control. Why not run the saline injections in parallel all the way out to the 25th time bin? Because what you would find is that in light cycle the animals basically go to sleep. Wait they are testing in the dark, right? ….back to the paper. OMG it doesn’t really directly say and all we have to go on is testing in dark versus light vivarium cycle. Another factor, gaaaah. Testing in dark or light versus the circadian cycle.

The point is not to ask why they didn’t test absolutely everything but to point out that even a fairly effortful “replication” study of an exceptionally simple phenomenon gets complicated in a huge hurry.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: