The NIH says investigators must incorporate sex-differences analyses in their studies

May 14, 2014

For some reason I am having a DOI error on the actual comment from Clayton and Collins. So until that is resolved, the sourcing is from the journalists who got the embargoed version.

Apparently Janine Clayton and Francis Collins have issued a commentary on a new policy that the NYT describes as:

The N.I.H. is directing scientists to perform their experiments with both female and male animals and include both sexes in sufficient numbers to see statistically significant differences. Grant reviewers will be instructed to take the sex balance of each study design into account when awarding grants.

Yeah, that sounds pretty clear. My studies just doubled…which means really that they were just cut in half. I’m cool with that. I actually agree that it would be good if we did almost everything as a sex-differences study.

There’s the money though. Sex difference studies in a behaving animal are not just a doubling as it happens (and as I inaccurately described it just above). From a prior post on this topic entitled: The funding is the science II, “Why do they always drop the females?

As nicely detailed in Isis’ post, the inclusion of a sex comparison doubles the groups right off the bat but even more to the point, it requires the inclusion of various hormonal cycling considerations. This can be as simple as requiring female subjects to be assessed at multiple points of an estrous cycle. It can be considerably more complicated, often requiring gonadectomy (at various developmental timepoints) and hormonal replacement (with dose-response designs, please) including all of the appropriate control groups / observations. Novel hormonal antagonists? Whoops, the model is not “well established” and needs to be “compared to the standard gonadectomy models”, LOL >sigh<.

The money and the progress.

Keep in mind, if you will, that there is always a more fundamental comparison or question at the root of the project, such as “does this drug compound ameliorate cocaine addiction?” So all the gender comparisons, designs and groups need to be multiplied against the cocaine addiction/treatment conditions. Suppose it is one of those cocaine models that requires a month or more of training per group? Who is going to run all those animals ? How many operant boxes / hours are available? and at what cost?

Oh, don’t worry bench jockeys. According to the NYT article:

Researchers who work with cell cultures are also being encouraged to study cells derived from females as well as males, and to do separate analyses to tease out sex differences at the cellular level.

“Every cell has a sex,” Dr. Clayton said in a telephone interview. “Each cell is either male or female, and that genetic difference results in different biochemical processes within those cells.”

“If you don’t know that and put all of the cells together, you’re missing out, and you may also be misinterpreting your data,” Dr. Clayton added. For example, researchers recently discovered that neurons cultured from males are more susceptible to death from starvation than those from females, because differences in the ways their cells process nutrients.

“Encouraged”. Okay, maybe you CultureClowns have an escape clause here. Animal model folks are facing “demanded” language.

Final observations are ridiculous:

But [the new policies] are likely to be met with resistance from scientists who fear increased costs and difficulty in performing their experiments. Studying animals of both sexes may potentially double the number required in order to get significant results.

“There’s incredible inertia among people when it comes to change, and the vast majority of people doing biological research are going to think this is a huge inconvenience,” Dr. Zucker said.

Margaret McCarthy, a neuroscientist at University of Maryland School of Medicine who studies sex differences, agreed. “The reactions will range from hostile — ‘You can’t make me do that’ — to ‘Oh, I don’t want to control for the estrous cycle,’” she said.

This has nothing to do with whether a scientist “wants” to or not.

Let me be clear, I want to do sex-differences studies. I am delighted that this will be a new prescription. I agree with the motivating sentiments.

What I “fear” is that grant applications will be kicked in the teeth if they include sex differences comparisons. What I “fear” is that my research programs will be even less productive on the main area of interest, to the tune of a lot of extra work that will simply confirm a lot of what we already know. For example, female rats tend to self-administer more drug than males do. A lot of my colleagues have been working on these topics for a long time. The identification of those areas where it actually matters (i.e., sex difference effects that haven’t yet been detected) are going to come along with a lot of negative findings. What I “fear” is that when we are interested in a certain thing, there is a bit of sex-differences literature and the hypothesis is going to be “males and females are the same” or even “females are more/less sensitive to drug” that this is going to bring down the holy hells of reviewer wrath over what hypothesis we are testing.

I fear a lot of things about this. What I don’t fear is my own interest in the topic. What I don’t fear is the “inconvenience”. I don’t even fear “difficulty”. It just isn’t that difficult to add female groups to my studies.

What it takes is additional grant funding. Or tolerance on the part of P&T committees, hiring committees and grant review panels for apparently reduced progress on a scientific topic of interest. And those things are not at all easy to come by.

The funny thing is, we’ve been taking steps in the lab toward this direction in the past year anyway. So I should be grateful I have at least that little tiny bit of a head start on this stuff.

21 Responses to “The NIH says investigators must incorporate sex-differences analyses in their studies”

  1. im not a statistician but i dont think that just doubling your n is enough. that would be enough if we wanted to confirm an effect in both sexes. but for a comparison we need much more stat power.

    that is, n in normal studies is presumably the minimum required to detect an effect, vs baseline of zero (for reduction in animal use reasons).

    and there’s no reason why detecting a sex difference would require the same n again.

    to distinguish effect sizes between male and female may require substantially more than doubling n if the size is small, as many sex differences will surely be.

    like, my hunch is, maybe 10x the number of subjects in many plausible cases?


  2. anonymous postdoc (shrewshrew) Says:

    I think the first goal is not to detect all sex diffs down to the ittiest bittiest ones, but to see whether the grand architecture of mechanism BSDs have concocted hold up even a little bit in female organisms and cells. Because if they don’t, the translational science is no longer translational, and the *basic* science we think we understand…we don’t.


  3. Dave Says:

    Grant reviewers will be instructed to take the sex balance of each study design into account when awarding grants

    What the fuck does that mean? Typical Collins bullshite. Do we just have to say that we will study males and females and that’s it? Easy enough to state that.

    What’s next from Collins? NIH-mandated lab dress codes? Micro-managing lunch breaks?


  4. Former Technician Says:

    Well, um, in vaccine research we already use females. The only time we use males is when we are using a breed that produces few offspring. We have current studies in mice, guinea pigs and non-human primates that are all in females.

    I do not look forward to doing immune duration studies in male animals. The animals get grumpy and aggressive after a few months of poking and prodding. Not looking forward to adding testosterone into the mix.


  5. anonymous postdoc (shrewshrew) Says:

    That’s a little bit of an offensive comparison. I don’t give a shit when you take a lunch break, I give a shit if the pharmaceutical I am being given is going to have the intended effect, or not.

    As I mentioned on Twitter, Debra Bangasser showed that intracellular signaling of CRF receptor 1 is utterly different in male and female brains. I saw a poster at SFN (2010? 2011? a few years ago) indicating something similar in cannabinoid receptor/s. The rate of male-biased neurodevelopmental disorders alone tell us that a cell’s complement of sex chromosomes dictates brain function and development prepubertally. This is all chronically understudied but the existing findings are sobering, and indicate that male and females cells are potentially very different.

    So that’s actually a super-duper double dog offensive comparison.


  6. Dave Says:

    How was that offensive? Because I don’t think the NIH should be telling us how to do our research when budgets are already stretched to the limit? What’s offensive is that they can be frustratingly passive in many important areas, but then they come out with stuff like this.

    And for the record, in our field, males and females are used in almost all human clinical trials. Human IRBs require strong justification for excluding males or females from studies, but perhaps you were not aware of that. There is a big world beyond rodent neuroscience studies, believe it or not.


  7. yikes Says:

    I think one can understand that there are sex-specific differences in biology, but at the same time concede that these differences will vary in their importance from one context to the next. Should every study measure the importance of sex differences? I don’t think it is chauvinistic to think this is a waste of resources for many projects. I don’t study brain biology, behavior, or anything else where sex differences have been shown to be anything bigger than fine tuning. If you really think the goal is to detect massive differences (“hold up even a little bit”) then these would be patently obvious if you included any number of males and females in the data set (which we do – literally “any number” as we randomize but do not even try to match them; maybe this introduces a degree of bias or noise, but the effects we measure FAR outweigh this).

    I think differences between species, or even mouse strains, far outweigh the biologic impact of XX vs XY in my field. And the biggest determinant of phenotype, by orders of magnitude, is the engineered (autosomal) genotype in our mutant strains.

    Bottom line – I think this is a bad case of central planning; more sex-specific research should be done, but not with a brush that is 100% broad.


  8. Wowchem Says:

    Lol, we work so hard, ie single gene changes to cancer cells and now crispr knockdowns, to make two cells exactly the same except the target protein. So throw another cell in there and it will have a different baseline IC50. What does it mean: female cells or any other cell has different membranes permeability, proteins, ect. It makes no sense in early discovery


  9. anonymous postdoc (shrewshrew) Says:

    “I think differences between species, or even mouse strains, far outweigh the biologic impact of XX vs XY in my field. ”

    I think this attitude (and you are by far not the only one to express it) is what is cheesing me off. Lots of people insisting that sex differences cannot play as large a role as some other manipulation. This is like arguing that the effect of a drug is going to be way more important than the time of day you give it. The entire field of circadian biology would argue that for many important systems, this is simply untrue. We don’t like to think that there’s no baseline, that all biological systems are in a constant state of flux, but that is the reality.

    Simply put, the resistance people are expressing to even testing for sex differences, and their insistence that *if* they did, they probably wouldn’t even find anything so why bother, is exactly the reason this mandate needed to be handed down from NIH. Leaving it up to the discretion of PIs was allowing a bunch of assumptions that sex differences don’t exist, aren’t important, and are “special cases” of biology to become received wisdom.

    “And the biggest determinant of phenotype, by orders of magnitude, is the engineered (autosomal) genotype in our mutant strains.”

    How can you be so sure if you haven’t tested it?


  10. Dave Says:

    This is primarily an animal study issue, as large clinical human trials most often study males and females together, although mechanistic intervention studies (n = 10 – 20/group) may use either males or females only for justifiable reasons. You are missing that point entirely.

    Rodent studies/findings are very rarely translated to the human anyway (this IS a much larger problem), so I would also argue that this ‘directive’ will not yield much in the way of improving human disease, ultimately. If one has the resources, it is worth doing, otherwise I think the value to humans of studying male and female mice and cells is limited.


  11. eeke Says:

    Has the NIH issued anything more than just a commentary? Is there anything official yet that mandates inclusion of both sexes in basic research studies? Just curious, since I am preparing another grant proposal for upcoming cycle.

    In one of my recent manuscripts, a reviewer criticized us for using only females. Can’t win.


  12. […] Drugmonkey has an excellent post about the latest <achtung!> command from LordHighCollins of the The NIH. […]


  13. Joe Says:

    I could imagine this being a small deal, like the questions that you have to answer on the human use forms or animal use forms currently.
    At study section after the grant has been discussed and just before the re-statement of scores, the chair asks the primary reviewer, “Are they using vertebrate animals, and is the use OK?” The usual answer is, “Yes, all 5 questions answered appropriately.” Then, “Are there humans, and are they OK?” The answer is something like “Yes. Males and females. OK. No children. OK. The codes are…” Then, “OK. Restate scores”. So if you explained your use of only women (as I do for certain studies), and your field is accepting of that for this purpose, then its just another paragraph you have to write.
    However, if it is as Neuro PB states in #1 above, and every phenotype is going to have to be tested in enough males and females to prove or disprove a difference, then it is going to be super expensive.


  14. yikes Says:

    @shrewshrew – I think you are assuming a lot about my assumptions.

    I include males and females in all my experiments – it’s cheaper that way. I study a phenotype that is lethal by age 3 months. *All* the mutant mice die, male or female. All the non-mutant mice live, male or female. I am not assuming the mutation is the biggest predictor of phenotypic variation, I am concluding it based on data, even if not subjected to a formal statistical analysis.

    If I powered a study appropriately, and asked the question, maybe I could determine if sex influences the phenotype – maybe it changes lifespan by 1-2 weeks? (there is not enough spread in the pooled data for it to be anything bigger). Would that matter? Certainly not to me. I am interested in the lethality, which cuts across sexes. The biggest effect I could possibly see would be too small to investigate any further. Not because it is based on sex. Because it would be too small an effect.

    Similarly, we can avert lethality with a drug. All the treated mutant mice live; all the untreated mutant mice die (see! it’s reproducible!). The response rate is 100%, and I can see this even if I have n=3M/7F in one group and 6M/4F in the other group. If I wanted to determine rigorously whether the sex influenced the severity of the phenotype in treated or untreated mice, I would have to use a lot more mice. And the difference would not be large – which I define as life vs death.

    I guess I am saying is that informal, weakly powered experiments are still enough to reveal *huge* differences if they exist, and since I have those to work on, I want to figure them out first before looking into other real, but more modest effects beyond the mutation and drug at hand. Strain effects are in this category too – very real and meaningful for translation (which strain is the best model for outbred humans?), but I should have the leeway to focus on larger effects that outweigh these impacts in early studies.

    I am not asserting that this framework is applicable to all animal experiments. YMMV. But I think it is inappropriate to assume I am likely to be missing an important difference.

    This (proto-) edict should be subject to cost-benefit analysis, post-hoc if nothing else.


  15. SidVic Says:

    Stuff like this makes me despair for the future of western civilization. Must everything now be politicized? A couple of points;

    Does this mean that the initiatives (PAs etc..) that focus/support upon gender differences research will go by the wayside? Now that they are essentially forcing, via mandate, everyone into area. I agree with DM that sex differences are fascinating, and offer full throated support for gender studies and comparative physiology, but that is not what we are dealing with here.

    I study protein handling/folding and in my humble opinion it is quite unlikely that sex differences exist in the parameters that I examine. Of course, that is the real root of the issue. My judgment can’t be trusted. Hell, left to my own devices I would be cruel to mice(hence the onerous regulatory scheme in place). Meanwhile joe sixpack can trot down to hardware store and buy gluetraps. Due to my sexist blinders I would ignore potential gender differences in aliphatic helix formation.

    Of course, this nonsense will just require another workaround. It’s not that hard to insert “female and male derived cell lines will also be compared in this series of studies”. Cell lines, really? I, for one, have grown tired of navigating this environment. Essentially they have made a liar of everyone, at least anyone who wants to actually accomplish something.

    Sry for rant, consider my ass frosted.

    PS- I know I may sound nutty, but I reallly believe that this is trickle down from the war against women meme promulgated by you know who.


  16. becca Says:

    Honest question- does anyone have two neuro cell lines from the same *type* of cell, that are cultured in the same fashion, from a male and a female individual?

    I like my cell lines. But I like them *whilst keeping in mind* that they are not only *not* models for organisms, but also not even particularly accurate models of a very specific type of cell found in an actual organism- the process of being in culture makes them freakish.

    @Dave- actually, when you take a lunch break probably does matter. Circadian rhythm effects on some of this stuff ought to at least be allowed for (i.e. the time of day you do your experiment is a crucial variable in much animal work).

    @yikes- I think in many cases, the differences between strains are indeed far greater than those between sexes. For example, even in the case of lifespan (where there are dramatic sex effects in humans), the interstrain differences far exceed the sex differences (which often aren’t statistically significant),4&brieflooktool=&projhint=Yuan2
    Ditto for EtOH preference;,4&projhint=Finn1

    (as an aside: I notice that many of the strains have dramatic differences between sexes, while others have no difference)

    That said, it pays to never assume too much that the engineered difference is the key difference, even when you have “all the mutants die” phenomenons (all the mutants die WHEN and of WHAT?)…


  17. DJMH Says:

    It seems odd that with the power of the NIH budget at his disposal, Collins adds a bureaucratic step that will only encourage people to do underpowered meaningless t-tests on M v F. Why not put out some more PAs or RFAs specifically on sex differences? Scientists will flock to the grant dollars, and actually be interested in the results as opposed to the rest of us who just try to get an N of 4 female mice so we can claim to have complied.

    IOW, use the carrot, not the stick.


  18. drugmonkey Says:

    I agree DJMH. Just say every legit sex-comparison app gets 5 extra percentile points on the payline and see what happens.


  19. AcademicLurker Says:

    Just say every legit sex-comparison app gets 5 extra percentile points on the payline and see what happens.

    It seems to me that a policy like that would have a lot of potential complications.

    It might be fine for you mouse jockeys, but how does a crystallographer include “legit sex-comparisons” in a proposal?


  20. drugmonkey Says:

    If your work doesn’t compare sex differences, no problem, you just get an unchanged regular payline. Simple.


  21. […] there are a lot of factors. They aren't all that complicated either, since they boil down to scientists who want to so […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: