Thoughts on the NIH policy on SABV

There is a new review by Shansky and Murphy out this month which addresses the NIH policy on considering sex as a biological variable (SABV).

Shansky, RM and Murphy, AZ. Considering sex as a biological variable will require a global shift in science culture. Nat Neurosci, 2021 Mar 1. doi: 10.1038/s41593-021-00806-8. Online ahead of print.

To get this out of the way, score me as one who is generally on board with the sentiments behind SABV and one who started trying to change my own approach to my research when this first started being discussed. I even started trying to address this in my grant proposals several cycles before it became obligatory. I have now, as it happens, published papers involving both male and female subjects and continue to do so. We currently have experiments being conducted that involve both male and female subjects and my plan is to continue to do so. Also, I have had many exchanges with Dr. Shansky over the years about these issues and have learned much from her views and suggestions. This post is going to address where I object to things in this new review,for the most part, so I thought I should make these declarations, for what they are worth.

In Box 1, the review addresses a scientist who claims that s/he will first do the work in males and then followup in females as follows:

“We started this work in males, so it makes sense to keep going in males. We will follow up with females when this project is finished.” Be honest, when is a project ever truly finished? There is always another level of ‘mechanistic insight’ one can claim to need. Playing catch-up can be daunting, but it is better to do as much work in both sexes at the same
time, rather than a streamlined follow-up study in females years after the original male work was published. This latter approach risks framing the female work as a lower-impact ‘replication study’ instead of equally valuable to scientific knowledge.

This then dovetails with a comment in Box 2 about the proper way to conduct our research going forward:

At the bare minimum, adhering to SABV means using experimental cohorts that include both males and females in every experiment, without necessarily analyzing data by sex.

I still can’t get past this. I understand that this is the place that the NIH policy on SABV has landed. I do. We should run 50/50 cohorts for every study, as Shansky and Murphy are suggesting here. I cannot for the life of me see the logic in this. I can’t. In my work, behavioral work with rats for the most part, there is so much variability that I am loathe to even run half-size pilot studies. In a lot of the work that I do, N=8 is a pretty good starting size for the minimal ability to conclude much of anything. N=4? tough, especially as a starting size of the groups.

The piece eventually gets around to the notion of how we enforce the NIH SABV policy. As I have pointed out before and as is a central component of this review, we are moving rapidly into a time when the laboratories who claim NIH support for their studies are referencing grant proposals that were submitted under SABV rules.

NOT-OD-15-102 appeared in June of 2015 and warned that SABV policy would “will take effect for applications submitted for the January 25, 2016, due date, and thereafter“. Which means grants to be reviewed in summer 2016, considered at Council in the Fall rounds and potentially funded Dec 1, 2016. This means, with the usual problems with Dec 1 funding dates, that we are finishing up year 4 of some of these initial awards.

One of the main things that Shansky and Murphy address is in the section “Moving forward-who is responsible?“.

whether they have [addressed SABV] in their actual research remains to be seen. NIH grants are nonbinding, meaning that awardees are not required to conduct the exact experiments they propose. Moreover, there is no explicit language from NIH stating that SABV adherence will be enforced once the funds are awarded. Without accountability measures in place, no one is prevented from exclusively using male subjects in research funded under SABV policies.

Right? It is a central issue if we wish to budge the needle on considering sex as a biological variable. And the primary mechanism of enforcement is, well, us. The peers who are reviewing the subsequent grant applications from investigators who have been funded in the SABV era. The authors sortof mention this: “Researchers should be held accountable by making documentation of SABV compliance mandatory in yearly progress reports and by using compliance as a contingency for grant renewals (both noncompetitive and competitive).” Actually, the way this is structured, combined with the following sentence about manuscript review, almost sidesteps the critical issue. I will not sidestep in this post.

We, peer scientists who are reviewing the grant proposals, are the ones who must take primary responsibility to assess whether a PI and associated Investigators have made a good faith attempt to follow/adopt SABV policy or not. Leaving this in the hands of Program to sort out, based on tepid review comments, is a dereliction of duty and will result in frustrating variablity of review that we all hate. So….we are the ones who will either let PIs off the hook, thereby undercutting everything NIH has tried to accomplish, or we will assist NIH by awarding poor scores to applications with a team that has not demonstrably taken SABV seriously. We are at a critical and tenuous point. Will PIs believe that their grants will still be funded with a carefully crafted SABV statement, regardless of whether they have followed through? Or will PIs believe that their grant getting is in serious jeopardy if they do not take the spirit of the SABV policy to heart? The only way this is decided is if the peer review scores reward those who take it seriously and punish those who do not.

So now we are back to the main point of this post which is how we are to assess good-faith efforts. I absolutely agree with Shansky and Murphy that an application (competing or not) that basically says “we’re going to follow up in the females later“, where later means “Oh we didn’t do it yet, but we pinky swear we will do it in this next interval of funding” should not be let off the hook.

However. What about a strategy that falls short of the “bare minimum”, as the authors insist on in Box 2, of including males and females in 50/50 proportion in every experiment, not powered to really confirm any potential sex difference?

I believe we need a little more flexibility in our consideration of whether the research of the PI is making a good faith effort or not. What I would like to see is simply that male and female studies are conducted within the same general research program. Sure, it can be the 50/50 group design. But it can also be that sometimes experiments are in males, sometimes in females. Particularly if there is no particular sense that one sex is always run first and the other is trivially “checked, or that one sex dominates the experimental rationale. Pubs might include both sexes within one paper, that’s the easiest call, but they might also appear as two separate publications. I think this can often be the right approach, personally.

This will require additional advocacy, thinking, pushback, etc, on one of the fundamental principles that many investigators have struggled with in the SABV era. As is detailed in Box 1 and 2 of the review, SABV does not mean that each study is a direct study of sex differences nor that every study in female mammals becomes a study of estrous cycle / ovarian hormones. My experience, as both an applicant and a reviewer, is that NIH study section members often have trouble with this notion. There has not been, in my experience on panels, a loud and general chorus rebutting any such notions during discussion either, we have much ground still to cover.

So we will definitely have to achieve greater agreement on what represents a good faith effort on SABV, I would argue, if we are to advocate strongly for NIH study sections to police SABV with the firm hand that it will require.

I object to what might be an obvious take-away from Shansky and Murphy, i.e., that the 50/50 sample approach is the essential minimum. I believe that other strategies and approaches to SABV can be done which both involve full single-sex sample sizes and do not require every study to be a direct contrast of the sexes in an experimentally clean manner.