Maybe the problem in NIH grant award is in topic diversity?

January 29, 2014

I had a thought occur to me over the past few days. It’s been growing along at the back of my mind and is only partially crystallized.

What if PIs of a given class of interest, whether that be sex, ethnicity, nation of origin or whatever, are not randomly distributed across the various topic domains supported by the NIH? What if a PI of characteristic X tends to work on Topic B using Model M whereas a PI of characteristic Y tends to work on Topic A using Model H?

What if the funding rates for Topic X differed from those for Topic Y? Or if applications using Model M consistently succeeded differently compared with applications using Model H?

I didn’t see any covariates for topic domain or even the funding IC in the Ginther report.

Surely someone at NIH is thinking about this. Surely?

I have two anecdotes for your consideration.

First, as with many areas of science, the ones dear to me suffer from a sex bias. There is a huge tendency to do the animal studies in male animals. Any study using female animals is very frequently a sex comparison study and is proposed explicitly or implicitly as a comparison with the default, i.e. male. I’ve talked about this before. The NIH also takes pains to fix the generalized reluctance via their most functional technique, the call for applications for a dedicated pool of money. In theory, the awarding of grants on sex-differences or on issues specific to women’s health will then spur additional work. Perhaps create a sustained program or even a career of work on this topic.

My anecdote is that I’ve noticed over the years (possible confimation bias here) that women in my field have a greater representation than men in these sorts of studies. Sex-differences models and womans’ health issues in my fields of interest seem to have women as the driving investigators more often than their overall representation.

If this generalizes, then we will want to know if the competitive success of such grant applications because of topic is contaminating our estimation of women PI’s success.

The second anecdote is older and comes from my long history participating on the “Diversity” committees of various academic institutions. Back in the dark ages I recall an incident where a Prof in the experimental sciences had to go to war with a Dean who was in charge of undergraduate summer research funds for underrepresented individuals. The Prof had a candidate who wanted to work in the experimental science, but the awards were generally being made to kids who wanted to work on academic topics related to underrepresented groups. The Dean thought this was the most important thing to do. In this case the prof won his battle in the second year of trying, over the objections of the Dean. I keep in touch with some of my undergraduate professors and I can say that said undergrad went on to become a NIH funded investigator (who still fails to work on issues directly related to underrepresentation). I have no idea if any of the other underrepresented summer research students went on to glorious academic careers in their respective disciplines, perhaps they did. But this is not the point. The point is that perhaps I am a little too glib about the pipeline implications of Ginther. Perhaps the grooming of underrepresented minority undergrads for a career in academics is itself not topic neutral. And the shaping and shifting from that very early stage may dictate field of study and therefore the eventual success rate at the NIH game.

Assuming, of course, that Topic X enjoys differential success rate from Topic Y when the grants are under review at the NIH.

__
Doctoral Degrees to African Americans by topic

36 Responses to “Maybe the problem in NIH grant award is in topic diversity?”

  1. Former Technician Says:

    The first thought that comes to my mind is that we rarely work with male mice. Our vaccine work is done on females exclusively. I cannot recall in previous labs that I have worked on in molecular or microiology ever working with male mice. Perhaps this is a difference in the disciplines?

    Like

  2. Dr Becca Says:

    So I just did a fun little PubMed game.

    “sex differences”[Title/Abstract]) AND “Nature”[Journal] (repeated for Science and Cell)

    The results?

    Nature: 27 since 1947, or one every 2.5 years (but the journal’s been around since 1869)
    Science: 48 since 1904, or one every 2.3 years
    Cell: 1. Literally one paper in the journal’s 40-year history.

    By comparison, putting in “epigenetic” for Nature turns up over 200 pubs in the last 35 years, i.e. over a tenfold difference. Somehow I’m not surprised that no dudes are doing sex differences research.

    Like

  3. drugmonkey Says:

    The fact that there are a total of 17,117 returns for sex differences and 31,088 for epigenetic is a little surprising, I will admit. Did you check out those “girls can’t tell time” Science pubs from 1904 and 1905?

    Like

  4. sciencedude Says:

    “What if PIs of a given class of interest, whether that be sex, ethnicity, nation of origin or whatever, are not randomly distributed across the various topic domains supported by the NIH? What if a PI of characteristic X tends to work on Topic B using Model M whereas a PI of characteristic Y tends to work on Topic A using Model H?
    What if the funding rates for Topic X differed from those for Topic Y? Or if applications using Model M consistently succeeded differently compared with applications using Model H? ”

    DM, you are venturing into dangerous territory! People have been brought before the inquisition for less heretical ideas than the questioning of the foundation of our faith that racial prejudice must be behind all occurrences of racial disparity! Do not forget what happened to Galileo!

    Like

  5. Ola Says:

    So my N=1 anecdote on this is cardiovascular risk differentials between African American and Caucasian populations. The result would appear to contradict your hypothesis – most of the people active in this research area are not themselves African Americans. There is a concentration of such research in the deep south, which has both higher CVD levels as well as larger AA populations to study, but the researchers doing it are largely white.

    Like

  6. anonymous postdoc Says:

    Yes. A friend of mine (not in neuroscience noted) to me a few years ago how puzzled he was that some of the coolest findings he could think of weren’t in CNS or even Genome Biology – they were in very specialized journals, because their tissue of interest was the placenta. Obviously the placenta is for girls. Except when placental defects cause the male fetus to experience disproportionate harm – I notice that sometimes gets people’s attention.

    The saving grace is that publishing and funding are not driven by the same people or goals. The big journals may not want to publish sex differences, but NIMH had a whole administrative supplement just to please do the same R01 experiments you proposed in males in some females too pretty please?

    I know a lab which has been pretty decently funded lately to do experiments on a girl topic (thus this work represents a high research priority to NIH and foundations, considering general funding rates). However, when submitting these same experiments and their fantastic results, even society-level journals tell them ‘this is good quality but low priority work, so even though you could address these comments, just drop dead.’

    Like

  7. flyoverprof Says:

    I know a lab which has been pretty decently funded lately to do experiments on a girl topic (thus this work represents a high research priority to NIH and foundations, considering general funding rates). However, when submitting these same experiments and their fantastic results, even society-level journals tell them ‘this is good quality but low priority work, so even though you could address these comments, just drop dead.’

    This is precisely why the C/N/S paper requirement for hiring is crazy. Restricting yourself to hiring in overcrowded topics is foolish from a grant-getting standpoint, and for many NIH-funded departments, this should be their primary concern going forward. This represents an arbitrage opportunity, but I don’t know of anyone (explicitly) doing ‘unsexy’ field hires.

    Like

  8. girlparts Says:

    Speaking of placentas, do you know that there is only one NIH study section devoted to “Pregnancy and Neonatology?” That is miscarriages, pre-term labor, extended labor, pre-eclampsia, gestational diabetes, fetal biology (growth, development, metabolism, pharmacology, infection), and neonatology (jaundice, SIDS, low birth weight etc.). Considering that about 80% of women will be pregnant in their lifetime, and we’re all born to one, that seems like an astonishing lack of emphasis on this “girl topic” to me.

    Like

  9. DrugMonkey Says:

    Somehow I expect grants on those topics get reviewed elsewhere as well. Have you checked RePORTER for funded grants?

    Like

  10. girlparts Says:

    I don’t know if there is a good way to look up grants by study section as opposed to awarding institute, but of course you’re right – gestational diabetes can be reviewed by a diabetes-focused section, or fetal brain development by a neuro section. Still, that is the only section that explicitly looks at pregnancy complications and outcomes.

    We all think our own study area is the most important thing ever and woefully underfunded. Still, looking at the outcry over insurance being forced to cover childbirth, I think we have a weird cultural idea that pregnancy is an obscure niche, and it even affects in biomedical science.

    Like

  11. . Says:

    Interesting topic as I got manuscript reviews in just last week in which we had used only female rats and the reviewer demanded that we fully justify the use of only females – in a research area in which women are the most rapidly growing subgroup.
    I also note that one of my male colleagues quipped “if sex differences were important we would know about them already”, so there is no reason to fund the (my) work.

    Like

  12. drugmonkey Says:

    I don’t know if there is a good way to look up grants by study section as opposed to awarding institute

    There absolutely IS. RePORTER has a field. But what you should do is look up by keyword and then start looking at individual grants and see what study section funded it.

    Like

  13. girlparts Says:

    Thanks for the RePORTER tip. I’m going to go find some more places to send grants.

    Like

  14. anonymous postdoc Says:

    To more directly address the focus of the post, the best way to ensure that URM trainees are going to be successful is to direct them to the most overtly “important” topics in the field. If they are working on something everyone can agree is big, it is harder to dismiss their research contributions, which is what Ginther is about. Essentially funneling them into the pipeline of working on URM-relevant topics is to shunt them into a research ghetto.

    As the above discussion on girl topics demonstrates, if the field at large has decided that your research is on “special cases” (like the 50% of the population which is female, or the 100% of the population which has previously gestated or is currently gestating), then being disadvantaged in any other way is the death sentence for your prestige, if not your career.

    Like

  15. DrugMonkey Says:

    Before you go charging off note that I am speculating. It would require some data to confirm….

    Like

  16. Dr Becca Says:

    Here is some data – the list of officers for the Organization for the Study of Sex Differences.

    Currently the president is male, but previous and president elect are women. Out of the remaining 17 officers, 15 are women.

    Like

  17. Joe Says:

    Dr. Becca, I don’t know how big a part it plays, but some of the federally-funded women’s health centers have as part of their mission the promotion of women scientists. Thus they preferentially fund women for their projects.

    Like

  18. DrugMonkey Says:

    That echoes my second anecdote Joe. It’s a great tactical goal…huge in fact. But *if* there are unanticipated consequences then maybe some tweaking is in order.

    Like

  19. The Other Dave Says:

    Why does everything have to be some nefarious conspiracy? Do you work with mammals? Females have estrus cycles, which unless controlled for can add immense variability to results. So you can measure from males and be done, or measure from females and have everyone ask you whether it’s a cycle-dependent effect, or you can measure from males and females and have no direct comparison and differing effects. If you’re just figuring out the initial biology, which you gonna choose?

    As for different fields having different sex ratios and racial mixes… uh… duh. What, did you go to your first interdepartmental party?

    Like

  20. Dr Becca Says:

    So you can measure from males and be done

    Ah, but there’s the rub. Are you actually done? Is your work funded by NIH? Do female humans make up 50% of the population whose health NIH seeks to improve?

    The answer is no, you are not done. It sucks that female mammals with their pesky fluctuating hormones create such an inconvenience for your experimental design. But let’s not kid ourselves that once you’ve got your data in males, you’ve the problem in a truly translatable way.

    Like

  21. anonymous postdoc Says:

    Jesus fuck, there should be a “Why I don’t study females or sex differences” bingo card.

    It would definitely have “but they’re so vaaaaaaaaariable!” and “why don’t we just figure out the biology first?”.

    The Other Dave, do you study mammals? You know about testes? It turns out that a significant source of variability in males on a variety of tasks, including anxiety-like behavior, learning and memory, and for SHIZ social behavior is the fucking social dominance hierarchy that develops in the cage of group housed males? Which is communicated in large part by variation in androgen levels, such as testosterone?

    (Oh, and if your animals are singly housed, you know they are as depressed as fuck, right? Because they are social? So you don’t win by that method either.)

    I am going to start asking in every paper I review whether the authors controlled for the influence of circulating sex hormones by carefully assessing the dominance hierarchy in their cage, or by gonadectomizing and replacing their males with various levels of T, and if they didn’t their results are uninterpretable. HAHA LOLOL SIKE I’m not going to do that, because I am just a postdoc and if I started pulling that shit no editor would ask my crazy ass to review anything again. Yet this exists as an accepted argument supporting the idea that females are too difficult to study.

    With regards to the “let’s just figure out the biology first (in the default sex, i.e. doodz)” argument: I have some really cool data right now, where a particular manipulation has made my males fucking stupid as shit, but the females are one hundred percent little Rosalind Franklin geniuses. Even you, TOD, can see the implication, but I will spell it out for you anyway:

    If I went about “just figuring out the biology” or only behaviorally testing males, I might have concluded that this manipulation is fundamentally damaging to the function of the brain region supporting this behavior, rather than communicating severe risk to males only, based on absolutely no evidence. You can imagine that these kinds of findings might have pertinent implications to a number of sex-biased mental disorders.

    Like

  22. The Other Dave Says:

    @Becca: The NIH is not trying to cure mice either. But there we are.

    @anonymous postdoc: Fair enough. You win.

    Like

  23. Dr Becca Says:

    @anonymous postdoc: I love you.

    Like

  24. The Other Dave Says:

    @DM: When you talk about diversity, which are super important topics, your posts often have an ‘us vs them’ tone. I get the impression that you think it’s obvious what needs to be done: If stupid grey beard racist misogynists would just get out of the way, the world would be all fixed. Really? Is it that simple?

    According to my high school physics book it is easy to build a fusion generator too, so why are the oil companies keeping the world from cheap endless energy? It must be a conspiracy, right? That’s the way you’re coming across.

    There are an awful lot of very smart very successful people who have researched diversity and gender equality issues, thought hard about them, and are working hard to implement change. There is a giant literature on the topic, written by genuine experts (you can start in Google Scholar). There are whole journals devoted to research on the topic (NWSA, Hypatia…), and a lot of books (social scientists write a lot of books). You might find some of it fascinating and informative.

    You’ve got a great readership. Over the last decade, DM, you’ve built your blog into a valuable platform. I think you’ve spent yourself when it comes to grantsmanship. Maybe you could now give a little summary of something from the diversity literature each week, raising some issues, prompting discussion. Sort of like a diversity journal club. I am sure that your local WISEST rep or Gender & Women Studies Program rep could make some good suggestions, if you can’t find good stuff on your own. You know those profiles of drug researchers you used to do for a while? How about profiles of highly successful minority & women scientists? You could even interview them about the topic!

    Like

  25. DrugMonkey Says:

    90% of my posts have an “us vs them” tone dude. Do you only notice when you feel like you aren’t in the “us”?

    OTOH, I think I make considerably more suggestions beyond getting some class of person to “get out of the way”.

    Like

  26. Alex Says:

    I want to know which high school physics book said that fusion generators are easy to build.

    Like

  27. DrugMonkey Says:

    TOD is seriously confused over the difference between “conspiracy” and “entirely predictable emergent property”

    Like

  28. The Other Dave Says:

    Everyone but DM who doesn’t want to remain willingly ignorant on these issues is encouraged to do some reading, just like you would for any other subject.

    In the mean time, DM is going to wait for Dr. Isis to start blogging intelligently about the subject. Dr. Isis might be feeling a little inconsequential right now, but she is a strong and wonderful person and will use her fire to show people what she can do, especially since she has a blog that is an inspiration to many young female scientists.

    If Isis hasn’t already, she could march over to Molly Carnes’ office. She could ask Molly about some of the literature and fascinating work in this area. Isis can then use her blog as a platform to change the world. Eventually, Nature will profile Isis as a world changer. But in the mean time, DM is going to read what Dr. Isis learns and writes about, because he is unwilling to actually read the literature for himself. Maybe DM will believe Dr. Isis.

    DM is lazy because he thinks he knows all the answers already. He would rather spout opinions than humble himself and learn anything about a very active field of intellectual activity populated by thousands of people. None of whom he seems to know or care about. This is an attitude typical of people who despite their own best intentions, unconsciously enforce the status quo.

    Like

  29. DrugMonkey Says:

    I note you don’t have any specifics as to where I am wrong. You simply point to your preferred straw man version of me, contaminated by guilt or confirmation bias of your own. Try harder if you have an actual point.

    Nobody is claiming this shit is easy.

    Like

  30. The Other Dave Says:

    DM: You imply, in your post above, that no one at NIH is thinking about things like association between subfield funding rates and racial/gender participation. Are you kidding me? Next time you sit down to write about how totz unjust the wurld iz to make the girls smile at you, at least fucking do a Google or Quertle search.

    Since you like reports, you can start by reading this one, also conveniently available online:

    Click to access Gender_Differences_in_NIH_grants_report.pdf

    You will note, if you make it as far as the Preface, that it is REQUIRED BY LAW, as part NSF’s reauthorization, to “examine differences in amounts requested and awarded, by gender, in major Federal external grant programs.” That includes NIH.

    When you are ready to start tackling the primary social studies literature, you can begin with this classic:

    Click to access Merton_Science_1968.pdf

    I think you will like that paper, because it draws together the common threads of many of your posts, including bias against young investigators, women, and scientifically underrepresented races.

    The modern, gender-specific version, of that is the Matilda effect. This example will get you fired up (and laid, if that’s your goal, which I am beginning to think is the extent of it):
    http://sss.sagepub.com/content/42/2/307.short

    But there are a LOT more nuanced, data-driven, and informative works out there. The aforementioned Molly Carnes has written some nice things, if you want to stay within the small circle of fellow biomedical scientists who write about this stuff. This is what she’s thinking about lately:
    http://projectreporter.nih.gov/project_info_description.cfm?icde=0&aid=8548830

    But wait. I know. You’re going to say that I’m forgetting the point. I was supposed to prove that you were wrong about NIH not thinking about this stuff.

    First, it’s important to note that NIH has a more narrow scientific mandate than NSF, which is very active in supporting this area, in many ways, including but not limited to:
    http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5383
    http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5475

    So most of the work about bias in NIH funding might actually be supported by NSF. Also, when NIH focuses on disparities, it’s focused on health disparities. Which honestly to me seems reasonable. Who cares if a minority PI doesn’t get a grant when 100,000 minority children are dying because of inadequate access to health care? There are craploads of RFAs for info on racial and ethnic discrimination preventing adequate health care.

    But back to the question at hand… Does anyone at NIH think about bias?

    Is this proof that at least one person at NIH thinks about this stuff?

    Click to access NIH_Update_Women_in_Science_V5_I6.pdf

    Gasp. She might actually read things instead of blog opinions.

    I thought you read Rock Talk, DM:
    http://nexus.od.nih.gov/all/2011/08/18/new-nih-study-on-diversity/
    Oooh, you do. You commented on that very post, and pointed out that scientists probably have higher IQ than other people. No bias in you, dude. Clearly.

    So what. A couple newsletters.

    WTF is this? An NIH task force on the status of women in science?

    Click to access OIRSecondTaskForceSummary.pdf

    A SECOND one, no less!

    Probably just a committee. They don’t do ‘nuthin, surely.

    Click to access AllRecommendations.pdf

    Damn.

    Got it, DM? Your blog entries would be a lot more impressive if they showed that you had at least a little actual curiosity about the subject.

    Are YOU biased?

    https://implicit.harvard.edu/implicit/research/

    Every reader should try that last link. Fascinating stuff.

    Now, I’ve got a superbowl party to go to.

    Like

  31. drugmonkey Says:

    You imply, in your post above, that no one at NIH is thinking about things like association between subfield funding rates and racial/gender participation.

    I implied no such thing. In fact I assume that there are indeed people thinking. My criticisms of the NIH have more to do with the institutional inertia and the fact that when push comes to shove the institution prefers to remain as ignorant as possible and to seek nonsolutions that only appear useful. This says nothing whatsoever about the poor folks at the NIH who are trying to do the right thing. After all the Gither report involved many NIH folk.

    there are a LOT more nuanced, data-driven, and informative works out there

    so? are you unclear on what a blog is? after all this time hanging about here and commenting?

    Does anyone at NIH think about bias?

    Why do you so insist on the straw man?

    Your blog entries would be a lot more impressive if they showed that you had at least a little actual curiosity about the subject.

    Imagine my concern with your opinion.

    Are YOU biased?

    Yes. As is everyone. Kindof a central theme around these parts, actually. The key is in coming up with structural fixes that accept this invariant reality instead of pretending it magically doesn’t exist because we don’t want it to for certain tasks. Like NIH grant review.

    Every reader should try that last link. Fascinating stuff.

    Yes they should and yes it is. You post that like it hasn’t appeared on this blog with frequency.

    Like

  32. The Other Dave Says:

    Har. Brilliant rebuttal, DM. For the audience out there: DM’s clever riposte would be known as “argument ad hominem“. It is typically only used on the internet and in fights among pre-pubescent children. It doesn’t work anywhere else.

    Like

  33. drugmonkey Says:

    I think you need to go look up “ad hominem” dude. not even close.

    Like

  34. The Other Dave Says:

    DM, you should know that I am totally smiling through all this. I have been an on-and-off reader of your blog since almost the beginning, back when PP and BM were regular contributors. Obviously I wouldn’t keep checking in if I really thought everything you had to say was shallow blather. Seriously: Kudos to you for this site.

    But… Dude! We are totally talking past each other in this thread. I don’t get it. Totally baffles me. But it doesn’t matter. What matters is the issue. I may be an annoying asswipe, but the stuff I am trying to get you to look at really is good info. Seriously. Check it out. At least my first and last links. The report is informative, the implicit bias work is fascinating.

    Like


  35. […] has a well-documented problem with racial biases in funding rates. Here is an intriguing idea: maybe the lower level of diversity in those who are funded by NIH might reflect a lower diversity in… of proposals. Maybe scientists from the underfunded groups are interested in different stuff that […]

    Like


Leave a comment