Disparity of NSF funding

You are familiar with the #GintherGap, the disparity of grant award at NIH that leaves the applications with Black PIs at substantial disadvantage. Many have said from the start that it is unlikely that this is unique to the NIH and we only await similar analyses to verify that supposition.

Curiously the NSF has not, to my awareness, done any such study and released it for public consumption.

Well, a group of scientists have recently posted a preprint:

^{Chen, C. Y., Kahanamoku, S. S., Tripati, A., Alegado, R. A., Morris, V. R., Andrade, K., & Hosbey, J. (2022, July 1). Decades of systemic racial disparities in funding rates at the National Science Foundation. OSF Preprints. July 1. doi:10.31219/osf.io/xb57u.}

It reviews National Science Foundation awards (from 1996-2019) and uses demographics provided voluntarily by PIs. They found that the applicant PIs were 66% white, 3% Black, 29% Asian and below 1% for each of American Indian/Alaska Native and Native Hawaiian/Pacific Islander groups. They also found that across the reviewed years, the overall funding rate varied from 22%-34%, so the data were represented as the rate for each group relative to the average for each year. In Figure 1, reproduced below, you can see that applications with white PIs enjoy a nice consistent advantage relative to other groups and the applications with Asian PIs suffer a consistant disadvantage. The applications with Black PIs are more variable year over year but are mostly below average except for 5 years when they are right at the average. The authors note this means that in 2019, there were 798 awards with white PIs above expected value, and 460 fewer than expected awarded with Asian PIs. The size of the disparity differs slightly across the directorates of the NSF (there are seven, broken down by discipline such as Biological Sciences, Engineering, Math and Physical Sciences, Education and Human Resources, etc) but the same dis/advantage based on PI race remains.

Fig 1B from Chen et al. 2022 preprint

It gets worse. It turns out that these numbers include both Research and Non-Research (conference, training, equipment, instrumentation, exploratory) awards. Which represent 82% and 18% of awards, with the latter generally being awarded at 1.4-1.9 times the rate for Research awards in a given year. For white

Fig 3 from Chen et al 2022 preprint FY 13 – 19;
open = Non-Research, closed = Research

PI applications the two types both are funded at higher than the average rate, however significant differences emerge for Black and Asian PIs with Research awards having the lower probability of success.

So why is this the case. Well, the white PI applications get better scores from extramural reviewers. Here, I am not expert in how NSF works. A mewling newbie really. But they solicit peer reviewers which assign merit scores from 1 (Poor) to 5 (Excellent). The preprint shows the distributions of scores for FY15 and FY16 Research applications, by PI race, in Figure 5. Unsurprisingly there is a lot of overlap but the average score for white PI apps is superior to that for either Black or Asian PI apps. Interestingly, average scores are worse for Black PI apps than for Asian PI apps. Interesting because the funding disparity is larger for Asian PIs than for Black PIs. And as you can imagine, there is a relationship between score and chances of being funded but it is variable. Kind of like a Programmatic decision on exception pay or the grey zone function in NIH land. Not sure exactly how this matches up over at NSF but the first author of the preprint put me onto a 2015 FY report on the Merit Review Process that addresses this. Page 74 of the PDF (NSB-AO-206-11) has a Figure 3.2 showing the success rates by average review score and PI race. As anticipated, proposals in the 4.75 (score midpoint) bin are funded at rates of 80% or better. About 60% for the 4.25 bin, 30% for the 3.75 bin and under 10% for the 3.25 bin. Interestingly, the success rates for Black PI applications are higher than for white PI applications at the same score. The Asian PI success rates are closer to the white PI success rates but still a little bit higher, at comparable scores. So clearly something is going on with funding decision making at NSF to partially counter the poorer scores, on average, from the reviewers. The Asian PI proposals do not have as much of this advantage. This explains why the overall success rates for Black PI applications are closer to the average compared with the Asian PI apps, despite worse average scores.

Fig 5 from Chen et al 2022 preprint

One more curious factor popped out of this study. The authors, obviously, had to use only the applications for which a PI had specified their race. This was about 96% in 1999-2000 when they were able to include these data. However it was down to 90% in 2009, 86% in 2016 and then took a sharp plunge in successive years to land at 76% in 2019. The first author indicated on Twitter that this was down to 70% in 2020, the largest one year decrement. This is very curious to me. It seems obvious that PIs are doing whatever they think is going to help them get funded. For the percentage to be this large it simply has to involve large numbers of white PIs and likely Asian PIs as well. It cannot simply be Black PIs worried that racial identification will disadvantage them (a reasonable fear, given the NIH data reported in Ginther et al.) I suspect a certain type of white academic who has convinced himself (it’s usually a he) that white men are discriminated against, that the URM PIs have an easy ride to funding and the best thing for them to do is not to declare themselves white. Also another variation on the theme, the “we shouldn’t see color so I won’t give em color” type. It is hard not to note that the US has been having a more intensive discussion about systemic racial discrimination, starting somewhere around 2014 with the shooting of Michael Brown in Ferguson MO. This amped up in 2020 with the strangulation murder of George Floyd in Minneapolis. Somewhere in here, scientists finally started paying attention to the Ginther Gap. News started getting around. I think all of this is probably causally related to sharp decreases in the self-identification of race on NSF applications. Perhaps not for all the same reasons for every person or demographic. But if it is not an artifact of the grant submission system, this is the most obvious conclusion.

There is a ton of additional analysis in the preprint. Go read it. Study. Think about it.

Additional: Ginther et al. (2011) Race, ethnicity, and NIH research awards. Science, 2011 Aug 19; 333(6045):1015-9. [PubMed]

Research colonialism, analyzing an RFA

I recently fielded a question from a more junior scientist about what, I think, has been termed research colonialism with specificity to the NIH funding disparity known as the Ginther Gap. One of the outcomes of the Hoppe et al 2019 paper, and the following Lauer et al 2021, was a call for a hard look at research on the health issues of communities of color. How successful are grant proposals on those topics, which ICs are funding them, what are the success rates and what are the budget levels appropriated to, e.g. the NIMHD. I am very much at sea trying to answer the question I was asked, which boiled down to “Why is it always majoritarian PIs being funded to do research with communities of color?”. I really don’t know how to answer that or how to begin to address it with NIH funding data that has been generated so far. However, something came across my transom recently that is a place to start.

The NIH issued RFA-MD-21-004 Understanding and Addressing the Impact of Structural Racism and Discrimination on Minority Health and Health Disparities last year and the resulting projects should be on the RePORTER books by now. I was cued into this by a tweet from the Constellation Project which is something doing co-author networks. That may be useful for a related issue, that of collaboration and co-work. For now, I’m curious about what types of PIs have been able to secure funding from this mechanism. According to my RePORTER search for the RFA, there are currently 17 grants funded.

Of the funded grants, there are 4 from NIMHD, 4 from NIDA, 2 from NIA, 1 each from NIMH, NIHNDS, NINR, NICHD, NIGMS, NIDCD, and NCCIH. In the RFA, NIMHD promised 6-7 awards, NIDA 2, NIA 6, NIGMS 4-6 so obviously NIDA overshot their mark, but the rest are slacking. One each was promised for NIMH, NINDS, NICHD, NIDCD and NCCIH, so all of these are on track. Perhaps we will see a few more grants get funded by the time the FY elapses on Sept 30.

So who is getting funded under this RFA? Doing a quick google on the PIs, and admittedly making some huge assumptions based on the available pictures, I come up with

PI/Multi-PI Contact: White woman (2 NIA; 1 NCCIH; 3 NIDA; 1 NIDCD; 1 NIGMS; 1 NINDS); Black woman (1 NIDA; 1 NICHD; 1 NIMHD); Asian woman (1 NIMHD; 1 NIMHD; 1 NINR); White man (1 NIMHD; 1 NIMH)

Multi-PI, non-contact: Asian woman (1 NIA, 1 NIDA, 1 NIMHD); Black woman (2 NIDA, 1 NIMHD); White woman (1 NIDCD; 1 NIGMS; 1 NINR) Black man (1 NIGMS; 1 NIMH); White man (2 NIMH)

I would say the place I am most likely to be off in terms of someone who appears to me to be white but identifies as a person of color would be white women. Maybe 2-3 I am unsure of. I didn’t bother to keep track of how many of the non-contact PIs are on the proposals with white Contact PIs versus the other way around but….I can’t recall seeing even one where a non-contact white PI was on a proposal with a contact PI who is Black or Asian. (There was one award with three white men and one Black man as PIs and, well, does anyone get away with a four PI list that includes no woman anymore?) Anyway… make of that what you will.

I suspect that this RFA outcome is probably slightly better than the usual? And that if you looked at NIH’s studies that deal with communities or color and/or their health concerns more generally it would be even more skewed towards white PIs?

Ginther et al 2011 reported 69.9% of apps in their sample had white PIs, 16.2% had Asian PIs and 1.4% had Black PIs. Hoppe et al 2019 reported (Table S1) 1.5% of applications had Black PIs and 65.7% had white PIs in their original sample. So the 11 out of 17 grants having white PIs/Contact MultiPIs matches expected distribution, as does 3 Asian PIs. Black PIs are over represented since 1-2% of 17 is..zero grants funded. So this was not an opportunity that NIH took to redress the Ginther Gap.

But should it be? What should be the identity of PIs funded to work on issues related to “racism and discrimination” as it applies to “minority health and health disparities”? The “best” as determined by a study section of peer scientists, regardless of applicant characteristics? Regardless of the by now very well established bias against applications with Black PIs?

Someone on twitter asked about the panel that reviewed these grants. You can see from the funded grants on RePORTER that the study section reviewing these proposals was ZMD1 KNL (J1). Do a little web searching and you find that the roster for the 11/15/2021-11/17/2021 meeting is available. A three day meeting. That must have been painful. There are four chairs and a huge roster listed. I’m not going to search out all of them to figure out how many were white on the review panel. I will note that three of the four chairs were white and one was Asian (three of four were MDs, one was a PHD). This is a good place for a reminder that Hoppe et al reported 2.4% of reviewers were Black and 77.8% white in the study sections reviewing proposals for funding in FY2011-2015. I would be surprised if this study section was anything other than majority white.

Research Opportunities for New and “At-Risk” Investigators to Promote Workforce Diversity

NIDA, NIMH, and NINDS have issued a Program Announcement (PAR-22-181) to provide Research Opportunities for New and “At-Risk” Investigators with the intent to Promote Workforce Diversity.

This is issued as a PAR, which is presumably to allow Special Emphasis Panels to be convened. It is not a PAS, however, the announcement includes set-aside funding language familiar to PAS and RFA Funding Opportunity Announcements (FOA).

Funds Available and Anticipated Number of Awards The following NIH components intend to commit the following amounts for the duration of this PAR: NINDS intends to commit up to $10 million per fiscal year, approximately 25 awards, dependent on award amounts; NIDA intends to commit up to $5 million per fiscal year, 12-15 awards, dependent on award amounts; NIMH intends to commit up to $5 million per fiscal year, 12-15 awards, dependent on award amounts; Future year amounts will depend on annual appropriations.

This is a PA typical 3 year FOA which expires June 7, 2025. Reciept dates are one month ahead of standard, i.e., Sept (new R01) / Oct (Resub, Rev, Renew); Jan/Feb; May/Jun for the respective Cycles.

Eligibility is in the standard categories of concern including A) Underrepresented Racial/Ethnic groups, B) Disability, C) economic disadvantage and D) women. Topics of proposal have to be within the usual scope of the participating ICs. Eligibility of PIs is for the familiar New Investigators (“has not competed successfully for substantial, NIH (sic) independent funding from NIH“) and a relatively new “at risk” category.

At risk is defined as “has had prior support as a Principal Investigator on a substantial independent research award and, unless successful in securing a substantial research grant award in the current fiscal year, will have no substantial research grant funding in the following fiscal year.“

So. We have an offset deadline (at least for new proposals), set aside funds, SEPs for review and inclusion of NI (instead of merely ESI) and the potential for the more experienced investigator who is out of funding to get help as well. Pretty good! Thumbs up. Can’t wait to see other ICs jump on board this one.

To answer your first question, no, I have no idea how this differs from the NINDS/NIDA/NIAAA NOSI debacle. As a reminder:

Notice NOT-NS-21-049 Notice of Special Interest (NOSI): NIH Research Project Grant (R01) Applications from Individuals from Diverse Backgrounds, Including Under-Represented Minorities was released on May 3, 2021.

The “debacle” part is that right after NIDA and NIAAA joined NINDS in this NOSI, the Office of the Director put it about that no more ICs could join in and forced a rescinding of the NOSI on October 25, 2021 while claiming that their standard issue statement on diversity accomplished the same goals.

I see nothing in this new PAR that addresses either of the two real reasons that may have prompted the Office of the Director to rescind the original NOSI. The first and most likely reason is NIH’s fear of right wing, anti-affirmative action, pro-white supremacy forces in Congress attacking them. The second reason would be people in high places* in the NIH that are themselves right wing, anti-affirmative action and pro-white supremacy. If anything, the NOSI was much less triggering since it came with no specific plans of action or guarantees of funding. The PAR, with the notification of intended awards, is much more specific and would seemingly be even more offensive to right wingers.

I do have two concerns with this approach, as much as I like the idea.

First, URM-only opportunities have a tendency to put minority applicants in competition with each other. Conceptually, suppose there is an excellent URM qualified proposal that gets really high priority scores from study section and presume it would have also done so in an open, representation-blind study section. This one now displaces another URM proposal in the special call and *fails to displace* a lesser proposal from (statistically probable) a majoritarian PI. That’s less good than fixing the bias in the first place so that all open competitions are actually open and fair. I mentioned this before:

These special FOA have the tendency to put all the URM in competition with each other. This is true whether they would be competitive against the biased review of the regular FOA or, more subtly, whether they would be competitive for funding in a regular FOA review that had been made bias-free(r). […] The extreme example here is the highly competitive K99 application from a URM postdoc. If it goes in to the regular competition, it is so good that it wins an award and displaces, statistically, a less-meritorious one that happens to have a white PI. If it goes in to the MOSAIC competition, it also gets selected, but in this case by displacing a less-meritorious one that happens to have a URM PI. Guaranteed.

The second concern is one I’ve also described before.

In a news piece by Jocelyn Kaiser, the prior NIH Director Elias Zerhouni was quoted saying that study sections responded to his 2006/2007 ESI push by “punishing the young investigators with bad scores”. As I have tried to explain numerous times, phrasing this as a matter of malign intent on the part of study section members is a mistake. While it may be true that many reviewers opposed the idea that ESI applicants should get special breaks, adjusting scores to keep the ESI application at the same chances as before Zerhouni’s policies took effect is just a special case of a more general phenomenon.

So, while this PAR is a great tactical act, we must be very vigilant for the strategic, long term concerns. It seems to me very unlikely that there will be enthusiasm for enshrining this approach for decades (forever?) like the ESI breaks on merit scores/percentiles/paylines. And this approach means it will not be applied by default to all qualifying applications, as is the case for ESI.

Then we get to the Oppression Olympics, an unfortunate pitting of the crabs in the barrel against each other. The A-D categories of under-representation and diversity span quite a range of PIs. People in each category, or those who are concerned about specific categories, are going to have different views on who should be prioritized. As you are well aware, Dear Reader, my primary concern is with the Ginther gap. As you are aware, the “antis” and some pro-diversity types are very concerned to establish that a specific person who identifies as African-American has been discriminated against and is vewwwwy angweee to see any help being extended to anyone of apparent socio-economic privileges who just so happens to be Black. Such as the Obama daughters. None of us are clean on this. Take Category C. I have relatively recently realized that I qualify under Category C since I tick three of the elements, only two are required. I do not think that there is any possible way that my qualification on these three items affects my grant success in the least. To do so would require a lot of supposing and handwaving. I don’t personally think that anyone like me who qualifies technically under Category C really should be prioritized against, say, the demonstrated issue with the Ginther gap. These are but examples of the sort of “who is most disadvantaged and therefore most deserving” disagreement that I think may be a problem for this approach.

Why? Because reviewers will know that this is the FOA they are reviewing under. Opinions on the relative representation of categories A-D, Oppression Olympics and the pernicious stanning of “two-fers” will be front and present. Probably explicit in some reviews. And I think this is a problem in the broader goals of improving equity of opportunity and in playing for robust retention of individuals in the NIH funded research game.

__

*This is going to have really ugly implications for the prior head of the NIH, Francis Collins, if the PAR is not rescinded from the top and the only obvious difference here is his departure from NIH.