What would the NIH do if it wanted to make things really hard for Asian and Black PIs to get funded?

January 17, 2014

The takeaway message from the report of Ginther and colleagues (2011) on Race, Ethnicity and NIH Research Awards can be summed up by this passage from the end of the article:

Applications from black and Asian investigators were significantly less likely to receive R01 funding compared with whites for grants submitted once or twice. For grants submitted three or more times, we found no significant difference in award probability between blacks and whites; however, Asians remained almost 4 percentage points less likely to receive an R01 award (P < .05). Together, these data indicate that black and Asian investigators are less likely to be awarded an R01 on the first or second attempt, blacks and Hispanics are less likely to resubmit a revised application, and black investigators that do resubmit have to do so more often to receive an award.

Recall that these data reflect applications received for Fiscal Years 2000 to 2006.

Interestingly, we were just discussing the most recent funding data from the NIH with a particular focus on the triaged applications. A comment on the Rock Talk blog of the OER at NIH was key.

I received a table of data covering A0 R01s received between FY 2010 and FY2012 (ARRA funds and solicited applications were excluded). Overall at NIH, 2.3% of new R01s that were “not scored” as A0s were funded as A1s (range at different ICs was 0.0% to 8.4%), and 8.7% of renewals that were unscored as A0s were funded as A1s (range 0.0% to 25.7%).

I noted the following for a key distinction between new and competing-continuation applications.

The mean and selected ICs I checked tell the same tale, i.e., that Type 2 apps have a much better shot at getting funded after triage on the A0. NIDA is actually pretty extreme from what I can tell- 2.8% versus 15.2%. So if there is a difference in the A1 resubmission rate for Type 1 and Type 2 (and I bet Type 2 apps that get triaged on A0 are much more likely to be amended and resubmitted) apps, the above analysis doesn’t move the relative disadvantage around all that much. However for NIAAA the Type 1 and Type 2 numbers are closer- 4.7% versus 9.8%. So for NIAAA supplicants, a halving of the resubmission rate for Type 1 might bring the odds for Type 1 and Type 2 much closer.

So look. If you were going to try to really screw over some category of investigators you would make sure they were more likely to be triaged and then make it really unlikely that a triaged application could be revised into the fundable range. You could stoke this by giving an extra boost to triaged applications that had already been funded for a prior interval….because your process has already screened your target population to decrease representation in the first place. It’s a feed-forward acceleration.

What else could you do? Oh yes. About those revisions, poorer chances on the first 1-2 attempts and the need for Asian and black PIs to submit more often to get funded. Hey I know, you could prevent everybody from submitting too many revised versions of the grant! That would provide another amplification of the screening procedure.

So yeah. The NIH halved the number of permitted revisions to previously unfunded applications for those submitted after January 25, 2009.

Think we’re ever going to see an extension of the Ginther analysis to applications submitted from FY2007 onward? I mean, we’re seeing evidence in this time of pronounced budgetary grimness that the NIH is slipping on its rather overt efforts to keep early stage investigator success rates similar to experienced investigators’ and to keep women’s success rates similar to mens’.

The odds are good that the plight of African-American and possibly even Asian/Asian-American applicants to the NIH has gotten even worse than it was for Fiscal Years 2000-2006.

Advertisements

26 Responses to “What would the NIH do if it wanted to make things really hard for Asian and Black PIs to get funded?”

  1. A. Tasso Says:

    So… why don’t any of the “minority” programs benefit BOTH AfAms and AsianAms?

    Like

  2. drugmonkey Says:

    I dunno but the announcement I saw put “Asian American Native American Pacific Islander Serving Institutions (AANAPISIs) ” right up there with institutions serving other URM populations. Other items link back to the NSF (http://www.nsf.gov/statistics/showpub.cfm?TopID=2&SubID=27) but it isn’t clear from this site what the takeaway message is supposed to be.

    I rather suspect the burden will be on applicants to define and demonstrate how they are addressing “Underrepresented Populations”.

    Like

  3. drugmonkey Says:

    Oh, and there was a paper disputing the central findings of Ginther, btw. Links in here

    http://scientopia.org/blogs/drugmonkey/2013/02/05/new-study-concludes-no-bias-in-nih-grant-review/

    Like

  4. qaz Says:

    Wait… I thought you LIKED getting rid of the A2! Now you’re back saying the A2 was a good thing because it let people learn how to write grants? Welcome back to the pro-A2 side.

    I always liked the A2 because it meant you had a much better sense of whether you were in line or not than having a random lottery every time. It did mean that you were less flexible because you had to plan two years ahead because there was a set of planes circling the airport waiting to land. But it also meant that new investigators and other people who had trouble writing fundable grants (*) could learn how to please this set of reviewers and get their grants funded.

    * As we’ve been discussing, we don’t actually know why certain minorities are having trouble getting grants funded. I always come back to a lesson one of my senior faculty taught me when I was a junior faculty. (There were still A2’s at the time) “When study section is right and you’re wrong, you’re wrong – fix the grant. When you’re right and study section is wrong, study section is still right – fix the grant.” Of course, that depends on the assumption that you are working with a set of reviewers that you are trying to please. Having the A2 in a standing study section goes a long way to providing that.

    PS. Note that this argument is (and has always been) independent of whether the revisions made change the actual science of the grant. I think it does sometimes and doesn’t other times. The question of the A2 looks very different from the perspective of the study section members (“just fund the good science – don’t put people in a line to get funded”) vs from the perspective of the applicant (“you know you’re in line; you can learn how to fix the grant for this set of reviewers”).

    Like

  5. DrugMonkey Says:

    Don’t be obtuse, qaz. It would be *better* if all categories of PI had equivalent chances if being funded on the A0 and A1. Since this is not the case, removing the A2 very likely increased the bias.

    Like

  6. qaz Says:

    It’s not obtuse. It’s saying that those of us who LIKED the A2 were right. (And, in fact, right for the right reasons – it gave people a chance to learn how to get a grant past a specific set of reviewers.) And that those who argued for getting rid of the A2 were wrong.

    It’s a practical question. Is the A2 a good or a bad thing? You argued that the A2 was bad because it set up air traffic control delays. It turns out that air traffic control delays are a good thing because they give everyone a fairer chance. (In addition to providing for more stable funding planning.)

    Like

  7. sciencedude Says:

    The answer is simple qaz. DM wants A2s, and perhaps A50s, for AA applicants only. I think the answer to the funding disparity is also simple. Since there are plenty of NIH-funded scientists who are pro-racial preference, for example DM, those folks should have the option to donate there own grants to unfunded URM applicants. This could be another box to check. “In the event your grant is funded, would you be willing to donate your award to an unfunded URM?” Come to think of it, this scheme would work for racial preferences in education and employment, as well.

    Like

  8. DrugMonkey Says:

    qaz- really this isn’t difficult. Everyone treated fairly, grants funded on first proposal b/c of ideas, instead of NaCl concentration bullshittery.

    Like

  9. DrugMonkey Says:

    sciencedude- you also are being stupid. I said before that I am completely *against* the racial preferences that white PIs enjoy at present. How is this unclear to you?

    Like

  10. qaz Says:

    DM – in a utopian world, sure. I think it’d be great if grants were funded fairly without bias, and people could live on one R01, and people could plan, and we didn’t have to live with the Glamour Mag problem, and, and, and…. But we still don’t know WHERE the bias is actually coming from (as you point out Ginther controls for a lot of the obvious possibilities). And the uncertainty makes living on one R01 difficult. And the people fighting the GlamourMags all made their names in GlamourMags.

    In practice, many grants are not going to be funded on the first proposal and the A2 and air-traffic-control-delays were a pretty good system for getting funded. I agree completely that study section should trust individuals more and complain about details less, but removing the A2 didn’t help that at all. (What did help was cutting the grant size from 25 to 12 pages, which was a GREAT idea. What would help even more is having four or five people reading each grant instead of three.)

    BTW, in practice, in the old system with the A2s and preferences given to resubmission, things were closer to a what we’re asking for. (New investigators and other biased-against groups did poorly in the A0/A1, but often got funded in the A2. Renewals got preference, which meant that you could live on one R01. Etc.) If your A0 and A1 scores were really bad, you could start desperately putting in multiple grants, but you had three shots to renew your one R01 and would often know you were in line.

    It’s funny after all the ranting, we find ourselves defending the old system before all the changes. It’s like the scoring system – the old system had lots of problems, but the new system turns out to be worse.

    And sciencedude – trolling does not actually help the discussion.

    Like

  11. DrugMonkey Says:

    You are completely delusional and overlooking the *considerable* number of problems with the air traffic pattern. Particularly for the newbs.

    Like

  12. DrugMonkey Says:

    2.5 years from initial submission to funding is good qaz? I mean FFFS.

    Like

  13. qaz Says:

    Yeah, FFS. I lived through that and it worked. I hated it at the time and complained bitterly about it, but that’s because I didn’t know how much worse it would be without it. It sucked, but truth be told it sucked less than it does now. It worked because you worked towards a goal, and you received clues that you were actually progressing towards the goal. That’s what startup money was for. The assumption was that the university put some startup money (skin in the game) that would be enough to get you through the air traffic control delay.

    For noobs: FFS tell me that current noobs aren’t living with much more than 2.5 years from initial submission to funding!

    For noobs today, there is no air traffic control pattern anymore. There’s just cr*pshoot lotteries that you hope you win. So some people get funded in time and survive. Others don’t and die. That’s a much worse system than air traffic control that gave them (and their departments!) evidence that they were making progress.

    For non-noobs, it meant that you started applying for your R01 renewal in early year 4 of your 5-year grant. If you did get through, you continued your grant. If you didn’t, you had a second shot during the no-cost extension. And if you needed a third try but had a good score, well, that’s what bridge funding was for. If you didn’t get a good score, you had to start putting in extra tries for multiple grants. Tell me that any normal PI isn’t planning 3 years ahead. (At a minimum!)

    The problem is that the logic of I must get funded or I’ll get scooped assumes that you cannot switch to do something new in the middle of a grant cycle. But that’s not what the system was. That’s not what the system is. And that’s not how science works. If a new thing comes along halfway through your cycle, you just go do it. Who cares if it’s exactly what’s funded? As long as its vaguely in the ballpark, everyone is fine with it. And the truth is (and always was) that a lot of really good ideas got funded on the A0 try. So lots of things didn’t take 2 years to get funded. But it doesn’t matter if it takes 2 years to get funded. As long as your general plan is proceeding. And as long as you’ve planned ahead.

    The current system is a mess – since you never know what’s going to happen, you have to play the cr*pshoot lottery and hope that the right proportion of grants you submit get funded. It’s a bad thing if 0/5 get funded when you need 1. But it’s also a bad thing when 4/5 get funded when you need 1. Our labs now live in boom and bust cycles. And it sucks.

    And, of course, we waste our time writing (and reviewing) grants that never get funded instead of actually doing science.

    Like

  14. qaz Says:

    PS. I went through the ESI/NI people in the most recent study section data I had access to. They have been faculty for an average of 4.6 years, with only two PIs (both A0) with less than 4 years of faculty experience. FFS how is that better than a 2.5 year air traffic control pattern?

    Like

  15. DrugMonkey Says:

    I didn’t say that preventing A2s *worked* and in fact I have been arguing that little has changed in review culture. “A0” is as likely to be an actual A2,3 or 4. Or so I bet. And predicted from the start.

    Like

  16. qaz Says:

    You did predict that A0s would be fake A2s. But I still see no evidence that that is what is happening. What I see happening is people sending in five grants over two years hoping to get one funded. Which is a very different thing.

    Are other people sending fake A2’s (repurposed grants as A0s) to the same study section? I have never seen a fake A2 in a study section (where we got the A1, rejected it, and then saw a repurposed A0). Assuming that people are going to new study sections, then they are not learning to please a given set of reviewers. And we are back to cr*pshoot lotteries of hoping that our proposal matches the reviewer’s taste (*).

    In order to send an A0 as a “rebuilt A2”, you need to change enough of the aims so that CSR doesn’t mark it as a “rebuilt A2”, which means that you are not just “improving the grant”, you are also (in many ways) starting over. This is not air traffic control. This is trying to find a cr*pshoot lottery of finding a proposal that matches the reviewer’s taste (*).

    * As has been said in many places, 25% of the submitted grants would be great science, and 50% are probably good science. (It may well be 50% great and 75% good.) With a 10% funding line, we are in random gambling territory of matching your proposal to a reviewer’s taste. While I feel comfortable that reviewers can judge 25% lines (top quarter, mid 50%, bottom quarter), anything within that is noise. I suspect a lot of that noise is grant-reviewer matches, although some of it may well be due to many other factors.

    Like

  17. The Other Dave Says:

    Politics. As long as politics plays a role in funding decisions, none o’ this is going to change.

    As long as having buddies on study section helps you, people without lots of buddies are not going to be funded as well.

    As long as people can whine to POs about anything and everything and have those POs listen, none o’this is going to change.

    Science funding, despite what people wish, is NOT a meritocracy. This is no real mysterious secret.

    And because of that, and the fact that various segments of society have been shitting on various other segments of society ever since there was society, non o’ this is going to change.

    Jiggling NIH application procedures is not going to fix society.

    If we want more funded minority and women scientists, then the fastest most effective way to do it is to stop all the hand-wrining and farting around, and just have different paylines. period. End of problem.

    Oh, that sounds too unfair?

    To whom?

    Like

  18. sciencedude Says:

    To me.
    DM, from a few posts back, I think you mean the assumed preferences that white, hispanic, and Asian (native-English-speaking) PIs enjoy. It is not clear to me because you want to substitute an assumed preference for an overt preference. I still think that voluntary affirmative action is the fairest way to go. Of course, it is a lot harder to be so gung-ho for preferences when it is your own ox that is getting gored.

    Like

  19. sciencedude Says:

    BTW, there actually is a fairly objective test for addressing the question of whether black PIs are victims of racism or the grants they are submitting are, on average, not as good. Look at the individual review scores for black vs. non-black applicants. If racism is the problem, then AA PI scores should show a broader range of scores then non-AA PIs. In particular, they should show some scores that are unusually low compared to the median. If the range is no different than for other PIs, you either conclude that racism (intrinsic or otherwise) is not an issue, or the entire review panel is racist.

    Like

  20. drugmonkey Says:

    What part of the 40% versus 60% triage rate was not clear to you sciencedude?

    What part of the “disparity still persists despite controlling for these other obvious variables” was not clear to you?

    Like

  21. sciencedude Says:

    quoting from Ginther, …”unscored applications did not fully explain differences in award probability.”
    I have not seen the raw data from Ginther, so we have to take the authors’ word about how well variables were controlled for.
    I know you take it as a matter of faith that racism is the only possible explanation for the disparity. Any thoughts on a more objective test?

    Like

  22. drugmonkey Says:

    When did I say that I believe “racism” (whatever that means to you may not be the same thing it means to me, anyway) was “the only possible explanation”. I have asserted “death of a thousand cuts” as “the” explanation in several venues. Including this blog if I am not mistaken.

    There is not going to be any singular “the reason” for this disparity. life is far too complicated. The search for “the” reason, consequently, is a cynical effort to do nothing.

    This is also a reason that the sort of fix that I recommend is going to have the fastest and most significant effect because it is agnostic to “the” cause of the disparity.

    Like

  23. sciencedude Says:

    Sorry, I think the word you have used all along is “bias” in the review process, which to me implies intrinsic or extrinsic racism. But in any case, if the disparity in awards is for any other reason than “bias” in the review process, it would be unfair to non-black PIs to say, “sorry, this guy got a lower score, but we prefer the way he looks, so we are funding his grant and not yours.” This is the same sort of stupidity that is now screwing over Asians trying to get into college:
    http://www.nytimes.com/roomfordebate/2012/12/19/fears-of-an-asian-quota-in-the-ivy-league/discrimination-is-obvious
    Maybe that should be your next cause du jour. And while you are at it, I am also very concerned about the serious under representation of whites in professional sports, particularly basketball.

    Like

  24. drugmonkey Says:

    the word you have used all along is “bias” in the review process, which to me implies intrinsic or extrinsic racism

    You are quite incorrect in this. A disparity of outcome reflects a bias in the system. The source or nature of that bias is not identified by the fact that it exists.

    if the disparity in awards is for any other reason than “bias” in the review process, it would be unfair to non-black PIs to say, “sorry, this guy got a lower score, but we prefer the way he looks, so we are funding his grant and not yours.”

    In a zero-sum game such as the award of NIH grants it is really hard to have one set of individuals under a negative-outcome bias in which some other group is not under a positive-outcome bias. The grant that doesn’t go to PI #1 goes to PI #2.

    In this case, Ginther provides a roadmap for assessing which “nonblack PIs” are under which aspects of positive bias in the system and which ones are under similar negative bias. Really, go read the entire thing. It makes the case quite clearly.

    This is the same sort of stupidity that is now screwing over Asians trying to get into college:

    Colleges and universities rightfully assert that a diverse student body is a benefit to education. The process of reaching an appropriate balance of students of various backgrounds (ethnic, economic, regional, national, etc) is not differentially inappropriate based on who it appears to limit and who it appears to encourage. This is because the “merit” of an application is decided by the University, not by the applicant. Asians are not being “screwed over” at Harvard, and that white woman suing in Texas was not being “screwed over” because they did not “merit” admission. It isn’t up to them to decide admissions policy. You don’t individually deserve a place at Harvard.

    That’s process. What about outcome and threshold setting? Well, it is pretty hard to argue with “approximates the national demographic mix” as a baseline. IMO. Going above that would be great, but one has to balance what sort of hit that would put on the other demographics.

    So if the allegation in the story you link to is that the Ivy League’s 17% is horrible, well it is in considerable excess of the 4.8% of the US population as of 2010 (according to the census). How far away from the national average can you get without putting a dent in other demographics?

    There is a comment in the story you link about the college age population so that would, I suppose be another argument to have. These are credible issues to discuss. Pointing to test scores, grade point averages and various competitions is, however, not. At all. I refer you back to the fact that the general-purpose colleges and universities have asserted a diverse student body as an explicit goal for a very long time. It is a good goal. I say that as a general member of the public, as a person that valued a diverse student body when I was in college and as a parent distressingly close to the college process for offspring. A college body made up exclusively of like-scoring individuals on one particular metric (of academic achievement or anything else) would be inferior for my kids. Substantially so.

    I am also very concerned about the serious under representation of whites in professional sports, particularly basketball.

    start here, ignoramus. The barriers in sport, and the performance measures, are not the same as for college admission or for NIH grant award. The incentives for getting the best of the best are different as well.

    Like

  25. sciencedude Says:

    At the end of the day, it comes down to having a group of people sufficiently diverse to “please the eye” of Drug Monkey. Dude, you’re deranged. Your inability to recognize your own hypocrisy is pathetic.

    Like

  26. DrugMonkey Says:

    Why is this “hypocrisy” and what makes you suggest it is about pleasing *my* eye? I didn’t write or commission the Ginther report and I for sure didn’t invent University and college admission standards. Affirmative action existed long before I became interested in such things. etc.

    What it *does* appear to come down to is that you do not believe in redressing biases. You have company in this, it just isn’t very good company.

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: