The cumulative probability of getting NIH grants

One of the career strategies we have discussed numerous times in various contexts is how many grant applications one should be submitting to the NIH. I have been a consistent advocate for …more. This is a recognition that success rates on a per-application basis have been below 20% for most of my career. Obviously this particular target number varies a lot. Sometimes we are talking about paylines, since that seems to be a hard target for success. Recent paylines from the NCI have been in the high single digits- 7-9%. Or we may talk about NIH-wide success rates overall, accounting for not just payline funding but pickups above the payline. These numbers change from year to year but mid to upper teens is a pretty fair general estimate.

My usual argument is that investigators who want to get funded should start with the assumption that they are no better than anyone else and need to base their strategy off the average success rate….at the very least.

Dumb old me, math challenged in the extreme, may have even expressed this as something like “If the payline is 10%, you need to put in 10 applications to even be in the game”. The more math savvy of you immediately chime in to correct me about calculating cumulative probabilities. This is not a difficult concept. I get it. But my mind likes to forget about it and I’ve never taken a blog whack at this issue directly, that I can recall.

Thanks to this handy binomial probability tool I googled up, we can now contemplate the abyss. Let us suppose a per-application success rate of 17.7%. Mid to upper teens, a decent place to start our discussion. And let us gate on the cumulative probability of at least one award. Well, if you put in 5 applications, your odds of one of them funding is 62.2% and if you put in 10 applications, this is 85.7%. Not too shabby. But it puts a very fine point that probabilities of award do not add. Fifteen applications are required to get to a 95% cumulative probability of at least one being awarded.

Reminder: We are not talking payline here. We are talking the 17.7% hit rate for the NIH wide average of everything that goes into awards including all of those various pickup behaviors. If you want the assurance of making the payline at a place like NCI, well…..Lord help you, amirite? That sounds totally rough and brutal and even unfair.

Now. Suppose that for some reason, say that your skin reflectance categorizes you as Black instead of white, your success rate was 10.7% instead of 17.7%.

Your chances are, of course, somewhat different. The cumulative advantage of putting in more grants, aka working harder, accrues less surely here as well. I’ve color coded a few ~equivalent cumulative probabilities for convenience. Using the Hoppe success rates, a Black applicant has to put in 9 proposals to get approximately the same 62% chance a white applicant achieves with only 5 applications. This same PI would have to put in 18 proposals to approximate the 86% hit rate the white PI gets with only 10 applications. About 25 proposals to get the 95% hit rate enjoyed by white applicants who put in 15 proposals.

Insert my favorite staring eyes emojii available on other platforms.

I would estimate that many Black folks, in the academy and elsewhere, are somewhat used to the idea that they need to grind a bit harder to achieve. At some level it probably doesn’t even really bother some on the day to day.

But this is a LOT harder. Putting in just one grant proposal is not easy. Particularly when you are a brand new Professor. But it is not trivial, even when you are a Full professor with some time and experience under your belt. [ Oh, btw, sidebar: Ginther et al 2018 has a little comment that I probably missed originally. “In results not reported, being a full professor increased the probability of NIH funding by 11.9 ppt (p < .001), but the funding gap remained -12.8 ppt (p < .001).” Yeah, age doesn’t help. ] When we are talking 5, 10, 25 applications it is maybe easy to overlook the sweat that goes into making new and credible proposals. Sure, some can be revised proposals and some are retreads of various prior proposals. But they take work and sweat to make them competitive. You are not going to enjoy the NIH-wide average hit rate with consistently below-average proposals!

This brings me back to a related issue that appeared in the Ginther et al 2011 report. “Applications from black and Asian investigators were significantly less likely to receive R01 funding compared with whites for grants submitted once or twice……black and Asian investigators are less likely to be awarded an R01 on the first or second attempt, blacks and Hispanics are less likely to resubmit a revised application, and black investigators that do resubmit have to do so more often to receive an award.“. I will admit I still don’t quite understand what they are presenting here at the end. It reads as though they are gating on Black investigators who do eventually win an award and do so on revision, not the A0 (this sample was back in the A2-permitted days, iirc). This whole passage, however, can be received as “well, just work a little harder to compensate” and to appear as if we’re only talking about an extra revision or two. I probably received this in this way myself on initially seeing the 2011 paper. And I have to say the “1.7 fold advantage” that is discussed in Hoppe for the per-application success rates comes in the same way. It can be received as, well, you just have to write two to get one. Because it focuses us on the “given you did get an award” instead of what it takes to get that award, statistically speaking.

But looking at these cumulative probability graphs really hits differently.

Black PIs don’t have to work just a little harder.

Black PIs have to work a LOT harder.