An update on NIH grant success rates
March 14, 2011
A query at PhysioProf’s blog from cackleofrad asks:
What has been the % chance of funding over the course of your career? Does your strategy of 2 RO1′s, on average, per year mean that you spend a great deal of time tweaking each one? I ask since my contemporaries are sending in ~4-8 RO1′s per year with the chance of funding ~10% or less.
As a reminder, the payline is not exactly equal to the “chance of funding”. Reason being, in simple terms, that the paylines (where published for various ICs) are conservative. They tend to be the level at which the IC has overwhelming confidence they are ok if they fund all the grants that are reviewed as being at that percentile or better. They end up funding more grants than fall within the payline simply because of this conservativeness. If you want to know why they would be conservative, well it is one heck of a lot easier to deal with PIs who get funded absent an expectation of funding than to deal with PIs who do not get funded who have a good expectation that they would.
There is also the consideration that Program staff are intentionally conservative with the payline to facilitate funding proposals out of order– aka “exceptions” or “pickups”. As demonstrated by the NIGMS funding data, the grant funded above an apparent (or published) hard payline are not selected randomly, the correlation with overall impact score is still pretty good. Nevertheless, you can see that in the margin between “everything gets funded” and “essentially nothing gets funded” there are quite a number of awards being made. So this is where the success rate gets much higher than the payline.
So in terms of your realistic chance of funding, the success rate is a better answer. The NIH almost always trumpets the success rate which is (almost) the number of grants funded in a given Fiscal Year divided by applications received for that Fiscal Year of funding. (I say “almost” because there is a bit of mumbo with applications that are revised within the fiscal year.) It is good enough for gov’mint work, as they say, when it comes to addressing cackleofrad’s query.
This slide is from FASEB, via writedit. It gives a recent slice of the “success rate” picture. (For more see this, as well as this and this.)
Getting back to cackleofrad’s question in the context of PhysioProf’s observations, I note that the success rate for those without prior NIH awards during the real salad days of the doubling (~98-03) did not benefit as much as did the success rate of the experienced investigators. About 21-22% versus 25-26%. Four percentage points might not seem like much, but it is a 15% hit on the success rate.
Now go back to the origin of the data series here, 1995, when the doubling was just having an effect. New investigators enjoyed only a 19% success rate. In the most recent years, this number is anywhere from 17-19%. Yes, thanks to a number of efforts that have raised the newbs onto the trendline enjoyed by experienced investigators. Which is a good thing, true. But it sure looks like the success rates for new investigators right now compare favorably with the situation pre-doubling.
All I’m trying to point out with this is that if you want to know “how hard Prof X had it” when she started her career compared to how hard the new investigators have it today, you need to consider the newb success rates then and now, not just the overall NIH omnibus success rates.
DrugMonkey 
March 14, 2011 at 12:06 pm
Back in the 80s, I was a technician in a small lab at an elite university. The PI survived on between 1 and 2 R01s or equivalents. I don’t recall this PI writing 2-5 applications per year, and I know there were years when no grants were written so that we could “get the papers out.” This PI was not a particularly good grant-writer, so it usually took 1-2 resubmissions to get the money. This PI survived just fine.
I’m trying to reconcile this experience with the post-1995 data quoted by DM above. For how long has it been the case that PIs need to write 2 or more R01s or equivalents every single year they are running a lab in order to survive? Has this always been “the bar” or has the bar been raised?
LikeLike
March 14, 2011 at 12:21 pm
The bar has been dramatically raised. RPG success rates for established investigators were in the high 30s/low 40s throughout the 1980s.
LikeLike
March 14, 2011 at 1:38 pm
http://scienceblogs.com/drugmonkey/2009/09/nih_historical_success_rates_e.php
The first “this” link answered your question Beaker….
LikeLike
March 14, 2011 at 2:36 pm
DM,
your link:
http://scienceblogs.com/drugmonkey/2009/09/nih_historical_success_rates_e.php
does not seem to be working.
Thanks
LikeLike
March 14, 2011 at 3:12 pm
Sb has been undergoing a DDOS attack which has left some IPs banned. Are you having a general problem with Sb or just that post?
LikeLike
March 21, 2011 at 8:31 am
So the data say that it’s no longer harder for 1st-timers to get R01’s than established investigators, but on the other hand it’s harder for *everyone* to get R01’s now than it was 10 years ago.
In fact, it’s so hard to get grants now that people are sending in 4 or more R01s per year (how is this humanly possible?) whereas before they sent in 2 or fewer. My question is, what is the point of making it easier for newbies to gain access to a system that then forces them to waste so much time writing grants that they can’t get any real work done? Maybe they’d be better off in industry. Maybe that’s the case for a lot of us, whether we are “established” investigators, newbies, or in between.
LikeLike
March 16, 2012 at 5:14 am
I put in 7 in the past 12 months Grumble. It is not just “possible” but rather, it is necessary to be able to do this at times.
LikeLike
March 16, 2012 at 8:53 am
Experienced PI, I am curious about your strategy. I presume that some of your applications don’t end up funded, and under the new rules, those ideas must be re-framed as new applications. Did you submit 7 new projects in the past 12 months and are you able to support all 7 with track record, publications, and prelim data in sufficient amounts to give all those applications a shot at funding? I realize that it probably wasn’t 7 new projects because some were probably resubmissions, but even if it was only 3 new projects (or 1 renewal and 2 new projects), how do you find time to do any actual research? How many are in your lab? Do you subscribe to the “throw everything at them and see what sticks” philosophy?
Do you think a new investigator is smart to adopt your approach in lieu of the fact that it is unlikely that they can support 7 applications with prelim data? My impression is that good but risky ideas aren’t very persuasive in study section these days. As funding lines drop, risk aversion rises.
Regardless, 7 in 12 months is amazing, and my hat is off to you.
LikeLike
March 16, 2012 at 9:09 am
Did you submit 7 new projects in the past 12 months and are you able to support all 7 with track record, publications, and prelim data in sufficient amounts to give all those applications a shot at funding?
They are not all “new”, no. Some revisions, some continuations and some brand new projects. A mix. Obviously I wouldn’t have submitted them if I did not think they had “a shot” at funding.
how do you find time to do any actual research?
Grant writing is “actual research”. But taking your point, when I am writing grants it is just one of the many tasks that I have to accomplish sequentially and concurrently. How do you find time for all the myriad of things that are under your domain? You make time when it is necessary.
How many are in your lab?
I have submitted at this rate at times when there were only two members of my lab and when there are many. This has nothing to do with the number of ideas you can generate. If I were the only person or if I had a team of 50 I doubt I would ever exhaust the number of scientific ideas that I think valuable to pursue.
Do you subscribe to the “throw everything at them and see what sticks” philosophy?
That is a considerably loaded question that cannot reasonably be answered unless y0u define your terms more closely.
Do you think a new investigator is smart to adopt your approach in lieu of the fact that it is unlikely that they can support 7 applications with prelim data?
As one matures, one realizes that “prelim data” are only one of the tools in the grant writing toolbox. I have written grants with lots of highly applicable preliminary studies and been triaged. I have written grants with the barest skin of tangentially related data and had them funded.
As far as new investigators go, well, if you are telling me you are in your first year or three and cannot imagine writing and submitting 7 applications in 12 months to the NIH….this is not going to go well for you.
Regardless, 7 in 12 months is amazing, and my hat is off to you.
It is not “amazing”, it is necessary. You are better off working out how to match (or exceed) this rate rather than bowing in respect.
LikeLike
March 16, 2012 at 9:48 am
Experienced PI, thanks for the frank answers.
In terms of “throw everything at them and see what sticks,” I don’t mean glibly pursing shallow half-concepts and thinking that the study section won’t recognize them as such. I mean to say that it is easier to generate good ideas for research projects than it is to provide ample justification and support for those ideas in the form of data, papers, and track record. This difference is especially large for people just starting their independent careers. Under the current system, those fully developed project ideas get two shots before they undergo a major overhaul or get discarded.
The fact that you have had grants funded with the “barest skin of tangentially related data” indicates either that your track record was highly persuasive or else that funding grants has a stochastic element that can only be counteracted by increasing the number of submissions. Or both.
LikeLike
March 16, 2012 at 12:56 pm
than it is to provide ample justification and support for those ideas in the form of data, papers, and track record.
I believe you are getting overly focused on what is “ample justification”.
Or both.
Another element that you seem to be ignoring is that perhaps a really fantastic idea will be sufficient to obtain a fundable score despite a lack of what you imagine to be “ample justification” on the more pedestrian aspects such as preliminary data and track record.
fully developed project ideas get two shots before they undergo a major overhaul or get discarded.
One of the most common errors of newer investigators is to throw the kitchen sink into each proposal. Understandable but it is rare an application gets criticized for having too little in it, much more common to see criticism for an excess of ambition. Perhaps those “fully developed” ideas could support more proposals than you think.
LikeLike
March 16, 2012 at 1:53 pm
Well, I’m a burnt child. My reviews come often come back with comments such as, “this would be great if you could do it, but you haven’t show us that you can.” This is a stock criticism that young investigators often hear from the study section. In 13-page format, one cannot expect the preliminary data alone to be persuasive. The solution, I think, is to publish it, even if only in PloS One. Just get it out there, because there ain’t space in the application.
Yes, you pretty much got me pegged. Do experienced PIs get criticized as harshly for excess ambition? If no, should they be?
LikeLike
March 16, 2012 at 5:35 pm
I put in 7 in the past 12 months Grumble. It is not just “possible” but rather, it is necessary to be able to do this at times.
It sounds illegal to do this. You are not allowed to use federal dollars to lobby the government for federal dollars so if you’re already significantly supported by grants you can’t use this much of your effort to write grants.
LikeLike
March 16, 2012 at 6:54 pm
The Fed pays me for 40 hrs a week, whimple. The rest is my own spare time.
LikeLike
March 17, 2012 at 2:22 am
You have no idea what percent of his effort is supported by federal grants and what percent is being devoted to writing grant applications. It only “sounds illegal” if you pull numbers out of your ass.
This is completely utterly 100% false, and is a serious misreading of the Federal laws and regulations governing sponsored research.
The Federal Government pays you for a percent of your professional effort. If 100% of your professional effort is supported by NIH grants and you spend 90 hours per week exerting professional effort, then NIH is paying you for all 90 of those hours. And any time spent writing NIH grants is absolutely considered professional effort.
LikeLike
March 17, 2012 at 6:30 am
Wow, I’m surprised any of you pre tenure Assistant Professors know how things work. A gold star for Comrade Physio”proffe”.
LikeLike
March 17, 2012 at 1:41 pm
This so-called “Experienced PI” sounds like a delusional Walter Mitty blowhard to me. I bet he’s a glasswasher in a Hughes lab.
LikeLike