NIAID publishes their FY2013 paylines

April 19, 2013

NIAID is one of the NIH ICs that actually publishes a payline. According to their website, as of April 19 the R01s from experienced investigators will have a payline of 8 percentile. The payline for new investigators will be 12 percentile. By way of comparison these were 10%ile and 14%ile in the prior two Fiscal Years for NIAID.

Mechanisms such as the R03, R21 and R15 will have to get a 20 overall impact score, or better, to fund but these are still listed as “interim” criteria.

So from a statistical basis, you need to have put in 13 proposals to NIAID this year in order to have a fighting chance to get one.

Lovely.

Advertisements

45 Responses to “NIAID publishes their FY2013 paylines”

  1. Ola Says:

    NHLBI is 6% for RO1s

    Like

  2. mikka Says:

    Is NIGMS still publicizing theirs or did that end after Berg?

    Like

  3. NIH Budget Cutter Says:

    Such wonderful news!

    The Budget Control Act of 2011 and Sequestration should insure such paylines for the next 5 years. But we still have a ways to go; after all, these paylines are still several points above zero.

    And no one is coming to your rescue. Your darling in Congress, Arlen Specter, is all gone. Add to that incoming budget pressures from underfunded entitlements, and you are about to get hit with the Perfect Storm. My advice: Take action now and run for the hills, or you will finish end up like this guy:

    http://discovermagazine.com/2011/apr/30-how-bad-luck-networking-cost-prasher-nobel

    Like

  4. Dave Says:

    These paylines look good compared to some ICs.

    Like

  5. physioprof Says:

    This is from the NINDS February Council minutes:

    NINDS paid almost all grants from October 2012 Council from its FY12 budget allocation and is currently expediting grants to the 12th percentile. Dr. Landis indicated that the Institute hopes to be able to award grants to the 15th percentile; however, the ability to do so is contingent upon the outcome of ongoing federal budget negotiations.

    I have been told by program staff that since sequestration was not reversed, the payline for FY13 will remain at 12%ile.

    Like

  6. Eli Rabett Says:

    You are doing the math wrong again. Each proposal you put in has a probability of (1-0.08) of not being funded. If you put in N proposals the probability that they all won’t be funded is (1-0.08)^N which for N=13 is 34%, e.g. you have a 2 in 3 chance of winning.

    Like

  7. zb Says:

    And I would like to see the data of probability of funding as a function of number of grants submitted, which probably doesn’t follow the random probability rules. The research institutes lobby had a number like that. Does anyone have a quick and dirty way to get such an estimate? would a DM survey give us anything? How about internal surveys?

    I’m guessing stopping rules might influence the stats, though maybe they shouldn’t (I.e. your rate of submission should not depend on whether you get a grant.)

    Like

  8. zb Says:

    Good article on Prasher. But, I think one that shows the skills needed to do science that get underplayed to the young and naive, rather than a systemic failure. Scientists do have to have an entrepreneurial spirit, sell their science, find collaborators and “investors” and ask for help and support. And, the have to have a fighting spirit.

    Does science loose when people who don’t have all those characteristics in addition to brilliant analytic minds, great lab skills, and perseverance? Maybe, a bit, but many people have both.

    Like

  9. Dave Says:

    And I would like to see the data of probability of funding as a function of number of grants submitted, which probably doesn’t follow the random probability rules

    This is where I have a problem with the “you need to submit 13 applications to stand a chance” stuff. It just doesn’t work like that because this assumes that each application has an equal chance of success, which cannot be the case when you are submitting 13 R01s in one year, and it doesn’t take into account A0/A1s, the investigator etc.

    Like

  10. physioprof Says:

    Also, there is likely to be a decent correlation between the scores of a given PI’s grants. Not really high, but it is there. So the PIs who are accomplished and doing good science and good grant-writers don’t need to submit as many grants as the ones who need a lightning strike to get funded.

    Like

  11. DrugMonkey Says:

    Everyone who has a good run thinks they deserve it for their awesome science and excellent grantsmithing, PP. Until they hit a dry spell and then it is all about the incompetent and biased reviewers who make errors of fact.

    Like

  12. DrugMonkey Says:

    The NINDS thing sounds dodgy. Using end of the year funds to get a jump on the next FY? And sooooo not the thing to do to stop the grant churning.

    Like

  13. toto@club-med.so Says:

    Eli: of course you do realize that, by formalizing “fighting chance” into “1 in 3 probability of failing even though you wrote 13 goddamn applications”, you’re not exactly lifting our spirits.

    Like

  14. physioprof Says:

    The NINDS thing sounds dodgy. Using end of the year funds to get a jump on the next FY? And sooooo not the thing to do to stop the grant churning.

    I disagree completely. The goal was to smooth out the effect of sequestration so that paylines did not have to plummet in 2013. This makes sense from an equity standpoint, as the vagaries of exactly when an application is submitted should not have a massive effect on likelihood of funding.

    Like

  15. The Other Dave Says:

    Also, there is likely to be a decent correlation between the scores of a given PI’s grants.

    There better be. Otherwise, peer review is just a waste of time, and they might as well just pick randomly from the submitted proposals.

    DM is correct statistically — each PI needs to submit on average 13 proposals. But DM is wrong from an individual PI perspective. As many have pointed out already, some PIs will submit only one or two and get funded, and others will submit several dozen before they get funded (or never get funded). The best predictor of future success is always past success.

    Like

  16. yikes Says:

    @Eli: You are doing the math wrong again.

    DM is doing the math right. So are you. You are asking different questions.

    This scenario seems to be a straightforward description of a geometric distribution. The expected value for the number of applications needed to get the first success will be 1/p, where p is the success rate. In this specific example, 1/0.08 is about 13. Of course, as pointed out by TOD, if your success rate is better than most, then go ahead and substitute your own personal p in there. Good luck quantifying just how special you are, though. It might not be so unreasonable to expect that your probability of success with an individual grant is about the same as everyone else’s. Especially since this includes the decision making properties of butthead reviewers (there are always some in the study section, right?).

    A bad thing about this math is that the curve is asymptotic – as success rates get lower, the number of applications needed shoots to infinity (n = 1/p).

    Eli, you are answering a different question, which is really about the cumulative distribution instead the expected value – what’s the chance of having at least one success in 13 applications? I agree with your calculation. But, like others, I am not feeling oh so reassured by it. There is an implied value judgement here – at what point should I feel good about my chances? 2/3 after 13 tries does not make me happy. Let’s say I need to be 90% certain I will get a grant, in order to plan my career. Then I need 27 or 28 applications at a success rate of 8%.

    If the success rate were 20% (and I’m doing the math right…might not be the case), this would fall to just 10 or 11 applications to have a 90% chance of at least one getting funded.

    Like

  17. The Other Dave Says:

    This is what I am thinking…

    1) Whatever the success rates, no one ain’t never gonna get squat unless they apply.

    2) If NIH and NSF don’t want to deal with so many applications, they need to stop making it necessary to send in so many applications.

    3) Statistically, I have about the same odds of getting an R01 as getting a paper into Cell, Nature, or Science. Which is better?

    4) About 90% of the time people spend working on proposals could be spent watching TV or masturbating, with no overall loss of productivity.

    5) If my time is worth $50/hour, and I spend 50 hours working on a proposal, then that’s worth $2500. If I buy $2500 worth of Powerball tickets, I have about a one in one-hundred thousand chance of winning. That’s still worse than NIH. But it gets closer the longer it takes me to do a proposal or the more my time is worth. So I should probably work at getting an NIH grant. But once I get that R01, I can invest all $250K of my direct costs in the Powerball. That will give me about a one in one thousand chance of never needing to apply for funding again. Not really high enough odds to make it worthwhile. Therefore I should apply for a non-modular grant so I can invest more in the Powerball.

    6) Instead of writing proposals, maybe I should be day-trading or selling real estate and funding my science with the proceeds. Might be more efficient.

    7) I should hire 13 postdocs and make them each write an R01. But I can’t pay them all off the one R01, so they will have to work for free. How can I get them to write proposals for free? Hmmmmm… I know: I’ll let them join the lab contingent upon ‘their’ (my) grant getting funded. And then while they’re in the lab I’ll make them write more proposals. It’s good training, right?

    …I don’t think #7 is that original.

    Like

  18. GAATTC Says:

    Wow, that was the best post ever. Well done TOD. I’m printing this thing out and putting it on the wall of my office next to teaching and research awards that don’t matter ’cause we live in a “what have you done for me lately?” world.

    Like

  19. TwoYellowsMakeRed Says:

    So from a statistical basis, you need to have put in 13 proposals to NIAID this year in order to have a fighting chance to get one.

    Umm…. no. Your 13 applications are not independent or random events. If they are, then you shouldn’t be in science. The chances of making the payline are probably bi-phasic. If you’re not funded in your 1st application, your chances on the 2nd may be better because it’s amended*. The 3rd may be better too, because you’ve had the benefit of 2 reviews to focus on the most important parts. After a number of failures, though, the study section(s) doesn’t think your idea is significant and/or well executed, so your chances start going down. Now, given that you can’t submit 13 sequential applications in 3 cycles, there will be overlap and different study sections and all sorts of confounders to figuring out the probability.

    * let A=success of 01, B=success of 01A1
    You could say P(B)>P(A), but if the 01 is successful, then there is no 01A1, so you’d have to say that P(B|Ā)>P(A), i.e. the probability of a successful 01A1 given the failure of the 01 is greater than the probability of a successful 01.
    Boom! Science, bitchezzzzzzz!

    Like

  20. Dave Says:

    I like option 4

    Like

  21. AP Says:

    I’m a NI/ESI and missed the NIAID R01 by 1%tile. I got a 13%tile with a 20 Impact/Priority score. Vagaries of life, indeed.

    Like

  22. DrugMonkey Says:

    Ugh. Very sorry to hear that. Cross fingers for end of year pickup!

    Like

  23. whimple Says:

    Ask your PO to R56 you.

    Like

  24. DrugMonkey Says:

    Good point Whimpie!

    Like

  25. lurker Says:

    What’s the cost benefit of 2-4 weeks total angst to write each of those 13 grants versus the miniscule effort to buy powerball tickets. Whatever to the notion that grant writing helps you focus your research plan, when it has become more salesmamship and going with the grain than proposing true innovation. At least powerballs gives you tiny sprit of endorphins before drawing. Endless grant writing is soul sapping and still a pricey ticket to play the NOA lottery……

    Like

  26. Grumble Says:

    A big billboard on my way to work advertises that the current powerball jackpot is $104 million. Isn’t that the amount that Obama promised for the brain mapping project? I momentarily entertained a fantasy of winning the jackpot, endowing my lab with the entire proceeds, and telling the media that Obama’s proposal is mere peanuts.

    Like

  27. Dave Says:

    I momentarily entertained a fantasy of winning the jackpot, endowing my lab with the entire proceeds

    Minus the IDCs……of course

    Like

  28. anonymous postdoc Says:

    I already play the powerball with the hope of creating an endowed position for myself. My PI has admitted occasionally doing the same thing. What I’m saying is you people are behind the times.

    Between taxes and IDC’s, funding an entire lab operation seems unlikely unless its a mega jackpot, since you would want to work off the interest rather than drawing down the principal. Depending on the size of the award, though, you could fund several fellowships in addition to one’s own salary.

    Running constant behavioral assays leaves a lot of time to work out these details.

    Like

  29. Grumble Says:

    Why I believe the stories that no one’s life was every made happy by winning the lottery:

    Mrs. Grumble: What would you do if you won the lottery?

    Prof. Grumble: Oh, that’s easy. I’d donate it to a foundation run by and for me, and I’d never have to worry about funding my research again.

    Mrs.: [stares]

    Prof.: Er, um, well …

    Like

  30. hatcat Says:

    I was just speaking to a program officer at NICHD. Their payline is 5th percentile for R01s

    Like

  31. NatC Says:

    @The Other Dave
    Whatever you end up planning to do, remember that #7 is problematic because of the government’s rule that you cannot use federal (including R01) funds to pay someone to lobby for money from the government (which includes writing those R01s).

    Like

  32. The Other Dave Says:

    @NatC: Oh, we have an understanding. They write the proposals before they are allowed to join the lab, or on their ‘own’ time once they join the lab.

    I would of course also never pay someone to work on something not directly related to the currently funded proposal. Gosh and golly, that would also technically be mis-use of funds, wouldn’t it?

    RAPE THE TAXPAYER! RAPE THEM!

    My only regret is that I wasn’t smart enough to go into the defense industry. Or medical insurance. Or banking. Academia is for losers. Although economics doesn’t sound too bad. At least I wouldn’t have to worry about data or knowing how to use Excel properly.

    …assuming there is a ‘proper’ way to use Excel. That program is retarded. If you use Excel for science, you are announcing that you are retarded. NO SERIOUSLY. I AM NOT JOKING.

    I do like a nice 3D pie graph now and then, however. More NIH proposals should have 3D pie graphs, in my opinion. And clipart. There is not enough clipart in science proposals. Or papers. I would pay for color clipart in my figures.

    Like

  33. anonymous postdoc Says:

    The Other Dave is providing a revealing glimpse into the stream-of-consciousness of all Third Reviewers everywhere.

    Like

  34. whimple Says:

    [write proposals] ‘or on their ‘own’ time once they join the lab’

    …also not allowed.

    Like

  35. Dave Says:

    That’s it TOD!! I’m putting clipart in my next proposal. I might at least draw a few chuckles.

    Like

  36. Dave Says:

    …assuming there is a ‘proper’ way to use Excel. That program is retarded. If you use Excel for science, you are announcing that you are retarded. NO SERIOUSLY. I AM NOT JOKING.

    Damn it. I always suspected that I was retarded. This confirms it.

    Like

  37. miko Says:

    I think the Other Dave is that crazed sorority emailer.

    Like

  38. Mike Pollard Says:

    I have a real problem with the idea that if the payline is X percent then all you need to do is submit Y applications to get funded. It just does not work that way. If the payline of your NIH institute is 10% (and many don’t publish paylines) then you need to write an application that is at least better than 90% of other applications.

    The study sections that I have sat on have never looked at an applications and said “OK, so the payline for [NIH institute if your choice] is X% and this is [your name]’s Y application so we need to give him/her a score that is better than X%ile”. Study sections don’t know how many applications each investigator submits or the final paylines for institutes.

    The best way to give yourself the fighting chance at funding is to identify the study section most appropriate to your work, and hopefully where there are people that know you and your work, and then submit the absolute best application you have ever written. And it does not hurt to get to know the appropriate PO and SRO assigned to your application, especially as you send in your A1.

    Like

  39. drugmonkey Says:

    the idea that if the payline is X percent then all you need to do is submit Y applications to get funded.

    Not what I meant at all and hopefully not what I said. It is more that random factors all have to align properly for you to make it over the fund line. You have to get the shake of the right reviewers, in the right mindset, with the right pile of other (shittier) applications in front of them. There is only one way to beat this game and that is to buy a lot of lottery tickets.

    The best way to give yourself the fighting chance at funding is to identify the study section most appropriate to your work, and hopefully where there are people that know you and your work

    This is undoubtedly the starting point. I have certainly secured the most triages and some of my best scores ever from the same study section. The former tells you that this is not enough. There are inevitable human factors having to do with appearing “too greedy”, the concept that “we just GAVE you one”, cultural factors having to do with certain types of application (say, risky versus workmanlike) and their biases about how a particular type of experiment “must” be performed. Not to mention their biases having to do with what aspect of Bunny Hopping is the best. And the fact that if you have more than one proposal in within a round, they are in direct competition with each other.

    , and then submit the absolute best application you have ever written.
    This is demonstrably nonsense. The “absolute best application” I’ve ever written is not funded. Probably the ones that did get funded don’t even line up as the next-best. I suspect that if I showed you a few dozen of my apps you’d be hard pressed to pick out the funded ones from the triages from the meh-percentile ones. Then the tales I can tell about the applications that have been reviewed well in sections that I’ve been on…

    It is a huge mistake for applicants to let perfect be the enemy of pretty-damn-good.

    And it does not hurt to get to know the appropriate PO and SRO assigned to your application, especially as you send in your A1.

    POs are independent of study section, of course. And yes, it is a good idea to get to know POs and SROs. But why put your eggs in a single basket there either? Some SROs that I am friendliest with have seriously screwed me over on reviewer assignments and some have clipped off the bad reviewer just in time to save my ass. They have assignment issues, man, and the idea that they are going to be bailing your ass out just because you talk to them is insane.

    Like

  40. Genomic Repairman Says:

    I think we should just say fuck it to the NIH and grovel at the feet of E Perlstein so he can help us crowdsource a mighty war chest of what $25k?

    Which by the way where is he going to do his work and to whom and for how much does he make out the check for indirect costs on that money?

    Like


  41. I suspect that if I showed you a few dozen of my apps you’d be hard pressed to pick out the funded ones from the triages from the meh-percentile ones.

    Dude, I bet I could pick out the dogs and lions!

    Like

  42. DrugMonkey Says:

    Without RePORTEr, genius?

    Like

  43. Mike Pollard Says:

    Having read your reply to my post I have to say that you sound amazingly bitter. How so? So some of your grants not been funded? Join the crowd. PO and/or SRO not bow and scrape at each of your emails/phone calls? They don’t actually work for YOU. Study sections not identifty your work as stellar? That makes it you….and everyone else.

    You may view getting an NIH grant as a lottery. But that is simplistic because it means that each application has an equal chance at success. That sort of logic is insane. It does not explain why there those who have maintained funding for decades while there are others who have little to no success. When you work at a soft money institution, like I do, it does not pay to base your career on lottery tickets.

    And, So from a statistical basis, you need to have put in 13 proposals to NIAID this year in order to have a fighting chance to get one does mean if the payline is X percent then all you need to do is submit Y applications to get funded.

    Like

  44. drugmonkey Says:

    you sound amazingly bitter.

    The word you are looking for is most likely “cynical”. Of the things that I am “bitter” about in my career, the way the NIH grant review game works is not one of them.

    . But that is simplistic because it means that each application has an equal chance at success.

    Of course it is a simplistic analogy. Applications do not have an equal chance in the sense that lottery tickets do. But neither are the chances lined up in strict accordance with some unified concept of omnipotent merit.

    It does not explain why there those who have maintained funding for decades while there are others who have little to no success.

    These decades that you refer to have seen success rates from established investigators change from damn near 50% to something under 20%. There is also a correlation of seniority with improved success that still is maintained. And yes, it is only recently that some of the most senior folks have finally come under the Lottery Rules that many others of us have had operating on our situation for years.

    So what? I don’t blog for them. They are not my audience, for the most part, and stuff that works for them doesn’t work for the younger crowd.

    When you work at a soft money institution, like I do, it does not pay to base your career on lottery tickets.

    When you work in a soft money job category like I do, it is insanity itself to think that this NIH funding game is one you can survive in by submitting one “perfect” grant to the same study section every four years.

    does mean if the payline is X percent then all you need to do is submit Y applications to get funded.

    You have serious reading comprehension issues, my friend. “fighting chance” is in no way equal to “guarantee”.

    Like

  45. Mike Pollard Says:

    Doesn’t matter how you want to slice and dice my comments, your bitterness just shines right through.

    it is insanity itself to think that this NIH funding game is one you can survive in by submitting one “perfect” grant to the same study section every four years.

    Well this is pretty much the approach that has worked for me for more than 20 years.

    No, please don’t respond. It will only take time away from your next submission to your imaginary lottery.

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: