Time to put on your big kid pants, NIH applicants!

November 12, 2010

Some person named Buds is fretting about the recent NIH policy which only permits a single revision of an unfunded application.

I have recently been through the “new” submission protocol at NIH. Its an extremely frustrating process. It almost seems that a requirement for “new submission” is that you change your line of investigation…….that you cannot do really, and that what sinks many. My grant (initially not funded at 18th percentile, last attempt) was administratively withdrawn and it seems that once they make up their mind, they wont change it does not matter how you try and point out the differences between the new grant and the old one. Seems like someone else knows your grant better than you do.

So have new sp aims which basically means a new direction of research.

Emphasis added. To point out that this is nonsense. Time to revisit, first of all, NOT-OD-10-080.

A new application is expected to be substantially different in content and scope with more significant differences than are normally encountered in a resubmitted application. A new application should include substantial changes in all sections of the Research Plan, particularly in the Specific Aims and the Research Strategy sections. There should be fundamental changes in the questions being asked and/or the outcomes examined. Changes to the Research Plan should produce a significant change in direction and approach for the research project.

Emphasis added. “More significant differences” in comparison with the original application. Nowhere is this calling for you to do a wholesale revision of your entire direction of research in your laboratory. If you see it this way, perhaps you are writing your Specific Aims too broadly, and your approach reads more like the laboratory program description than a specific project.

So knock that shit off. Now.

The DM and CPP are constantly going on about how you should have multiple grant applications going in at the same time. The only way to do this that I can see is if you manage to describe projects narrowly enough so as to permit breathing room for the other proposals.

And to do that, requires a simple trick in the mind of the PI. Stop thinking of a grant as a contract. The purpose of the grant application is to get funded for a general area of investigation. It is not to dictate your every scientific move for 5 years. Get the money and work on the topic as best you see fit. Given that, it should not be too hard to write up some new Aims and then, if you get lucky, go right ahead and work on what you see fit to work on.

Publish some cool papers and nobody is going to say boo.

After all, the days of competing continuation applications are just about finished.

Advertisements

No Responses Yet to “Time to put on your big kid pants, NIH applicants!”

  1. Malone Says:

    Amen brother! Probably everyone complaining about new applications should read this.

    JM

    Like

  2. CoR Says:

    “The purpose of the grant application is to get funded for a general area of investigation. It is not to dictate your every scientific move for 5 years. Get the money and work on the topic as best you see fit.”

    Thanks for that, seriously. Kinda shifted my perspective.

    “The DM and CPP are constantly going on about how you should have multiple grant applications going in at the same time. The only way to do this that I can see is if you manage to describe projects narrowly enough so as to permit breathing room for the other proposals.”

    Was my plan, but was just told by a PO to not submit 5-6 proposals per year (I haven’t done that so far) as PO has seen young PI’s do – PO suggested I submit fewer, better grants.

    So when you say multiples, what are you specifically suggesting?

    Like

  3. BikeMonkey Says:

    Never ever, young Cackle, should you submit an insulting pile of crap. Assuming you can write it into pretty good shape?.. keep ’em coming until you get your first notice of award. Rule of thumb? Try to keep it to no more than one per study section per round.

    If you start by hitting one grant per round (to 2+ study sections) this gives you multiple things in play at a time. Over time you’ll have some minor revisions to put back in- this can be your first time to hit more per round because the submission dates are offset.

    I also favor the one big / one small approach. For newbs especially, it makes sense to me that you may not have a lot of prelim data. So after the R01 goes in, try to put in an R21.

    Like

  4. BikeMonkey Says:

    Crap. Let me try that again.

    Starting off, one grant per round might be the best you can do. In say the first four rounds, try to mix up the study actions a little bit. Later, you will have revisions to write- if only minor revisions, a good time to submit two R01s in a single round (receipt dates offset by a month, plenty of time to revise).

    Like

  5. physioprof Says:

    Over the course of my career, I have averaged two R01 submissions per year.

    After all, the days of competing continuation applications are just about finished.

    Evidence? I seriously doubt this will happen. If you think about the fact that many of the changes in peer review over the last half-decade have been to cater to the outrage of the established investigators, not newbs.

    And even if it does, established PIs will simply write their new grants as if they are competing renewals, and the study sections wil review them as such.

    Like

  6. gc Says:

    “Stop thinking of a grant as a contract. ”

    Thank you for spelling this out. I had the exact same discussion with a collaborator yesterday who said the same thing…and I was thinking otherwise 2 days ago.

    Like

  7. BikeMonkey Says:

    PP, I mean the technical version of continuation. *Of course* the senior folks are going to get new grants on the same old topic. But a new grant is going to take even *less* fire for straying from the prior Aims.

    They are over because the timing of one revision and the low funding lines gives less room for error. So the continuations are going to get picked off on a steady, incremental basis.

    Like

  8. physioprof Says:

    My experience on multiple study sections is that competing renewals get a big scoring bonus for prior productivity. If you look at Jeremy’s stats, you’ll see that 40% of competitive renewals score within the 15%ile, while less than 20% of new grants from established investigators do so. (ESI and NI new grants do even worse, of course.)

    Now there has gotta be a selection bias here: If you weren’t productive in the prior project period, you sure as fucke aren’t gonna submit a competitive renewal.

    Like

  9. Tman Says:

    Nice post DM. It will be good if you could expand more on why a proposal funded by the NIH is not per se a “contract”. Particularly, in terms of the views and expectations of those who performed the scientific review and of the funding institute.

    Stumbled upon this when I found out Arlenna moved to scientopia and when I was reading her posts.

    Like

  10. arrzey Says:

    NIH grants have never been “contracts” – there are contract mechanisms for that. Tman, “not a contract ” means that you are not obligated to follow the line of research for which you get funded. If something more exciting comes along that is PUBLISHABLE, go for it. It means when you apply for the next funding, that study section won’t give a hoot about what your last grant was FOR. All they care about is What You Have Done Since Then. Study sections care about your effort and accomplishments. It’s also good to try and keep it vaguely within the mission of the institute which funded you. Or target a different institute with the next submission.

    There is a tricky bit to this advice. If you write grants to the exclusion of getting papers out the door, then it doesn’t matter, cause if you’re not publishing you’re not going to get funded. And, if you don’t have a big lab full of energetic postdocs/fellows/students who are producing results & papers, then writing grants may not be the best strategy.

    And finally – if your multiple grants suck, it doesn’t matter how many are submitted. Make sure you get advice elsewhere (ie senior funded people who are comfortable enough to tell you when you’re full of it) and a reality check on whether you are sending garbage or, worse, hanging onto something too long after its ready to go.

    Like

  11. Tman Says:

    arrzey, that is comprehensive, and I like every word of it ranging from why it is not a contract to how to be successful in renewals (publish papers) to having someone in the stature of a mentor to point out the crap. [it sounds obvious when I say it like that, but I can see how PIs who publish tons of good papers have continuous funding from the NIH].

    Just to gain more understanding, why then who gets funded by the NIH is based on a proposal which has “very” definite directions and specifics (many times, we get penalized for the lack of specifics in the proposal)? And why proposals gets scored based on the “specifics” in the proposal? So then, are scientific reviews intended to pick the best PI with the best direction judged based on institute’s priority and not necessarily based on the best idea? I guess I intuitively understand what’s going on, but would love to hear more.

    Like

  12. drugmonkey Says:

    As with much of life, young padawan, the NIH “system” of funding extramural research is laden with many contradictions and paradoxes.

    Like

  13. arrzey Says:

    @Tman – beyond the contradictions & paradoxes (oh so true), there is also answering the question that all study sections ask: Can the PI do the work? And then, (if we give ’em great big gobs of $$), will they? Publications makes it easy for them to answer this question.

    Institutes have priorities. Talk to NIH staff before you submit (which has been said here, many times). Find out if they want your project. The priorities come if you fall into the nebulous grey area (go back and read DM’s posts on funding data).

    But NIH staff does NOT score the grants. Reviewers also have priorities:”this area is hot!” or “this is a good proposal, but why should I be interested in this question?” One of the most important things to remember (which has been said here, many times) is that you want your reviewers to be your advocates. You want them thinking that you and this project (and both are valid criteria for funding) are just the best thing ever.

    When I talk to the peeps (mentees) about how to write proposals, the advice I got (back in the Eocene, but now modified for current forms): 70% of your time on the Specific Aims, 20% time on Significance and Innovation and 10% on the rest. Really.

    The final answer to your implied question is: grants are judged on a number of things, and the revision of reviews and forms is a small attempt to make the rules align with reality.

    Like

  14. Tman Says:

    Thanks aarzey for the in-a-nut-shell responses.

    I don’t necessarily agree with the 70%, 20% and 10% split you mentioned, though.

    Like

  15. Malone Says:

    For what it is worth, I listened to a talk by Dr. Kitt, CSR deputy director at UCSD. She gave two suggestions regarding new submissions: 1. ask a slightly different question, and 2. slightly change the direction. The reason being, A1 not funded means that reviewers aren’t directly saying but implying that the specific questions / directions need adjustment.

    That sounds reasonable, fair and could be followed. Unless, there is an Einstein out there thinking…. “fuck the reviewers, I am going to continue working on my general relativity theory”.

    Like

  16. BikeMonkey Says:

    Well put, Malone.

    Did Kitt say anything else interesting?

    Like

  17. Malone Says:

    The talk was deceivingly known and a common theme about CSR submission; but I guess one always picks up the nuances as understanding improves. Here are some things slightly new:

    1. CSR’s future plan to the error correction window to 0 days

    2. CSR’s future plan to stop accepting supplementary progress or data from submission to IRG review. The reason being, not everyone submits supplementary data, the reviewers are not required to read them and so it doesn’t provide a level ground.

    3a. When someone asked about priority score calculation, once and for clarified that the “overall impact score” given by each assigned reviewer is not an average of the 5 criterion scores listed in the SS. The overall impact score is not listed in the SS. And every member in the study section are eligible to participate in the discussion and give an overall impact score (even though some may be checking their e-mails, which is not allowed, during discussion). Priority score is just a mean of the overall impact score. I could see why she went on to explain the details because I still see posts in forums complaining that the priority score calculation (as the average of criterion score) is wrong or something.

    3b. Criterion scores are not discussed during IRG review, so they don’t reflect what was discussed. Only the overall impact scores are discussed and adjusted.

    4. In connection to the “new submission” issue: someone asked why did they started the A1 only rule. She said that somehow the way grants were funded were based on an unwanted, undesired and an unintentional law of “if you wait long enough you will get funded”. So, there is a long line and that became the norm. The people most affected by this waiting game are the young investigators. So, if not funded by 1 resubmission, then the reviewers and institute aren’t saying directly but implying that the PIs need to seriously consider revising the questions and their directions. No need to change the research area, but critically evaluate the questions and/or the directions.

    5. Everybody knows this, but she insisted that PIs should check the previous rosters to understand the composition of each study section to identify which could be most suitable study section. Any preferred study section and any information about multidisciplinary nature of the grant should be mentioned in the cover letter. Seems obvious, looks like many people are not doing that.

    6. And reinstated that reviewers check whether the proposal can exert a “sustained and powerful influence in the chosen research field” to determine the overall impact. Straight forward too.

    Like


  18. Criterion scores are not discussed during IRG review, so they don’t reflect what was discussed. Only the overall impact scores are discussed and adjusted.

    Good SROs remind reviewers that it is appropriate to revise criterion scores and critiques after the meeting if scores/opinions have changed based on the discussion at the meeting.

    Any preferred study section and any information about multidisciplinary nature of the grant should be mentioned in the cover letter.

    It is malpractice not to discuss (via phone or e-mail) your application with the SRO of the study section you think you want to target to determine whether your thinking actually makes sense. Good SROs are happy to help with the targeting process, and I have had extensive discussions with SROs about this. On occasion, the outcome of such a discussion has been a revision of my thinking about where I want my application to go.

    Once you have reached a common understanding with an SRO that her study section is appropriate for your application, you should then include in your cover letter the fact that you discussed the application with Dr. So-and-So, and she agrees that your application should be reviewed by her study section. The people in receipt and referral see a reference to the SRO like this and just immediately grant the request.

    Like

  19. Malone Says:

    I know this is a wrong thread for this question, but posting anyways.

    Just realized that rolling the FY11 budget back to FY08 level means the NIH budget level without the stimulus money. I think that seems reasonable to me given that the stimulus money is never intended to stay forever. And it will defeat the purpose of the stimulus concept and next time if the federal gov. wants to give an insulin shot, it will have go to a very high level of stimulus. I am not conservative, but this rolling back to FY08 level makes sense to me (of course adjusted for inflation). Am I missing something here?

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: