Extensive guidance from NIH on Unallowable Resubmissions of Grant Applications

March 2, 2011

As we’ve discussed before, the change to permit only a single revision of a NIH grant has thrown a lot of uncertainty into the extramural research force. Although one obvious solution would be to just submit another “new” grant that is a thinly disguised re-working of the prior unfunded version, the CSR of the NIH is on to us. They have been working diligently to convince us that they will weed out such dodges with extreme prejudice.
I have just noticed that they have an extensive description of their screening process up on a web page. Complete with some examples and scenarios.
Key things that caught my eye include:

Which Applications Are Problems?
· Applications submitted as new (A0) but appear to be resubmissions (A1)

Yikes. So even if you get triaged and have criterion scores in the 7s and 8s, you can’t just start over with an A0? The decision isn’t up to you. You have to resubmit.
More after the jump…

When we compare applications, we examine all parts, particularly the Specific Aims and Research Strategy sections using text comparison software and the expertise of those mentioned above. Subsequent versions of an application may be quite different and are still considered to be overlapping if many of the hypotheses, aims, and/or outcome measures are the same or highly similar–particularly when the “new” application is responsive to the critiques of another application.

emphasis added.
Geez! text comparison software? This is BS! I wouldn’t be surprised in the least if I 1) have stylistic elements to my writing that don’t change much and 2) have grantsmithing habits that I deploy explicitly because they seem to have been successful in prior grant reviews. Not to mention I freely suggest phrasing to my colleagues who are looking for grantsmithing advice.
The last bit makes me go…hmmmmm. I mean this is pouring salt in the wound. I have the distinct impression that to whatever degree my grantsmithing has improved generally over the years, it is in no small part due to being “responsive to the critiques of another application”. I mean seriously. You have a study section harp on about five specific issues….you are going to tend to have other ideas that go to that study section….and you aren’t going to apply what you learned so painfully with the last go-round to your current proposal? Gaah! (As always, a particular little stiff-arm to the newbs.)

Examples of Changes that Could be Substantial
* Changing the disease model

Brings up some interesting implications for those of us in the substance abuse fields, does it not? Alcohol and “all other drugs” currently enjoy the distinction of different ICs. Pretty clear evidence of being a different “disease model”, right? Yet many of our colleagues work on showing the similarities of the addiction process, regardless of the drug of choice (including Alcohol). Much of this understanding is powering the upcoming merger of NIAAA and NIDA into one Institute. Even within NIDA’s current portfolio, different drugs could be seen as different “disease models”. Are they? Sufficiently different I mean? Are two different viral infections different? Two closely related bacterial infections? How different do the cancers have to be to constitute a different “disease model”?
Grrr.
You are allowed to contest the rejection of your application.

The Arbitration Board is consulted if an applicant rebuts the previous decision to the CSR Director. This board includes 12 scientific staff from across NIH, meets once a week and adds additional consistency to decisions. No members of the Scientific Overlap Committee or Division of Receipt and Referral staff serve on this board. CSR’s Director finalizes the decision of this board and communicates the result to the applicant. The Division of Receipt and Referral takes action as needed.

hahah, excuse me if I don’t fall all over myself believing this process will be that useful. As with any other situation in which PIs wish to appeal NIH decision making, my assumption is that there is no real motivation for NIH to do anything other than stonewall.
One last interesting bit:

NIH now has a 37-month limit for resubmissions. If you do not submit an A1 version of your application within the 37 months of your initial submission or if your A1 is not successful by the end of this period, a future application you submit on the same general topic will be considered a new application. Given the pace of scientific discovery, NIH would expect this application to manifest substantial changes in its scope and approach.

Yay! If you can just hang on for three years, you can try again. lolsob, as the kids say, lolsob.
I do like the fact that it is tied to the original submission because what this really means is a much shorter timeline. Say your original went in this round, Feb 5, 2011. Reviewed in Jun/Jul. Say your summary statement comes out in time for you to get your revised application in by Nov 2011. You will know your dismal fate one year after original submission and your next possible submission date would have been 17 months on. Realistically, however, we should insert an extra round in between the first two versions. Assuming additional data have been requested, teaching loads, etc. Its a good bet that reality will have you pushing it back to a Mar 2012 resubmission of the A1, review in Jun/Jul 2012. When that review disappoints, you’d be looking at Oct 5, 2012 as your next available new grant date. This is a 21 month timeline. When you look at it that way, they are only (“only”) inserting another 16 months delay. Sounds better than 37 months anyway*.
__
*hey, I’m trying to look at the bright side here 😦

31 Responses to “Extensive guidance from NIH on Unallowable Resubmissions of Grant Applications”

  1. becca Says:

    This is the part where the students laugh and wonder how long it will be before profs complain that journal articles are going through text comparison software.
    You brought it upon yourselves, really. With turnitin. Serves you right.

    Like

  2. Lorax Says:

    Im pretty sure I can say, told you so, but I would take me too long to find the appropriate post/comment.
    *curls up in the corner and ponders the horror of full-time teaching of med students*

    Like

  3. Neuro-conservative Says:

    In an era of single-digit paylines, this policy is patently absurd. It accentuates and reifies the essentially arbitrary distinction between a 9th percentile A1 and an 11th percentile A1, banishing the latter into oblivion.
    At the very least, there should be some sort of “manual override” — for example, I would propose a letter of support from Program, as is currently required for applications in excess of $500K (direct) per year.

    Like

  4. whimple Says:

    In an era of single-digit paylines, this policy is patently absurd.
    Not really. In the era of single-digit paylines, ALL policies are absurd. It is unsustainable to have a 90% discard rate on highly trained, heavily invested personnel. It’s just a question of the manner in which the system crashes at this point.

    Like

  5. Orac Says:

    Beautiful. I have a lot of what I call “canned text” that I use for background and justification, not to mention canned text that describes methodologies that I use. These have been honed to a fine degree over years of submissions and resubmissions. What this policy is more likely to do is to subject the reviewers to more and more poorly written grants, as PIs try to jigger good text to make it less likely to trigger the software.

    Like

  6. BugDoc Says:

    “hey, I’m trying to look at the bright side here :-(”
    Yay, there’s a bright side! I had been looking for one, but hadn’t found it yet.

    Like

  7. Odyssey Says:

    This really is going to favor existing multi-R01 (or equivalent) labs. They’re the ones most able to devote resources to developing new projects in the event an A1 is not funded. The corollary of course is it’s really going to screw the n00bs.

    Like

  8. Eric Lund Says:

    What Orac said. In any proposal, no matter which agency you are submitting it to, there is a certain amount of boilerplate text that has to be there: why the proposal is suitable for the program, the methods you are using, who you are and what expertise you bring to the problem, etc. Stuff which changes only slightly, or not at all, from one proposal to the next.
    @becca: I don’t know about your field, but quite a few established people in my field would say, “Bring it on!” I’ve seen cases of entire paragraphs lifted lock, stock, and barrel from other people’s work, including mine. They forget that the old Tom Lehrer song is supposed to be a joke:

    Plagiarize Let no one else’s work evade your eyes Remember why the good Lord made your eyes So don’t shade your eyes But plagiarize, plagiarize, plagiarize! Only be sure always to call it please, research.

    Like


  9. Whimple (#4) wins the internet. Resources are so scarce, any system will appear arbitrary and capricious.

    Like

  10. DrLizzyMoore Says:

    There are only so many ways that you can certain key elements in your grant..which are unlikely to change, even with your changing projects. So I guess: MFO displays a biphasic developmental cycle will become: MFO exhibits a sequential growth phase shift? Does that even make sense? WTF, NIH! Sometimes, I think that they hate science.

    Like

  11. drdrA Says:

    1. Eric Lund- I think I love you. Anyone who can quote Tom Lehrer wins in my book.
    2. DM- Seriously. SERIOUSLY. Remember that recent hubbub about bringing back the A2.. …. in part because the definition of ‘substantially’ changed is open to interpretation. And some bloggers waved their hands and said- just repackage, recycle, re-invent.
    At that time apparently some bloggers did go back and read the actual guidelines for what “substantially changed” means- and the definitions you cite are precisely the reasons why certain individuals were so worked up that an unfunded 9th percentile A1 would get dumped over the cliff.
    !!!

    Like

  12. DrugMonkey Says:

    What can I say, doubledoc, I was wrong. They seem to be quite serious about this.

    Like

  13. drdrA Says:

    Well DM. My bad for not posting the link to the csr page in the comments at the time we were having that discussion. I actually remember feeling shocked that guidelines were so strict.
    Somehow I got the link from from somewhere- I *thought* that it was in a blog post at either rock talk, nexus, the niaid blog, the niaid funding newsletter that I get, or writedit’s place- but for the life of me I can’t find the link looking back.

    Like

  14. anonymous Says:

    Is it possible to submit a declined A1 as a new RFA grant?

    Like


  15. Dude, why the fucke are you scaring people unnecessarily with this nonsense? It is *not* hard to meet these standards! For one thing, you emphasized the wrong fucken text in the standard. Try this:

    if many of the hypotheses, aims, and/or outcome measures are the same or highly similar

    In order to be considered a resubmission, *many* of the *hypotheses, aims, and/or outcome measures* have to be the same or highly similar. All this gibberish about how fucked you are because you use boilerplate in your background/significance section or stylistic wording of your aims is fucken ridiculous.

    Like


  16. So even if you get triaged and have criterion scores in the 7s and 8s, you can’t just start over with an A0? The decision isn’t up to you. You have to resubmit.

    BULLSHITTE! Unless you are a complete doofus about it, you can definitely start over with an A0. Talk to your Program Officer and SRO and they will help you navigate the process of making your application different enough to pass muster. It is not that fucken difficult!

    Like

  17. DrugMonkey Says:

    ’cause I’m on the scared train now, PP. Maybe I’ll feel better about it next week….

    Like


  18. Don’t be such a motherfucken fraidy cat, holmes! This kind of shitte just gives those of us with a steady hand and nerves of steel a further competitive advantage against the fearful hordes.

    Like

  19. becca Says:

    “This kind of shitte just gives those of us with a steady hand and nerves of steel SROs and POs that are already on our side from a copious infusion of cookies a further competitive advantage against the fearful hordes.”

    Like

  20. lylebot Says:

    You guys are making an awful lot of assumptions about how that “text comparison software” will work. I seriously doubt it would be based on simple word overlap. If anyone has put any thought into it, it will not match two proposals on two different topics that happen to be written by the same PI.
    Very smart people spend their whole lives researching this sort of thing, you know.
    Now if I was relying on the NIH for my livelihood, I definitely wouldn’t be happy about them using “text comparison software”. But let’s not jump to unwarranted conclusions about what it will do.

    Like

  21. halp Says:

    What do you do when the reviewers and committee freakin’ LOVED your grant (according to all the feedback you got) but then didn’t fund it? Do you veil and resubmit with more preliminary data and differently worded specific aims, with the idea that if they loved it before, they’ll love it again? Or do you submit something utterly and completely different, despite the fact that you have been told they would love to fund this research but it just missed the cutoff?

    Like

  22. Gruffi Gummi Says:

    “they will weed out such dodges [applications] with extreme prejudice
    First, the NIH should weed out the bumbling clowns on the review panels, also with extreme prejudice. No lip service to “enhancing peer review” (another newsletter from our dear Jeremy Berg in the mail today) is going to substitute integrity, professionalism and an institutional culture supportive to these virtues.

    Like

  23. crystal doc Says:

    hahah, excuse me if I don’t fall all over myself believing this process will be that useful. As with any other situation in which PIs wish to appeal NIH decision making, my assumption is that there is no real motivation for NIH to do anything other than stonewall.

    Anecdatum: My “new” R01 was recently rejected on grounds of being a resubmission, and I went through the appeal process. I spoke with a program director at NIH who vetted my 1 page argument in advance and thought I had a very good case, but no dice, as it turns out.

    if many of the hypotheses, aims, and/or outcome measures are the same or highly similar

    I would read this as OR rather than AND, and be aware that hypotheses and aims here may be interpreted VERY generally– e.g. if you work with a protease and look for substrates that get cleaved, or a kinase and look for substrates that get phosphorylated, to impact upon some downstream process or phenotype, you might totally change the methodology you use, the hypothesis with regard to candidate substrate molecules and pathways involved, and CSR may well regard that aim “unchanged”!

    You have a study section harp on about five specific issues….you are going to tend to have other ideas that go to that study section….and you aren’t going to apply what you learned so painfully with the last go-round to your current proposal? Gaah!

    Yeah, this is the shitz. I talked to deputy director of division of receipt and referral after my appeal was denied, and was told that the biggest problem was my *many* changes were all viewed along the lines of too responsive to the past critiques. My past summary statement was a grab bag of like a bazillion things the different reviewers hit on; there wasn’t a real consensus. Almost any change *might* have been construed as responsive to *some* comment on that statement, especially when CSR is being very loose about what they view as “responsive”. Did even one reviewer think there was not sufficient translational emphasis to a previous grant, and now you have a whole shiny new aim bringing in clinical collaborators and patient samples? Gee, this looks directly “responsive”. You didn’t have a good antibody and took some flack, but now you’ve got a better one in your preliminary data? It couldn’t just be that you’d been working on this all along, knew it was a weakness from the start, and finally have got ahold of a better reagent; no, you’d never have been trying for better Westerns were it not for that brilliant reviewer who pointed the way. Responsive, responsive, responsive. My current guess based on this n=1 experience is that you need not one but *two* wholly new aims, or else just wait out the 3.4 years.
    Final note: I was told by division of receipt and referral that at present the comparisons are all manual, not text comparison software– not sure if that may be something they roll out soon, or just a scare tactic.

    Like


  24. What they absolutely must be doing is as a threshold analysis looking at the submitted Specific Aims page and comparing it to previously submitted Specific Aims pages from the same PI. And only if they see something suspicious, do they then go and start looking at old summary statements to see if you were “responsive”. There simply isn’t time for them to be doing the “responsive” analysis for every single new grant submitted, and that is certainly not a kind of analysis that could ever be done automatically.
    Look, the bottom line is that you’ve gotta make your specific aims look as different as possible, so that you don’t trigger that textual threshold and attract a more detailed substantive analysis. So change the verbs, change the nouns, refer to things in as different a way as is possible.

    Like

  25. DrugMonkey Says:

    And less we lose sight of this in our grant geekery…
    Sorry to hear you got screwed on this crystal doc. Really, really rough out there. And thanks for sharing your story with us, no doubt you’ve saved someone some similar pain.

    Like

  26. whimple Says:

    The upside is maybe they’ll stomp out those 5+ different R01’s the crystallographers are awarded to determine the structures of 5+ different molecules.

    Like

  27. DrugMonkey Says:

    As far as I can tell, whimple, this policy does nothing about preventing substantially similar applications from different investigators being funded.
    (or did you mean 5+ search/replace grants within a single group? does that happen?)

    Like


  28. (or did you mean 5+ search/replace grants within a single group? does that happen?)

    I once reviewed a P01 where two of the subprojects were search/replace jobbes with the same experimental approaches and hypotheses, only in two different parts of the brain. It was kind of an interesting discussion in the study section. On the one hand, they were interesting experiments, and they were worthy of performing in these two different parts of the brain, and they probably did require the full budget resources of the two subprojects. But we just couldn’t get past the irritation factor of the copy/paste search/replace jobbe, and we shitcanned the whole fucken thinge.

    Like

  29. whimple Says:

    or did you mean 5+ search/replace grants within a single group?
    Yes.
    does that happen?
    Yes, it does.
    So the question is whether CSR also strikes applications that look similar to currently funded grants from the applicant, in addition to looking at the other pending and unfunded applications from the applicant. They should have started doing that years ago.

    Like

  30. DrugMonkey Says:

    I guess I have to agree with whimple to the extent that it seems best for the advance of science in tight budget times to get very serious about overlap in the entire OER portfolio. OTOH, the specifics don’t match. The A(2) scrutiny is directed at ideas that have been judged insufficiently important for funding. whimple refers to ideas that *have* been found worthy of funding. So it isn’t quite the same thing.

    Like

  31. whimple Says:

    whimple refers to ideas that *have* been found worthy of funding
    Not quite. I refer to ideas that have been found worthy of funding in the absence of this kind of scrutiny

    Like


Leave a comment