No more supplementing NIH applications with post-submission updates
August 4, 2010
As I discussed in a prior post, the NIH used to permit PIs to submit additional materials in support of their application between submission and review. This policy was formalized at one point and perhaps I should have given more consideration to a suspicion that some of that formalization reflected annoyance on the part of the SROs.
Well, those days are over. A Notice (NOT-OD-10-115) in the NIH Guide informs us that while some administrative things can still be updated, these cannot:
Unacceptable post-submission materials (for all applications except those listed under Exceptions below) include:
* Updated Specific Aims or Research Strategy pages
* Late-breaking research findings
* New letters of support or collaboration that do not result from a change in senior/key personnel due to the hiring, replacement, or loss of an investigator
( Interestingly these items are still allowed for submissions for RFAs which have only a single receipt date. )
Impact? As always, hard to say. First of all, not everyone submits an update..IME on study section it was a small minority. Less than 10%? I’ve taken advantage of this policy on probably less than 20% of my applications…and probably fewer. Then you have to think about whether such updates have a meaningful effect on the review of the proposal, I would doubt it is a high number.
However. It seems inevitable to me that IF someone is going to get a boost from submitting work in progress in the 3-4 month interval between submission and review, it will be the most junior investigators who are just starting to get their labs up and running. So I think this policy is a BadThing.
And, oh, NIH? As usual, your policy changes would go down a heck of a lot easier with some simple data. What fraction of applications are supplemented with additional data? Of these, what is the success rate compared with the success rate of unsupplemented applications? Do these numbers differ between newer and more established investigators?
August 4, 2010 at 8:59 am
I really wonder if the NIH ever tries to solicit feedback on ideas from outside the IC’s?
LikeLike
August 4, 2010 at 9:14 am
I find this change to be a step in the wrong direction. In these fast-moving times, who doesn’t generate useful new data during a given 4-month interval. It seems very 20th Century to just hold a proposal static for 4 months.
LikeLike
August 4, 2010 at 9:28 am
Like I said, though, N-c, from my experience I don’t think everyone takes advantage of this with every application. So it is a question (answer buried in the CSR database) as to how much real impact this has.
I think from the CSR perspective this is all about streamlining and headaches.
LikeLike
August 4, 2010 at 9:33 am
Absofuckenlutely. And now that the supplemental materials are available on IAR from the exact same Web page from which you download the original application, it has been even easier for reviewers to keep track of and access the shit.
LikeLike
August 4, 2010 at 9:58 am
Some might argue that new investigators get a huge bump anyways…or perhaps the preliminary data is not as important on their apps? (Not sure I would agree with said people…just trying to play devil’s advocate)
LikeLike
August 4, 2010 at 10:19 am
NSF has done the same thing as of the July deadlines this year, but it is likewise unclear what the impact will be.
LikeLike
August 4, 2010 at 10:53 am
The New Investigator / ESI “bump” is entirely at the Program level. All data I’ve seen suggests that at the study section level, the noobs are still taking a hit. I would prefer to see study sections responding to the instructions about ESI / New I applications.
LikeLike
August 4, 2010 at 12:25 pm
Agreed that this is bad, bad, bad for new investigators. My PhD advisor swore by putting in supplemental materials. As far as I know he put one in for every single grant he submitted and he advised all his trainees to do the same. Guess those days are over now.
LikeLike
August 4, 2010 at 12:36 pm
Did this person ever mention whether or not it appeared from the summary statements that the supplemental materials were critical to the review? or even noticed?
LikeLike
August 4, 2010 at 12:43 pm
Yes, all the time. In fact, I read almost all of those summary statements (or they were read to us in lab meeting) and they were consistently mentioned in the reviews. I was in the lab for 5 years and the PI was(is) a grant writing machine.
LikeLike
August 4, 2010 at 12:44 pm
Are these little faces we get for commenting assigned to us as we continue to comment across threads? CPPs is quite fitting! I can’t make out what mine is… either way, very cute little feature.
LikeLike
August 4, 2010 at 1:06 pm
I’ve always sent supplementary material – new preliminary data, new papers published, and almost always the reviewers have commented on it. In fact I was told that you absolutely have to send this supplementary data, otherwise it looks like you are not being productive. And in both study sections I’ve dealt with I usually get an email from the SRO reminding people to send the supplementary material by a certain date. So the process is somewhat formalized.
LikeLike
August 4, 2010 at 1:07 pm
Why is my little icon so grumpy!
LikeLike
August 4, 2010 at 1:13 pm
Sorry about the gravatars folks, there are indeed a large number of grumpy ones. i’ll try one of the other options..
LikeLike
August 4, 2010 at 1:53 pm
Thanks for the heads up, DM! Just checked out the notice, and I’m happy to see at least that papers newly accepted for publication are still allowed as Supplementary info.
LikeLike
August 4, 2010 at 1:54 pm
zomg, am I a Christmas tree????
LikeLike
August 4, 2010 at 2:59 pm
Oh, I’m so glad my happy green monster is back and that angry fucken thing went back where it came from.
LikeLike
August 4, 2010 at 3:10 pm
This is messed up! First, I liked CPPs angry whatever that was, thought it suited him well. Second, i was some cool looking purple thing before and now I’m a huffy pacman ghost.
LikeLike
August 4, 2010 at 3:50 pm
Tough shit, dickweed.
LikeLike
August 5, 2010 at 5:20 am
I was told that new PI’s actually DO get a break in study section….the amount of preliminary results expected was lower…and you were given some credit for potential. Although, I would expect that this kind of behavior is VERY study section dependent…whereas the programmatic bump is more broad (although still subject to variability)
LikeLike
August 5, 2010 at 7:22 am
Sure, pinus, there are going to be reviewers that take the n00b instructions seriously. And times when an entire study section goes along. But is this *systematic*? there we have to rely on the bits of data that come out of CSR. For the most part those show inferior scores for noobs.
LikeLike
August 5, 2010 at 7:28 am
For new readers, start here for reference to the New Investigator outcome discussion
http://scientopia.org/blogs/drugmonkey/2009/09/25/nih-historical-success-rates-explain-current-attitudes/
LikeLike
August 7, 2010 at 8:57 am
Just found this post (still playing catch up with the blog merry-go-round this past week). I’ve seen several summary statements for K grants where the update made a HUGE difference in the scoring of the grant, although this change doesn’t appear to apply to K’s and training grants. I can see this having an even bigger impact on newbies applying for R01’s and fighting for every last piece of new data…info to back up this kind of decision would be nice.
LikeLike
August 7, 2010 at 1:36 pm
My reading is that it definitely *does* apply to K awards and individual F fellowship awards, but not to institutional training grants.
LikeLike
August 7, 2010 at 5:45 pm
In my experience on NRSA fellowship study sections, the only supplemental material that ever made a difference was news about publications being accepted. But that looks like it’s still permitted. In my experience on R01/R21 study sections, I’ve never seen supplemental be discussed (even when it did exist, which was rare). Nevertheless, at least we don’t have to have those “should we send it in now and hope we can send in the proof we can do it as a supplemental” discussions in the lab anymore!
LikeLike