Delay, delay, delay
March 20, 2018
I’m not in favor of policies that extend the training intervals. Pub requirements for grad students is a prime example. The “need” to do two 3-5 year postdocs to be competitive. These are mostly problems made by the Professortariat directly.
But NIH has slipped into this game. Postdocs “have” to get evidence of funding, with F32 NRSAs and above all else the K99 featuring as top plums.
Unsurprisingly the competition has become fierce for these awards. And as with R-mechs this turns into the traffic pattern queue of revision rounds. Eighteen months from first submission to award if you are lucky.
Then we have the occasional NIH Institute which adds additional delaying tactics. “Well, we might fund your training award next round, kid. Give it another six months of fingernail biting.”
We had a recent case on the twttrs where a hugely promising young researcher gave up on this waiting game, took a job in home country only to get notice that the K99 would fund. Too late! We (MAGA) lost them.
I want NIH to adopt a “one and done” policy for all training mechanisms. If you get out-competed for one, move along to the next stage.
This will decrease the inhumane waiting game. It will hopefully open up other opportunities (transition to quasi-faculty positions that allow R-mech or foundation applications) faster. And overall speed progress through the stages, yes even to the realization that an alternate path is the right path.
DrugMonkey 
March 20, 2018 at 12:25 pm
Thanks for pointing this out, DM – many people have no idea how long the current process takes or how frustrating it is. I’m 645 days post A0 submission (discussed, not fundable), 372 days post A1 submission (under previous paylines), and 268 days post study section, and all I have is a “maybe”.
And yes, my F32 would have funded 12 months after submission of an A1 (year 3 of postdoc), except I’d been awarded another fellowship.
Since I don’t see the NIH process speeding up, the only reasonable solution is a single submission – we all need to move on, one way or another.
LikeLike
March 20, 2018 at 12:46 pm
Agreed generally that the speed of grant review is incompatible with the nature of short training periods. My best argument against your proposal is that it’s very valuable for trainees to have a chance to learn from and respond to study section comments. That said, they get plenty of chance for that type of interaction with their manuscripts, so overall I think your proposal is a good idea.
LikeLike
March 20, 2018 at 3:05 pm
I agree with one-and-done for F applications, provided you get one shot per training opportunity (Pre-doc, Postdoc 1, Postdoc 2 [if needed]). K awards are a big enough deal that A0 and A1 applications are more appropriate. That said, I could see an argument for replacing K99/R00 with NIAID K22 or R00-only style awards that fund the grant winner when they receive a faculty position without having a required 1-2 years of additional training.
LikeLike
March 20, 2018 at 3:37 pm
I agree with cjb. The rationale for the K99 phase (i.e., the need for more training) is often a contrivance anyway. If you’ve published enough to be competitive for a K99, you’re probably ready to be on the job market. However, it’s becoming more and more common for job ads to have funding requirements. Every school in the country wants to improve their NIH rankings and the easiest way to do that is to recruit people who have funding. So less K99/R00 and more old-style K awards makes sense to me if you want to actually get people out of their postdocs faster.
LikeLike
March 20, 2018 at 3:38 pm
it’s very valuable for trainees to have a chance to learn from and respond to study section comments
They can learn this once they get to the R-mech stage and extrapolate from their paper-responding training and what they have learned from PIs along the way. An extra 9-12 months in the postdoc just to see the reviewers didn’t really budge their score much? Not such a good tradeoff.
provided you get one shot per training opportunity (Pre-doc, Postdoc 1, Postdoc 2 [if needed])
One shot per mechanism. Move along the path. F32 > K99 > K-other > R-mechs.
LikeLike
March 20, 2018 at 9:46 pm
I applied for an F32 in august, 2016. I got a score of 28 and got my reviewer comments in mid-November (one week before thanksgiving). We talked to my PO 2 days before thanksgiving. I revised my F32 furiously over the next two weeks, and resubmitted on Dec 8. It was reviewed in March and I got a score of 11, and it was funded. My F32 started one year and one month after my initial application.
If there were a “one and done” policy, I would not have a postdoc, as I needed that funding to support my position.
Also of note, I had no help with my application. I knew no one who had done an F32, nor did my mentor. My department had no F32s. I convinced some complete strangers to share theirs with me, which was helpful – but it is no substitute for having support or mentorship from people who have experience with writing an F32. Contrast this to some postdocs I met at one of the other universities in my city (private school – I was at a state school). They had mentors who had experience mentoring postdocs in writing F32s, they had postdocs around them who had successfully gotten F32 funds – they had tons of support and help. A one and done policy might be fine for them given their access to resources. For applicants who do not have those resources or support, being able to apply again likely is invaluable.
LikeLike
March 20, 2018 at 10:00 pm
How many A1 apps did the IC fund the round that you got a 28?
LikeLike
March 20, 2018 at 11:41 pm
I get the F32, most fields have competitive national postdoc awards/fellowships. But can someone explain to me what is the point of the K99/R00? Why should postdocs focus on their own independent research lines before starting a faculty position? And why isn’t a startup package good enough to get the ball rolling once you get a TT position?
I can imagine that departments responded by prioritizing candidates with K99 and lowballing startup once NIH decided to fund this mechanism, but why was it necessary in the first place?
LikeLike
March 21, 2018 at 12:17 am
Because there was a perception that Universities were not hiring and to give an incentive to hire less-experienced trainees. The ultimate goal, I believe, was to try to fix the terrible optics of the “42 years of age before first R01” statistic. It failed to do so, of course.
LikeLike
March 21, 2018 at 4:31 am
DM – have you seen any recently updated age data from NIH? A quick search on teh googles shows only data through 2015
LikeLike
March 21, 2018 at 10:04 am
I thought the five year limit was supposed to solve this issue? As in there is a ticking clock for all the post-doc mechanisms that starts from when you complete the PhD. Five years out and your eligibility is up. Is that no longer in effect? Because it had a huge effect on my ability to even find funding sources to apply for as a post-doc, since I did adjunct teaching for a year post PhD, thus “wasting” one of those 5 precious years.
LikeLike
March 21, 2018 at 12:45 pm
No, 5yrs is no longer in effect. They reduced it to 4yrs post completion of Ph.D.
LikeLike
March 21, 2018 at 12:57 pm
“I can imagine that departments responded by prioritizing candidates with K99 and lowballing startup once NIH decided to fund this mechanism …”
This is specifically forbidden by NIH. K99 awardees have to apply for the transition to the R00 phase and the department chair has to submit a letter detailing the institutional commitment to the candidate, including a start up package that is similar to other recent ones, protected research time, facilities, and how the institution will support the new PIs efforts to obtain R01 funding. I don’t know whether any institutions have attempted to subvert the start up requirement or how strongly it is enforced by NIH, but the rules are very clear.
I don’t have a strong opinion on the value of the K99/R00 award, but the idea is clearly to try to pick rising stars relatively early in their careers and ease their transition from postdoc to R01-funded PI.
LikeLike
March 21, 2018 at 2:28 pm
I disagree DM. Basically, I have a problem with instituting rules that will end up screwing people under the label of helping them. Yes, reviewers and agencies create a holding pattern. Only giving half the fuel to the planes is going to improve things slightly for some, and result in a lot more crashes. There are other reasons for not getting funded in A0, other than the holding pattern. Same for the five or four year max post phd. Ultimately, the policy can still be made, but arguing that it is to help the postdocs is mean.
LikeLike
March 21, 2018 at 2:28 pm
A question: I have seen several cases recently where A0’s receive OK, but not fundable scores. People revise and resubmit, get a very good score on A1… then NIH funds the A0. My guess is that institutes are doing this to make things look better on paper, as if they are funding more A0’s and with worse scores. They can argue that thry are pulling people with poor scores into funding. But this is only in the books. It’s impossible to know, but it sure looks like those A0’s would not have been funded without the improved score on the A1. Isn’t this just Holding pattern version 2.0?
LikeLike
March 21, 2018 at 3:19 pm
Basically, I have a problem with instituting rules that will end up screwing people under the label of helping them.
This because you are over focused on the individual and in particular on that subset of individual that you (and others here in this thread) feel would wash out under one-and-done and will succeed* under endless-attempts rules.
The major change in thinking that you need to have for this is that the odds improve** for the first submission when you no longer have revisions and other multiple-attempts.
*If there really is some identifiable suspect class of applicant that can only get through on revision then perhaps we need to address why this is and apply some specific fixes. (Ginther showed that black PIs had to revise more to get into the funding band, even taking into account the overall disparity in the funding rate. So that was a case where both things need fixing. )
**after the painful recovery from the current process, of course
LikeLike
March 21, 2018 at 3:31 pm
People revise and resubmit, get a very good score on A1… then NIH funds the A0. My guess is that institutes are doing this to make things look better on paper, as if they are funding more A0’s and with worse scores.
They do and I have the same suspicion. Note that they also (and more frequently imle) will pull the A0 to fund if the A1 gets a worse score. I always assumed that was all part of their driving need to be able to tell Congress they only fund the most meritorious science.
it sure looks like those A0’s would not have been funded without the improved score on the A1.
Maybe they would have, maybe they wouldn’t have.
Isn’t this just Holding pattern version 2.0?
I’m sure this is part of it.
I also think that some of what this reflects is a sort of rolling-the-dice. Program may hold off on an “almost” score hoping that the revision gets a superlative score. Then not only would it make the stats look good (if anyone was averaging priority scores) but it also changes the political capital scenario.
Rgrants are easier to consider because of the diversity in POs who are assigned and this may be less clear if K99s are debated for funding all as a pool. But ultimately, you have a scenario where different POs (or competing interests within the same POs head may function the same) are advocating for different apps. The easiest thing is when they have an app with the best score and/or well within the tentative payline. Arguing for exception pay for grey zone stuff has to involve the expenditure of a more limited type of Program capital. This is what is behind the PO zombie mantra when you are arguing they should pick up your just-missed score, assuming that they really in their heart of hearts want to see it funded.
LikeLike
March 21, 2018 at 6:57 pm
“How many A1 apps did the IC fund the round that you got a 28?”
Not a single F32 was funded that round.
LikeLike
March 21, 2018 at 8:21 pm
another great reason for my plan. they can’t do this nonsense either. if they can’t afford F32s, no reason to dangle people. move them along.
LikeLike
March 22, 2018 at 7:02 pm
“This because you are over focused on the individual “. Agree. I am focused on the individual.
How about the issue raised in several recent posts about review variability being a feature, not a bug? Why should trainees only have one shot at rolling the dice? The more variability there is, the less it is fair to let a single try be deciding of a career. I am not saying that it is a guarantee that a second, or third, try will get a fundable score. But it is possible, and it isn’t always a holding pattern on purpose.
“So that was a case where both things need fixing. “, exactly. In the absence of a comprehensive fix, just putting time limits does end up biasing things. If anything, the time limits and holding pattern may not be the bigger problem. They may, however, be the problem that affects BSD’s and their progeny, so there may be more support for doing something there.
I understand that some of the number games played by PO are unavoidable, and done with good intentions.
LikeLike
March 22, 2018 at 8:05 pm
“A question: I have seen several cases recently where A0’s receive OK, but not fundable scores. People revise and resubmit, get a very good score on A1… then NIH funds the A0”
In all the cases I’m familiar with, this is simply due to timing.
Let’s say the PI submits an R01 in cycle II (June 5th), which would be reviewed in October. If the application just misses a published payline, some NIH Institutes may have other funding mechanisms such as Select Pay but that would have to go to Council. For cycle II, Council would meet in January and the grant likely won’t get paid until February.
If that same PI was lucky they may receive the summary statement in time for them to submit the A1 application by November 5th (or the PI was a new PI or was continuous submission eligible which allows them to submit past the Nov. 5 deadline and still be reviewed in cycle III). The A1 application would be reviewed in February, which would go to May Council, and paid in June.
In the scenario above (which happens quite often), the PI won’t know if the A0 is likely to get paid until just before the review of the A1. Because crazy things do happen, no sane person will withdraw their A1 application until after the NOA is awarded, which often doesn’t happen until just after the A1 is reviewed.
Also, sometimes the A0 gets paid after the A1 is reviewed because the published payline shifts a point or two, allowing for the A0 to be funded. Even if the A1 receives a fundable score, often it’s faster to simply pay the A0 in this case.
In this case, there isn’t any grand conspiracy to pad stats – sometimes it’s just faster to pay the A0 rather than the A1. As far as stats are concerned, when the NIH calculates success rate, they already lump the A0 and A1 as one submission if within the same fiscal year. Withdrawn applications don’t count towards stats only if they’re withdrawn prior to review.
LikeLike
March 22, 2018 at 8:37 pm
“Grand conspiracy” Exaggerating a little? Try WITCH-HUNT!!
LikeLike
March 22, 2018 at 8:53 pm
I’m talking about at least some situations where the PO says explicitly they’ve gone back to the A0 because the score was better.
LikeLike
March 25, 2018 at 8:29 am
Wow, DM. You really do believe in the hunger games.
I disagree with you on this one completely. In my experience, resubmission is not the process that slows students down. Instead, it provides them with learning experiences. One of my best PhD student ever took four (yes, four) tries to get his F30. That push to carry him through was part of his grit and speaks to his abilities to write grants. He learned a lot from this process and it made him a better scientist. (Before you start saying we should have trained him better, the specific story of why it took four is complicated and not one of training failures. He’s definitely an exception – most students get it in one or two rounds.) The important point is that this multiple stage revision did not slow him down on his scientific progress at all. We continued to do great science and continued to write papers along the way. Of course, that was part of the lesson. We actually had the backup money available, but he wanted that F30. The key was that we taught him how to keep the grants in the air, improving them each time, without slowing down the important thing, which was the science. The process will make him a better PI when he becomes one.
The problem is that too many people treat these mechanisms as grants instead of training opportunities, and too many labs do not provide backup money. What needs to happen is that people need to have the money to do the experiments anyway as backup. If the student gets an F3x or whatever, then the lab can expand to bring in a new person, but the original student has to not be dependent on that F3x. (That was what these training grants were originally intended for.)
The problem is not that we allow multiple submissions. The problem is that labs are holding people in positions because they are waiting on funding. The solution is to base going-on-to-the-next-stage on paper productivity and science rather than funding!
This one-and-done hypothesis is also a problem for diversity. It removes the ability to get a second-chance and makes it such that only already-well-prepared students will get those grants. While there are certainly diverse students who are well-prepared, we all know that students coming from difficult backgrounds are often not as well-prepared. Those students need the opportunity to get those second chances. Allowing those second chances helps us bring students who are less prepared into success.
PS. For all the people who are complaining about not getting it on the first round and having to do a second round, realize that the proposal is not that you get it funded on the first round – the proposal is that you never get funded!
PPS. DM, I don’t know why you still propose this. We have data from the attempt to apply this logic to the R01 mechanisms. Getting rid of the A2 did not improve the likelihood of getting funded on the first round. It didn’t work then and it won’t work now. It was such a failure, that they rescinded it and now effectively allow as many R01 resubmissions as you need.
LikeLike
March 27, 2018 at 7:47 pm
The problem is that labs are holding people in positions because they are waiting on funding.
Why do we always see the same problem and come to completely different diagnoses about the cause?
LikeLike
March 29, 2018 at 6:06 pm
Just a note that they basically extended the K99 phase of the K99/R00 by allowing one year NCE’s on a more routine basis now. https://grants.nih.gov/grants/guide/notice-files/NOT-OD-16-092.html
Which, while I benefited from it, seems to stick it to people who are in labs with less funding. They should have just moved it back to 5 years for you eligibility. It makes planning much easier.
Have any of you guys reviewed trainee awards? I would imagine they are a lot easier to review than full grants. There’s really no excuse for such long, long delays, tied to the 4 month NIH grant cycle. It’s unacceptable to have a grant that you have to apply to 2 years before you lose eligibility. And then they wonder why people have to extend it.
LikeLike
March 29, 2018 at 7:02 pm
“There’s really no excuse for such long, long delays”, really? feeling entitled Today? Remember that reviewing is still voluntary. Those participating still have all their other stuff to do.
“I would imagine they are a lot easier to review than full grants. ” not necessarily. You may not be considering the multiple documents that have to be reviewed.
Time needed to do a review is not only affected by the difficulty of the task.
LikeLike
March 29, 2018 at 11:33 pm
jmz4, let me clarify: I agree that the process of review and funding takes too long, and on the need for longer eligibility if the process takes this long.
LikeLike
March 31, 2018 at 9:17 am
In my experience, Fxx/Kxx training grants are usually much harder to review than R01s.
First, training grant study sections are usually very very broad, while R01 study sections are usually more field-specific. Serving on a Fxx/Kxx study section likely means you are reading grants far outside your area of expertise.
Second, training grants are supposed to be scored on a very complex set of issues which depend on backgrounds and interactions of trainee, mentor, project, resources, and future expectations.
1. Is the trainee likely to succeed? (Whatever that means)
2. Is the mentor a good mentor? (Hard to tell – is previous success from an ability to bring in good trainees or a sign of good mentorship? What proportion of previous people succeeded – usually we get lists of individuals who succeeded? What to do with young mentors without a track record?)
3. Is there a good training plan? (This depends greatly on what the trainee needs, which may or may not be well communicated.)
4. Is there an interesting project that is likely to produce impact and be something that will lead to good training?
5. Are the resources available to make this project work?
And finally, how much should you take into account the fact that the trainee is still learning how to write grants? And the fact that the trainee is learning to write grants means the grant is probably poorly written!
In contrast, an R01 is scored on whether the research is interesting and likely to provide impact.
LikeLike