R01 acquisition by successful and unsuccessful K01 applicants
September 13, 2011
The latest post at Rock Talk is fascinating. This figure in particular. It depicts the fate of successful and unsuccessful applicants for the quasi-faculty level training mechanisms, the “mentored scientist” K awards K01 (mostly PhDs) the K08 (MDs, basic and translational research) and K23 (MD’s, patient-focused research).
Apart from the specifics here, I think it is good to see some followup on the fate of unsuccessful applicants for a given mechanism. It lets us get a much better handle on the failure rate. An unsuccessful application rate only goes so far in telling us abut the state of the industry, so to speak. Lots of worried handwringing about “closing labs” because of dismal success rates for R01 grants can really only be evaluated by knowing how many labs have *actually* disappeared from the applicant pool. But I digress.
These differential outcomes on subsequent R01 or other NIH Research Project Grant funding for the un/successful applicants for each K mechanism are interpreted by the head of OER as reflecting the baseline need for training between the typical applicants for each type of K award.
“the benefits were more modest, as might be expected from a group that already has significant research training and experience.”
I would suggest that the more likely explanation is to be found in the “No Subsequent NIH Activity” rates for unsuccessful K08/K23 applicants. They are generally MDs. They can simply return to the practice of medicine if the whole “research thing” isn’t working out for them. Probably with fewer headaches and greater overall compensation as well!
Does it matter how you interpret this? Well, maybe. If the NIH is really keen to hook in the MDs to careers as NIH funded PIs then they have to adjust to this lower threshold for giving up. Bump up the success rates. If the disparity is indeed more to blame on the lack of prior training, they need to try to create more mechanisms to intervene before MDs reach the K-applicant stage.
DrugMonkey 
September 13, 2011 at 1:25 pm
I think the message for PhD types like me is that if you are at the K01 career stage, if your award is not funded, there is a 31% chance that you never try the independent route wheras if it is, its only a 16% chance. So basically failing a K01 doubles your chance of never trying for independent funding (unless i am misunderstanding what RPG is). On the other hand, getting a K01 vs not getting one basically has a very slight effect on your ability to get grants in the future (R01 + RPG is 43% vs 39%). So… why waste your time feeling bad about your poorly scored K01.
LikeLike
September 13, 2011 at 2:11 pm
Agreed Dave. Unsuccessful K01 applicants should keep moving forward, submit R grants as soon as possible.
LikeLike
September 13, 2011 at 2:55 pm
Comparing funded K## applicants to those K## applicants who fail to get funded is totally fucken stupid if you want to assess the effect of actually receiving a K##. This is because you’re just comparing two populations that have been pre-segregated into one that has been proven to be more effective at grant writing than the other. Rather, you need to compare the future grant-getting of funded K## applicants to people who never applied.
Think about it like the ill-fated R29–FIRST–program. If you compared those who applied for and received an R29 to those who applied for and failed to receive an R29, I’ll bet the applicants who received the R29 fare better at getting and keeping R01s later on. But the relevant comparison is between those who had an R29, and those who never applied. As we know, the effect reverses, and those who had R29s did more poorly in the aggregate later on.
LikeLike
September 13, 2011 at 4:47 pm
What is the point of the K01 mechanism, if it doesn’t promote the success of securing an R01 (27 vs 25%)?
LikeLike
September 13, 2011 at 4:51 pm
To delay it by a year, apparently.
LikeLike
September 13, 2011 at 4:54 pm
Joking aside, there is a longer report http://grants.nih.gov/training/K_Awards_Evaluation_FinalReport_20110901.pdf
LikeLike
September 13, 2011 at 5:02 pm
These statistics are painfully meaningless to the question whether having a K01 makes a particular person more likely to become a long-term R01-funded PI.
LikeLike
September 13, 2011 at 5:33 pm
…but they totally let the NIH LOOK BUSY! Am I right?
LikeLike
September 13, 2011 at 6:17 pm
I look forward to see the trajectory of K99 (both funded and non-funded) applicants. I am sure I won’t be the only one.
LikeLike
September 14, 2011 at 9:33 am
Rather, you need to compare the future grant-getting of funded K## applicants to people who never applied.
which population includes those that would be effective and ineffective grant writers, in theory. So you really have a two factor analysis in which you can’t identify the second factor within one of the groups of the first factor. seems to me that you are being overly demanding and insisting on an impossible analysis.
say you take the population of R01 applicants and work backwards to group them by whether they ever sought a K or not. You still have the contamination of who seeks a K- in theory those that think (or have mentors who think) that they *require* some Mentored Development. If they do, then they might be weaker-looking scientists than those who never tried for the K. Alternatively (and this is my bet-especially the K01s, I’m not familiar with the clinical mechanisms as much), those Ks are disproportionally coming from people with less-than faculty level appointments. Those would therefore be at a disadvantage on the openR market…
LikeLike
September 14, 2011 at 9:43 am
So the bottom line is that there is *no* way to ever know if receiving a K award actually does anyone any good at all.
LikeLike
September 14, 2011 at 9:53 am
I just think you need to look at the data you have available and make tenative conclusions while keeping in mind the necessary caveats. And dreaming up additional analyses you could do to triangulate on the question of interest. Kind of like science, you know…..
Now if you are asking me if I think the K awards “work” like they are intended to do on the face? no. I think they have two values- support faculty level scientists in transition and “evidence of prior grant support”. Neither of these values is specific to the K mechanisms as conceived but they are nevertheless a value.
The data Rockey put up identify one clear negative, which is the delay to first R01 by a year. My memory of policy is hazy, but wasn’t there some movement across this survey interval in terms of when/what awards K awardees were allowed to seek/hold? Something to do with the amount of effort that had to remain on the K even if the person had their *own* R award? so perhaps that has been fixed with more recent policy changes….
http://grants1.nih.gov/grants/guide/notice-files/NOT-OD-08-065.html
http://grants.nih.gov/grants/guide/notice-files/NOT-NS-09-015.html
LikeLike
September 14, 2011 at 10:45 am
Comparing the NIH K01 versus K08/K23 programs is comparing apples to oranges.
The K08/K23 programs are used by most of the NIH Institutes, and are used in the exact (or nearly the exact) same way.
In contrast, the K01 grant mechanism is used by a smaller subset of the NIH Institutes, and each Institute seems to have different uses/goals for their own specific K01 program.
For example, 14 ICs participate in PA-11-190 (K01 mechanism), and each IC seems to have different purposes for this program (see Institute-specific information here: http://grants1.nih.gov/grants/guide/contacts/parent_K01.html).
LikeLike