The NIGMS Director Jeremy Berg is da CandyMan!
September 14, 2010
He just keeps that data crack flowing for the grant geeks in the audience. Today’s analysis breaks down the NIGMS funded and unfunded applications by Investigator status. Early Stage Investigator and New Investigator (collectively never had a major R01 award before, distinguished by ESI’s having to be no more than 10 yrs out from terminal degree) applications are identified as are Type 1 (new submissions) and Type 2 (competing continuation applications for established investigators.
A plot of the overall impact score versus the percentile for 655 NIGMS R01 applications reviewed during the January 2010 Council round. Solid symbols show applications for which awards have been made and open symbols show applications for which awards have not been made. Red circles indicate early stage investigators, blue squares indicate new investigators who are not early stage investigators and black diamonds indicate established investigators.
Yowsa, data galore!
Go over to his post, he has a cumulative percentile breakdown that is pretty fascinating too.
Director Berg is once again to be congratulated for really leading the way at the NIH. Transparency of this type cannot help but accrue to the benefit of Institutes that are making funding decisions in a reasonable way. Sure, there are going to be cases to explain such as the poor sucker with the fantastic score that didn’t get funded. But these data point to the relative consistency with which their apparent exceptional funding decisions (aka “pickups”) are made. These types of plots (especially the cumulative one he included but I’ve not poached here) also help us to assess the degree to which study sections are disadvantaging ESI or NI apps in their scoring and how so. Are the truly excellent top 10% getting their due but then things break down in the grey zone? Are study sections assigning good ESI/PI apps to the scored but decidedly not-good ranges of 25-35th%ile land? Do these relationships change over time as the NIH commitment to ESI/NI investigators takes hold?
DrugMonkey 
September 14, 2010 at 8:21 pm
I met Berg a couple of years back. Now imagining him as Gene Wilder/Willy Wonka (http://vimeo.com/4571649). Can’t. Stop. Giggling.
Also, feel really bad for poor 3% square.
LikeLike
September 15, 2010 at 1:10 am
Re: 3% new investigator. I was on study section with a grant from an elite senior investigator (foreign who came to the states) and checked the “new investigator” box. While technically true, this was not the intended recipient. I suspect something like that was happening here. (or their mouse/human sections were woefully inadequate and lacking protections).
LikeLike
September 15, 2010 at 2:04 am
What amazes me is that it allegedly took so long for NIH to catch on that senior, established folks were sucking up all that schweet NI money…
Creation of the ESI was ok but punished a class of folks they didn’t need to punish. Why didn’t they just make NI status depend on less than 4-6 years in faculty appt? Would’ve been fairer IMO.
LikeLike
September 15, 2010 at 2:08 am
Because that wouldn’t have been as effective at reversing the trend of increasing average age of first R01. Of course, it does this by helping to ensure that some poor old fuckers *never* get an R01, and thus don’t contribute to the rising average.
LikeLike
September 15, 2010 at 3:59 am
I know, I know. When are they going to start bragging on that trendline finally turning down while giving a big ol’ face kick to the unlucky NI’s too old for ESI…
LikeLike
September 15, 2010 at 2:18 pm
Off-topic: In case you do not already know, your site move makes a huge number of your links to older posts inoperable.
LikeLike
September 15, 2010 at 2:37 pm
If you mean links to scientopia.org, Sean, yes we are aware of this problem. Sorry about that.
Something is going wrong with the redirects ever since we switched hosts recently. We’re trying to hold tight in hopes this problem can be resolved rather than telling people to change their feeds and re-editing all of our posts.
in the mean time if you substitute scientopia.org where the link is for scientopia.org you can find the older posts.
LikeLike
September 15, 2010 at 8:42 pm
I feel bad for the dude out at the 70th percentile. If he sees that plot he’ll know he’s the worst of the worst.
Maybe that’s not a bad thing…
LikeLike
September 16, 2010 at 12:02 am
actually you raise a question, Odyssey. Should be about 50-60% of these suckers triaged. Not clear if Berg is including all discussed (voted Overall Impact Score) with the not-discussed (would have to just be the average preliminary Impact Score from the assigned reviewers) or just the discussed applications. If the latter, why in the heck are so many applications getting discussed that wind up with such bad scores?
LikeLike
September 16, 2010 at 12:43 am
The triaged grants are not shown, as per Jeremy’s response to my query at his post about the proportions of triaged grants for the four classes of applicantions. The reason that the percentiles go up to the level they do is because study sections tend to review a *minimum* of 50% of applications in rank order and frequently up to near 60%, since you have to review *all* of the applications with the same preliminary average impact score, and then because you also add in a few that are “pulled up” at the discretion of any single panel member. I only count 13 applications above 60 %ile, which is less than 2% of the total. So this looks unremarkable.
LikeLike
September 16, 2010 at 1:18 am
where the heck is whimple to insist there must be shenanigans of people pulling up their buddy’s clearly triage-ready app for discussion?
LikeLike
September 17, 2010 at 8:22 am
What bugs me the most is to see unprepared members of the study section voting *blindly* for proposals submitted by recognizable names from recognizable institutions. 😦
LikeLike
September 17, 2010 at 9:34 am
why is it “blindly” if they have just listened to the three assigned reviewers give their spiel?
LikeLike
September 17, 2010 at 11:12 am
Dude, obviously Proteus is a mind-reader and knows exactly what scores all the reviewers write on their secret score sheets and what they are thinking when they write them.
LikeLike
September 21, 2010 at 9:52 pm
Thank you Dr Berg for the candies. Keep the good work and congratulations for the Award.
LikeLike
May 4, 2012 at 7:54 am
[…] only IC that, to my knowledge, made their funding data available was the NIGMS. We grant geeks were big fans, even those of us who don't seek funding from that particular Institute of the […]
LikeLike
November 25, 2013 at 11:02 am
[…] well, familiar to our readership as the prior head of NIGMS, blogger and provider of much grant-funding […]
LikeLike