Shorter R01 Applications
June 1, 2007
In a recent post, I suggested that official NIH-speak to the effect that a “majority” of respondents to a NIH Notice on the topic of shortening the R01 application was disingenuous. I was interested in the specific numbers and now we have them. There’s a nice PowerPoint by Robert Finkelstein (NINDS) and Don Schneider (CSR) up on the April 19, 2007 minutes of the Peer Review Advisory Committee (PRAC) site. From this we learn that of their sample (it is not clear if data are from the whole 5078 or from the 500 “randomly selected” for full analysis) 27% wanted to keep the 25 page format, 43% wanted to go to 15 pages, 22% favored 10 pages and 5% favored 5 page applications (3% no-response). So my suspicions are mostly unfounded but it still bears answering why they didn’t just report the bloody numbers in the first place. Nevertheless this bears some examination. 5 pages? I think we can safely assign this 5% to the unserious/nutcase/random response circular file. I mean, 5 pages, really. That leaves us with 71% of the serious sample in favor of shorter and 29% in favor of not changing. So yes, this is a pretty good majority in favor of shorter applications. My suspicions were unfounded thus far.
I’ve pointed out before why I think shorter apps will disadvantage the younger investigator. Despite the fact that in the PRAC data New Investigators are, if anything, more in favor of the shorter applications, 27% of the sample agrees with me that the New Investigator would be disadvantaged (43% say no disproportional impact on any of a number of categories). We also learn that 65% of respondents think that review criteria should be changed to emphasize ideas and impact (29% nay). Indeed. This is absolutely essential. Perhaps things are going to go in the right direction on this one.
DrugMonkey 
June 3, 2007 at 4:32 pm
I hope the emphasis on ideas and impact does help reviewers cut to the chase and focus on the proposal rather than the proposer. However, I am concerned, based on the proposals that come my way, that you need really, really good reviewers to separate the wheat from the crap (which usually happens when the reviewer sees the applicant is capable of designing a sound study, collecting the appropriate data, and analyzing-interpreting the data correctly). Or when the reviewer knows personally the applicant is probably capable of pulling it off … which puts us back where we started.
Also, I just noticed that I never went back & added the PRAC link to my original post. My bad.
LikeLike
June 3, 2007 at 10:06 pm
I don’t agree that it takes anything out of the ordinary in terms of reviewers to judge proposals. For me the evidence is the almost frighteningly good agreement in terms of initial scores that I see in my section. When there are significantly disparate views amongst the three reviewers it isn’t a wheat-from-chaff argument. It’s usually a “I like barley not wheat” or a “I like cake flour, not whole wheat flour” type of argument. Or perhaps “sure this is good wheat from Farmer Unknown but dang, Old MacDonald’s been producing good wheat for 15 years so we better not risk Farmer Unknown”.
LikeLike
June 5, 2007 at 6:10 pm
Oh, you haven’t read scores of summary statements over the past 2 decades or had PIs ask you if the 3 assigned reviewers all read the same proposal … or read/understood it at all (based on the irrelevant comments raised). And it is exceedingly easy to pick the crony out of the bunch who gave it generic high marks based on familiarity with the person rather than the proposal. Count your lucky stars & mental health that you have not come across such a review, which, to be fair, seems to be declining in number based on what comes across my desk anyway. I do very much like your above analogy though, especially Old MacDonald.
LikeLike
June 6, 2007 at 9:08 am
5 pages?!? That is insane. I wrote a sample grant proposal for one of my classes recently that was 11 pages, and it only had two specific aims and practically zero preliminary data. I would guess even 10- and 15-page-long R01s with any substance whatsoever would be exceedingly difficult to write.
LikeLike
June 6, 2007 at 9:15 am
bdf, let’s not be naive here. the review of lots of applications basically boils down to 3 pages.
1) Cover page: okay, who’s the PI, yeah, s/he’s great. I read their stuff all the time
2) Aims page: hmm, what are they going to do this time? yeah, looks cool.
3) Progress report listing of papers in last funding interval: yeah, they keep generating papers.
okey dokey let’s give ’em the money.
the rest is just window dressing…
so I may have been a bit flippant about dismissing the proposal for 5 pages. There is a very large category of investigators who would benefit from this.
LikeLike
June 6, 2007 at 9:19 am
Sounds good! Who needs data anyway? 🙂
To be clear, I’ve neither written nor reviewed an R01, but there has to be more to just that, no?
LikeLike
June 6, 2007 at 9:37 am
Yes and no. I have some comments here and there that touch on the issue of how senior/luminary grants are reviewed versus New Investigators. and some on why.
There are times when a grant is in year 20+ of competing continuation, the PI is a luminary in the field and the actual proposal is demonstrably crap. Certainly in comparison with other proposals from less-luminary scientists. Often the grant gets a good score because the reviewers end with “yeah it is a bad proposal but I know good stuff will becoming out of this lab in the next 5 years”. It boils down to three key pages and perhaps just the cover page when it comes right to it. Particularly when the grant is on A2 revision!
LikeLike
June 13, 2007 at 5:44 pm
[…] Indeed, I’ve been known to rant quite a bit in person. In essence I think that the deck is stacked in favor of the well-established investigator and against the newer or untried investigator. In a […]
LikeLike