Your Grant in Review: Competing with the same proposals

August 13, 2007

Looking over my pile of applications to review this time, I’m struck by a disconcerting point. Your proposal is not merely in micro-competition with others that are closely related to yours, you are sometimes going to be competing with exactly the same grants, round after revision round. It works like this…

The “micro-competition” part refers to the implicit or explicit competition of grants within a reviewers’ assigned pile in each round. The explicit version of this is a request from the chair or SRA to “rank and spread” your scores within your pile of, say, 7-10 R01 applications. If each reviewer tries to do this with their own pile, the thought is that the eventual distribution will be flat as well. The goal to reach a flat distribution of scores is so that the decisions are easy for Program. If you have relatively few applications at a given score line, you don’t have to make tough calls. If there are twenty applications within one point of the funding line, well, then Program has to make the call on which to fund and which not to fund. [Interestingly, I don’t think that reviewers, who collectively tend to cluster scores around the perceived funding line understand the extent to which they are abandoning their influence on what is funded by this behavior.] The bad part of this is that for any given application it introduces a degree of random chance. You might have a great application but if the reviewer has a superlative one, well, yours is going to suffer. Now calm down, hardly anyone sticks to a hard line version of rank-n-spread and these situations are almost always resolved in the discussion process. After all, if yours is really that good, it will by chance be the best in at least one reviewers’ pile and (at least in our section) the person who is scoring you low because they have a better one will most likely raise their post-discussion score. Nevertheless you are, in aggregate, going to be competing with other applications.

Although perhaps obvious, the assignment of grants is far from random within a study section. They pay the SRAs the big bucks (!) to appropriately choose which reviewers have the appropriate expertise to review the grants assigned to that study section. As you can imagine, it breaks down by topic, models, scientific domain, etc so that it is very likely that you are going to be competing with proposals that share some, if not many, critical features with your own. Also, SRAs make some attempt to maintain continuity in review of revised proposals. I have three revisions that I’ve reviewed previous versions of in this round, for example. This is what brings me to my point.

Since everyone has the nose to the grindstone and is putting ’em back in as soon as possible, the revisions of two given proposals are coming back in the same every-other-round order. Thus, they continue to be in lockstep competition with each other.

Implications? I don’t know really, I am just fully appreciating this factor now. Obviously the scientific merit issues get contaminated by revision status (A2 might bump ahead of an A1) or a difference in “responsiveness to critique”, the sixth major criterion for review. Reader thoughts invited, obviously. Would you want to mix your application’s competition up randomly each and every time? To avoid continued comparison with any specific proposal in particular? What if this meant a more explicit attempt to assign your proposal to a new set of reviewers each time?

2 Responses to “Your Grant in Review: Competing with the same proposals”

  1. PhysioProf Says:

    In relation to grant review, this made me laugh my ass off:

    http://www.sciencemag.org/cgi/content/full/317/5840/880/F1

    Like


  2. […] 17th, 2007 Frequent commentor Physioprof has the call: this made me laugh my ass […]

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: