Peer Review: Opinions from our Elders

One of the most critical hurdles for the young scientist is getting a “fundable score” on a grant proposal from a study section, the first and most important link in the peer review process. If you receive a fantastic score from a study section (currently about the upper 7-10%ile), you will almost certainly get funded; receive a bad score and there is no chance of funding. Unfortunately the deck is stacked against the young investigator in many ways. One of the biggest problems, and something that can be changed, is the representation of young and transitioning scientists on panels. In short, they are not represented. The official NIH guidelines for selecting peer reviewers, indicate that one must already have been awarded a grant to serve. That means that scientists who are attempting to transition to independent work, and likely those in the first 2-3 years are unrepresented. There is also an unstated criterion that participation from those scientists who have not yet acquired tenure/associate professor rank is to be minimized. Interestingly, the official guidelines also state that “There must be diversity with respect to the geographic distribution, gender, race and ethnicity of the membership.” So why should there be an attempt for “peer review” to be explicitly unrepresentative of career status?

A 2006 commentary on NIH peer review illustrates one of the problems. The author asserts that grant review has decreased in quality of late and that this problem may be attributed to the participation of too many junior scientists on panels. This is a not uncommon perspective of older scientists. The director of the NIH Center for Scientific Review (CSR) comments in his Sept 2005 report to the Peer Review Advisory Committee that assistant professor participation was up in 2005 over 1998. Then the CSR attempted to limit assistant professor participation to 10% of reviews in the fall rounds of 2006.

Here’s the question. Where are the data? What evidence is there that assistant professors or younger investigators give consistently lower quality reviews? Especially when you take out the previous-experience-reviewing factor, which is obviously circular. Where is the evidence that the more-junior scientists that do participate on study sections have any functional impact? One reviewer out of the three tasked to review a given proposal does not have a categorical impact. If the assistant professors are limited to 10% or less, this means that on average one out of every three proposals will have one out of three reviewers of junior status. It is absurd that the NIH brays about the need for helping the next generation of scientists while ginning the system against them in the place where it most counts.