What? The NSF allows unfunded people to review their grants?

October 13, 2010

Prof-like has a pair of questions up.

1) should unfunded PIs be included on panels or study sections?

2) Should postdocs (if funded by Federal funds) be included on panels or study sections?

The implication here seems to be that the NSF allows professorial rank people without NSF (or other major governmental?) awards to review proposals. Perhaps also that postdocs (or perhaps research scientists?) that do have funding to review grants. If so, this is most unlike the NIH where the vast majority of reviewers have to be of Associate Professor status or higher. Also unlike the NIH expectation that reviewers have to have been awarded a grant similar to those which s/he is reviewing. My answer got a little long so I thought I’d pop it up as a post.

I’m on record in favor of PIs who are not yet funded by the NIH being represented on review panels. So Yes on #1. I throw out a “maybe even some senior postdocs as well” but I always figured that was an extreme Overton shifting position. Are you telling me that NSF lets postdocs review research grants? Interesting.

I’m in favor of this because it seems like basic fairness, one, and the only way to combat biases, two.

Look at it this way- The NIH has explicit rules that study sections must have diversity. Check this link

There must be diversity with respect to the geographic distribution, gender, race and ethnicity of the membership.

In my experience this seems to be taken quite seriously. Ethnic minorities would appear to be well represented on panels on which I’ve served. Women run about 35-40% I think at one point I check this for my most frequent section against the CSR overall numbers. Through conversations suggesting reviewers to an SRO or discussing why so-and-so had been ad hoc’ing for two years and not appointed, it became clear to me that the geographic distribution is a pretty hard line.

Notice anything missing in this “diversity” statement? No? Well how about this comment…
There is a need for balance in the level of seniority represented among members of a study section. Too many senior-level reviewers are just as problematic as too few.

Right on. And too few junior reviewers are as problematic as too many….what? Where is that statement? Not to be seen…

So why do we have diversity requirements if not to make things *fairer* for all applicants? What would be the point of requiring a diversity of reviewer backgrounds, perspectives, seniority, geography, etc, if not to ensure fairness through the competition of biases? hmm?

So why would one suspect class of applicant be overtly and intentionally excluded? The NIH made a lot of noise recently about purging assistant professors off the panels. Their justification for doing so was almost entirely unstated and for damn sure free of any backing data.

Advertisements

No Responses Yet to “What? The NSF allows unfunded people to review their grants?”

  1. Namnezia Says:

    NSF uses a bunch of ad hoc reviewers which do not necessarily sit on panels. Basically the way it works is that once you submit an application you are in their database. And what happens is that, funded or not, you’ll suddenly get sent an email with a link to a grant and a request to send in a review in 4 weeks. So then you log on to “Fastlane” download the grant, read it and upload your review. It is a bit weird t just get someone’s grant emailed to you randomly. So when you get your reviews back you will get something anywhere from three to eight reviews, and you can’t really tell which are from actual panel members and which are the random external ones.

    Like

  2. qaz Says:

    I reviewed NSF grants ad hoc as a postdoc. (Actually, I reviewed an NSF grant as a graduate student. I don’t remember what happened with it, but I know I took it extremely seriously and probably spent longer working on that review than I ever have since.) So I know for a fact that they used to do it. I think they still do. I’ve served on NSF panels (and regularly do several ad-hoc reviews a year) even though I’ve never had an NSF grant. (On the other hand I do now have NIH grants, so maybe that “counts”.)

    In general, my understanding is that NSF’s ad-hoc reviewers are more like journal reviewers – they are whoever the editor thinks will give a good review. I think panel members are more like study section – they are people who have an understanding of the funding process.

    Like

  3. proflikesubstance Says:

    I was specifically asking about people actually on panels, not ad hoc reviewers. My experience so far is at odds with Qaz’s assertions regarding panels.

    Like

  4. becca Says:

    Because of the influences of academic medical researchers (read: credentialism obsession ala MDs), I think NIH tends to be more status structured than NSF.
    It seems to me perfectly reasonable to have either unfunded PIs or postdocs as ad hoc reviewers. For that matter, I don’t see why an occasional one on the panel would hurt, though obviously they’d have to be balanced with significant expertise.

    Like

  5. Kim Says:

    For ad-hoc reviews, I know NSF was deliberately trying to broaden its pool of reviewers at one point. (That is, they were asking senior scientists for suggestions of early career scientists who would be appropriate reviewers.) As for panels, I know they include people who aren’t currently funded. (Many highly respected senior scientists don’t have a federally funded grant all of the time, and requiring that everyone be currently funded to be on a panel would exclude many of the good reviewers for RUI (Research at Undergraduate Institutions) grants, which are included on panels with discipline-specific proposals from graduate programs.) I believe they also can include people who haven’t been funded before – early in my career, I often heard the advice to “get on a panel” to learn about the funding process from the inside, and I think some of the people who have been on panels with me were unfunded early career people.

    The dynamics of panels mean that the opinions of people with more experience may carry more weight (not formally, but just through the way the panel members interact). However, everyone on a panel brings a different expertise, and a young scientist may be the one person who has experience with a particular technique. My experience has been that the junior people are respected and valued on the panels, and I think that’s very healthy for science.

    Like

  6. Carl Riedel Says:

    Speaking as a corporate materials professional, I fail to see the problem with non-granted researchers being a member of either a board or panel member. There are seemingly more projects that involve materials or surface engineering as their core aspect. Why would the relevant board not want to include a materials subject matter expert? That SME might not have had a grant given to them from the NIH but are still acknowledged experts in their field and can still actively participate in weighing the potential benefits or pitfalls of a particular line of research.

    Like

  7. Joe Hourclé Says:

    I don’t have a doctorate, and I’ve participated in NASA grant reviews. (Am I allowed to admit to that? I know I’m not allowed to say which ones specifically, or which proposals I’ve reviewed) I’ve never been on a grant proposal as anything other than an unfunded collaborator, unless someone submitted my name without my knowledge.

    Of course, my area of expertise is a smaller field (scientific data management), and there aren’t too many people who they could find with sufficient expertise, and without a conflict of interest. On each panel there were at least a dozen people with either hard science or computer science experience, either being current or previously funded via NASA grants, or civil servants, so I only judged based on what I knew. (so I might’ve been harder on people who left out things like commitment letters from the Co-I who was going to be doing all of the work; but NSPIRES now has checks in place for that)

    Like


  8. I agree with Joe–requiring panelists to be funded would exclude government and industrial scientists, who are both qualified and have a valuable perspective to offer. At my old National Lab, there are several eminent scientists who are National Academy members and/or society fellows who have been government scientists for their whole careers, and are therefore ineligible for NIH or NSF funds. They certainly would be a great resource for NIH/NSF panels even though they are “unfunded”.

    Like

  9. Jay Malone Says:

    The question about letting graduate students and postdocs in grant review panels would translate approximately to something like having a well funded PI who doesn’t know a squat about a particular application in a study section. The effect would essentially be the same when one doesn’t know the literature related to a particular application. This seems to be common in NIH study sections and I see practical difficulties in getting tailor made reviewers for each grant.

    But what a person who has been a PI brings to the table, is perspective and the first hand knowledge of the restrictions and practical difficulties with such grant series.

    I have no problem if a PI would make his graduate student worker or the postdoc to review and provide feedback for the PI. But the contribution from graduate students should stop right there. The final report should be from the PI.

    It may help advanced postdocs to sit in panels, but still I don’t see how this is going to help the postdoc transition faster without getting distracted with this review business.

    -JM

    Like

  10. drugmonkey Says:

    Joe, Prodigal-

    In the NIH system I have seen two exceptions to being NIH funded as a requirement for service. Intramural / Gov lab and industrial scientists. The former with frequency, the latter only on occasion.

    Like

  11. anon Says:

    I am absolutely in favor of non-NIH funded scientists sitting on study sections. It should be a requirement! It would broaden an SRO’s ability to find appropriate experts and possibly result in less “NIH-club” biased reviews.

    Like

  12. qaz Says:

    I don’t see the problem with having an occasional post-doc or unfunded PI on the panel (or study-section). We’re not talking about the whole panel. At most, I would be shocked if we see more than one on any given panel. In practice, a newbie on a panel has very little direct influence because of the primate dynamics of human groups.

    In answer to JM – getting on a panel is often directly correlated with learning how to write grants. What I’ve seen (in a lot of junior faculty, including myself when I was learning) is that what everyone is trying to tell you when they try to teach you how to write grants just doesn’t make sense until one sees how a panel works. Although I’ve never quantified it, there appears to be a strong correlation between getting on a panel (any panel, NSF, NASA, NIH, NRSA or R01) and one’s ability to get grants. [Yes, I know there could be other things correlated with it, but everyone comes back saying “That’s what you meant!” I know I sure did.]

    Like

  13. Jay Malone Says:

    qaz, Agreed.

    Like

  14. James Says:

    Yes…what qaz said. In fact, what if there were 1-2 slots on study sections for postdocs and (10%? 25%? all?) NRSA fellows were required to do 1-2 cycles of reviews, as part of their “training” and “service to the NIH”? (Ignore the associated logistical problems of high personnel turnover for now.) Or would senior PI’s just view this as amateur hour?

    Like

  15. ex-hedgehog freak Says:

    It’s incredibly important to allow this opportunity to postdocs as part of their training and continuing professional growth. After all, I am sure many of you all here, like myself, were used as peer reviewers for neuro journals as postdocs. Why shouldn’t the same be true for study sections? After all as one of only two peer reviewers, we probably have more influence in that regard than on study section

    Like

  16. Psy Says:

    The principle of having unfunded/very junior scientists on NSF panels sounds great. The practice… I’ve never seen more random, uninformed reviewers. And at least in my field (neuro) I hear the same from others. I still don’t have funding but having sent 4-4 grants to NSF & NIH I’ve been shocked at the low quality and sheer randomness of reviews at NSF. YMMV of course and this is likely to be program dependent.

    Like


  17. I’ve never seen more random, uninformed reviewers.

    I still don’t have funding[.]

    Multiple possible causal scenarios present themselves.

    Like

  18. drugmonkey Says:

    Psy, unless NSF is very different from the NIH, you have no idea of the career or funding status of the “uninformed” reviewers.

    More to the point, if you are finding that reviewers at a particular agency are always “uninformed” about your great science, it is possible that you need to think a little harder about tailoring your style of presentation to the likely reviewers of your proposal. That might smooth out the “random” nature of the reviews you have been receiving..

    Like


  19. I don’t even know how I ended up here, but I believed this submit was great. I don’t know who you’re but definitely you’re gonna a well-known blogger in the event you aren’t currently 😉 Cheers!

    Like

  20. ghd website Says:

    I confess, I have not been on this internet site for a long time, however, it was another enjoyment to see such an excellent items and overpass it. Thanks for helping making people more aware of good points.

    Like

  21. Erick Boehm Says:

    This entire body has laid down well-defined rules and norms for both capital and funds market.

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: