Take your NIH PO's comments with a grain of salt
September 29, 2010
As my disclaimer, I am one of those that reminds new (and not so new) applicants to the NIH for research funding to talk to the Program Officer. Early and often, just like voting. In the pre-submission phase you want to identify the line POs down in the Division and Branch structure of one of the funding Institutes or Centers of NIH who might be interested in your work. After review, the PO who was actually assigned to your application can give you invaluable feedback about how the review of your application went down. So I recommend calling this person.
Take their comments, however, in context. They don’t know everything. Even if they are sitting in the room, paying attention to the discussion of grants, this does not always mean they truly understand what is going down.
One of the times I get really frustrated with POs is when they are so fixated on the objective truth of peer review. They often act as though they believe that the review process really does work nearly perfectly. Most often when it comes to newbies. Consequently if you are coming up short on your proposals, in their worldview they think you are “not writing well enough”. Or need to (somehow without funding) provide more/better preliminary data.
So this advice gets reflected back on the poor applicant who 1) drives herself crazy trying to “improve” her writing (which is just fine already) or 2) goes back to the lab to find that perfect figure which which will guarantee this grant gets funded (no such thing).
Disastrously, this prevents said newbie from doing what she really needs to be doing which is to submit multiple good-enough grants. In the face of budgets which allow the funding of only a subset (a third? quarter?) of the grants which are excellent and interesting and impactful and all that jazz, review becomes variable. Meaning the difference between making it into a fundable score and just missing a fundable score takes on the appearance of chance. The only way to beat such odds is to give yourself more chances at the game. This means writing and submitting multiple applications (on different topics, of course).
Don’t mistake me. There IS a learning process for grant writing and it remains good advice for the new (and not so new) investigator to seek feedback from peers prior to submitting a grant and when doing the post-mortem after receiving the summary statement. But this can’t be taken too far. At some point, you are just driving yourself crazy with conflicting (good) advice from people who are in different situations from yourself or each other. And anyway, you can apply the stylistic and structural advice you have received to new application just as well as to the revision of your existing application, right?
September 29, 2010 at 9:07 am
When I spoke with my PO after my first pre-doctoral NRSA submission got a crappy score (but still a score!), she told me I’d probably never be a successful scientist. Thanks, PO lady!!!
LikeLike
September 29, 2010 at 3:13 pm
Excellent advice. From having been on study section, one can pick out the top 25-30% of grants. Its very hard to differentiate better than that. Even if you’re grant is in that top 30%, there’s probably only a 1 in 2 or 1 in 3 chance it will be funded. The only way to overcome these odds is to write multiple top 30% grants.
LikeLike
September 29, 2010 at 4:41 pm
It is *impossible* to distinguish better than that, because there exists no objectively real unidimensional ranking of the grants within that range. This is why the whole Improving Peer Review dealio at NIH was a total scam. It was sold to the disgruntled extramural community–all of whom are convinced that the reason their grants that are genuinely in the top quartile don’t get scored in the top decile is because of a “deficiency” in peer review–as a way to make sure their grants get reviewed “fairly” (i.e., in the top decile where they are convinced they belong). In reality, what it has achieved is not increased “fairness” or “accuracy” at identifying the top 10%–an impossible task–but increased efficiency at reviewing the dramatically increased number of grants being submitted.
LikeLike
September 30, 2010 at 6:09 am
“…talk to the Program Officer”
Ha! The fucking PO I was assigned doesn’t bother going to study section meetings. He has no idea what’s going on, and usually feeds me lines that are straight off the website. The SRO is only slightly more helpful (which isn’t saying much).
LikeLike
September 30, 2010 at 8:27 am
Unfortunately POs don’t actually go anymore, IME. Or at least as often. When I started on study section there were always a good half dozen POs in the room at all times. Not anymore.
Now, they have some fancy call in conference system so they are mostly sitting somewhere else, barely listening, and working on some other task. One hopes they actually pay attention when the grants to which they are assigned come up.
In a previous time, it was not uncommon for a representative from the branch or division within the Institute to cover study sections for the whole branch/division. Usually in those cases though, the assigned PO would just refer you to whomever was in attendance.
The SRO is not really supposed to tell you much after review, not in their job description. And prior to review, their job is different anyway. Are you asking PO type questions of the SRO? Not surprising that they would be unhelpful.
usually feeds me lines that are straight off the website
yes, there is a lot of zombie-mantra* talk that you have to put up with from POs. There are reasons for this of course. Your goal is to keeps developing the conversational relationship (on the phone and, importantly, in person at meetings) so that now and again this person will stray off the beaten path and tell you something interesting.
*”we advise you to revise and resubmit to improve your score”
LikeLike
September 30, 2010 at 11:58 am
DM, thank you for the info.
I’ve asked the SRO whether he could clarify some of the reviewers’ comments. For example, “why didn’t the PI propose to use a different method described in other publications?” I asked the SRO if the specific publication or method was mentioned. In this case, he said no! In other cases, the SRO has been helpful in providing general advice along the lines of whether the study section was appropriate for my application, etc.
If the PO is not involved in the review process, what use is it to bother speaking with that person? Is it only for schmoozing to get points during a council meeting, or do some of them actually try to provide helpful advice beyond the “zombie mantra” as you describe?
LikeLike
September 30, 2010 at 12:42 pm
yeah, that isn’t a question a SRO is supposed to address, that’s PO business.
the PO is *supposed* to be involved. the degree to which they do so, and the degree to which they might want to convey specifics on your particular application is up to them, however.
do remember, as I mentioned somewhere or other recently, that POs *are* motivated by their own career issues to have scientists in their domains of interest be successful…there is *some* hope, in other words.
LikeLike
September 30, 2010 at 3:22 pm
I’ve heard of POs that get extremely irritable about applicants going to the SRO “over their head”. I ended up going to my SRO after spending 2 weeks trying to get hold of my PO when I got my last summary statement, and I don’t think that I’ve gotten punished for it. It did seem like he was only half-listening when we finally did talk, however.
LikeLike
September 30, 2010 at 8:05 pm
The situation when your PO relationship matters is when your grant comes in right at the funding line. This is the space where the PO can actually influence whether you get funded. If you score in the 2nd percentile, your grant is funded independent of programatic issues. If you got triaged, no PO input can save you.
In the case of a triage, I question the value of the PO input (even if they do provide a thoughtful response to your query). If your grant got slammed that bad, then you should be thinking more about why the study section didn’t like it. The PO might suggest that you focus on health issues that are lacking in their portfolio, but your problem lies more with your score in one or more of the review criteria (approach), grantsmanship, or both
LikeLike