Sometimes CSR just kills me

The Peer Review Notes for September 2014 contains a list of things you should never write when reviewing grants.

Some of them are what we might refer to as Stock Critique type of statements. Meaning that they don’t just appear occasionally during review. They are seen constantly. A case in point:

7. “This R21 application does not have pilot data, which should be provided to ensure the success of the project.”

Which CSR answers with:

R21s are exploratory projects to collect pilot data. Preliminary data are not required, although they can be evaluated if provided.

What kind of namby-pamby response is this? They know that the problem with R21s is that reviewers insist they should have preliminary data or, at the least only give good scores to the applications that have strong preliminary data. They bother to put this up on their monthly notes but do NOTHING that will have any effect. Here’s my proposed response: “We have noticed reviewers simply cannot refrain from prioritizing preliminary data on R21s so we will be forbiding applicants from including it“. Feel free to borrow that, Dr. Nakamura.

Another one:

“This is a fishing expedition.”

CSR:

It would be better if you said the research plan is exploratory in nature, which may be a great thing to do if there are compelling reasons to explore a specific area. Well-designed exploratory or discovery research can provide a wealth of knowledge.

This is another area of classic stock criticism of the type that may, depending on your viewpoint, interfere with getting the desired result. As indicated by the answer, CSR (and therefore NIH) disagrees with this anti-discovery criticism as a general position. Given how prevalent it is, again, I’d like to see something stronger here instead of an anemic little tut-tut.

One of these is really good and a key reminder.

“The human subject protection section does not spell out the specifics, but they already got the IRB approval, and therefore, it is ok.”

Response:

IRB approval is not required at this stage, and it should not be considered to replace evaluation of the protection plans.

And we can put IACUC in there too. Absolutely. There is a two tiered process here which should be independent. Grant reviewers take a whack at the proposed subject protections and then the local IACUC takes a whack at the protocol associated with any funded research activities. It should be a semi-independent process in which neither assumes that the approval from the other side of the review relieves it of responsibility.

Another one is a little odd and may need some discussion.

“This application is not in my area of expertise . . . “

I find that reviewers say this in discussion but have never seen it in a written critique (even during read phase before SRO edits eliminate such statements).

The response is not incorrect…

If you’re assigned an application you feel uncomfortable reviewing, you should tell your Scientific Review Officer as soon as possible before the meeting.

…but I think there is wiggle room here. Sometimes, reviewers are specifying that they are only addressing the application in a particular way. This is OKAY! In my experience it is rare that a given application has three reviewers who are stone cold experts in every single aspect of the proposal. The idea is that they can be primary experts in some part or another. And, interestingly given the recent statements we discussed from Dr. McKnight, it is also okay if someone is going at the application from a generalist perspective as well. So I think for the most part reviewers say this sort of thing as a preamble to boxing off their areas of expertise. Which is important for the other panel members who were not assigned to the application to understand.

Thought of the Day

Some days…

I tell you, one of the most hilarious parts of this blogging gig is this.

On the one hand, any time I try to gently suggest that postdocs could work a little harder, might need to actually produce a little more, need to figure out how to close out “projects” into submittable manuscripts AND THAT THIS IS A TRAIT THAT THEIR PIs HAD AS POST DOCS…..

I get pushback.

From the hordes of Internet postdocs who are all brilliant, wonderfully productive, practically PIs themselves in all significant ways and are, sadly, only held back by their current rat bastige PIs who fail to help them in some egregious manner.

On the other hand,I have recieved a lot of commentary behind the scenes from PIs. Who tell me the most hilarious stories about postdocs who fail to produce, are completely delusional about their own efforts, accomplishments and/or levels of effort.

There is only one possible conclusion.

The postdocs who read my blog and the PIs who read my blog are entirely independent sets with no possible areas of population overlap.