Your Grant in Review: How not to respond to criticism
August 10, 2007
I posted before on the tendency to apple polish in the response to prior review of your grant. I didn’t really think it would be necessary to address todays’ point. Looking over some revised grants, though, I am reminded that people still need to be steeped in the basics, i.e., the basics of decent human social behavior, nevermind the basics of grantsmanship…
You know that initial reaction that you have to a critique of a paper or grant? Where you run on for a hour about how bad the review comments are, how the reviewer surely must be an idiot, it is all this one complete moron’s fault and so the editor should just accept it (or Program fund it!) anyway? Yeah? Well, don’t turn this response into your formal reply. Who in the heck would do this? How is this supposed to go down with the reviewers anyway?
I can’t imagine how it is going to be helpful to rant about a bad reviewer. If your points are in fact valid, the subsequent reviewers will notice. If not, you look like a jerk. What if you end up with the same bloody reviewer again? He/she sure as heck isn’t going to find an excuse to help you out this time…
In grant revision this is all about keeping your eye on the prize. You are wasting valuable space that could otherwise be used for, oh, I don’t know, actually responding to criticism?
DrugMonkey 
August 10, 2007 at 4:16 pm
Yeah, you’d think this kind of thing would be obvious.
In addition to my scientific training, I also have a patent prosecution background, where the tradition is to be very obsequious in a formal manner to the patent examiner in responses to office actions, which are exactly like grant reviews except they are analyses and decisions on the allowability of the application for the patent. You go so far as to always capitalize “Examiner” when you refer to him or her in the response. And you always say at the end of the response, “We respectfully submit that this response overcomes the Examiner’s rejections and places this application in condition for allowance.”
I tend to do this kind of thing in my Introduction to a revised application (without capitalizing “the reviewers”). I start with something like: “We thank the reviewers for their perceptive comments concerning our application, and appreciate their enthusiasm for the significance of our Specific Aims.” I end with: “We respectfully submit that these revisions to the application meet the reviewers concerns and hope that enthusiasm for the application will accordingly be increased.”
On the one hand, I know that these tropes are meaningless, and they don’t have persuasive power. On the other hand, I feel that they lend an air of gentility to the process, and maybe signal a salutory degree of formal respect for the process. Sometimes I think that reviewers probably read this stuff and wonder what kind of whacked-out pompous freak I am. No one has ever complained about it, I get grants funded, so I keep doing it.
LikeLike
August 10, 2007 at 10:02 pm
so, um, triage it. life’s too short and funding too tight to put up with a-holes. how close can you come to actually calling someone an asshat in a critique, anyway?
LikeLike
August 11, 2007 at 7:07 am
actually this particular one isn’t in my pile to review. thankfully. i can’t imagine that with the combative tone in the Intro that the PI has actually responded to critique very well…
Physioprof, one or two boilerplate sentences are, as you say, obviously part of a nod to the process. but if this type of syrup pervades each and every response, well, that rubs me the wrong way.
LikeLike
August 11, 2007 at 10:05 am
[B]ut if this type of syrup pervades each and every response, well, that rubs me the wrong way.
Once I get into the substance, I try to keep things simple and factual.
How do you react when applicants express gratitude for specific positive things that were said in the last round of reviews? E.G.: “We are pleased that Reviewer #1 considered the experimental design of Aim #2 to be ‘elegant’ and ‘highly likely to lead to important findings regarding blah blah blah’.”
LikeLike
August 11, 2007 at 12:08 pm
I used to do that sort of quoting of the positives at the start of the Intro. At one point I got a second hand transmission of frustration from someone I assumed was my usual advocate on a particular panel. Among the gems was “oh, and tell DM to stop wasting space with the compliments, we already see the summary statement ourselves”. I hadn’t realized that of course. nor had I really thought about the fact that in some large fraction of cases that reviewer *wrote* those comments or reviewed them extensively as one of the other reviewers. so now I just don’t bother especially since it uses space that could be put to better use.
as a reviewer, i dunno. i think my eye just skips right over that because i’ve already looked at the prior critique and/or will do so right after looking over the Intro.
LikeLike
August 11, 2007 at 12:17 pm
I always include those “positive” statements right up top because you simply can’t count on the reviewers actually reading the previous pink sheets carefully. Unfortunately, not every reviewer is as diligent as you, DM :>) — Also, you can’t count on exactly the same set of reviewers, and it helps establish a baseline for anyone coming on to the panel for the first time. (Although perhaps my reasoning is completely off the mark).
LikeLike
August 12, 2007 at 7:01 am
Gee, let me guess here.
He’s (and it’s gotta be a he) is older, been doing this since at least the 70s. always been the golden-haired, never had to revise a grant ever. now those whippersnapper have the gall to question his piece o crap grant which basically says, “I have a great CV and I’m going to do some experiments so give me the money” but otherwise is a complete failure as a proposal.
(cry. river.)
“Something must be wrong with review because everyone tells me what a great scientist I am”
DM, everybody has this guy in their department, everybody. In most cases, program rolls over and say “Oh, yes, there must really be a problem” instead of saying “that’s nice but we’re in the proposal business and yours, er, sucked.”
LikeLike
August 12, 2007 at 8:47 am
“In most cases, program rolls over and say ‘Oh, yes, there must really be a problem’ instead of saying ‘that’s nice but we’re in the proposal business and yours, er, sucked’.”
I am skeptical about this.
LikeLike
August 13, 2007 at 7:32 am
Piled Higher likely overstates the problem here. As far as I hear, Program is a little sensitive to how it “looks” meaning that I’ll hear things like “Well, if the score was just under 200, maybe we could do something”. From this I conclude that the closer it is to the funding line, the easier it is for them to make pickups. But look at the R56 mech and the stated purposes. Look at who they are awarded to. Essentially this situation, HoaryAuldProf who (gasp) doesn’t get a fundable score on a first submission.
Now, are you skeptical that Program listens more to their HoaryAuldProf types when they complain about a bad review? That’s just naive. I had one recently where we were trying to induce a pickup. Massaging of the PO had no effect. About a week after my local BigCheez had a chance to mention the situation to the Director, we got a lot more interest and got to put in a full rebuttal. Denied in the end but the treatment was clearly different even with the slightest of HoaryAuldLobbying.
LikeLike
August 17, 2007 at 7:33 am
Physioprof, if this isn’t the way the NIH sees things then why do we have all this crap about peer review being “broken” and “flawed” and all that coming from Zerhouni and Scarpa? Why do we have the Bridge mechanism? Why are program staff talking about having to save “their” investigators?
If you are a younger/less tried investigator, according to program staff study section behavior is perfect and there couldn’t possibly be anything like a bad review. If you are one of “their” boyz/girlz, well, then they’ll agree that you got screwed and put it up to council anyway.
LikeLike
August 17, 2007 at 8:30 am
Actually, my read on what Zerhouni and Scarpa are saying is not that the current system of peer review is “flawed” or “broken”, but rather that it is being asked to do something that it wasn’t designed to do. It works fine for distinguishing the top quarter of grants from the rest. It fails at distinguishing the top ten percent from the second ten percent.
And as far as the R56 “Bridge” mechanism goes, I have two points. First, it is supposed to be used only in cases where the investigator has little or no other substantial funding. Second, my impression is that the amount of money being spent on R56 awards is much less than the amount of money being spent funding New Investigator awards that are below the nomimal payline.
Those who are left to fend for themselves are investigators competing for grants beyond a single R01. If you are competing for a second R01, there is no reason that program should fund it unless it is inside the nominal payline. And I have seen no evidence that program is doing anything other than that. And in my “home” institute, I have seen affirmative evidence that they are, indeed, doing just that.
Given the current funding situation, this seems like a reasonable approach to take: try to keep the greatest number of PIs alive. You might not agree with it. (In fact, it could be argued that it would be better to just let a bunch of labs die, under the assumption that the long-term trend is for a retraction in the scale of the US biomedical research enterprise.) But if the expectation is that funding for NIH is in a temporary lull, and will start increasing again, then the approach is not unreasonable.
My prediction is that if the budget woes continue, NIH is going to enact policies that–either explicitly or implicitly, but effectively–will cap the amount of NIH funds and/or number of grants permitted to be awarded to a single PI. The New Investigator and Bridge policies already do this, to some extent, as does the policy of not reaching below the nominal payline to fund additional grants for a PI who already has R01-scale funding.
As I pointed out elsewhere here (or maybe at Writedit’s pad; I forget where), if you look at the supporting information coming from the top-level committees that are going to compile all the information that NIH is now seeking from stakeholders, you find language like: “Researchers are going to have to adapt to a new reality in which NIH funds are only one component of what is necessary to support their research programs.” (to paraphrase).
The writing is on the wall.
LikeLike
August 17, 2007 at 9:47 am
“Researchers are going to have to adapt to a new reality in which NIH funds are only one component of what is necessary to support their research programs.” (to paraphrase).
If true, people at soft money institutes that act like mine does are well and truly screwed. It is bad enough that we suffer reputation issues (“they have all kinds of money so what do they need another grant for?” type comments leak back to us from study section all the time) already.
Those of us at the bottom of the food chain get ZERO institutional support while suffering from the reputations of those above us in terms of how well-off our programs are. Not to mention that IC directors look askance at our higher-than-ave indirect rate.
I believe the cutoff for R56 is $200K in directs. But do they account for hard-money-prof who has his/her salary covered versus soft-money-prof who does not? $200K is not always equal to $200K and the NIH has *always* had trouble grappling with the realities of soft-money institutions/job categories in my view.
on the other hand, as you say, the writing IS on the wall and all the moaning won’t change it. Perhaps this explains why we’ve dropped 20% of our investigators in the past couple of years. and all up and down the tenure ranks, not just the junior people.
LikeLike
August 29, 2011 at 10:14 am
[…] shouldn't have to mention this but also resist the urge to talk schmack about the prior review(ers). This doesn't go […]
LikeLike