This doesn't sound "mean" to me
March 27, 2012
Some commenter at Rock Talk complained about a recent grant review:
I just received the most terrible of reviews, where the reviewer was not only biased but highly inflammatory, prejudicial and aggressive. I must say I was totally taken aback. When you say things like “…terribly convoluted approach”,…”PI has clearly no clue…” how something works, trashes my published work by saying these pubs “are a gross exaggeration”….the list goes on and on. Even as a relatively senior investigator, I was very shocked by the mean-spirited nature of the comments. I cannot imagine how it would destroy a new investigator.
I am having trouble seeing it. I mean sure “no clue” is directed at the applicant rather than the application, but it’s pretty tame stuff. If a reviewer thinks your papers exaggerate? Presumably in wild speculative interpretation that runs beyond your data? Seems okay and even obligatory to express this. The “terribly convoluted approach” comment is a pretty inoffensive way to get to the heart of this common failing of grants as well…I’m not seeing how you could put it more “nicely”.
DrugMonkey 
March 27, 2012 at 4:26 am
Without seeing the rest of the review or knowing what it was about, it’s difficult to make a conclusion about how nasty the reviewer really was. I would agree with the commenter in that if those statements were directed at me, I would take offense and feel as though it was personal. There are definitely more constructive ways of communicating to the applicant “PI has clearly no clue…” Trashing manuscripts that have gone through peer review and are already published seems counterproductive to criticizing a proposed project. As you assume DM, the “exaggerated” papers (and I wouldn’t know what the fuck that meant) may have had something to do with an interpretation of data. If this interpretation subsequently lead to a hypothesis that the reviewer felt did not make sense or was not adequately supported, he or she could have spelled that out in a constructive fashion. Anyway, that’s my 2 cts. Maybe the anonymous poster can wade through the lousy review and its apparent attacks to see what the real problem is.
LikeLike
March 27, 2012 at 5:12 am
I have to disagree with you here DM (it doesn’t happen very often). Some grant reviewers have never liked a grant they didn’t write themselves. This sounds like one of those reviewers.
LikeLike
March 27, 2012 at 5:29 am
..” Trashing manuscripts that have gone through peer review and are already published seems counterproductive to criticizing a proposed project.
Oh boy. This is a big mistake. The fact that your paper has been published doesn’t mean a reviewer has to believe it constitutes the level of evidence you think it does. A reviewer is under no obligation to start from the assumption that if published, she has to accept every nuance of interpretation believed by the authors.
LikeLike
March 27, 2012 at 5:38 am
As a journal AE I always ask the reviewer what exactly they *mean* by empty nonsense like “terribly convoluted”. Unless the reviewer is willing to share how they would make things simpler, it’s not clear if things can be “less convoluted” and still work. But that’s in a journal situation where there is feedback between author and reviewer. In a proposal situation where there is no possibility of rebuttal, such comments serve no useful purpose other than bullying.
LikeLike
March 27, 2012 at 5:46 am
Bzzt, wrong. It tells the applicant to simplify the plan b/c the intricacies aren’t working for this reviewer. “convoluted” is pretty damn clear language…if you can’t work out what the reviewer means, maybe you need to get someone smarter to look over your proposals before you submit them.
LikeLike
March 27, 2012 at 5:59 am
Or, alternatively (and more likely, imho), such reviewers are like Austrian Emperor Joseph II in “Amadeus” who complained that Mozart “was using too many notes”. He wasn’t amused by Mozart’s request to clarify exactly which notes should be cut, but if the complaint had any validity that would be a reasonable request.
LikeLike
March 27, 2012 at 6:06 am
Hey thanks, I keep mistakenly crediting Salieri for that…
Look, Amadeus was genius. You could take the same tack on Jimi or on yngwe malmstein….thing is, for every genius there are a couple hundred doofs. And when it comes to grant review, not everyone getting told to simplify their approach is a misunderstood genius.
LikeLike
March 27, 2012 at 6:08 am
I disagree with Drugmonkey and think that he’s being convoluted in his approach to the evaluation of the review and doesn’t have a clue about appropriate language used for communicating information. But then his previously published posts suggested that he is often mistakes meanness for straight talk, so this is not surprising.
PS: I think that “”PI has clearly no clue”, assuming that’s an exact quote, shouldn’t have gotten past anyone’s filter. It’s meaningless blather and ad hominem attack, not communication. Even assuming that “convoluted” is a grant code word for more complicated than necessary or explained, it is not improved by adding “terribly.” If previous publications involve “exaggeration” those exaggerations should be specified (if not all of them, at least a couple to make clear in what area the issue lies. And, adding “gross” doesn’t help clarify in any way. Did the reviewer include any exclamation points? For example, previous publications have involved (like) gross exaggerations!!!!!!!!
LikeLike
March 27, 2012 at 6:09 am
It is also the case that sometimes grant reviewers who say this are not buying the “how many angels in the head of a pin” esoterica of a given BunnyHopper subsubsubfield. They are saying that they’d prefer to see less internal self-gratification and more meaningful progress.
LikeLike
March 27, 2012 at 6:18 am
Did you just call me mean, zb?
LikeLike
March 27, 2012 at 6:37 am
There are two interpretations of this kind of review.
(1) DM’s interpretation, i.e., the reviewer is completely reasonable and the applicant should attempt to figure out what the hell the reviewer is trying to get at.
(2) the reviewer is not reasonable and is just dinging the applicant because he/she feels like it.
Either way, the language of the review (obviously just the part we are told about) is unprofessional. Even if the reality was more like #1, I think it behooves reviewers as much as applicants to clearly describe their score driving objections to the grant in constructive language and where possible to specify the key problems. It seems hypocritical to expect applicants to write a well organized and well reasoned grant proposal, but not to hold reviewers to the same professional standard. If the reviewer does not adequately support their reasoning, then unfortunately, interpretation #2 is just as possible as #1, meaning the applicant may spend time and effort revising their proposal with no useful outcome if the proposal goes back to the same reviewer.
LikeLike
March 27, 2012 at 6:55 am
Someone remind me why we, as scientists, get pissy and cry “ad hominem attack!” when someone says “PI clearly has no clue” but feel justified in walking a little taller when we get “ad hominem praise” like “PI has clarity of mind and a simple, elegant approach to this complex problem”? Is the Nobel Prize not ad hominem praise? If we want the glory, we have to be able to accept the shame.
I’m curious… which of the following are an ad hominem arguments? Which are unprofessional?
a) PI should rot in hell
b) PI clearly has no clue
c) PI is misinformed and approach lacks clarity
d) PI’s approach is misinformed and lacks clarity
e) PI has an interesting approach
f) PI’s approach is sensible and well-informed
g) PI understands scope of problem and has sensible approach
h) PI clearly gets this shit
i) PI should have my babies
LikeLike
March 27, 2012 at 7:28 am
DM you say you’re “not seeing how you could put it more “nicely”? What about simply putting things *more productively*?
“…terribly convoluted approach” -> “a more elegant approach than that proposed would be…”
“PI has clearly no clue…” -> “PI seems unaware of *these relevant studies*/*this easier approach*/*this clearly superior approach*/*how unlikely these experiments are to be conclusive*/*how these studies fail to address the important question they say they relate to*”
pubs “are a gross exaggeration” -> “do not demonstrate what is claimed because…”
It’s only of tangential relevance whether the commentary is “nice”. It’s hard to know how much of the uselessness of the quotes is because they are such short out of context snippets, but it does not look good. When people are annoyed enough by what they are asked to give feedback on, the quality of their feedback tends to suck (understandably “this isn’t redeemable, so why should I give suggestions?”).
You seriously might as well just write “this grant is http://msp274.photobucket.com/albums/jj261/silverio16/poop.gif” 47 times. It would waste less of everybody’s time.
LikeLike
March 27, 2012 at 7:36 am
BugDoc-
I didn’t say the reviewer was “completely reasonable”.
I am saying I don’t agree that “gross exaggeration” and “terribly convoluted” are clear evidence of “unprofessional” language.
Furthermore, and this is my real point, if *this* is your level of evidence for review being “highly inflammatory, prejudicial and aggressive” then you need to grow the hell up. Or you are going to spend so much time wringing your hands and feeling sorry for yourself that you can’t possibly compete. Not to mention, having this stance deafens your ears to understanding the underlying deficiencies with your application. And if you can’t hear the *real* criticism because you are so focused on the tone in which it is presented, you are in a big old hole come revision time. Also a hole for you next new grant.
LikeLike
March 27, 2012 at 7:41 am
becca-
“elegant” is for pedantic assholes and is even more laden by “eyes of the beholder” crapola than is “convoluted”.
I agree the “no clue” is ad hominem and I already said that in the OP. It is still tame stuff though.
“gross exaggeration”, well maybe your way is nicer 🙂
When people are annoyed enough by what they are asked to give feedback on, the quality of their feedback tends to suck
Is this not an important thing to know about your grant? One thing I saw a time or two on review (and, ahem, more than once as applicant) was the multiple-revisions-but-PI-still-not-getting-it response. After saying the same thing nicely two times (or seeing the nice way ignored in the prior review), the reviewers might get a bit more forceful to emphasize that they are serious. Not saying this is great but within “nice” and “professional” language it can be challenging to express “no really, you dumbfucke, this thing isn’t getting funded without you listening to us and changing your writing/approach/etc”
LikeLike
March 27, 2012 at 9:03 am
Just curious, to what extent are SRO’s responsible for curtailing aggressive/unprofessional language in a review? Are they obligated to post reviews exactly “as is”, regardless of how blatantly offensive? Along those lines, if you have a reviewer in a study section who is consistently negative (for example, Clay’s comment above: “Some grant reviewers have never liked a grant they didn’t write themselves. “), is there any mechanism to keep these types of reviewers in check, or just have them booted out altogether?
LikeLike
March 27, 2012 at 9:16 am
DM,
The fact that certain statements do not sound “mean” to you doesn’t mean that they are not “mean” or felt as such by others. Diversity is a fact and we all should be required to recognize it and take care of it in its multiple forms and expressions. It is not necessary to make derogatory/demeaning-like remarks in a scientific review to alert the applicant on the weaknesses of her/his proposal, no matter how big they might be. Directness and consideration on other people’s feelings are totally compatible.
NIH should make a better effort in requesting all reviewers to pay attention to this because this is being going on for many years and it happens with new and with established investigators.
Adversity is part of life and everybody has to face it sooner or later. Why make it even harder, adding unnecessary elements?? Having a grant rejected is a very difficult event in a scientist’s life. Yes, being open to criticisims is in the essence of every scientific work. But it is not in the essence of science to “color or decorate” critiques to convince the applicant that the reviewer’s analysis is to take it seriously. This is how I see it.
LikeLike
March 27, 2012 at 9:29 am
“if *this* is your level of evidence for review being “highly inflammatory, prejudicial and aggressive” then you need to grow the hell up.”
It’s not an either/or proposition. The “grow the hell up” suggestion is good advice in general. But I still think that there is not much oversight, feedback or incentive to keep reviewers constructive (thankfully, most reviewers do a fine job in any case). I would argue that if you find it challenging to express “no really, you dumbfucke, this thing isn’t getting funded without you listening to us and changing your writing/approach/etc” more professionally, maybe you need more practice in writing. I think there are ways to say this clearly without being rude and personal, e.g. “Overinterpretation of key preliminary data, e.g. [cite specific example], is a significant flaw in this proposal since alternative interpretations such as [xyz] would undermine the aims.” or “While the hypothesis is interesting, the experimental approach seems unnecessarily complicated, prompting concern that the results would be very difficult to achieve and interpret.”
LikeLike
March 27, 2012 at 9:36 am
maybe you need more practice in writing
My point is to ask how the average reviewer would respond if the polite version has been tried multiple times and the PI still doesn’t seem to be getting it. I have seen this result in a minor uptick in the “meanness” of the language…as I said both by sitting on a panel and as a clueless (hardheaded?) applicant
Just curious, to what extent are SRO’s responsible for curtailing aggressive/unprofessional language in a review?
My experience talking in depth with a small number of SROs has been that they most decidedly do screen through summary statements for language. To the extent of having macros set up to flag keywords, even. I am not sure the extent to which they simply edit it themselves or demand the original reviewer clean up his/her act since I’ve never had one ask me to fix my reviews for language issues…
and of course they throw down repeated admonitions to reviewers to keep it professional, avoid ad hominems, etc in the weeks leading up to the review meeting. There are training documents as well.
LikeLike
March 27, 2012 at 9:43 am
@anon
The SRO obligation is to post the critique “as is”. However,
“if you have a reviewer in a study section who is consistently negative …….”, the SRO should give him/her a call and kindly request if reviewer would consider rephrasing “the statements” as to leave the scientific content intact eliminating the accessories. If the reviewer decides not to do it and he’s recidivist he should be ousted, no matter how “genius” he might be in his science.
LikeLike
March 27, 2012 at 9:48 am
The SRO obligation is to post the critique “as is”.
that ain’t the way I heered it……
LikeLike
March 27, 2012 at 9:55 am
“if the polite version has been tried multiple times and the PI still doesn’t seem to be getting it…”
Then it is the applicant’s sole responsibility to open his ears and his eyes and he/she will because there is only 2 chances for any given grant to get funded. If he/she wants to spend the rest of his life having a rejected proposal re-packed but unchanged on more and more tries, that is his decision. The paternalistic “decorative approach” by reviewer to make the applicant cease and desist is not right.
LikeLike
March 27, 2012 at 10:00 am
DM,
“that ain’t the way I heered it……”,
could you please rephrase it in standard English?. I don’t understand it. Sorry!.
LikeLike
March 27, 2012 at 10:18 am
The paternalistic “decorative approach” by reviewer to make the applicant cease and desist is not right.
It may not be right but reviewers tend to have a least a part of them pulling for applications. As I and others with review experience say repeatedly, there are a whole lot of applications very deep into the pile that are good. Interesting at the very least. Exciting (in part), quite frequently. When a reviewer gets excited about the bones of an application but the dingbat applicant can’t manage to write a decent response to review it can be frustrating.
Plus, *any* time you make a fair effort to review a grant and the applicant appears to be blowing you off, well, that doesn’t go down well. Simply human psychology.
So the fact that reviewers may escalate their rhetoric after trying it the “nice” way might be “not right” but it is an inevitable part of the way humans behave. This is why I think this guy’s examples are pretty piss poor ones. This language doesn’t represent anything too far over the line, imo.
LikeLike
March 27, 2012 at 10:19 am
I don’t understand it.
“That is not the way I heard it. “
LikeLike
March 27, 2012 at 11:28 am
“So the fact that reviewers may escalate their rhetoric after trying it the “nice” way might be “not right” but it is an inevitable part of the way humans behave.”
Seriously, are you saying that applicants should grow the hell up, but it’s okay for reviewers to escalate rhetoric based on their personal frustration with the applicant? Is it okay for reviewers to escalate their rhetoric based on other personal frustrations like “this person is too junior to have a clue” as you have often implied occurs when people review new investigator applications (hence the new investigator bump – a good thing IMO)? Reviewers should stick to their job and aim to provide a clear, constructive and professional review, with specific points to support their criticisms. Then it’s up to the applicant to respond to their comments. If they can’t do that, well, you can only lead the horse to water……
LikeLike
March 27, 2012 at 12:09 pm
Dude! this is a very, very, very, VERY tepid “escalation” we are talking about. I’m not in any way suggesting reviewers use the sort of language that emerges from PhyzzyPoof’s piehole.
Nor am I suggesting reviewers “should” do anything other than what you suggest. I am saying I find it completely understandable that reviewers get frustrated and sharpen their tone. More on point, I find it absolutely ridiculous that this guy (and you, apparently) think that this language we are discussing from his example amounts to evidence of “highly inflammatory, prejudicial and aggressive”, not to mention “mean-spirited” reviewer comments that should be expected to “destroy” a noob (and not so noob wilting violets) .
LikeLike
March 27, 2012 at 12:17 pm
The question of whether it is “mean” or “ok” for reviewers to use language like that is a complete red fucken herring, and is completely orthogonal to whether the reviewer was “biased” or “unfair”.
(1) Some of the most “unfair” criticisms I have ever seen in a critique have been written extremely politely, or even couched as fucken compliments but intended to influence the panel against the grant and/or applicant.
(2) Even if the reviewer was “mean” and the statements were not “ok”, the fact is that the reviewer didn’t like your fucken grant. If the reviewer liked your grant, she wouldn’t have been “mean”. This is a “you problem”, and you need to write a fucken grant so that reviewers like it and aren’t “mean” to you.
LikeLike
March 27, 2012 at 12:45 pm
“That is not the way I heard it. ”
Me, neither. The quote from an SRO that I use in seminars is, “We remove the egregious personal remarks.” In the context of the OP, I wonder about the rest of the sentence indicated by the elipses ”PI has clearly no clue…” If someone proposes something that has been tried and published, and doesn’t indicate that they know it has already been done, then “no clue” doesn’t seem like a mean remark, but a less-than-tactful way of pointing out ignorance. And there’s no excuse for ignorance of that scale in a proposal.
LikeLike
March 27, 2012 at 1:04 pm
I guess I’m not particularly trying to weigh in on whether the reviewer was “mean” in that particular case, so I think you’re mistaking my concern as being about people’s hurt feelings. All along, I’ve been trying to make the GENERAL point that providing a useful review to applicants (which I hope you’ll agree is an important part of the peer review process) is better done when reviewers make their points specifically and clearly without provocative language. I know with you guys, it’s always about how to sackke up and deal with it, which is fine on an individual basis. In fact, there’s really no other option, so it’s sort of a no-brainer. But I think it should be possible to have an opinion about issues with the general review process without it being all about “me” and my “fucken” grant. Discussion about science should be open and constructive; writing comments with a rude or snarky tone underserves the process of review. Is it reality that some people are asshats? Sure. Is there a problem with calling it out? And yes, Physioproffe, being polite has nothing to do with being fair (cookie for you!). But it’s a good start.
LikeLike
March 27, 2012 at 2:01 pm
wow. if this stuff is “mean”, more of us are Reviewer #3 than i had previously thought. or maybe it’s just me…
LikeLike
March 27, 2012 at 2:04 pm
Wading in delicately (since I’m not really so much in the science-grant-applying or science-grant-reviewing game these days) on the more general issue of feedback that is of potential use to its recipient, especially from the point of view of revising:
Sometimes the content is close enough to where, ultimately, one thinks it should end up that one can suggest straightforward ways to make what is proposed less convoluted, or can succinctly identify reasons that cited paper X does not demonstrate (or say) what the grant proposal author suggests that it does (e.g., by citing a subsequent publication that undercuts it), or can easily label a particularly important hole in the literature review, etc.
Other times, however, what one is reviewing may be so disconnected from the relevant literature, so methodologically tortured, or so downright incoherent that the most reasonable thing for the reviewer to do is simply identify it as such rather than trying to fill in the gaps in the thinking that the person writing the proposal ought to have done in the first place.
And, I’m prepared to assert that it’s not prima facie mean to refuse to do someone else’s homework here.
LikeLike
March 27, 2012 at 2:23 pm
All along, I’ve been trying to make the GENERAL point that providing a useful review to applicants (which I hope you’ll agree is an important part of the peer review process) is better done when reviewers make their points specifically and clearly without provocative language.
The person complaining at Rock Talk in the first place had presumably extracted the most “provocative” phrases they could possibly find in the critique and left out all the parts where reviewers “made their points specifically and clearly”.
LikeLike
March 27, 2012 at 2:36 pm
Wait… Was that you trolling me at RockTalk!!????!!??
LikeLike
March 27, 2012 at 3:28 pm
“And, I’m prepared to assert that it’s not prima facie mean to refuse to do someone else’s homework here.”
Yes! And in addition, NIH policy officers state clearly that it is not the reviewer’s job to write the proposal for the applicant. The reviewers are asked to use bullets to point out the strengths and weaknesses, and provide a paragraph that discusses the main issues that drove their Overall Impact score. They are directly discouraged from offering alternative approaches.
LikeLike
March 27, 2012 at 4:52 pm
At least you can get some information from potentially mean-spirited comments.
“PI has no clue about bunnyhopping” or the like is easier to interpret than what I received under Innovation from one recent reviewer:
Strengths
1. positive comments on PI’s experience with innovative approaches now applied in new area
2. positive comments on approach for said study
Weaknesses
“None noted.”
Criterion score: 6
Any suggestions here how to respond to that in the resubmission?
I can interpret that I should get a clue about bunnyhopping or whatever–read more and get it right, but how does one interpret the above? I’ve seen this a few times where the score doesn’t match the comments. 5 for an Environment that was “good” with “no weaknesses.” If it said, “your department sucks and doesn’t offer any support, you could look for a new job.
LikeLike
March 28, 2012 at 9:22 am
I would much rather read a review that tells me how the reviewer feels about my application than some carefully-couched, nice-sounding drivel that doesn’t make clear the problem with the application. When I was a n00b and got my first summary pink sheets, I was quite unclear about why my applications were scoring poorly. I needed a senior colleague to read and interpret them, and it was some time before I could read and write reviewer-ese.
The number one rule of grant writing is: Don’t tick-off the reviewer. Clearly this person has. Writing a “convoluted”, unclear, poorly-explained approach is an easy way to mess up. “Gross exaggerations” of the significance of your work or a particular interpretation of your results is a good way to tick-off your whole field as well as the reviewer of your application. Have some humility. State the limitations of your approaches.
LikeLike
March 28, 2012 at 9:28 am
Years ago, I applied for a local grant, and recommended a Guru to review it. I was utterly surprised when the Guru gave a strong negative review of my application. In addition, I was not able to understand some of the comments. After stewing around a while, I called the Guru (who was a friend and coauthor). He asked if I would be at the meetings. He said one of his students would present a paper which would answer all my questions. OK, I ‘m at the session ready to take notes. The student gets up and announces that he is not going to give the presentation listed, but rather a different one. So I was never enlightened, and have gone on ever since traveling down unrecognized erroneous paths.
LikeLike
March 28, 2012 at 10:06 am
Ass Prof-
Agreed that sounds like a poorly calibrated reviewer…did the other reviewer scores make more sense? In any case, finding local (or field specific) friends to look it over for you can help.
LikeLike
March 28, 2012 at 10:08 am
And you never asked Guru what was up, JT??!!??
LikeLike
March 28, 2012 at 3:09 pm
“Gross exaggerations” of the significance of your work or a particular interpretation of your results is a good way to tick-off your whole field as well as the reviewer of your application. Have some humility. State the limitations of your approaches.
It’s substantially more complicated than this. I agree that you have to stay humble when it comes to particular interpretations and approaches, but you absolutely must not be humble when it comes to explaining the significance of your proposed studies to the field at large. If it sounds ludicrous to assert that successful pursuit of your aims will have a major impact on your field, then your aims sucke.
LikeLike
March 29, 2012 at 7:22 am
“you absolutely must not be humble when it comes to explaining the significance of your proposed studies”
Agreed. Your aims should change the field, and you should do a good job of explaining that. But there are people that grossly exaggerate the significance of their results in their publications and write papers that are so self-congratulatory that they are painful to read. Each field I am in or have been in has a couple of such jerks. Everyone loathes them.
I have also had the experience of reviewing applications that were written too arrogantly. There is a difference between an application that is bold and confident and one that represents the applicant as the messiah that will save the field from the ignorant plodders.
LikeLike
March 30, 2012 at 6:35 am
I wouldn’t write something as forthright as that in a review, and I think I’m nice! It is still easy to get the message across without using the language used above. [Insert caveats about not knowing the whole grant history, cheery picking the nasty comments etc here.]
LikeLike