Oh grow up
June 15, 2010
It recently occurred to me that after all my years of study section service there is only one PI that appears to hold a grudge. Meaning to the extent s/he can’t even acknowledge my presence. And yes, this was a shift that developed after this person had a grant go through a section I was on.
Oh, I’m sure there are others who suspect, rightly or wrongly, that I am to blame for their disappointing grant score. But they seem to be able to act like grownups about the situation.
DrugMonkey 
June 15, 2010 at 12:38 pm
Ah, but you are not a scientist unless you have an archnemesis! 🙂
LikeLike
June 15, 2010 at 12:44 pm
You know, we all try to do our best to be as objective as possible when reviewing to the point that sometimes a grant doesn’t do well because, well, you know, it wasn’t that good.
To blame you is hysterical. But to not understand that you may very well likely *not* have been one of the three reviewers shows a lack of understanding of the process.
I can’t tell you how many times I’ve not been assigned grants that I consider to be right up my alley.
Bottom line: trying to guess who reviewed your grant is often futile. Holding a grudge is childish.
And, hey, have you ever had anyone try to thank you who may have guessed that you championed their grant application?
I didn’t think so.
LikeLike
June 15, 2010 at 2:12 pm
And, hey, have you ever had anyone try to thank you who may have guessed that you championed their grant application?
I didn’t think so.
No, but presumably they are worried about appearances of conflict, violation of privacy of review, etc. It is pretty clear you are not supposed to discuss the proposal with individuals on the panel prior to review. It is okay (as I understand it) to speak in general terms afterward. As in “Hey, you guys really liked it, eh?” or “Geez, you guys were killing me”, just so long it is not trying to ferret out who exactly the assigned reviewers were or anything privileged from the actual discussion.
LikeLike
June 16, 2010 at 10:27 am
Another version of this: a member of the study section you are on is talking to an applicant and proclaims (at a volume that can be overheard by you standing nearby) that s/he thought the applicant’s proposal was fantastic but that there were “those” on the SS that couldn’t see its value. [Eyes are then diverted in your direction, at this point, for maximum effect].
LikeLike
June 16, 2010 at 10:46 am
David, that would appear to be a violation of the confidentiality of review, IMO.
LikeLike
June 16, 2010 at 12:36 pm
Perhaps reviews shouldn’t be confidential. Being personally accountable for what a reviewer writes would improve the quality of reviews, and if some jerk holds a grudge because of an honest review, then screw him.
LikeLike
June 16, 2010 at 12:53 pm
Depends on what you mean by ‘quality’. I suspect that honesty and deserved criticism would suffer. This would further obscure the discrimination between applications and likely introduce even more disconnect between review language and priority score.
LikeLike
June 16, 2010 at 1:25 pm
DM: By “quality” I mean eliminating nonsense, pseudoscience and obvious ignorance. Reviewers would think twice before signing bullshit with their names. Would this obscure the discrimination between applications? No, IMO it will only force the discrimination to be more merit-based (and thus MORE honest), instead of being based on whatever else, and accompanied by “rationales” (painfully manufactured to fill the space in summary statements) that look like missing chapters from “Alice’s Adventures in Wonderland”.
LikeLike
June 16, 2010 at 2:17 pm
Yeah, because whenever you get a bad review, it’s gotta be “nonsense, pseudoscience and obvious ignorance” and not “your grant sucks”.
LikeLike
June 16, 2010 at 2:30 pm
Yeah, CPP, maybe you wrote the following cretinism personally:
“Weaknesses
(…)It seems likely that a malevolent group engineering a disease vector would adopt as a first stratagem
enabling it to resist the current antibiotics and this would largely bypass the stratagem being developed
in this proposal.”
The same Reviewer, in the immediately following paragraph
“APPROACH
Strengths
This approach would be antibiotic independent.”
This is, excuse my language, fucking Alice in the fucking Wonderland, and it is another Mad Tea Party logic if you believe that the NIH peer review system is sound.
LikeLike
June 16, 2010 at 3:57 pm
Gummibears, 20 grants and nothing? Have you ever considered you are a crappy grant writer?
LikeLike
June 16, 2010 at 4:12 pm
I suspect that honesty and deserved criticism would suffer [without reviewer anonymity].
I agree, and I further suspect that it would suffer in a way that would benefit the more established players.
Consider two hypothetical scientists, the well-established Dr. Big and the up-and-coming Dr. Young. In the normal course of business, at least in my field, Dr. Young may be asked to review a proposal (or a manuscript) by Dr. Big, or vice-versa. Dr. Big need not worry about offending Dr. Young as there is nothing Dr. Young can do to derail Dr. Big’s career. However, if Dr. Big chooses to hold against Dr. Young it will adversely impact Dr. Young’s career. So if reviewers are named, Dr. Young would have to be circumspect in his reviews of Dr. Big’s proposals/manuscripts, instead of calling out any BS that Dr. Big may be shoveling in. Dr. Big, meanwhile, can continue to trash Dr. Young’s work on any grounds that are remotely plausible. I, the program officer or journal editor, see negative comments on Dr. Young’s work by the Dr. Bigs of the field, but neutral to positive comments on Dr. Big’s work by the Dr. Youngs of the field. Whose proposal/paper do I fund/accept for publication?
LikeLike
June 16, 2010 at 4:18 pm
Anon: 20 and nothing at NIH. 2 and one funded on the first try at NSF. The same PI, the same idea, from the hard core NIH programmatic area. Draw conclusions.
LikeLike
June 16, 2010 at 4:22 pm
Really E. Lund, like the Unicorns and rainbow fairies would ever let that happen.
LikeLike
June 16, 2010 at 4:28 pm
Gummibears, you should quit academic science. How many years of your life went down the toilet writing 20 unfunded grants? You sound like a creative person… can you come up with a better way to make use your time?
Regarding NIH peer-review, it would be nice if the reviewers didn’t collude with each other outside the study section meeting. I know several examples of chummy reviewers keeping their various sub-fields closed to interlopers by agreeing to damn grants that come their way with faint praise. So easy and so untraceable to keep the dollars at zero by giving out 3’s and 4’s. 🙂
LikeLike
June 16, 2010 at 4:34 pm
Eric: the argument is correct, but only partially. The lack of anonymity would be a strong deterrent against writing offensive/groundless critiques in the first place. There are extreme cases of people unable to accept even meritorious and polite criticism, but they are not prevalent enough, IMO, to prevent us from considering the idea of introducing transparency to fix the currently broken system at NIH.
Of course, this is not a black and white situation: “transparency good, anonymity bad”. NSF can produce meritorious critiques in spite of the anonymity. There are, however, different checks and balances there to prevent a misconduct (strong position of the Program Officer/Director, who usually is an experienced scientist; more variable composition of the review panels). So, I am not saying that the transparency of reviews is the only possible solution for NIH. But it is one of the options.
LikeLike
June 16, 2010 at 5:08 pm
“How many years of your life went down the toilet writing 20 unfunded grants?”
Quite many, whimple, and that’s why I have a score to settle with the NIH institutional culture.
As for leaving science: the imbeciles at NIH have almost driven me to that point, but now it is moot: I got decent funding from NSF, I can work on my stuff, and NIH can rot. I only hope that, since NSF is forced to fund the NIH programmatic area (due to the parochialism at NIH), the money for these agencies will also be appropriated accordingly.
LikeLike
June 16, 2010 at 5:23 pm
Whimple, it’s disappointing that you can’t make the connection between your two statements (#15): advising Gummibears to leave science because he can’t get funded from the NIH, and then to say “I know several examples of chummy reviewers keeping their various sub-fields closed to interlopers by agreeing to damn grants that come their way with faint praise.” How is any new investigator supposed to penetrate that culture?
I quote Max Planck, “Science progresses one funeral at a time.” In this case, it would have to be early retirement for those chummy NIH-funded douchebags.
LikeLike
June 16, 2010 at 5:32 pm
Anon, b/c even whimple knows that kind of conspiratorial chummy-buddy stuff is not the norm. It is bad that it goes on at all, sure. But I’m not buying it as common
LikeLike
June 16, 2010 at 5:37 pm
Eric, I’d like to add something extra to your scenario – appreciate your thoughts:
Consider that Dr Young does come up with criticism of Dr Big’s approach (valid or invalid, it doesn’t matter). Dr Big is offended either way, and calls the journal editor/program officer to complain on the grounds that Dr Young is unqualified in the field and doesn’t know what the hell he/she is talking about. Dr Young’s credibility is brought into question, and Dr Big has reason to trash him further in the future. This already happens with journal peer reviews all the time, although because of blinding it’s generally because of Reviewer 1. Now imagine if either of the processes we are describing here is unblinded.
LikeLike
June 16, 2010 at 5:38 pm
Anon: I believe the retirement of the club members will not be enough, as institutional culture tends to be more persistent (and infectious). NIH needs a shock therapy, I’m afraid. One possibility would require introducing transparency of the review process. Another idea may involve hiring “super Program Officers – NSF-style established scientists, who would be more difficult to bully by the panels. Third: an appeal venue independent of the NIH. Alternative ideas may also be proposed, but they will need to be quite radical. Much more so than the lip service having been given for some 20 years by the NIH to “improving peer review”.
LikeLike
June 16, 2010 at 6:41 pm
I didn’t advise “leave science”, I advised “leave academic science”, although Gummi, that you would do the same failed experiment 20 times does not inspire high confidence in your scientific abilities. I hope you didn’t at least go 0 for 20 all with the same study section. An important part of science is recognizing when “you can’t there from here” and going somewhere else. Waiting for NIH institutional change is likely to be a very long wait.
DM, collusion between reviewers does not necessarily have to be overt, or even overtly conscious. Don’t you know of any Golden Children of impeccable pedigree in your own field that turn out science of disproportional non-brilliance to the magnitude of their extensive funding? How do you think that happens?
LikeLike
June 16, 2010 at 6:55 pm
Don’t hold out whimple, that sounds like a good gig. How do I get in on that?
LikeLike
June 16, 2010 at 7:03 pm
You have to do a postdoc with the right person. I can give you a list of names if you’d like. 🙂
LikeLike
June 16, 2010 at 7:42 pm
Whimple, the things that kept me trying at NIH (different study sections) for so long were the match with their declared programmatic area, and my naive view that the system largely works as advertised. My rants here are community service: I hope someone currently unfunded and wondering what the hell is going on (I assume that this someone has decent scientific ideas) will not repeat my mistake and waste so much time.
I am wondering how many people applying to NIH are still unfunded and driven to believe that they write crappy proposals. My situation was quite unique (method development, heavily borrowing from non-biological fields), and, paradoxically, the pure-nonsensical bullshit the NIH review panels produced helped – I could believe that my ideas were crappy if they were trashed using a sound argumentation, but not when it was obvious that the reviewers were in fact totally clueless. But in other cases, when people send proposals more comprehensible by the NIH study sections, the panels may produce shit that looks convincing.
In any case, such unfunded PIs should try applying elsewhere, and not be discouraged by the lower average success rates (e.g. NSF). At least they will be competing on a more level playing field.
LikeLike
June 16, 2010 at 7:44 pm
Now that I think about it, enterprising young scientists could look at this list:
http://www.hhmi.org/cgi-bin/scientist_search/search.pl?kw=&it=ECS&x=48&y=12
and do a post-doc with just about any of the people these people did postdocs with. Pick one that’s not likely to die soon though. Boo-yeah! 🙂
LikeLike
June 16, 2010 at 8:44 pm
Gummi —
One day last month, I really wanted a taco. So I went to McDonald’s and asked for a taco. And they wouldn’t give it to me!
So I went back the next day, and asked for a taco. And they still wouldn’t give it to me! I went back to McDonald’s everyday for 3 weeks, and still no taco.
Finally, I thought — what the hell, let me try Taco Bell. And I got a taco!!!!!
Clearly, the institutional culture of McDonald’s is broken.
LikeLike
June 16, 2010 at 9:14 pm
That’s classic Neuro-conservative!
Let’s name names on the study sections. I nominate GMPB – Gastromucosal Pathobiology study section as my top crony study section.
LikeLike
June 16, 2010 at 9:19 pm
Listen dumbass, we’ve explained this already to you a gazillion times: If study sections are always “misunderstanding” your grants and are always “clueless”, IT MEANS YOU ARE WRITING YOUR GRANTS POORLY. IT IS YOUR FUCKING JOB TO EXPLAIN YOUR FUCKING SCIENCE TO THE FUCKING STUDY SECTION.
LikeLike
June 16, 2010 at 9:30 pm
NeuroC: With one correction: I kept asking for a hamburger at Mac Donald’s, week after week, and no hamburger. Finally, I thought –what the hell, tried Taco Bell and they gave me my hamburger!
(hamburger=drug discovery method development and experimental validation using examples of selected pathogens; want to tell me that this is not the NIH programmatic area?)
LikeLike
June 16, 2010 at 9:33 pm
It could also mean the study section doesn’t care what you wrote, for a whole pile of reasons, some science based, others not. Once the “SUCKS” stamp goes on your application, the incentive to come up with rational critiques goes way down, particularly for triaged grants.
LikeLike
May 14, 2011 at 5:00 am
hey, just ’cause a rabbit is smaller than a human, doesn’t mean they don’t matter as much and aren’t worth having feelings for or crying about.
for more info please visit:
http://nationalliteracyconsortium.org/index.php?title=Main_Page/
http://iibcc.org/index.php?title=Main_Page/
LikeLike