Question of the Day
April 2, 2018
How do you assess whether you are too biased about a professional colleague and/or their work?
In the sense that you would self-elect out of reviewing either their manuscripts for publication or their grant applications.
Does your threshold differ for papers versus grants?
Do you distinguish between antipathy bias and sympathy bias?
DrugMonkey 
April 2, 2018 at 11:45 am
If I see good work from a colleague in the field, I do have a natural bias in favor: I can appreciate it more than work out of the field, because I know the context, and because I naturally think it is interesting. There is nothing wrong with championing good work in one’s field; indeed I hope to see colleagues in my field both reviewing my papers and my grants.
I self-select out of grant and paper reviews under two conditions: if the abstract indicates that they are doing the same experiments I am (this is super rare); or if I have a bias AGAINST them because I feel they have reviewed me unfairly in the past- even if this is just a suspicion. Better to decline to review than to risk being perceived as biased. Again this is rare, but it has happened.
Also, twice I have declined to review papers from a colleague I have gotten too many recent paper review requests from. I think it is healthy to get reviews from multiple sources.
LikeLike
April 2, 2018 at 12:42 pm
@baltogirl – yeah that last one gets me a lot. Nothing worse than having a good dinner with a seminar speaker, then having every. single. one. of their papers come across your desk for the next 6 months. Wish they’d be a bit more imaginative in their reviewer suggestions. In these cases I just respond to the editor to “spread the love” and have someone else review instead.
To DM’s question, I’d say self-selection recusal threshold is much stricter for grants. To review a paper when one has a beef with the author is easy, especially if it’s in a low rank journal because who gives a shit anyway? You may not even get to see the other reviews (because who the hell follows up to see what other people thought about that paper you rejected?) To do the same for a grant is more difficult and the consequences far greater – you’re required to justify your biases in front of a room of 30 people. I guess a lot comes down to the question of whether people are watching – my gut feeling is maybe folks are more willing to recuse themselves if peer review is open or in-person. If there’s a chance you may get called out for a COI, you’re more likely to declare a COI and recuse. If it can go under the radar, then keep calm and carry on screwing people over. This is partly why I hate those online NIH review panel things. Eye contact is the killer!
LikeLike
April 2, 2018 at 12:44 pm
So TL/DR, like everyone else on the planet my moral compass is determined by who’s watching. Anyone who claims to be immune from this effect is lying.
LikeLike
April 2, 2018 at 1:16 pm
Such an interesting topic. If I see the abstract of a ms I am asked to review and feel it would benefit me personally to reject it then I declare a conflict of interest. I do not apply the same rubric in the positive — that is, if some paper would benefit me by being published, I tend to think that I can fairly evaluate. For grants I have only recused myself on the basis of being too close a collaborator, so I haven’t excluded myself for anticipated antipathy yet.
LikeLike
April 2, 2018 at 1:53 pm
Ola-
Thinking back over review experiences I most often find myself not questioning *why* a reviewer seems unusually favorable or unfavorable about a review. Maybe once there was a clear bias (for “person from my home country”) that I saw. And that one was only made obvious because the person an d I had served on multiple rounds together and this particular review really was unusually positive for that other reviewer.
Most of the time I assume it could all be a legit scientific difference of opinion. A bias based on the science, not the PI or other team members.
So I’m wondering how general your idea is that “people are watching” dictates or influences COI self-election. The watchers would have to know about the conflict.
LikeLike
April 2, 2018 at 2:38 pm
I focus more on the sympathy bias side. If I don’t feel like I can provide honest critique because of my relationship with the person, I would decline review. I don’t worry about the antipathy side because I think all negative critique should have a scientific justification.
I recently came across a new to me rationale to decline a paper review. The paper dealt with a topic that has two incompatible schools of thought. One reviewer said he was from the other school of thought than the authors and would be unable to provide a fair review. The wording implied (to me) that the issue wasn’t a lack of technical knowledge on the reviewers part, but instead based on the difference in opinion about the validity of the method.
LikeLike
April 2, 2018 at 5:09 pm
The wording implied (to me) that the issue wasn’t a lack of technical knowledge on the reviewers part, but instead based on the difference in opinion about the validity of the method.
I’m definitely torn on these. In the first sense, I think we should be able to see our method/approach biases and understand at some level where it is about preference as opposed to genuine right/wrong. And we should not litigate these larger issues for each and every paper by the other side, but try to review within that world view*. OTOH, sometimes we really, really, really believe those guys over there are doinitrong and ruining all the science. and we should be willing to fight for what we believe is justified/unjustified.
*historical examples in my field include the method to make rodents dependent on alcohol- sucrose versus saccharin fade procedures (fights were vicious), forced fluid/diet, schedules of access, preferring strains and vapor inhalation. all had proponents and I guarantee there was more than one paper review situation where arguments about method were really unnecessary and ridiculous.
LikeLike
April 2, 2018 at 8:24 pm
I recently reviewed a paper that I thought was so bad, so derivative, so non-rigorous and sloppy. I checked the box for “should not be published anywhere” instead of the one for “more appropriate for another journal.” I felt a little guilty, because what if it was some grad student’s first manuscript? Maybe I was too harsh. I always beat myself up about this kind of thing, and for peace of mind I should probably only review for authors who I am biased in favor of. Well, anyhow, the other reviewers hated it too and the paper got rejected. So I felt okay. It was unanimous.
Three months later the journal asks me to review THE SAME PAPER. Apparently the BSD corresponding author convinced the editor to reconsider. I told them no — “if they convinced you to reconsider, then I think you owe them a fresh set of reviewers.”
It’ll probably get published there. And if not there, then somewhere else. One more log on the pile of open-access crap.
LikeLike