On not using the suggested reviewers
August 27, 2012
There’s a good one up at retraction watch. An author suggested reviewers for his manuscripts using email addresses he had access to, then supplied his own reviews. Apparently suspicions were aroused by the 24h return of reviews- an obvious sign of fakery.
My learned colleague Odyssey opined that this situation strengthened his resolve to never select the suggested reviewers when acting as an Associate Editor.
I think this is ridiculous. A few bad apples, blah, blah. But more importantly, it seems simple fairness that if a journal is going to request suggestions for reviewers then they should use them. And not, as Odyssey is suggesting, as an exclusion list.
I think AEs should use one and only one of the suggestions.
DrugMonkey 
August 27, 2012 at 8:12 am
M’learned DM,
It is a two way street. I’d be more than delighted to use suggested reviewers if more authors could be trusted to suggest appropriate ones.
And damn you can type up a post quick!
LikeLike
August 27, 2012 at 8:13 am
Oh, and I’d be quite happy if journals did away with the requirement for suggestions.
LikeLike
August 27, 2012 at 8:17 am
I always appreciate a list of potential reviewers — though I don’t think the journals I edit for make it a requirement, because I don’t always get one.
When I do, I run ’em all through Jane and BioMedExperts, to see if there are any obvious COI’s such as co-pubs with authors, or potential red flags such as working at the same institution with one/more of the authors for several years. If there aren’t, I invite ’em all — but I make sure to invite an equal number of reviewers not on the list, too. I see it as an opportunity to load new reviewers into the database and improve review on this manuscript, not an opportunity to slack off on finding reviewers. I never make a decision on the sole basis of reports from author-selected reviewers.
LikeLike
August 27, 2012 at 8:32 am
If that is the journal’s policy, it should be noted. If I list the most qualified reviewers in the field, I don’t want most of them excluded because there is an unstated policy that only one would be considered.
LikeLike
August 27, 2012 at 8:36 am
It always seemed most logical to me that I wouldn’t be getting everyone on my list.
So I usually shot for suggesting ppl I thought wouldn’t be obvious to an AE…
LikeLike
August 27, 2012 at 9:10 am
I’d actually like to know how other people choose the “suggested reviewer” list. My advisor has always tended toward choosing people he knows (at least has had conversations with), and would never suggest someone he didn’t know, even if the potential reviewer had written relevant papers. I think this is less an attempt to game the system, and more to avoid being struck by random acts of reviewer assholeness. On the other hand, the suggestions do usually get us pretty relevant reviewers – you tend to know the people close to your subject.
If most authors choose this route, I would agree with DM – one and only one of the suggested referees. If significant numbers of people were really writing in to suggest referees of type “This person knows everything about the field, and will thrash you if you screw it up,” or just listing “here is everyone who’s written good papers on this topic in the last ten years, including our science enemies,” then maybe include more. But I don’t think that’s how most people suggest referees. Time for a poll?
LikeLike
August 27, 2012 at 9:35 am
As an editor, you are not fuckeing uppe by using a suggested reviewer. You are fuckeing uppe if you don’t at least look the people up you don’t know of on Pubmed or whatever and see what their qualifications are. And if the e-mail address provided by the author is not an institutional one, identify an institutional e-mail. This has nothing to do with suggested reviewers and everything to do with an editor being lazy and clueless.
LikeLike
August 27, 2012 at 9:37 am
Pinko Punko,
It’s not journal policy. It’s how I operate as an AE. AE’s have the freedom to choose reviewers as they see fit. The suggested reviewer policy was put in place to supposedly ease the work load for AE’s, but in my experience it doesn’t.
LikeLike
August 27, 2012 at 9:39 am
Important to remember that a “suggested reviewers list” is essentially an “excluded reviewers list” – 1, because no editor I’ve heard of will use more than one.
My own impression based on limited experience (but also shared by the editor of a large society journal) is that there is little benefit to authors in having a suggested reviewer used in terms of getting an “easier” or more positive review. At best you have made the editor’s job easier, which is not nothing, and which is the point.
LikeLike
August 27, 2012 at 9:40 am
And what CPP said.
LikeLike
August 27, 2012 at 10:57 am
Odyssey — what you are doing is wrong and asshole-ish. And I do not say that lightly — I don’t believe I have ever used that sort of language in a blog comment. You are simultaneously slacking on your responsibilities and violating the trust of your journal which established the policy in good faith. You should stop doing this, or withdraw from being AE if you cannot.
LikeLike
August 27, 2012 at 11:50 am
Neuro-C,
Why exactly is it wrong? I can understand authors not liking it and maybe thinking it’s asshole-ish, but there is nothing in the journal policy saying that AE’s will use reviewer suggestions. As I have tried to point out, I have adopted this policy only because too many authors abuse/game the system. I volunteer my time and effort to work as an AE – I’m not paid. All it really buys me is a line on my CV, and not one I need at this point in my career.
Do you really think my finding more “independent” reviewers imperils the publication chances of a manuscript? IME suggested reviewers are just as likely to unfairly shred a manuscript. I read every manuscript I handle before sending them out to review. I do my best to find reviewers who have the background necessary to review each manuscript. I read and consider each review. I have at times over-ruled reviewers asking for what I consider unnecessary revisions/new experiments. You want me to stop doing this?
LikeLike
August 27, 2012 at 12:05 pm
IME suggested reviewers are just as likely to unfairly shred a manuscript.
This. I remember a case when the authors wrote a rebuttal of the rejection of a manuscript I handled, suggesting that I purposely found biased reviewers, when the most negative reviewer was one they themselves suggested! It took great restraint not to just say that…
But in general, I find suggested reviewers to be a good idea and a time-saver — but, yes, I do a bit of research for COIs, and never rely entirely on suggested reviewers just in case they are all the the authors’ former drinking buddies or something.
LikeLike
August 27, 2012 at 12:14 pm
Odyssey — it is irrelevant what “too many authors” may or may not do. Your charge is to handle each manuscript individually. If you are too lazy or vindictive to do this, you should resign your position. Especially if you do not “need” the line on your cv and you feel resentful of the time demand.
LikeLike
August 27, 2012 at 12:33 pm
Regardless of what the right move on this is, there needs to be a consistent, established policy that’s made clear to authors.
It’s one thing for an AE to basically disregard my picks. But if the reviewers I get are anti-correlated with my recommendations, well, I really want to know about that.
LikeLike
August 27, 2012 at 1:04 pm
@poke
Is it really not crystal clear to authors that suggested reviewers are simply that, merely suggestions that the AE can use or not as they please? It’s not like they are called obligated reviewers. After all, even if the AE invites the suggested reviewers, there’s no guarantee that they’ll accept the invitation.
LikeLike
August 27, 2012 at 2:24 pm
@Jonathan Badger
I understand that suggested reviewers are just that: suggestions. That is absolutely clear. That’s not what I was complaining about in my post.
I’m fine with editors’ choices being independent of my input. But it’s a problem if the well-qualified reviewers I suggest are specifically being excluded.
My point isn’t that I should get to pick who reviews my papers.
I just don’t want people who could truly understand, appreciate, judge, etc. my papers totally eliminated from the process because some AEs are using the suggested reviewers list as a list of people to not send the paper to.
LikeLike
August 27, 2012 at 2:51 pm
I’ve been an AE for Middling Journal for two years and also have frequent issues with inappropriate people listed as suggested reviewers. These have included obvious conflicts of interest such as the reviewer being a recent trainee or mentor of the author and/or a recent coauthor and reviewers that are in the same department as the author in addition to reviewers that work in fields that are only marginally relevant to the paper and reviewers that are all from the same city as the authors. I run all of the suggestions through a database for which the name escapes me at the moment and the number of times a conflict of interest is flagged and/or the suggested reviewer only published one paper in the area 20 years ago is astounding.
Middling Journal is middling at best and not even close to Glamor Mag status. I get a lot of papers from (1) PIs for whom publishing in Middling Journal would be a career high and to a lesser extent from (2) established PIs who are looking for a home for a manuscript that, for whatever reason, hasn’t been accepted into “better” journals. It’s only a subjective observation, but the problems I have had with suggested reviewers tend to come more from Group #1.
LikeLike
August 27, 2012 at 2:59 pm
Odyssey — what you are doing is wrong and asshole-ish.
Dude, you’re fucken high. There is absolutely no obligation for an editor to choose to make use of a suggested reviewer, and all journals explicitly state that they may or may not make use of any suggested reviewers, at their absolute discretion. They also state explicitly that they cannot promise that they *won’t* use a reviewer the authors have “excluded”.
The purpose of allowing authors to suggest reviewers is *solely* for the convenience of the editor, and is not based on some right that the author possesses to influence who reviewers her paper.
LikeLike
August 27, 2012 at 3:03 pm
Your charge is to handle each manuscript individually. If you are too lazy or vindictive to do this, you should resign your position.
Dude, you’re fucken deranged!
LikeLike
August 27, 2012 at 4:54 pm
I’m fine with editors’ choices being independent of my input. But it’s a problem if the well-qualified reviewers I suggest are specifically being excluded.
This is the thing that worries me, here. If the papers whose authors are trying to game the system don’t overlap with those that the AE’s are trying to “ungame,” then everyone’s time has been wasted, and the manuscript may not be reviewed by the best possible people. If AE’s can’t trust authors to provide a conflict-free list of suggested reviewers (which it seems they often can’t), then maybe it shouldn’t be the authors’ responsibility in the first place.
LikeLike
August 27, 2012 at 5:16 pm
Indeed, Dr. Becca. As physioprof failed to note, Odyssey was specifically excluding suggested reviewers. That is a violation of trust in the review process. It doesn’t matter if Odyssey feels that “too many authors” have tried to game the system, he is being unfair to those who act in good faith.
LikeLike
August 27, 2012 at 9:12 pm
Wasn’t the sole failing in this case that of accepting the *email addresses* provided? Why not just have a policy of independently obtaining email addresses from institutional staff pages? Much harder to fake.
LikeLike
August 28, 2012 at 4:09 am
Neuro-C,
What violation of trust in the review process? The trust that you’ll get the reviewers you’ve suggested?
LikeLike
August 28, 2012 at 4:42 am
I have been publishing in a limited area, where there are only a handful of competent reviewers. We suggest each other as reviewers, and that is how it works. Occasionally an editor will slip in an outsider. It is a matter that we know each other and try to help each other rather than tear down.
LikeLike
August 28, 2012 at 5:53 am
B wrote: “I think this [suggesting reviewers the author knows or has met] is less an attempt to game the system, and more to avoid being struck by random acts of reviewer assholeness.” This certainly motivates my listing colleagues that I know, though I also list a few that I don’t but have the right expertise (rare, since my field is quite small and we all know each other).
Others have pointed out that sometimes those reviewers are actually the worst ones. Knowing that there’s a chance that the negative reviewer is actually someone I know makes them easier to bear, for me at least. I can usually tell that they’re trying to be constructive.
LikeLike
August 28, 2012 at 7:29 am
Really, Odyssey? How about the trust that the names will actually be considered and not actively excluded?
LikeLike
August 28, 2012 at 7:50 am
Neuro-C,
The suggested reviewers are… suggestions. I would suggest you go look that up in a dictionary, but since it’s only a suggestion I certainly wouldn’t suggest that my trust has been violated if you don’t.
LikeLike
August 28, 2012 at 7:57 am
“How about the trust that the names will actually be considered and not actively excluded?”
IMO, Odyseey is defending an appearing deceiving practice and might show doubtful respect for the scientific process.
If an author (s) are discovered using deceiving methods aimed at a non-independent but favorable review, the AE should directly address the problem with the Corresponding Author and make sure s/he understands there is/ will be specific sanctions for such a behavior. Period.
LikeLike
August 28, 2012 at 8:05 am
The suggested reviewers are… suggestions.
We understand that. Nevertheless when a journal asks for (and even insists upon) suggestions for reviewers and suggestions for reviewer exclusions it sets up an expectation. And that expectation is not that the suggested reviewers will be selected but certainly that they will not be universally excluded as if the two lists are the same. Or worse, I’m sure there’s some rogue AE out there who makes sure to solicit a reviewer from the exclusion list since, you know, that’s just a suggestion.
There is a simple practical issue in addition to the above. If I got wind that any of the AEs that might be handling my papers were doing this, it would be a simple matter to only suggest my collaborators and departmental colleagues or whatever. or totally and obviously inappropriate folks who are going to refuse to take the manuscript even if asked (like PhysioProf, hmm, think I’ll use his name. And Odyssey’s!).
Anyone else with half a brain would do the same. Thus, one rogue AE can take down the entire “help” system that the journal established.
LikeLike
August 28, 2012 at 8:22 am
Or worse, I’m sure there’s some rogue AE out there who makes sure to solicit a reviewer from the exclusion list since, you know, that’s just a suggestion.
There is at least one that I know of, and it is totally fucking unacceptable and unethical.
LikeLike
August 28, 2012 at 9:01 am
There is at least one that I know of, and it is totally fucking unacceptable and unethical.
I don’t know that I’m as bothered by this as by Odyssey’s roguery. Certainly if it is occasional, and because the AE decides individually to ignore the specific request for good reasons, I have no problem. Making it a universal policy bothers me a little bit less than the universal policy to never take the suggested reviewers.
This may be due to the fact that I very rarely ask for reviewer exclusions and do not feel as though I operate in a subfield in which evil competitive shenanigans affect my ability to get a manuscript published.
LikeLike
August 28, 2012 at 9:09 am
Thus, one rogue AE can take down the entire “help” system that the journal established.
I believe I’ve said at least twice that I would like to see the “help” system disappear.
LikeLike
August 28, 2012 at 9:56 am
So you have two pools of submitting authors. (A) Those who are trying to game the system by picking their buddies, maybe trading favorable reviews (explicitly or implicitly, whatever), and (B) those who just want their papers reviewed by competent, fair people with relevant expertise.
In a small field, an author in pool (B) torpedos their own reviewer pool if that author is unlucky enough that their paper is sent to Odyssey. In other words, in his attempt to defuse the malfeasance of pool (A) Odyssey is likely harming the good actors in (B).
Outstanding.
LikeLike
August 28, 2012 at 10:43 am
In a small field, an author in pool (B) torpedos their own reviewer pool if that author is unlucky enough that their paper is sent to Odyssey. In other words, in his attempt to defuse the malfeasance of pool (A) Odyssey is likely harming the good actors in (B).
That apparently is the consensus.
LikeLike
August 28, 2012 at 11:01 am
I think we’ve also determined that (A) doesn’t work anyway, because suggested reviewers do not give “easier” reviews in anyone’s experience.
A corollary to this is that I was told by the editor of a large journal that every time an author has accused a reviewer of bias and names who they think it is, they are wrong.
You can not “game” a system with a SNR < 1.
@DM… I also don't like to exclude reviewers as an author and am in a generally non-evil subfield, but as an AE I know that I don't know why someone is being excluded (authors are asked to state a reason, but it is usually something vague like "conflict of interest" or "competitor"). Unless the list is long or weird, I assume authors have their reasons, and I am highly averse to sharing someone's data with someone they have specifically asked me not to share it with. The potential for serious career damage is there, especially in those fields that are characterized by a high asshole frequency.
LikeLike
August 28, 2012 at 11:04 am
Well said, Miko.
LikeLike
August 28, 2012 at 5:18 pm
Odyssey: I believe I’ve said at least twice that I would like to see the “help” system disappear.
So what’s the problem? Just don’t look at either list (suggested/excluded), since you’re finding all the reviewers on your own anyway.
LikeLike
August 29, 2012 at 9:06 am
I believe I’ve said at least twice that I would like to see the “help” system disappear.
Why?
LikeLike
August 30, 2012 at 5:51 pm
I always assumed that editors largely respected the Do Not Use list itself, but that they also automatically selected a reviewer who was a past trainee of the DNU list. Because, duh, way to point a giant flashing arrow to the people who might have some interesting criticisms of a manuscript.
So, I shy away from putting names on the DNU list unless it is of extreme importance.
LikeLike
August 31, 2012 at 6:59 am
In at least one case I can think of, there’s a BSD PI who I routinely put on the exclude list. But I have *no* problem with his trainees getting our manuscripts, DJMH. Of course, most of his ex-trainees put Prof. BSD on the exclude list, too…
LikeLike
September 1, 2012 at 4:56 pm
I frequently use DNU lists as an author. I my field, shameless plagiarism by some bigshots is frequent and they tend to slow down other’s papers as much as they can.
As an AE I honor suggestions made by authors but I read the papers and ignore obviously non-critical reviews who neglect obviousflaws. But after all, you need to have a buddy system in place. There are too many neurotic reviewers out there who critize for no other reason but there ego. If a paper is good, it’s good and reviewers should say so and shut up.
And btw: I usually review papers within 24-48 hours. No matter ifI knowthe authors or not. I simply do so because I hate it when my papers sit on someone’s desk for six weeks.
LikeLike