Why cascading manuscript acceptance schemes can't work

For reference, Scicurious’ proposal

What if manuscript submission could be as good as a one-shot?

Like this: you submit a paper to a large umbrella of journals of several “tiers”. It goes out for review. The reviewers make their criticisms. Then they say “this paper is fine, but it’s not impactful enough for journal X unless major experiments A, B, and C are done. However, it could fit into journal Y with only experiment A, or into journal Z with only minor revisions”. Or they have the option to reject it outright for all the journals in question. Where there is discrepancy (as usual) the editor makes the call.

and the Neuroscience Peer Review Consortium.

The Neuroscience Peer Review Consortium is an alliance of neuroscience journals that have agreed to accept manuscript reviews from other members of the Consortium. Its goals are to support efficient and thorough peer review of original research in neuroscience, speed the publication of research reports, and reduce the burden on peer reviewers.

I think these schemes are flawed for a simple reason. As I noted in a comment at Sci’s digs….

Nobody bucking for IF immediately goes (significantly) down. They go (approximately) lateral and hope to get lucky. The NPRC is a classic example. At several critical levels there is no lateral option. And even if there was, the approximately equal IF journals are in side-eyeing competition…me, I sure as hell don’t want the editors of Biol Psychiatry, J. Neuro and Neuropsychopharmacology knowing that I’ve been rejected by one or two of the other ones first.

I also contest the degree to which a significantly “lower” journal thinks that it is, indeed, lower and a justifiable recipient of the leavings. Psychopharmacology, for example, is a rightful next stop after Neuropsychopharmacology but somehow I don’t think ol’ Klaus is going to take your manuscript any easier just because the NPP decision was “okay, but just not cool enough”. Think NPP and Biol Psych are going to roll over for your Nature Neuroscience reject? hell no. Not until their reviewers say “go”.

This NPRC thing has been around since about 2007. I find myself intensely curious about how it has been going. I’d like to see some data in terms of how many authors choose to use it (out of the total manuscripts rejected from each participating journal), how many paper are subsequently accepted at another consortium journal, the network paths between journals for those that are referred, etc.

My predictions are that referrals are very rare, that they are inevitably downward in journal IF and that they don’t help very much. With respect to this latter, I mean that I bet it is a further minority of the manuscripts that use this system that are subsequently accepted by the second journal editor on the strength of the original reviews and some stab by the authors at a minimal revision (i.e., as if they’d gotten minor-revisions from the original editor instead of rejection).

One fascinating, unknowable curiosity is the desk reject factor. The NPRC could possibly know how many of the second editors did desk rejects of the referred manuscripts based on the forwarded reviews. That would be interesting to see. But what they can’t know is how many of those would have been sent out for review if the reviews had not been forwarded. And if they had been sent out for review, what fraction would have received good enough reviews (for the presumptively more pedestrian journal) that they would have made it in.

analogies

my blather on the realities of being an NIH extramurally-funded investigator is to “game theory” what screaming and waving in panic is to dogpaddling over to the side and climbing out the pool ladder.

Idle thought

Relevant to Sci’s recent ranting about the paper chase in science…

Sorry reviewers, I am not burning a year and $250K to satisfy your curiosity about something stupid for a journal of this IF.