Circumstantial evidence

A recent post soliciting Open Laboratory 2007 nomination from Noah Grey of Action Potential Blog reminded me of a little commentary exchange that we were having over a post on “paranoia in research“. Inexplicably I let him get in the last word. Fortunately an opportunity presents itself to continue the discussion.

In very brief outline, the topic is the observation that paranoid secrecy over what one’s current scientific findings, experiments and hypotheses are is a bad thing for science. And yet it seems to be an increasingly common fact of life. My suggestion was that certain editorial practices of the most well regarded scientific publications contribute to scientific paranoia in explicit ways. Some of my prior points included:

The second category of silliness for which Nature policy is directly responsible is accepting a submission because “a paper on a similar hot topic is accepted at (competing journal X)and gee don’t you want one too”. or worse, accepting a me-too for co-publication that comes in, astonishingly, from the group you sent the other paper to for review.

The point is that you don’t have to determine who stole what in a specific case to recognize that systematic journal policies encourage scooping and project stealing.

In the series of exchanges Noah took a couple of positions which boiled down to 1) “Give explicit examples”, 2) “Well, we at Nature Neuroscience aren’t responsible for what may go on with other Nature titles” and 3) “I’m so offended that you would even suggest this! Stop making these accusations against us!”. Oh, and some semantic wibble that doesn’t fool anyone either. The bottom line is that taking offense and using vaguely threatening language in an attempt to avoid answering a critique is not a valid argument. Placing the burden of proof on someone to assemble overwhelming evidence of editorial shenanigans is a nice little trick but doesn’t in and of itself prove a damn thing. Particularly when one has one’s own set of experiences that may or may not be fodder for open blogging, for various reasons. Not to mention that it would have been simple for Noah to come right out and answer this query of mine:

can you tell me you never ever get PIs telling you that something related to their MS is accepted / almost accepted elsewhere? can you tell me that this never works to influence how a submission would be treated (from review to decision to print) in such a case?

The bottom line here is that only those on the inside of situations involving less than stellar behavior can really know what is going on. From the outside, all we usually have are partial pieces of the puzzle, i.e., from authors involved, reviewers involved, both, or from editors involved. After all, these people know things are a little dodgy and they sure as heck aren’t going to admit openly to their suspicious behavior. “Heck yeah, I got this paper to review and than had to sit on it until we could get our submission out”. Or, “Well, the reviews were pretty damning on this one but our competitors have related papers in acceptance so we’d just better take this one now”. But in many cases the casual reader can assemble some pretty damning circumstantial evidence.

Within the last month or so we had an all-too-familiar situation develop involving at least three of the very top journals. Let’s call them J1, J2 and J3. Also involved are a grand total of five articles (A1-A5) which all touch on a single scientific issue. The “received” dates for A1 (in J1) and A2 (in J2) are within a span of two weeks. Hmm, one of those interesting coincidences, no doubt. Those papers were then “accepted” for publication about 11-12 weeks later. So far, so good.

Where it gets interesting is looking at the submission dates for A3-A5, all in J3. The first “received” date was about 5 weeks after A1 and A2 were initially submitted and the last one was about 8 weeks afterwards. Well before A1 and A2 were accepted, to underline the point.

It gets better. A3 and A4 were accepted by J3 a few days before (A3) and after (A4) the range of dates over which A1 (J1) and A2 (J2) were found acceptable for publication; the poor authors of A5 had to wait another month.

What an amazin’ series of coincidences. Five labs generate manuscripts good enough for the very highest pinnacles of academic publishing on a single topic and happen to be all ready to publish within a span of two months time. Wow what a coincidence. And it must be rare, right? Because if a given scientific topic is so obvious and pedestrian that 5 groups are working on it at the exact same time, uh, well, maybe it isn’t really a topic that reaches the standard of the glamor mags top scientific journals? [Insert Emperor/clothes snark here.]

Or. Just maybe. Is it possible that the authors of A1 or A2 got wind of the other being ready to submit and rushed one out? Or that they agreed in advance to co-submit (it happens and this is a whole ‘nother issue. Short version this is bad because it also contributes to the paranoid culture.)?

And do you think? Just perhaps? Maybe one of the authors on each of A3-A5 was asked to review the A1 or A2 manuscripts? Ya think? Naah. That would be a violation of ethics, wouldn’t it? Well, only if they gave it a different (i.e., bad) review than they otherwise would’ve given it, right? I mean, if they waited until the first round of reviews were in, saw A1 or A2 was going to be accepted and only then submitted their work they’re clean, right. Um, no. What, like the paper was just sitting there all ready to go? Or did the rather large lab groups suddenly find that the PI had an inexplicable urgency to wrap up one of many ongoing stories in particular? And to shoehorn a not-quite-naturally-fitting series of experiments together?

And then we get to the behavior of the editorial team of J3. Completely clueless were they? Just amazed to get three submissions on this scientific topic within about a month’s time, weren’t they? “Gee” they thought, “This must be a hot new thing! We gotta get on it, man.” Really. Not one of the communicating or senior authors of A3-A5 just happened to mention, oh in passing really no biggie, that J2 and/or J1 had related manuscripts poised to be accepted?

As I said, we cannot know with any certainty what is going on from the outside. It could be a tremendous coincidence each time this sort of thing happens. Or, it might not be. And that is why it would be ever so helpful for Editors to state unequivocally that they will:

• Not shade the editorial decision or timeline for that decision and/or publication one iota based on rumor or hard evidence that a topic-related manuscript is close to submission, under review or accepted at a competing (or the same) journal.
• Ask very tough questions of authors should it ever appear even possible, nevermind likely, that a manuscript may have been rushed into submission following confidential knowledge of a competing paper due to peer review assignment.
• Retract an offer of acceptance, or even a published article, should it develop that ethical peer review standards have been breached.

Alternately, they could just ‘fess up to their behavior and at least then we could have an honest discussion as to whether they indeed should be doing this.