What does a retracted paper mean?
June 3, 2011
I’ve been having a little Twitt discussion with Retraction Watch honcho @ivanoransky over a recent post in which they discuss whether a failure to replicate a result justifies a retraction.
Now, Ivan Oransky seemed to take great umbrage to my suggestion in a comment that there was dereliction in their duty to science to intentionally conflate a failure to replicate with intentional fraud. Per usual, we boiled it down to a fundamental disagreement over connotation. What it means to the average person to see that a paper is retracted.
I rely upon my usual solution, DearReader. Select all choices that apply when you see a retraction or that you think should induce a retraction.
A retracted paper meansonline survey
Direct link to the poll in case you can’t see it.
My position can be found after the jump….
I think we need to be exceptionally clear in the business of science that a failure to replicate is not, in fact, evidence of fraud.
I don’t give a fig what any journals might wish to enact as a policy to overcompensate for their failures of the past.
In my view, a correction suffices (and yes, we need to concentrate on making sure that any search engine that lands upon official mention, especially PubMed and the official journal site, makes it clear that the paper was in fact corrected) in most cases where there is not fraud.
Retraction, to me, implies that there is reasonable evidence of some sort of shenanigans.
__
Related reading: Being later found to be right does not reduce your blame for faking it in the first place.
DrugMonkey 
June 3, 2011 at 12:37 pm
I’d interested on your take on this:
Should a paper be retracted when there are shenanigans that are not related to the data?
http://neurodojo.blogspot.com/2009/10/retracting-paper-when-science-is-sound.html
I don’t know. There seem to be no guidelines or consensus on what warrants retraction. I think the practices regarding retraction are changing, too, but haven’t done the analysis to demonstrate that yet.
LikeLike
June 3, 2011 at 12:43 pm
Shenanigans are strongly implied, but I wish they weren’t, since I see a great deal of use for full retractions in certain non fraud cases.
For example, if you write a whole paper stating Agent X signals through Pathway Y, and it turns out that all along your batch of Agent X was contaminated with Agent A that everyone knows signals through Pathway Y, the entire paper will be flawed.
Actually, I’m told that early on, before we had good endtotoxin tests, this kind of thing actually did happen in the TLR field. Perfectly good scientists, using reasonable methods, produced entire papers of junk data. Retraction would make more sense than ‘correction’
LikeLike
June 3, 2011 at 12:47 pm
Thanks @DrugMonkey for continuing the conversation here. That, after all, was the point of our post, to ask when a retraction is warranted. We’ve left a comment in response to yours on Retraction Watch, which includes several recent cases of retraction due to failure to replicate, as well as COPE’s guidance on when journals should retract: http://retractionwatch.wordpress.com/2011/06/03/so-when-is-a-retraction-warranted-the-long-and-winding-road-to-publishing-a-failure-to-replicate/#comment-3153
We would add here is that we completely agree that “that a failure to replicate is not, in fact, evidence of fraud.” We would go further and say that a retraction is not evidence of fraud. (That’s why we were careful in our post not to mention fraud or misconduct at all.) A retraction is evidence that, as @leonidkrugylak put it, “data in paper not to be trusted.”
Our answer to this is to make sure that retraction notices say quite clearly why a paper was withdrawn. Many don’t, and I think we’d agree that contributes to whatever perception there is about retractions suggesting fraud.
LikeLike
June 3, 2011 at 1:28 pm
As written the first choice would mean essentially every paper ever written should either be retracted or be retracted soon.
LikeLike
June 3, 2011 at 2:03 pm
I agree with Ivan that fraud or suspicion of fraud is not a necessary condition for triggering a retraction. Negligent errors also warrant retraction. I also know of cases where papers were retracted due to plagiarism, in which case the data and analysis might actually be right (because it was performed by the people whose work was plagiarized), and since plagiarism is a form of fraud I agree with that approach. If the error was nontrivial, failure to replicate is better handled by a correction than retraction, and of course much work is superceded by later work.
I can understand why not everybody would agree with this point of view, and I am aware of retractions due to failure to replicate. In any case, the reason for retraction should be clearly stated in order to avoid casting suspicion on people only guilty of honest error. The retraction letter should appear in the place of the retracted article, so that somebody who follows a citation to that article will see it.
LikeLike
June 3, 2011 at 2:15 pm
As long as the data are not fraudulent, we can learn from others’ mistakes. A correction is better than a retraction in that case. Even if the data were produced by a contaminated reagent, it is good to have that paper sitting there to remind us to check our reagents for contamination before publishing. Retraction should be used when the the only thing we can learn from a paper is that the author(s) were dishonest.
LikeLike
June 3, 2011 at 9:29 pm
DM, I enjoy your blog immensely, but I think you’re very wrong on this. Personally, I really didn’t get the sense that the Retraction Watch post “leaves a very inaccurate impression that it is the failure to reproduce prior results that is the major factor,” as you suggest. It seemed pretty clear to me that the the implied problem was not the failure to replicate by itself, but the fact that there appears to be a systematic and serious methodological problem that prevents replication even in principle and which wasn’t described in the original paper.
As to the question of whether the problem warrants a retraction or a correction, I think a reasonable case can be made either way, but personally I lean towards retraction. It’s patently false to suggest that only fraud justifies a retraction; as Oransky points out, retractions due to serious methodological problems are not uncommon in the literature, and with good reason. If a study is conducted with the best of intentions but the authors (or a third party) later discover that there’s an egregious error which renders the results completely worthless (and quite possibly misleading), correction serves absolutely no purpose, because it leaves the paper in the public record. Nobody actually bothers to read corrections precisely because people generally assume that they address minor issues. Retraction in such cases is the right course of action.
Now there’s obviously a gray area, here inasmuch as there are plenty of cases where there are methodological problems with a paper (I mean, pretty much every paper has problems) that could in theory invalidate the results, but that don’t warrant retraction or even correction. But that doesn’t invalidate the basic point that some errors are so bad that they clearly warrant retraction. To take a trivial example, DM, I’d certainly hope that if you published a set of findings which later turned out to stem from a completely miscoded dataset, implying literally no empirical support for your conclusions, you would issue a retraction. What’s the alternative? Issuing a correction saying, “hi, we’d like to correct our findings–they’re all bogus because my RA doesn’t know how to use Excel; now carry on as you were”?
I also think your poll is kind of loaded, because it doesn’t include an option reflecting the above interpretation. Saying “something was wrong with the data or analysis” is not the same as saying “something was definitely and utterly wrong to such an extent that it precludes any possible empirical support for the stated conclusions”. The latter justifies a retraction, the former does not.
LikeLike
June 4, 2011 at 12:50 pm
Failure to replicate results is not in and of itself a strong enough reason to retract a paper (or to ask someone to retract a paper). I think anyone who’s been in the lab for even a year can understand the difficulty of repeating a protocol from someone else, step by step, in precisely the same way and getting exactly the same answer-especially with biological systems.
That being said, I do think that an inherent problem with the data does warrant retraction-such as you’re not working with the reagents you thought you were or your readout of X was really a readout of D that is independent of X. In such cases, the retraction should detail the reasons explicitly (e.g. this one in PNAS. The orignal paper itself should not (and, I think, usually does not) simply disappear from the journal. It should remain, with clear designation that it has been retracted and attachment of the detailed notice. This approach allows us to learn from other’s mistakes but makes it clear that the orginal results are likely meaningless.
Perhaps there should be distinct designations of retractions-one for technical issues/honest mistakes and one for data shenanigans.
LikeLike
June 4, 2011 at 2:51 pm
I decided I had more to say.
LikeLike
June 4, 2011 at 7:44 pm
I agree with Ivan Oransky, Eric Lund, and tal that a retraction does not necessarily mean a fraud. While it is certainly best to carry out careful studies to avoid publishing unreliable results in the first place, if you find out that the paper you published is unreliable (whether because the reagent you used didn’t have the specificity you had expected or because the computer program you used had a serious bug), the honest thing to do is to retract the paper. It is embarrassing and it means one less paper for your CV. But it can happen without a fraud.
A correction would be acceptable if the error was minor. But I have seen at least one example of a “correction” that changed the major discovery/story of the paper that essentially made that paper meaningless. I felt that the authors were dishonest and they should have retracted the paper instead.
Also, even when a paper is retracted, it doesn’t mean that the information on that paper vanish from the record. You can find and read that paper. You can see the statement of why the paper was retracted. You can certainly learn from it if you want.
LikeLike
June 4, 2011 at 8:02 pm
BTW in the example of “correction” that I mentioned above, by “dishonest” I didn’t mean that the authors committed a fraud. What I thought was that it was dishonest to make a “correction” instead of total “retraction” when their paper about a very surprising result that they published on a prestigious journal became essentially worthless now that they know the very surprising result was wrong.
LikeLike
June 5, 2011 at 10:50 am
Right, but will you?
Also, ping.
LikeLike
June 5, 2011 at 12:14 pm
Hi,
Intrigued by this news report about this study, can you shed any light on which company did this research or where we can find results please?
http://www.kulr8.com/news/local/Mother-Worries-Son-Will-Struggle-to-Find-Marijuana-121603614.html
Regards,
Jacob
LikeLike
June 5, 2011 at 2:58 pm
For better or worse my initial gut response whenever I read “paper retracted” is to assume some sort of foul play. As has been mentioned, this may be due to the fact that the reasons for retraction are not always mentioned.
LikeLike
June 6, 2011 at 9:34 am
Look. I don’t care what you call it. Your distinction between retraction/correction seems OK I guess. But it needs to be absolutely clear to any new-and-learning student of the field that paper X’s results/conclusions/methods/whatevers are not correct as presented. I think if it’s found that paper X’s results are wrong for any reason (don’t give a damn what reason) this should be the first text that anyone sees when they click on the damn paper in pubmed, and this text must make absolutely clear which parts of the paper can be trusted and which cannot. Before the abstract, even.
Addendum: My adviser always said, if you find your mistake yourself, and you tell the journal, it’s a correction and not career-ending (or even terribly damaging unless there’s a pattern of sloppyness). If someone else finds it, you’re in trouble and it’ll probably end up being a retraction.
LikeLike
June 7, 2011 at 7:01 am
For better or worse my initial gut response whenever I read “paper retracted” is to assume some sort of foul play. As has been mentioned, this may be due to the fact that the reasons for retraction are not always mentioned.
LikeLike