When are data "publishable" and "not-publishable"?
July 16, 2008
A functional Magnetic Resonance Imaging luminary has accused two previous scientific trainees of dubious academic shenanigans if not outright scientific misconduct. Nikos Logothetis discussed his complaints in a news focus published in Nature. Specifically, he:
charged that two of his former research students took data from his laboratory without his permission and published scientifically incorrect interpretations of them against his advice.
and
claims that the journal involved, Human Brain Mapping, acted incorrectly by publishing the paper after he told them the data were inappropriate. He says the journal has denied him the right to a timely reply.
The trainees in question are Amir Shmuel and David A. Leopold who published a paper in a special issue of Human Brain Mapping.
Shmuel A, Leopold DA. Neuronal correlates of spontaneous fluctuations in fMRI signals in monkey visual cortex: Implications for functional connectivity at rest. Hum Brain Mapp. 2008 Jul;29(7):751-61
Okay, so the first issue here has to do with deciding when data are “publishable” and when they are not. First and foremost, we wish to know that the data themselves are “good” so to speak. That nobody is accusing anyone of fudging the numbers or inventing data in some underhanded way. This does not appear to be the case here although I’ve so far only seen this brief outline of the charges and complaint. The second question is whether the data really show what the authors think they do. And this is where I tend to rely on the peer review process as well as a basic assumption of probity for the authors. Not to mention the post-review process. Don’t believe the paper shows what it purports? Sack up, generate your own data (or analysis) and show why.
The second issue is where it gets a little trickier for my perspective. Who ultimately ‘owns’ the data and gets to make the publish/don’t publish call? Who gets to decide which interpretation will be used in the paper? “DM, don’t you always say it is the PI’s decision?”. Um, yeah, I do when it comes to settling authorship disputes within the lab. This sort of thing? Well, I’ve been in a case or two of similar nature in which I thought a certain set of data constituted a paper, one or more senior collaborators disagreed and overruled my “let’s let the peer-review process decide” arguments. It boils down to the smell test, I suppose. If there is a legitimate thought that the data do not really support the interpretation…I’m sympathetic. If the thought is that the data are just not good enough for a certain level of publication or are too far down the Least Publishable Unit slope….not so much. And if there is any whiff that some of the collaborating authors are in opposition merely because they do not like the interpretation or it questions some positions they’ve staked out….game on. I have been in at least one situation in which I felt quite strongly there was opposition to publishing because the data tended to cast doubt on a position staked out in a different paper generated by the collaborating lab. Actually ‘doubt’ is way too strong, it was a fairly minor issue in the grand scheme of things, but it did tend to imply some cherry picking was going on in the other paper. So the devil is most certainly in the details as far as this Logothetis/Shmuel/Leopold dustup goes in my view. I don’t know enough about the field to know if Logothetis is generally in opposition to the “theory about spontaneous brain activity” that drew his ire, if there are more legitimate debates about the appropriate imaging protocol to be used or what. Perhaps one of our phunctionalMRI phrenologist phriends will chime in…..
Okay, that brings me to a final concern. Shmuel and Leopold are now running their own programs. In fMRI / neuroscience. Of course. They are now junior scientists in direct competition with said luminary, Logothetis.
I’m sure I don’t have to belabor that particular point although, again, without knowing the players this scenario is of uncertain probability.
[h/t: PhysioProf]
—
Update: additional comment here.
DrugMonkey 
July 16, 2008 at 4:51 pm
After consultation with Logothetis, MPS vice-president Herbert Jackle wrote to the authors giving approval for the use of the data, but adding that Logothetis’s scientific concerns should be taken into account, in accordance with the MPS’s code of good scientific practice.
Oddly, I think I still have some of Herbert Jackle’s ancient data buried in my files…
At any rate, I would interpret that sentence exactly as the authors seem to have: you can publish but please pay attention to what Logothetis is telling you. Max Planck’s interpretation of “You don’t need to erase these data from your memory but you can’t publish without Logothetis’ explicit approval” never would have occurred to me. That’s without seeing the original text, though, obviously.
LikeLike
July 16, 2008 at 4:54 pm
“Okay, that brings me to a final concern. Shmuel and Leopold are now running their own programs. In fMRI / neuroscience. Of course. They are now junior scientists in direct competition with said luminary, Logothetis.
I’m sure I don’t have to belabor that particular point although, again, without knowing the players this scenario is of uncertain probability. ”
Yeah, knowing the players in this scenario, you should be belaboring that particular point. Fortunately the junior folks have some powerful people on their side and are no longer in Germany, where Logothetis is.
The scientific issue raised by Logothetis is important — he argues that the baseline activity they’re measuring here was not really a baseline, because it could have been influenced by the refresh rate of the monitor, even when no images were displayed on the screen. But it appears to me that the eyes closed/eyes open control in the manuscript address (though don’t completely eliminate) that concern. At that point, I really do think that this is an issue for “post-review.”
LikeLike
July 16, 2008 at 5:03 pm
It seems to me to be a simple case where Nikos Logothetis first agreed to the publication of the data by Shmuel and Leopold only to withdraw his permission once he had read their interpretation. There will be those who will claim that Shmuel and Leopold went over Nikos Logothetis when they submitted the paper for publication. However, Logothetis first gave his permission, although he probably was not a co-author, because the data most probably were produced by Shmuel and Leopold. One can assume that the two sent a copy of the manuscript for Logothetis’s review and comments and when he read their interpretation of the data, he became furious and withdrew his permission.
I find this behavior of Logothetis to be childish and vindictive. I don’t think I would behave differently from Shmuel and Leopold if my ex-boss had first permited me to publish my data only to withdraw this permission when he found out that my interpretation does not agree with his.
LikeLike
July 16, 2008 at 6:03 pm
“However, Logothetis first gave his permission, although he probably was not a co-author, because the data most probably were produced by Shmuel and Leopold.”
highly unlikely, given the players involved. Seems unlikelly that Logothetis would not be a co-author, because the “data most probably was produced by Shmuel & Leopold”, since that would be true for most of the data generated in a lab like his (i.e. generated by the research fellows).
LikeLike
July 16, 2008 at 6:54 pm
Yeah, knowing the players in this scenario, you should be belaboring that particular point. Fortunately the junior folks have some powerful people on their side and are no longer in Germany, where Logothetis is.
It’s just that I’m frequently on about the whole senior vs junior investigator thing and I didn’t have any specific knowledge. It does have a certain whiff about it, doesn’t it?
That comment from the Max Planck admin guy about how they gave permission to “use” the data but not publish it? What in the heck did that mean, one wonders? And that “pretend we didn’t actually fund the research” bit? very strange….. I thought the acknowledgment was pretty short and sweet. Although I guess maybe adding a point that Logothetis disagreed with the conclusions drawn might have been a reasonable thing to include in the acknowledgment statement. perhaps.
(oh, and don’t mind Sol. He’s got a slightly old fashioned view about authorship that doesn’t include the lab head getting on every paper….)
LikeLike
July 16, 2008 at 7:28 pm
just a note. It is well known that people typically have ‘problems’ in that particular lab.
LikeLike
July 16, 2008 at 7:35 pm
The affair is silly. Logthetis should make sure his name is not on the paper, and the if he feels the work is bogus, he should write a letter to the editor to say so.
LikeLike
July 16, 2008 at 7:37 pm
DM,
Your last dig, is that tongue in cheek or is it me not making myself clear enough where I am standing on Logothetis’s inclusion on that paper?
LikeLike
July 16, 2008 at 7:39 pm
A good discussion of the Hellinga case is here, including a ton of commentary from those supposedly in the know.
Aetogate here and here from the accusers.
I think this phunctional imaging one is going to start smelling like Aetogate……
LikeLike
July 16, 2008 at 7:48 pm
S.Rivlin @#8, It is just when you say
although he probably was not a co-author, because the data most probably were produced by Shmuel and Leopold.
I find this consistent with my recollection (possibly inaccurate) that you are not in favor of the modern convention for the lab head/PI to be on the papers, even if s/he had little to do with the specific experiments, analysis and writing other than to be, well, the lab head.
You have a tendency to suggest that people that follow this convention are personally culpable for an ethically dubious act. Fair enough? I tend to believe that when a convention is as common as this one, it is difficult to blame any one individual.
[And yes, I recognize that it is a slippery slope in which “heads” of organizational units of increasing size may demand to be on all papers under their administration. Hopefully we do not have to descend into organizational minutia to resolve that I am really referring to lab heads and not department heads being the senior author by default]
LikeLike
July 16, 2008 at 7:54 pm
whimple @#7: I quoted the section from the news article which indicates Logothetis is complaining in part that he was unable to submit such a letter to the editor. Unclear, of course, if this means at all or just that it wouldn’t appear in the special issue or if he was demanding a slot for a full competing article.
One interpretation of deep-anon @#2 saying “powerful people on their side” might be that there is some additional axe-grinding at work here…plenty of black hats to go around.
LikeLike
July 16, 2008 at 8:36 pm
DM,
Thanks for the explanation. If I hold any convention, it is very simple – if you have contributed scientifically to the work to be published, you should be one of the co-authors; if your contribution is monitary only or simply the high ranking of your position on the ladder at your lab-institution, then your name should not be on the published work. I am still angry about an incidence during my post-doctoral research, when the name of the head of the institution where the research was performed was added to a paper of mine by the head of the lab without consulting with me or even mentioning to me of such an intention.
LikeLike
July 16, 2008 at 9:05 pm
“One interpretation of deep-anon @#2 saying “powerful people on their side” might be that there is some additional axe-grinding at work here…plenty of black hats to go around.”
No, I think the junior folks are being defended by “white hats” to prevent their careers from being crippled by a lab that is widely rumored to be — what word should I use — dysfunctional?, to be perfectly clear. I brought them up because I think that’s why Schmuel & Leopold are not being shredded to dust by the power plays. There really are good guys (and gals) among the “seniors” out there, too, and sometimes they fight behind the scenes to protect the little guys.
LikeLike
July 16, 2008 at 9:10 pm
That comment from the Max Planck admin guy about how they gave permission to “use” the data but not publish it? What in the heck did that mean, one wonders?
I’m sure they were sincere in thinking that it adequately conveyed whatever they intended it to mean. But given the obvious ambiguity (again, relying on the story without having seen the letter itself) it seems like the level of indignation and retribution from Logothetis is excessive.
The Nature story also might have stated that mitigating factor a lot earlier. Most of the commenters there don’t seem to have read down that far and it’s a pretty crucial bit of the dispute.
LikeLike
July 17, 2008 at 11:11 am
Now why didn’t I think of that!? That truly is totally simple, as the determination of whether someone has “contributed scientifically” to a manuscript is easy fucking peasy!
LikeLike
July 17, 2008 at 6:20 pm
Easy peasy? PP are your phrases sliming into my subconcious or are you stalking me?
LikeLike
July 17, 2008 at 8:26 pm
PP,
As easy as it is for you to stick the F word in every sentence you utter. š
LikeLike
July 19, 2008 at 6:03 pm
According to Logothetis the first time heard about the paper was after it had been written, gone through editorial and peer review and was due to be published online in a few days. He was then offered co-authorship of a paper he had no input in creating and which he disagreed with. Peer review isn’t magic. Lots of bad stuff gets by peer review. That is a problem, a big problem, especially in a new field like fMRI where there is lots of room for bad science. The way the authors went about creating this paper is the wrong way to do science. If they had discussed this with Logothetis first, it would be different.
I think the editorial and peer review process failed in this case. The journal should give Logothetis approximately equivalent space to publish a rebuttal explaining why he thought the data was wrong to be used in this way. They should have held up publication of this article so that Logothetis’ rebuttal could be published at the same time. That the editors didn’t do this is (I think) about the egos of the editors and not about the science.
I don’t consider myself an expert, but I have read a lot about fMRI, and I take serious issue with the first sentences in their abstract.
” Recent studies have demonstrated large amplitude spontaneous fluctuations in functional-MRI (fMRI) signals in humans in the resting state. Importantly, these spontaneous fluctuations in blood-oxygenation-level-dependent (BOLD) signal are often synchronized over distant parts of the brain, a phenomenon termed functional-connectivity. Functional-connectivity is widely assumed to reflect interregional coherence of fluctuations in activity of the underlying neuronal networks.”
They are flat-out wrong. “Functional connectivity” relates to neuronal connectivity, not correlations of hemodynamics. That mistake shows a fundamental lack of appreciation of what fMRI is and what fMRI can tell us and actually is telling us about neuronal activity. I haven’t read the rest of the paper, but it doesn’t seem very promissing because their fundamental premise is wrong. That peer review and the editors let that gross mistake slip through shows a pretty serious flaw in the process. Either it is sloppy science, sloppy editing, or some of both.
fMRI only measures hemodynamics. Neuronal activity is (to some extent) independent of what the vasculature is doing and how blood is or is not flowing through it. It is fundamentally wrong to refer to correlations in fMRI as “functional connectivity” without being explicitly clear that the actual functional connectivity is between neurons and not through hemodynamics.
LikeLike
July 20, 2008 at 10:45 pm
The term “functional connectivity” has been used to refer to common hemodynamic patterns of fMRI signals for a long time. This seems to have a fairly good definition
http://www.scholarpedia.org/article/Brain_connectivity
Per that article, the first person to use “functional connectivity” in a similar manner was Friston 1994 Human Brain Mapping volume 2 pages 56-78.
Probably the first article to use the term for fMRI at rest is:
“Functional connectivity in the motor cortex of resting human brain using echo-planar MRI.”B Biswal, et. al. Magn Reson Med, Vol. 34, No. 4. (October 1995), pp. 537-541.
The fact that the current paper clearly defines the term instead of assuming a common definition and that the goal of the paper is to compare these hemodynamic changes to neural changes should address some of daedalus2u’s critiques.
Also, the Nature article didn’t note that the article was accepted into a special issue of the journal on “Endogenous Brain Oscillations and Networks in Functional MR”
http://www3.interscience.wiley.com/journal/119814868/issue
Delaying publication would have meant removing a paper from the special issue even though it passed peer review.
LikeLike
July 21, 2008 at 2:45 am
bsci, I have enjoyed your comments over at Protein Wisdom. The right-wing wackaloon assholes who hang out there wouldn’t know a logical argument if kicked them in the shin.
LikeLike
July 21, 2008 at 9:19 am
Thanks. I think I had a few too many random bits of free time this weekend. It’s fun to take their argument, accept as truth the core tenants of their explanation, and still show that their conclusion are wrong.
In this case I started with the assumptions that there may be some innate differences and everything should be run by free market interests. The conclusion is that you still want to remove obstacles that discourage your top candidates from taking or applying for a job. I think there are some people even on that site who agree with the general gist of my argument, but the last few are quite pitiful.
LikeLike
July 21, 2008 at 9:57 am
I think there are some people even on that site who agree with the general gist of my argument, but the last few are quite pitiful.
Why do I suspect that even those who agree will revert to type the millisecond you stop applying gentle rhetorical corrective measures?
LikeLike
July 21, 2008 at 10:16 am
Call me an idealist, but I at least hope that rational arguments affect the long-term opinions of at least a fraction of the commentors and uncommenting readers. It’s sometimes fun to argue just for the sake of arguing, but, considering you’re the one who devotes hours of his life to maintaining an issues-oriented blog, you either believe your writing can change people’s minds or your utterly and completely insane. š
LikeLike
July 21, 2008 at 5:32 pm
you either believe your writing can change people’s minds or your utterly and completely insane. š
uuuummmm. no comment.
LikeLike