The “whole point” of Supplementary Data
December 10, 2014
Our good blog friend DJMH offered up the following on a post by Odyssey:
Because the whole point of supplemental material is that the publisher doesn’t want to spend a dime supporting it
This is nonsense. This is not “the whole point”. This is peripheral to the real point.
In point of fact, the real reason GlamourMags demand endless amounts of supplementary data is to squeeze out the competition journals. They do this by denying those other journals the data that would otherwise be offered up as additional publications. Don’t believe it? Take a look through some issues of Science and Nature from the late 1960s through maybe the mid 1970s. The research publications were barely Brief Communications. A single figure, maybe two. And no associated “Supplemental Materials”, either. And then, if you are clever, you will find the real paper that was subsequently published in a totally different journal. A real journal. With all of the meat of the study that was promised by the teaser in the Glam Mag fleshed out.
Glamour wised up and figured out that with the “Supplementary Materials” scam they can lock up the data that used to be put in another journal. This has the effect of both damping citations of that specific material and collecting what citations there are to themselves. All without having to treble or quadruple the size of their print journal.
Nice little scam to increase their Journal Impact Factor distance from the competition.
DrugMonkey 
December 10, 2014 at 10:20 pm
You are right on this!
But pubs in Glamour Mags still come with an essential free pass to publish the same damned thing 2-3 times in regular journals…..at least if the finding holds up!
LikeLike
December 10, 2014 at 10:32 pm
Although I agree with the effect you are describing, I think the order went the other direction. I remember when one of my colleagues got a faculty job in the early 1990s with a couple of cryptic Science papers and everyone was asking “where’s the real paper?” The Science paper had become the real paper. The Supplementary data arrived later when reviewers started demanding that if the Glamour paper was the only one, then it had damn well better include the real data and the real controls. Because the web had become available, Glamour editors opened up the Supplementary Data floodgates.
DJMH however is right. The PROBLEM with supplementary data is that the Glamour publishers don’t want to spend a dime on it. Therefore its not edited and not real.
My favorite description of the problem with supplemental data is that if your paper is cited in the supplemental data but not in the real paper, then it doesn’t help your citation count / h-index / etc.
LikeLike
December 10, 2014 at 11:39 pm
I thought supplementary materials emerged as a way to accommodate extremely large datasets from genome-wide studies which were becoming more common at the time. Journals likely have had to spend a little more than a dime on it to maintain storage capacity.
LikeLike
December 10, 2014 at 11:58 pm
qaz- Oh yes, I keep meaning to tee off on that when I get the chance.
LikeLike
December 11, 2014 at 1:02 am
I love and hate supplemental material so much. Looking back at the old literature, there is so much where I see, “Details will be developed in a later paper” or something like that, and then no later paper. Of course, 90% of the time, people don’t give a fuck about it, the referees don’t actually read the SM, and the journal won’t make any attempt to look at it either. (Pet peeve: random word documents with terrible embedded equations, which no longer open properly – makes it clear the authors didn’t care either.)
We should just accept that 3-page papers are extended abstracts for a real paper. The problem with this is that PIs do not seem to be willing to do referee work for 20+ page papers. I don’t think this part is the fault of the glams.
Best practices for journals that actually care about this stuff? Definitely add the SM citations to the end of the paper (I think Biophysical Journal does this, maybe PNAS), and include the PDF+SI as easily as possible (PNAS gets this right at least).
LikeLike
December 11, 2014 at 10:44 am
Some journals have gone to the other extreme. J. Neurosci. refuses to host supplementary material. I would argue that this just hurts the journal, since in some cases enhanced figures (main figures with additional parts, for controls, say) results in better papers.
LikeLike
December 11, 2014 at 11:05 am
J Neuroscience walks with the angels on this. An awesomely principled move on their part.
LikeLike
December 11, 2014 at 11:05 am
It’s not just glamour journals though. In my latest paper, I have 12 supplementary figures (9 main figures) plus additional supplementary methods and results text. All of this is in addition to deposited raw data in the GEO database. The impact factor? 9. Almost all of the additional data is supportive and/or control type stuff, and the additional methods are very important to us because we like to be as transparent as possible there. Could the supplemental data form another paper entirely? Maybe in a lower tier journal, but that’s debatable.
And if it were up to at least one of the reviewers, we would have had to add another 2 – 3 figures of additional data, plus a ton of text to go with it. Personally I think reviewer experiments are a big part of the problem.
LikeLike
December 11, 2014 at 11:17 am
One of my favorite examples of idiocy at the moment is a journal that has gone entirely online and yet still has a page limit (you can pay for overage if you want). So they force “supplementary materials” for no good reason I can see.
LikeLike
December 11, 2014 at 11:22 am
YOU ARE MAKING ME MORE RESILIENT.
I still say the publishers couldn’t care less about screwing J Neurosci out of data–what they care about is money and prestige. This is super-cheap way to enhance prestige because now when your journal club tries to tear apart this Nature paper, the presenter will be forced to say, ‘Uh, I think they did a control experiment for that, it’s somewhere in the Supplemental, uh give me a minute….’ and everyone will end up thinking that unless they read 35 pages of unformatted crap they can’t criticize the paper.
LikeLike
December 11, 2014 at 11:29 am
So they force “supplementary materials” for no good reason I can see.
Because reviewers demand it most of the time, and in print journals, there is a limit to the number of main figures. So where else are you gonna stick it? It’s usually easier to add data than to go back and forth with reviewers and editors.
LikeLike
December 11, 2014 at 11:45 am
References in the Supplement STILL aren’t counted? I even blogged about this….6 YEARS ago. (wow, old).
http://junctionpotential.blogspot.com/2008/06/do-citations-in-supplementary-data-not.html
LikeLike
December 11, 2014 at 12:46 pm
[…] is all the fault of qaz. And long time reader Nat had a blog post on this ages […]
LikeLike
December 11, 2014 at 1:01 pm
Asking why glamour mags require extensive supplemental data is equivalent to asking why pretty girls tend to be mean. Because they can.
LikeLike
December 11, 2014 at 1:52 pm
Am J Physiol (at least, the flavors that I’m familiar with) doesn’t allow supplementary materials either.
LikeLike
December 11, 2014 at 4:12 pm
I don’t think the supplementary data are usually low quality, they just tend to be more boring experiments, either asked for by a reviewer, or there to back up the findings of a main figure (Cell actually requires this).
Any easy fix for the issue of editing is to have the entire paper go through review without any supplement, and only decide what goes into the supplement after the paper is accepted. I’m not going to put a bunch half-baked garbage next to my precious main figures, so I’ll have to clean it all up to an appropriate level.
LikeLike
December 11, 2014 at 5:22 pm
A tepid defence of the supplemental materials: As someone who reads out of field a lot, I appreciate policies that encourage writers to be concise, while referring to the supplemental for control experiments, large datasets, and other information that is less relevant to other readers. Sadly, this is often not how the resource is used.
LikeLike
December 11, 2014 at 6:46 pm
I believe that is called “citing other papers”, Liam.
LikeLike
December 18, 2014 at 4:44 pm
[…] The “whole point” of Supplementary Data Three Cheers for the Pom Pom Crab! An Evolutionary Battle Against Bacteria New research reveals how orcas attack baleen whales, and how the whales fight back ‘Rectal Feeding’ Has Nothing to Do with Nutrition, Everything to Do with Torture […]
LikeLike