Three Cheers for the de-GlamourMagification of J. Neurosci!!!

August 11, 2010

cross posting from DrugMonkey on Scientopia
Journal of Neuroscience:

Beginning November 1, 2010, The Journal of Neuroscience will no longer allow authors to include supplemental material when they submit new manuscripts and will no longer host supplemental material on its web site for those articles.

whew. calm down, DM, calm down. why are they doing it?

Although The Journal, like most journals, currently peer reviews supplemental material, the depth of that review is questionable. Most well qualified reviewers are overburdened with requests to review manuscripts, and many feel that it is too much to ask them to also evaluate supplemental material that can be as extensive as the article itself. It is obvious to editors that most reviewers put far less effort (often no effort) into examining supplemental material. Nevertheless, we certify the supplemental material as having passed peer review.

True, true. A concern to be sure. [stay calm, DM, stay calm…]

Another troubling problem associated with supplemental material is that it encourages excessive demands from reviewers. Increasingly, reviewers insist that authors add further analyses or experiments “in the supplemental material.” These additions are invariably subordinate or tangential, but they represent real work for authors and they delay publication. Such requests can be an unjustified burden on authors. In principle, editors can overrule these requests, but this represents additional work for the editors, who may fail to adequately referee this aspect of the review.
Reviewer demands in turn have encouraged authors to respond in a supplemental material arms race. Many authors feel that reviewers have become so demanding they cannot afford to pass up the opportunity to insert any supplemental material that might help immunize them against reviewers’ concerns.

w00000t!!!!1111!!!!ELEVEN!!!! YAYAYAYAYAYAY!!!! Damn tootin’!!!!!

Supplemental material also undermines the concept of a self-contained research report by providing a place for critical material to get lost. Methods that are essential for replicating the experiments, analyses that are central to validating the results, and awkward observations are increasingly being relegated to supplemental material. Such material is not supplemental and belongs in the body of the article, but authors can be tempted (or, with some journals, encouraged) to place essential article components in the supplemental material.

OMGWTFRUKidding me?????!!!??? YES!!!! YESSSSSSS!!!!!!!!!!!!!!! Exactamudo correcto!!! YES!!!!!!!!!!!!!!
Three cheers for the de-GlamourMagification of the Journal of Neuroscience!!
[h/t: Dr. Jekyll & Mrs. Hyde for alerting me to this important news]

15 Responses to “Three Cheers for the de-GlamourMagification of J. Neurosci!!!”

  1. Gerty-Z Says:

    HOORAY!!! It is about fucking time. I hope that other journals jump on that bandwagon-now I just have to start doing research that I can publish in J. Neuroscience.


  2. bob Says:

    Am I the only one that doesn’t mind supplementary information? I like the extra data when I really want to get into a paper. Instead of 20 supplemental figures, now we’ll just have 20 more “data not shown” lines.


  3. DrugMonkey Says:

    bob, I’m not a fan of data not shown either. bogus. If it is a point that requires data validation, it should be in the published record. The *real* published record. If it is supposedly too much to put in one paper, gee, maybe there should be several papers. Or perhaps a journal should be selected that doesn’t put artificially constraining length requirements on the table.
    GlamourMag-style science is a cancer and it has metastasized beyond the original offending tissues of N and S. JNeuro was one of the secondary or tertiary victims and has now stood up and excised the tumor. GOOD FOR THEM!!!!!!!


  4. Paul Orwin Says:

    A cheer from me as well (although I have no reason to care about J. Neurosci, i think glamourmagification of specialist journals is a problem). However, I will pick a comment nit about the Data Not Shown thing – in my experience (and I’ve used it) DNS is used for something that is an assertion of fact about the subject that is too trivial to present as data. An example from my work would be an observation about the growth rate of a bacterium under a certain condition – a figure or table showing growth rates under different conditions is not very interesting, but it might explain a technical aspect (like why the cultures were grown for 2d instead of 1d). Of course it can be nefarious, but I don’t think it’s bad a priori.


  5. DrugMonkey Says:

    If it is “too trivial to present as data” than it is too trivial to bother with “data not shown” .


  6. Paul Orwin Says:

    I disagree. If you make a statement of fact in a paper (not common knowledge), you have two options to back it up – primary data or citation. So if I say “Bacterium x grows 1/2 as fast in YE broth as in LB broth” I either have to show it with data, or present a citation from literature. Good luck finding a citation in literature for that! but if I put a graph or table of that in, a reviewer will say “who cares? why are you wasting my time and space with trivia?” but if I just leave it uncited or unbacked up, they may say “how do you know that”. DNS communicates that I have done the work to support it, but recognize that it is not a “discovery” that needs to be analyzed in depth. Of course YMMV 🙂


  7. Beaker Says:

    I generally support this, especially because (if you read the editorial), you find that the journal will continue to offer a means to link to movies via the journal website. Movies are good, and it irks me how little cred they get compared to some lame figure with error bars with stars over them.
    There are three outstanding issues related to supplements:
    1. Some -omics studies need to present Excel spreadsheets. These should be allowed somehow.
    2. In a broader picture, if any journal continues to allow supplementary data, then for God’s sake, make it one freakin’ PDF for all the data plus the figures –all in one easy-to -obtain unit. I am sick and tired of clicking multiple downloads for supplementary figures, plus a separate download for the legend (in the case of PNAS).
    3. CPP has made the argument that “more data always good.” This is a valid point, but somehow this purist concept has become twisted via the dynamics of peer review.


  8. Eric Lund Says:

    There are perhaps a handful of things, such as movies, which need to be supplemental material because the printed page cannot handle them. But anything that can be printed, should be in print. Human nature says that most readers will ignore supplementary figures, tables, and text.
    I’ll answer the objection that some may raise about material that will only be of interest to a few readers, because (at least for longer articles) we have a pre-computer age solution to that problem. It’s called an appendix, and it signifies “meaty details for those who really want to know, but other readers can skip this.” Once I have downloaded the paper, I can always go back and read the appendix if I later decide that I really do need to read it (or I did a first quick read and overlooked the additional material).


  9. Heather Says:

    Maybe there should be several papers. Or perhaps a journal should be selected that doesn’t put artificially constraining length requirements on the table.
    This is a tough call. Usually you are trying to reach a particular readership. While I really support this move on the part of J. Neurosci, are they not just pushing the burden of storing large files onto other servers, and not guaranteeing the perennity of said extraneous information?
    If they really enforce the self-contained nature of the articles, then my point is moot. Primary data from -omics studies can often be uploaded to community-recognized, permanent resources (none of us are afraid to lose PubMed, right? then no fear for GEO). And just the relevant charts included.
    I’ve published quite a few papers with appendices/supplementary material. But I do resent having data that I consider good get relegated there based precisely on that back-and-forth the editorial describes. Relegated, because as Eric Lund states, human nature is not to look at it unless there is an unusually vested interest.


  10. Bye bye baby. At least we don’t have to worry about that pesky bathwater any longer.
    Stupid, stupid idea!


  11. DSKS Says:

    ‘So if I say “Bacterium x grows 1/2 as fast in YE broth as in LB broth” I either have to show it with data, or present a citation from literature.’
    Not necessarily. The act of typing the results in full into the text, with appropriate sd/sem and stats if necessary, is sufficient in many circumstances if the technique and analysis is well-known and standard . And if a reader wants to see the curve they can email the corresponding author and ask for it. (e.g. in my field it’s not uncommon for channel permeability results derived from reversal potentials to be presented in the text, without presenting the raw current/voltage curves, when the data is merely incidental rather than central to the paper).


  12. Anonymous Says:

    I think that you have done your job, as a first author, if you are more proud of the supp. material than of the main text. The main text is just the spin. The supp. material is all the little details, explained in painful detail, about what you are saying is true. Besides, who decides when DNS is really trivial? I have seen papers where a crucial point was obfuscated and finally shown as DNS. 7 years later, I finally saw that D from DNS, and it was BS.


  13. DrugMonkey Says:

    Of *course* “data not shown are baloney. Otherwise they would show them. If it is trivial and obvious then there is no point of alleging data backup!


  14. qaz Says:

    A little late to the party, but DM’s HALLELUJAH pretty much summed up my reaction when I saw this too. I’m really pleased that JNeurosci has recognized that the supplemental material was not being reviewed to the same extent that primary data was. And that they decided to do something about it.
    An important note to remember is that JNeurosci does not have any limitations on numbers of figures or on length of the results sections. This means that you can ALWAYS show the data. So DNS is part of the same obfuscation cr*p that was supplemental data – a way to sidestep actually showing data. Personally, I agree with Drug Monkey that DNS is bogus, but the nice part about forcing authors to put “DNS” is that reviewers can always demand it get shown.


  15. daedalus2u Says:

    How is supplemental data not a part of the paper? Because trees were not ritually sacrificed? How about we go back to vellum? Or stone tablets. Or force people to memorize and recite from memory? Or draw pictures, grunt and point?
    If it is a paper I am interested in, I always check out the supplement. For some gene papers the supplements can be gigantic, MB files in excel. There is no reason, and no excuse for printing that stuff on paper. No one utilizes spread sheets on paper.
    Often supplements are available for free, even for papers that are behind a paywall.
    I don’t care if the data is in a supplement or in the paper so long as it is there. I think the whinging about reviewers is disingenuous.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: