On showing the data

September 5, 2013

If I could boil things down to my most fundamental criticism of the highly competitive chase for the “get” of a very high Impact Factor journal acceptance in science, it is the inefficiency.

GlamourDouchery of this type is an inefficient way to do science.

This is because of several factors related to the fundamental fact that if the science you conduct isn’t published it may as well never have happened.

Science is an incremental business, ever built upon the foundations and structures created by those who came before. And built in sometimes friendly, sometimes uneasy collaboration with peers. No science stands alone.

Science these days is also a very large enterprise with many, many thousands of people beavering away at various topics. It is nearly impossible to think of a research program or project that doesn’t benefit by the existence of peer labs doing somewhat-related work.

Consequently, it is a near truism that all science benefits from the quickest and comprehensive knowledge of what other folks are doing.

The “get” of an extremely high Impact Factor Journal article acceptance requires that the authors, editors and reviewers temporarily suspend disbelief and engage in the mass fantasy that this is not the case. The participants engage in the fantasy that this work under consideration is the first and best and highly original. That it builds so fundamentally different an edifice that the vast majority of the credit adheres to the authors and not to any part of the edifice of science upon which they are building.

This means that the prospective GlamourArticle authors are highly motivated to keep a enormous amount of their progress under wraps until they are ready to reveal this new fine stand-alone structure.

Otherwise, someone else might copy them. Leverage their clever advances. Build a competing tower right next door and overshadow any neighboring accomplishments. Which, of course, builds the city faster….but it sure doesn’t give the original team as much credit.

The average Glamour Article is also an ENORMOUS amount of work. Many, many person years go into creating one. Many people who would otherwise get a decent amount of credit for laying a straight and true foundation will now be entirely overlooked in the aura of the completed master work. They will never become architects themselves, of course. How could they? Even if they travel to Society Journal Burg, there is no record of them being the one to detail the windows or come up with a brilliant new way to mix the mortar. That was just scut work for throwaway labor, don’t you know.

But the real problem is that the collaborative process between builders is hindered. Slowed for years. The dissemination of tools and approaches has to wait until the entire tower is revealed.

Inefficiency. Slowness. These are the concerns.

Sure, it is also a problem that the builders of the average Glamour Article tower may not share all their work even after the shroud has been removed. It would be nice to let everyone know just where the granite was found, how it was quarried and something about the brand new amazing mortar that (who was that anonymous laborer again? shrug) created. But there isn’t really any pay for that and the original team has moved on. Good luck. So yes, it would be good to require them to show their work at the end.

Much, much more important, however, is that they show each part of the tower as it is being created. I mean, no, I don’t think people need to work with a hundred eyes tracking their every move. I don’t think every little mistake has to be revealed, nor do I think we necessarily need to know how each laborer holds her trowel. But it would be nice to show off the foundation when it is built. To reveal the clever staircase and the detailing around the windows once they are installed. Then each sub-team can get their day in the sun. Get the recognition they deserve.

[And if they are feeling a little oppressed, screw it, they can leave and take their credit with them. And their advances in knowledge can be spread to another town who will be happy to hire this credentialed foundation builder instead of some grumpy nobody who only claims to have built a foundation.]

The competition for Glamour Article building can’t really catch up directly, after all it takes a good bit of work to lay a foundation or create a new window design. They can copy techniques and leverage them, but there is less chance of an out and out scoop of the full project.

So if the real problem is inefficiency, Dear Reader, the solution is most assuredly the incremental reveal of progress made. We don’t need to watch the stirring and the endless recipes for mortar that have been attempted, we just need to know how the successful one was made. And to see the sections of the tower as they are completed.

Ironically enough, this is how it is done outside of GlamourCity. In Normalville, the builders do show their work. Not all of it in nauseating detail but incrementally. Sections are shown as they are completed. It is not necessary to wait for the roof to be laid to show the novel floorplan or for the paint to be on to show the craft that went into the floor joists.

This is a much more efficient way to advance.

It has to be, since resources are scarce and people in Society Burgh kind of give a shit if one of their neighbors is crushed under a block of limestone. And care if an improperly supported beam cracks and they have to get a new one.

This is unlike the approach of Glamour City where they just curse the laborer and draft three new ones to lift the block into place. And pull another beam out of their unending pile of lumber.

63 Responses to “On showing the data”

  1. namnezia Says:

    Oh get off it already.

    Like

  2. drugmonkey Says:

    Nobody forces you to read my blog homes

    Like

  3. Juan Lopez Says:

    The problems with GlamourCity go much further to destroy efficiency. Not only do they not publish often and rarely share techniques, but often actively sabotage the developments by other groups. Blocking their papers and grants, trash-talking them in conferences, etc.

    The small beans are too afraid of losing their funding. The big ones don’t see the urgency to change the system, it works just fine!

    It’s sad, but I don’t think they spend much time considering the “greater good”.

    I got another question for you. You write that “if the science you conduct isn’t published it may as well never have happened”. Can this be “if it wasn’t read, it may as well never have happened”? Work that is never read, did it make much of a contribution? Did it matter?

    Like

  4. Grumble Says:

    That’s a huge part of the problem, JL. Keeping up with the lit nowadays is like drinking from a fire hose: you ingest very little and the rest might as well not exist. But everyone reads the glams. So if your data supports a paradigm shift, it’s more EFFICIENT to publish it in a glam mag, because more people will read it and be aware of it and have the paradigm actually shifted inside their skulls.

    Like

  5. Ola Says:

    If we’re going with building metaphors, then here are some more:

    – Building a paper on SantaCruz antibodies is like building a house on sand
    – Beware of “off gassing” immediately following new construction
    – Don’t skimp on the “snagging” stage of a build project
    – Everything comes in late and over budget, just deal
    – It might have marble floors and 100 windows and a 5 car garage, but deep down everyone still knows you’re a wanker
    – Location location location
    – Nothing ever ends up exactly like the architect’s drawings
    – In order to build some things, some heavy duty demolition work on older structures is required
    &c.

    Like

  6. drugmonkey Says:

    Work that is never read, did it make much of a contribution? Did it matter?

    If the tree falls in the forest….

    Well sure, if literally nobody reads it, or it has absolutely zero impact on subsequent science, perhaps the contribution is null. The odds of that seem very slim to me.

    If you are asking relative audience size? yeah, I don’t really see where random grad students who “read every Science cover to cover” are really increasing impact.

    because more people will read it and be aware of it and have the paradigm actually shifted inside their skulls.

    total bollocks. What’s important is that people working on related stuff can find it (via database search engines like PubMed) when they need to. period, end of story. That’s real science. Rump sniffing something because it happens to appear in a Glam this week? pfah.

    Like

  7. Cynric Says:

    Nice analogy.

    As long as the funding for Normalville lasts, let the Glamourcity architects strut.

    Normalville is a much bigger, pleasanter, and more robust place to live, with fewer of the errors made in shoddy foundation-laying that causes so many Glamtowers to collapse.

    Like

  8. Cynric Says:

    And in this world, I guess CNS = Dubai

    Like

  9. qaz Says:

    Amen, brother! And yet, if you think you have a chance at a GlamourMag, you’ll publish your work there. Do you deny it? (Go back to an argument parallel chez Isis – if I have work that has a shot at a GlamourMag, do I put it in JNeurophys anyway?)

    For my part, even more than the time it takes to build the GlamourMag article is the time it takes to publish it. In my experience, the cycle of GlamourMag publishing is orders of magnitude slower than NormalScience publishing, not because of the time to build it – papers take similar time to build for us no matter where they are going, but instead because GlamourMags have a low acceptance rate which means that you have to take multiple shots at multiple GlamourMags. So you spend a year at BigGlamourMag-1 and then two years at BigGlamourMag-2 and then a year at … and by the time the article is *actually* published, it’s many years.

    And Cynric, I don’ t know how funding is in your field, but in the BunnyHopping field I’m in, if you aren’t at least trying for GlamourMags (or at least BabyGlamourMags) then funding is very very difficult to maintain. (This goes back to the “you can’t survive on one R01, anymore” that DM and CPP have railed about for years.)

    Like

  10. qaz Says:

    PS. And I hate how people writing for GlamourMags refuse to place themselves in the literature because that would “diminish” their content. As if they were the first one’s to think of putting one block on top of another to make a tower taller.

    Like

  11. qaz Says:

    content -> contribution

    sorry

    Like

  12. DrugMonkey Says:

    Yeah I never understood the theory of JIF dilution, qaz

    Like

  13. namnezia Says:

    Wow, I guess I’ve been living under a rock, safely away from those undercutting, progress-delaying, pubmed-spurning, grad student-churning, postdoc-abusing, science-hiding motherfuckers that publish in Glam Journals. I should totes be wary of any of my colleagues who publish in said journals and their self-aggrandizing secret agendas…

    Like


  14. I know this point has been touched on before, but I’m watching it right now: Glamour city kill our young. It’s a drug that seduces them, to keep trying and trying, and the inefficiency weighs them down, and tenure time roles around and they (if lucky) got 1-2 glam pubs, but don’t have the substantive Normalville pubs to support tenure. They are so busy keeping things under wraps, that they miss out on constructive collaborative opportunities. It’s sad. A brilliant woman, with potential out the wazoo, is going to crash and burn. I grieve.

    Like

  15. namnezia Says:

    On the other hand, a fellow I know, at a decent university, had a number of great Normalville pubs, in decent but short of glam journals, an R01, great letters bur was denied tenure. When he asked the dean about it the dean said, “let’s put it this way, we wouldn’t be here talking about this if you had a Nature paper”. Another fellow I know, different institution, was in pretty much the same situation but in addition he also had a glam pub. Alas, he also didn’t get tenure, when he asked the provost for clarification, he said “am I only to expect only ONE glam pub from you every five years?”. Mind you this happened not in Harvard or whatever, but in seemingly “normal” uni’s. So yes, you do need a slew of solid pubs, but ALSO you need one or two glam pubs to go with them, as well as funding, good letters, etc. etc.

    Like

  16. Cynric Says:

    Amen, brother! And yet, if you think you have a chance at a GlamourMag, you’ll publish your work there. Do you deny it?

    No, I don’t deny it. I have nothing against GlamourMags per se, and if I discover something groundbreaking I would (and have) sent it in. In a genuine mixed economy it’s no problem at all – let some folks focus on novel, impactful, paradigm-shifting BunnyHopping research, but leave space for quiet revolutions that arise totally from left-field, when people in the shrew-locomotion field discover ShrewSomersaults.

    But, I cordially dislike the fact that CNS is distorting the economy for everyone, by linking quality of science (and scientist) to the journal in which it’s published. It’s demonstrably unscientific, and corrupts both research philosophy, and hiring/promotion practices. For every groundbreaking paper in CNS there is an intellectually uninspired data dump in a “hot” field where the author has apparently blinded the editors with status and railroaded the paper in.

    It seems like a classic example of measurements distorting outcome: if JIF is the determination of academic rank, security and salary, then scientists (being human) try to do “high-JIF” work, instead of original, imaginative and clever work, or even, god forbid, quiet but important confirmatory and error-correcting work.

    Like

  17. SteveTodd Says:

    Amen, brother! And yet, if you think you have a chance at a GlamourMag, you’ll publish your work there. Do you deny it? (Go back to an argument parallel chez Isis – if I have work that has a shot at a GlamourMag, do I put it in JNeurophys anyway?)

    Spot on. Of course we will all take the GlamourMagPub (GMP) because not taking it is stupid as long as it counts for more. The question is why do we allow it to count for more?

    Like

  18. Dave Says:

    So yes, you do need a slew of solid pubs, but ALSO you need one or two glam pubs to go with them, as well as funding, good letters, etc. etc.

    ^Not sure if serious.

    Like

  19. Cynric Says:

    And Cynric, I don’ t know how funding is in your field, but in the BunnyHopping field I’m in, if you aren’t at least trying for GlamourMags (or at least BabyGlamourMags) then funding is very very difficult to maintain.

    I would say that that is definitely the general perception in the coffee room, but I’m also not alone in managing to keep up a steady stream of grant funding for a small lab (couple of postdocs, couple of students and a tech) despite publishing mainly in society journals.

    It may be that the days of the small town grocer are numbered, of course.

    Like


  20. “The participants engage in the fantasy that this work under consideration is the first and best and highly original. That it builds so fundamentally different an edifice that the vast majority of the credit adheres to the authors and not to any part of the edifice of science upon which they are building. ”

    Which is why GlamMagz restrict the number of references – someone might accidentally cite a paper that reveals how incremental the work actually is.

    Like

  21. Cynric Says:

    Spot on. Of course we will all take the GlamourMagPub (GMP) because not taking it is stupid as long as it counts for more.

    Also this.
    I do want a pay rise, and I do want my postdocs to advance into independent careers.

    Like

  22. BioDataSci Says:

    Nice analogy, DM!

    Another concern with the intense focus on GlamourPubs is that there is a huge temptation to keep going until you find something significant enough to publish in a GlamourPub. You may even get a positive result, but you sweep it under the rug and try sometime else to see if that will be “significant” enough for a GlamourPub. There are problems related to scientific integrity with this approach, but it also hurts the careers of grad students and postdocs if that big result doesn’t come. (Kind of what you said.)

    Like

  23. DrugMonkey Says:

    It’s an important elaboration BioDataSci

    BB stop raising my blood pressure!

    Cynric nails the part where the “get” of the paper acceptance is the lab goal, rather than any particular scientific interest. I HATE this.

    Like

  24. DJMH Says:

    Your argument seems to be that there’s no such thing as major advances in science, only incremental ones. I just disagree. I have worked on projects that make more incremental, masonry-type advances, and I am very proud of those projects. I have also worked on projects that offer an insight that changes how we should think about a much larger field/question, and these projects I think deserve more widespread audience.

    Agreed that CNS publishing policies are wretched, and that their absurd citation limitations aren’t helpful to science, but those things are in theory fixable. Having everyone publish everything in the same journal, which seems to be your preferred world, seems not to reflect the reality that some projects *are* more broadly interesting than others. It’s vaguely communistic.

    Like

  25. DrugMonkey Says:

    I am not arguing there are not major advances. I am arguing that keeping everything under wraps until it looks like a major advance is an inefficient way to pursue what is rightly and inevitably a massively collaborative effort.

    This approach also has the effect of unjustifiably undercounting the incremental advances…which discourages them…which again slows progress.

    Like

  26. DrugMonkey Says:

    DJMH – what does “deserve a more widespread audience” even mean?

    Like

  27. DrugMonkey Says:

    Isn’t anyone prepared to argue my fundamental point on efficiency is *wrong*?

    Or that inefficiency is well-justified on the basis of some benefit?

    Like

  28. DJMH Says:

    But your assumption is that all major advances == longer to publication. (a) That’s not necessarily true–I have had society and glamour-y pubs take the same amount of time to collect, write up, and publish; and (b) Sometimes the major advance does take longer, but publishing it earlier might not have accomplished much for science either, because the long time was spent in development of a technique or prep or whatever, and the results are effectively end-loaded. So publishing a paper at the half-way point would look something like, “We have tried X, Y, and Z to accomplish Q, and none of them have worked.” That is a lab meeting, not a paper.

    So (a) you’re wrong, and (b) even when you’re not wrong, you’re wrong. Happy?

    Like

  29. Dave Says:

    DM I agree in principal that glam chasing is perhaps an “inefficient” way to do research, but it’s tricky because there are many CNS papers that are of course very important. I think what you mean is that it is an inefficient way to run your career.

    Like

  30. DrugMonkey Says:

    So you are saying that there is not a significant number of glamour-y pubs rhs could have been published earlier, perhaps on chunks, in society journals. And that there is no way such decisions have slowed the efforts of another lab?

    Like

  31. Neuropop Says:

    DM: While Glamormag publishing may take the cake in terms of inefficiency, no journal is immune from it. Reviewers of society level journals can demand absurd numbers of experiments that often add little to the main point of the work but serve to “round out” the results. Check out Sol Snyder’s op-ed (in PNAS?) about the arms-race of scientific communication — “more experiments, more techniques, we want a complete story!!” Much as I despise Glams (although I admit freely to chasing pubs in them), they are not the sole culprit here.

    But I wonder if this is the particular purview of biomedical science. Most hard-core theoretical physics results (I used to be at the fringes before lapsing into biomed) are often in the main society level journal (Phys. Rev.). If the result is particularly meaningful, it is published in the glam of all Physics journal, Physical Review Letters with a longer version with all details to follow. If all one cares about is the worth of the idea, then glams don’t have a monopoly on them. However, they do have an unnatural hold on Deans, Study Sections and the like.

    Like

  32. DrugMonkey Says:

    Inefficient does imply the end result is useful. Otherwise I would have used “a waste”.

    Like

  33. DrugMonkey Says:

    Sure Neuropop but the relative cost of saying “screw it, I’m trying another journal” are lesser.

    Like

  34. Neuropop Says:

    DM:”Sure Neuropop but the relative cost of saying “screw it, I’m trying another journal” are lesser”

    Yes, but some(often?)times when even the Society level journals give you the runaround, that one might as well have chased the Glam to begin with. At what point do you say “screw it?” Before even choosing to submit at the glam?

    Like


  35. Nice troll, holmes. But you make some assumptions here that only apply to a subset of “glamour chasing”:

    This means that the prospective GlamourArticle authors are highly motivated to keep a enormous amount of their progress under wraps until they are ready to reveal this new fine stand-alone structure.

    My lab and almost all of the labs in my various fields that I am aware of present their progress on an ongoing basis at seminars, symposia, and conferences, including the progress that is intended for high-impact journal submission. Sure, there are some secretive douches in science, but those douches are secretive about everything, anyway.

    The average Glamour Article is also an ENORMOUS amount of work. Many, many person years go into creating one.

    Pulling shitte out of your asse as usual. Did you really go and look at a sampling of papers in Science, Cell, and Nature and try to estimate how much work went into each one? Many (most?) of the Science reports and Nature letters essentially represent a single sweet experimental observation. This is obviously just anecdote, but the papers I have published in Cell have all represented less than one year of work, with the overwhelming bulk of it performed by one or two trainees. You are pissed off about a *subset* of high-impact pub chasing, not the “average”.

    Sure, it is also a problem that the builders of the average Glamour Article tower may not share all their work even after the shroud has been removed.

    Pulling more shit out of your asse about the “average”. Most papers in high-impact journals include a vast amount of supplementary data which is “showing your work”, and there is abolutely zero evidence you have presented that the authors of papers published in high-impact journals are any less helpful to colleagues who wish to build upon the work than the authors of papers published in the sub-dump journals you patronize.

    Like


  36. Oh, and the longest fucken slogge through three revised versions to get a motherfucken paper published was with the goddamn motherfucken Journal of Neuroscience.

    Like

  37. zb Says:

    I blame Cell. Before then, my perception is that there were top out of site journals (Nature & Science) and most everything else (and, I suspect, some junk and some warehouse journals). But put Cell in the mix (and, did it join because molecular biology was in an explosion) meant there was a higher tier to aspire to, and then a crop of journals arose to fill the 2nd tier. Well, and then add the supplementary materials, which meant that long form work could be shoehorned intot he short format and you create the chaos we have today.

    Someone should do the analysis PP describes of leafing through Nature, Science, and the rest and estimating the timeframe of the project. In my field, I saw a steady change in the trajectory. The last paper I reviewed, I’d seen 4 times already, for different journals and seen the abstract of the work 5 years before it was published. The second paper (which was my own) went from first experiment to Science paper in less than a year.

    Like

  38. drugmonkey Says:

    My lab and almost all of the labs in my various fields that I am aware of present their progress on an ongoing basis at seminars, symposia, and conferences,

    Bully for you. Is this as efficient (and I believe you say that you only attend dinky little elite meetings where hardly anyone is there to hear you) as just publishing it already?

    You are pissed off about a *subset* of high-impact pub chasing, not the “average”.

    Alternatively, you are failing to include a comprehensive assessment of all the work that goes on in a laboratory that *might* be a Glamour Mag article but then never becomes one. This is valid accounting in my book.

    there is abolutely zero evidence you have presented

    oh, is this high school debate champeen time?

    that the authors of papers published in high-impact journals are any less helpful to colleagues who wish to build upon the work than the authors of papers published in the sub-dump journals you patronize.

    All I know is what I hear people complaining about. You yourself have admitted to frustrations when asking for things from other labs and being sent bullshittio. In comments on this very blog.

    I cannot think of one single time hearing a person who does my sort of work report this kind of shit from a colleague. It would amount to intentionally telling people the wrong way to run a behavioral experiment, sending intentionally fucked up code to run an experiment or to analyze a dataset. That sort of nonsense. Never heard of anything like this, ever. Not saying it hasn’t ever happened. It just isn’t as common as accusations of the wrong antibody being sent or whatall you glamourhounds fuck each over with all the damn time.

    Like

  39. drugmonkey Says:

    Before even choosing to submit at the glam?

    I think one should take a shot at what one thinks is the reachable top JIF range with a paper. Starting *every* paper at the very top seems stupid to me but I’m sure some would back this approach. After all, why refuse on their behalf, right?

    Yes, but some(often?)times when even the Society level journals give you the runaround, that one might as well have chased the Glam to begin with.

    I think this is a key point. If a given journal insists on a bunch of experiments that in my/your opinion would justify trying it at a fancier journal, there is no point in putting up with that request. Either send the manuscript essentially as-is to another journal, rebut (perhaps with a subset of the additional work), or do all of it and submit higher.

    Like

  40. Dr. Noncoding Arenay Says:

    “If a given journal insists on a bunch of experiments that in my/your opinion would justify trying it at a fancier journal, there is no point in putting up with that request. Either send the manuscript essentially as-is to another journal, rebut (perhaps with a subset of the additional work), or do all of it and submit higher.”

    ^^ this. I recently had a manuscript under review at a mid tier journal with a decent amount of data. The reviewers came back with a few reasonable comments and then some comments about “mechanism”…well I’m working on the mechanism, but I ain’t submitting to THIS journal when I complete that work! I would definitely go higher. Have reviewers stopped thinking before putting their comments to paper nowadays? I resubmitted the paper after addressing some minor comments and that’s all they are going to get.

    Like

  41. Cynric Says:

    Many (most?) of the Science reports and Nature letters essentially represent a single sweet experimental observation. This is obviously just anecdote, but the papers I have published in Cell have all represented less than one year of work, with the overwhelming bulk of it performed by one or two trainees. You are pissed off about a *subset* of high-impact pub chasing, not the “average”.

    I haven’t done this survey, but CNS is essential reading for most of us. Your description is pretty much what CNS should be (and used to be, like, 20-30 years ago), but nowadays most Nature letters I read have multi-multi-panel (plus insets) figures in the paper, plus a couple of dozen extra figures in supplements.

    My last ref report from Nature asked for all the whole cell patch experiments to be repeated in an awake KO mouse, but conceded that I’d have to do a few controls as the slice experiments were in rats. How many labs could pull that off in a year with one trainee?

    Obviously this was a bullshit comment because they didn’t like the paper, but the editor didn’t bat an eye at it being ever so slightly unreasonable.

    Like

  42. Cynric Says:

    The reviewers came back with a few reasonable comments and then some comments about “mechanism”…well I’m working on the mechanism, but I ain’t submitting to THIS journal when I complete that work!

    This is another problem with JIF – journals that used to serve scientific societies are now just as exercised over how to raise their perceived status (= IF) as a journal, as they are about communicating worthwhile results to the community.

    What I like best about Plos ONE is not having to concoct some angle as to why these results are so earth-shatteringly important that they’re bound to have huge impact. Just: here’s what we found, here’s why we think it’s important. Oh, and by the way, we found some evidence of a weird off-target effect of a common drug that’s muddied the waters a bit, but on balance we’re confident about the mechanism. When was the last time you saw any equivocation in a CNS paper?

    Like

  43. Cynric Says:

    Oh, and the longest fucken slogge through three revised versions to get a motherfucken paper published was with the goddamn motherfucken Journal of Neuroscience.

    Amen to that. And they charge you for the privilege of submitting to their referee shark pool.

    Never again.

    Like

  44. DJMH Says:

    DM: “I think one should take a shot at what one thinks is the reachable top JIF range with a paper.”

    Uh, so what exactly is our quarrel then? I make as educated as possible an estimate of top plausible (40-50%) JIF and submit there.

    Also, this discussion of JIF is making me hungry.

    Like

  45. drugmonkey Says:

    DJMH- our quarrel is over whether IF chasing is a good thing overall for science (and in your case whether there is any justification for this kind of stratification) versus a career reality that researchers can ill afford to ignore.

    All my wishing JIF chasing were not a reality for career success and reasons for why it is bad for science do not, at this present time, make it any less important for scientists to chase JIF points.

    In case anyone missed it, the NIMH Dir weighed in on DORA:
    http://www.nimh.nih.gov/about/director/2013/accessing-and-assessing-science-from-plos-to-dora.shtml

    Like

  46. Neuropop Says:

    PP: I don’t know what you are reading, but many Nature letters/Science reports that I come across in Neuroscience are far from one-off cute observation. Most seem to involve slice and in vivo recordings, some combination of optogenetic/DREADD/viral CRE stuff, some intricate behavior, analysis the folks at CERN would be proud of, and fucktillion inane controls. Gone are the days when the ePhys types could publish a really nicely designed, 3 month experiment with a new technique and intriguing result. Cell — well, talk to the poor postdocs in my department who slaved for 5 years and came out with one. Mercifully, they all got jobs.

    Like

  47. physioprof Says:

    When was the last time you saw any equivocation in a CNS paper?

    Our most recent paper published in Cell, the editor required us to include data that we wanted to leave out after reviewers asserted (probably correctly) that they were ambiguous. When I said no problem, we’ll put it in a supplemental figure, she said, no, put it in a main figure.

    You yourself have admitted to frustrations when asking for things from other labs and being sent bullshittio. In comments on this very blog.

    Yes, I have. These experiences have never correlated with the “glamourness” of the underlying science. I have gotten tubes of motherfucken bluescript in response to requests arising from papers in journals I never even heard of before.

    Like

  48. physioprof Says:

    And I should have added, the editor required us to explain exactly how and why the data were likely ambiguous.

    Like

  49. DrugMonkey Says:

    ambiguous? How come you didn’t tell your army of muppethugging postdocs to figure that shit out statte?

    Like

  50. Eli Rabett Says:

    The problem with the glamor mags is you either need to be a wonder rabbi* or have a wonder rabbi pushing your stuff. For anyone else it is simply a waste of time and will not even be sent out for review.

    * (noun) By metaphor from the Jewish religious role, an older, more powerful or higher-ranking person in the corporation where one works (but usually not in the chain of command) who can give good advice about office politics, and may be able to pull strings, remove heads, or otherwise provide protection from hostile forces.

    Like

  51. physioprof Says:

    The problem with the glamor mags is you either need to be a wonder rabbi* or have a wonder rabbi pushing your stuff. For anyone else it is simply a waste of time and will not even be sent out for review.

    This is completely utterly demonstrably false.

    Like

  52. Eli Rabett Says:

    So, demonstrate it.

    Like

  53. physioprof Says:

    This was not the case for me when I was a post-doc in the lab of a no-name, no-pedigree brand-new assistant professor with no rabbis pushing for us: we had one paper reviewed by Nature (not ultimately accepted) and one paper reviewed by Cell (ultimately accepted). It was also not the case for me when I started my own lab as a relatively unknown assistant professor without any rabbis pushing for me. And it has not been the case for many of my junior faculty colleagues over the last decade.

    High-impact journal editors are not idiots, and their goal is to publish work that is going to be exciting, influential to a field, and highly cited. They are not so stupid as to think that just because work comes from a famous lab it is going to be those things, and just because it does not, it is not. They take their jobs very, very seriously, and the vast majority of them are very conscientious and engaged with finding the most exciting science to publish, regardless of the identities of the authors.

    Of course, there is some influence of who authors are on the process, but it is nowhere near as dominant as you are making it out to be.

    Like

  54. Cynric Says:

    It is false that you have to have a rabbi, but the activation barrier for unknowns is a lot higher.

    As a counter anecdote, my postdoctoral boss (also little known) had a great manuscript identifying a channel through a battery of single channel recordings, pharmacology, and immuno that Nature dragged their feet over for ages, finally siding with a referee who insisted that it was just “correlative”

    About six months later a BSD published on the same channel using a dodgy antibody blocker (since shown to be non selective) and one inhibitor. In Nature.

    I’m not even sure this is misguided on the part of the editor, because the BSD paper will probably get more citations, as everyone cites him. But is is symptomatic of the positive reinforcement of established authors being “Nature worthy”.

    Like

  55. Cynric Says:

    They take their jobs very, very seriously, and the vast majority of them are very conscientious and engaged with finding the most exciting science to publish, regardless of the identities of the authors.

    Yeah, and some of them are jackasses who prop the bar at Gordon conferences revelling in having important scientists currying favour. And think they make as much of a contribution to the advancement of science as PIs, by helping them “improve” their work.

    Like

  56. Neuropop Says:

    Cynric: “Yeah, and some of them are jackasses who prop the bar at Gordon conferences revelling in having important scientists currying favour. And think they make as much of a contribution to the advancement of science as PIs, by helping them “improve” their work.”

    I’ve come across some snot-nosed ones that hold court at Gordon Conferences as well. More annoying is the fact that BSDs can sometimes get to “discard” certain referees comments but not so for the mere hoi-polloi.

    Like

  57. Eli Rabett Says:

    What Cynric said.

    Like

  58. CE Says:

    DM, to one of your original points, I’ve had my major postdoc paper out for its *first* round of review at a GlamMag for what is now starting the 6th month. I find this completely unacceptable and inefficient. How am I supposed to apply for grants and reference the shit I did? As you say, it’s almost like it didn’t happen.

    On the other hand, I’m applying for some young investigator awards – was invited to one today where last year’s winners all have 1 and often 2 first author CNS papers from their early career. Maybe this GlamourMag review process is f*ing me over, but I’m also f*ed without it.

    CNS is probably not good for science, but I don’t feel like I have a choice but to play.

    Like

  59. drugmonkey Says:

    CNS is probably not good for science, but I don’t feel like I have a choice but to play.

    exactly.

    Like

  60. Dave Says:

    On the other hand, I’m applying for some young investigator awards – was invited to one today where last year’s winners all have 1 and often 2 first author CNS papers from their early career.

    I have a K99 summary statement where one reviewer specifically made the comment that my “…publication record was excellent but, unfortunately (that word was used), competing candidates have Cell, Nature etc papers”. I actually appreciated the honesty.

    Like

  61. drugmonkey Says:

    That is unfortunate.

    Like

  62. anonymous postdoc Says:

    I also got dinged on my K99, for despite a respectable number of publications attesting my steady productivity, none of them had elevated enough JIF for the reviewers.

    But I don’t think a lot of neuroscientists actually have CNS papers (acronym irony). So certainly much of the competition is at the IF circa 15 Nature Neuro/Neuron/MolPsy level, and then from there the IF circa 8 BiolPsy/NPP/JNeuro level, which if you ask some people is slumming it, job/grant application wise, and for others is the highest realistic bar. The question is, is publication at the JIF 8 level acceptably high impact in this day and age?

    All I know is that molecular work alone is suitable for publication at the highest level, while behavioral work alone is some useless bullshit.

    Like

  63. Dave Says:

    I have always floated around in society-level journals, which in my area have IFs anywhere from 6 – 8. So to answer your question, no it is clearly not good enough if TT and Career Development grants are your aim. Sad but true. For me it is always a little under-appreciated just quite how cutthroat the selection of young scientists is right now. The key is in finding a mentor/department that understands that now.

    Like


Leave a comment