On dismantling the mystique of high IF journal publication

February 23, 2012

Stephen Curry has asked a key question in the wake of his post on the Welcome Trust’s Open Access policy. I was not previously aware that this policy launched an attack on GlamourMag science.

-specifically the Welcome Trust:

affirms the principle that it is the intrinsic merit of the work, and not the title of the journal in which an author’s work is published, that should be considered in making funding decisions.

Stephen eventually got around to asking how we scientists could reinforce this principle.

It is my view that merely asserting “there are great papers published in journals with more pedestrian IF ratings” will not be enough. This will require explaining just what is wrong with the enterprise of high-IF journal chasing “science”. Confrontation, if you will.

You may recognize that I have been pursuing this agenda on this blog for some time.

Advertisements

No Responses Yet to “On dismantling the mystique of high IF journal publication”


  1. This will require explaining just what is wrong with the enterprise of high-IF journal chasing “science”. Confrontation, if you will.</blockquote.

    It will "require" it, or you will "get a boner" doing it, because you hate those fuckers with a passion?

    Like

  2. Pinko Punko Says:

    One way to do it is not be a cobag on study section and use where the papers are published as a shorthand for their impact. It is pathetic.

    Like

  3. DrugMonkey Says:

    You arguing with the Welcome Bloody Trust PP? Seriously?

    Like

  4. DJMH Says:

    The main output of this research is new ideas and knowledge, which the Trust expects its researchers to publish in high-quality, peer-reviewed journals.

    I don’t see how this statement from the Wellcome people actually supports “confronting” high-IF journal practices. Sounds to me like they’re saying high-quality journals are the goal. And I don’t see any way in hell that a PLoS One model is going to count, given the explicit model that all scientific findings “judged to be technically sound” are accepted. I don’t think anyone would confuse that description with a “high-quality” journal.

    I read Wellcome’s statement as saying, “Your publication in PLoS Biology will be duly noted as not *necessarily* inferior to a Nature paper.”

    Like

  5. drugmonkey Says:

    “high quality” is as variable as the people who use the term.

    The NIH has phrasing like that and there is a boatload of NIH supported work in journals with IF’s south of 4.

    For some disciplines I hear their top field journals barely break IF of 1. In short, it depends.

    On what? Well for one thing, data that reflect an actual sample, with error bars and inferential stat analysis are higher “quality” than *any* “representative image/blot/band/gel bullshittio. Papers that rely on well verified methods that work in multiple labs are superior “quality” than the ones that rely on brand new gee whiz shit that can’t be replicated .

    So no, I doubt we can tell the intent of their “high quality” comment but in combination with the clear slap at GlamourMags I think my interpretation is better supported. Otherwise, why bother?

    Like

  6. DJMH Says:

    Just because the NIH has similar phrasing and yet some NIH-funded work ends up in low-IF journals, does NOT mean that you have an equal chance of funding success with high- and low-IF work. Obviously there is a lot of discipline-specific variation in IF, due to field size, which is (one of the many reasons) why it’s a crappy metric.

    Why would WT “bother” putting this statement out? To cover their asses, as institutions have done since time eternal. I took a quick look at their most recently funded science in the fields that I am familiar with, and it is 100% in labs with a stream of high-IF papers:

    http://www.wellcome.ac.uk/stellent/groups/corporatesite/@msh_publishing_group/documents/web_document/WTVM051669.pdf

    Like

  7. drugmonkey Says:

    My point was limited to the phrasing “high impact” and the provision of an example countering your general assertion. Obviously the specifics of what they fund give a clearer view of what WT means. I’ll have to look through for my subfields of interest…

    Like

  8. DJMH Says:

    Fair enough. I still think you’re conflating the OA and IF issues. I do read their statement as being clearly pro open access, but that’s not the same as saying IF doesn’t matter.

    Like

  9. drugmonkey Says:

    What on earth could “the title of the journal” possibly refer to if not the IF/Glamour falsehood?

    Like


  10. Just because the NIH has similar phrasing and yet some NIH-funded work ends up in low-IF journals, does NOT mean that you have an equal chance of funding success with high- and low-IF work.

    I took a quick look at [Wellcome Trust’s] most recently funded science in the fields that I am familiar with, and it is 100% in labs with a stream of high-IF papers[.]

    (1) Wellcome Trust is the British equivalent of HHMI, and allocates its funding accordingly. The overwhelmingly vast majority of working biomedical scientists have no connection to Wellcome Trust at all.

    (2) I have served on multiple cellular, molecular, genetic, and physiological neuroscience study sections. I can assure you that from either PI “productivity” or “scientific impact” standpoint there is absolutely zero greater weight given to publications in Neuron, Nature Neuroscience, Cell, Science, or Nature than to publications in Journal of Neuroscience. If you are only publishing in Brain Research or European Journal of Neuroscience, yeah, you may have a problem. But I have seen competing renewals with only two publications in the prior project period–but both in Cell, Science, or Nature–get savaged for poor productivity, and ones with five publications in the prior project period–all in Journal of Neuroscience–get lauded for excellent productivity.

    Like

  11. drugmonkey Says:

    Ok, I take y0ur point about WT being equivalent to HHMI and therefore of minimal practical application to the vast majority of scientists.

    But I have seen…..

    I have had similar experiences in my grant reviewing, albeit the ranks are downshifted. The point that generalizes is that grant review, ime, does a pretty good job at focusing on the actual science itself, the influence on the subfield, the linear progress from A to B, etc. Not surprising, of course, because it is being conducted by a focal audience. People that can take a stab at making those proximal quality judgements and indeed that have been selected for their ability to do so.

    Like

  12. DJMH Says:

    I was not referring to J Neurosci when I said “low IF,” but to PLoS One et al. I still read the WT statement as being pro-PLoS Biology without necessarily being pro Frontiers, as it were.

    Like


  13. I am a reviewing/academic editor at both PLoS ONE and Frontiers. PLoS ONE gets much better quality manuscripts in areas that I handle than Frontiers, the review process is more rigorous, and the editorial/reviewer/authorial Web-based management system is fuckeloades better. I recommend to anyone who asks that they submit to PLoS ONE and not Frontiers.

    Like

  14. drugmonkey Says:

    Wait…how is this a referendum on PloSONE?

    Now you are the one dragging in the OpenAccess issue. Glamour and Open issues are not at all the same and should not be conflated carelessly.

    Like

  15. Physician Scientist Says:

    I got my first PLOSone paper to review at the end of January. It was numbered something like 12-02850. I thought to myself “this can’t possibly be the 2850th paper submitted in 2012” so I contacted the editor. Turns out there is a 70% acceptance rate with approximately 300 papers in PlosOne published a week. By contrast a solid, society level journal like the Journal of Immunology publishes 1400 papers a year for an average of 26 papers a week and JBC publishes 4400 papers a year for an average of 85 papers a week.

    Given these numbers and the fact that PLOSone publication charges subsidize the whole PLOS library (and thus there is an incentive to take every paper), I’ll take my society level journal any day over PLOSone.

    Like

  16. drugmonkey Says:

    I note, Physician Scientist, that you’ve related nothing whatever about something being wrong with the substance of the articles published in PLoSONE….

    I have yet to run across a PLoSONE paper in my fields of interest and to think to myself “gee ,why did they accept THIS dog”.

    Like

  17. Physician Scientist Says:

    My impression has been a little different. I think the quality of work is not bad, but many of the papers in my field are just showing the same thing in a different cell type – very mundane stuff in my field.

    Like

  18. drugmonkey Says:

    Interesting. Because my impression so far is that the mystique of “solid work, no reference to importance/impact” has produced nothing particularly different from the usual journals in the fields I tend to follow.

    Maybe this is because these scientists haven’t latched onto PLoSONE as an alternative to an array of what might be considered methodological or dump journals?

    Like

  19. pablito Says:

    “…….the quality of work [in PLoS ONE] is not bad, …….very mundane stuff in my field.”

    My experience with PLoS ONE can be stated similarly; out of more than 1600 references in my Endnote bibliography library, only one is from that journal. I haven’t come across many examples that are important enough to cite.

    Like

  20. Pinko Punko Says:

    I think it seemed my comment was that the WT language was welcome- meaning “paper published in X does not equal paper of quality Y, so don’t use X as shorthand for Y”

    I think this is clear.

    Like


  21. […] unhealthy fixation on publishing in these journals, and this is a frequently discussed topic by Drug Monkey and Bjorn Brembs, among others. I am relieved it’s over. It degenerated into an irrational […]

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: