On dismantling the mystique of high IF journal publication
February 23, 2012
Stephen Curry has asked a key question in the wake of his post on the Welcome Trust’s Open Access policy. I was not previously aware that this policy launched an attack on GlamourMag science.
-specifically the Welcome Trust:
…
affirms the principle that it is the intrinsic merit of the work, and not the title of the journal in which an author’s work is published, that should be considered in making funding decisions.
Stephen eventually got around to asking how we scientists could reinforce this principle.
It is my view that merely asserting “there are great papers published in journals with more pedestrian IF ratings” will not be enough. This will require explaining just what is wrong with the enterprise of high-IF journal chasing “science”. Confrontation, if you will.
You may recognize that I have been pursuing this agenda on this blog for some time.
DrugMonkey 
February 23, 2012 at 3:23 am
February 23, 2012 at 4:56 am
One way to do it is not be a cobag on study section and use where the papers are published as a shorthand for their impact. It is pathetic.
LikeLike
February 23, 2012 at 5:22 am
You arguing with the Welcome Bloody Trust PP? Seriously?
LikeLike
February 23, 2012 at 6:14 am
I don’t see how this statement from the Wellcome people actually supports “confronting” high-IF journal practices. Sounds to me like they’re saying high-quality journals are the goal. And I don’t see any way in hell that a PLoS One model is going to count, given the explicit model that all scientific findings “judged to be technically sound” are accepted. I don’t think anyone would confuse that description with a “high-quality” journal.
I read Wellcome’s statement as saying, “Your publication in PLoS Biology will be duly noted as not *necessarily* inferior to a Nature paper.”
LikeLike
February 23, 2012 at 6:52 am
“high quality” is as variable as the people who use the term.
The NIH has phrasing like that and there is a boatload of NIH supported work in journals with IF’s south of 4.
For some disciplines I hear their top field journals barely break IF of 1. In short, it depends.
On what? Well for one thing, data that reflect an actual sample, with error bars and inferential stat analysis are higher “quality” than *any* “representative image/blot/band/gel bullshittio. Papers that rely on well verified methods that work in multiple labs are superior “quality” than the ones that rely on brand new gee whiz shit that can’t be replicated .
So no, I doubt we can tell the intent of their “high quality” comment but in combination with the clear slap at GlamourMags I think my interpretation is better supported. Otherwise, why bother?
LikeLike
February 23, 2012 at 7:49 am
Just because the NIH has similar phrasing and yet some NIH-funded work ends up in low-IF journals, does NOT mean that you have an equal chance of funding success with high- and low-IF work. Obviously there is a lot of discipline-specific variation in IF, due to field size, which is (one of the many reasons) why it’s a crappy metric.
Why would WT “bother” putting this statement out? To cover their asses, as institutions have done since time eternal. I took a quick look at their most recently funded science in the fields that I am familiar with, and it is 100% in labs with a stream of high-IF papers:
Click to access WTVM051669.pdf
LikeLike
February 23, 2012 at 7:57 am
My point was limited to the phrasing “high impact” and the provision of an example countering your general assertion. Obviously the specifics of what they fund give a clearer view of what WT means. I’ll have to look through for my subfields of interest…
LikeLike
February 23, 2012 at 8:03 am
Fair enough. I still think you’re conflating the OA and IF issues. I do read their statement as being clearly pro open access, but that’s not the same as saying IF doesn’t matter.
LikeLike
February 23, 2012 at 8:33 am
What on earth could “the title of the journal” possibly refer to if not the IF/Glamour falsehood?
LikeLike
February 23, 2012 at 8:56 am
(1) Wellcome Trust is the British equivalent of HHMI, and allocates its funding accordingly. The overwhelmingly vast majority of working biomedical scientists have no connection to Wellcome Trust at all.
(2) I have served on multiple cellular, molecular, genetic, and physiological neuroscience study sections. I can assure you that from either PI “productivity” or “scientific impact” standpoint there is absolutely zero greater weight given to publications in Neuron, Nature Neuroscience, Cell, Science, or Nature than to publications in Journal of Neuroscience. If you are only publishing in Brain Research or European Journal of Neuroscience, yeah, you may have a problem. But I have seen competing renewals with only two publications in the prior project period–but both in Cell, Science, or Nature–get savaged for poor productivity, and ones with five publications in the prior project period–all in Journal of Neuroscience–get lauded for excellent productivity.
LikeLike
February 23, 2012 at 9:39 am
Ok, I take y0ur point about WT being equivalent to HHMI and therefore of minimal practical application to the vast majority of scientists.
But I have seen…..
I have had similar experiences in my grant reviewing, albeit the ranks are downshifted. The point that generalizes is that grant review, ime, does a pretty good job at focusing on the actual science itself, the influence on the subfield, the linear progress from A to B, etc. Not surprising, of course, because it is being conducted by a focal audience. People that can take a stab at making those proximal quality judgements and indeed that have been selected for their ability to do so.
LikeLike
February 23, 2012 at 10:57 am
I was not referring to J Neurosci when I said “low IF,” but to PLoS One et al. I still read the WT statement as being pro-PLoS Biology without necessarily being pro Frontiers, as it were.
LikeLike
February 23, 2012 at 11:43 am
I am a reviewing/academic editor at both PLoS ONE and Frontiers. PLoS ONE gets much better quality manuscripts in areas that I handle than Frontiers, the review process is more rigorous, and the editorial/reviewer/authorial Web-based management system is fuckeloades better. I recommend to anyone who asks that they submit to PLoS ONE and not Frontiers.
LikeLike
February 23, 2012 at 12:14 pm
Wait…how is this a referendum on PloSONE?
Now you are the one dragging in the OpenAccess issue. Glamour and Open issues are not at all the same and should not be conflated carelessly.
LikeLike
February 23, 2012 at 1:00 pm
I got my first PLOSone paper to review at the end of January. It was numbered something like 12-02850. I thought to myself “this can’t possibly be the 2850th paper submitted in 2012” so I contacted the editor. Turns out there is a 70% acceptance rate with approximately 300 papers in PlosOne published a week. By contrast a solid, society level journal like the Journal of Immunology publishes 1400 papers a year for an average of 26 papers a week and JBC publishes 4400 papers a year for an average of 85 papers a week.
Given these numbers and the fact that PLOSone publication charges subsidize the whole PLOS library (and thus there is an incentive to take every paper), I’ll take my society level journal any day over PLOSone.
LikeLike
February 23, 2012 at 1:21 pm
I note, Physician Scientist, that you’ve related nothing whatever about something being wrong with the substance of the articles published in PLoSONE….
I have yet to run across a PLoSONE paper in my fields of interest and to think to myself “gee ,why did they accept THIS dog”.
LikeLike
February 23, 2012 at 2:29 pm
My impression has been a little different. I think the quality of work is not bad, but many of the papers in my field are just showing the same thing in a different cell type – very mundane stuff in my field.
LikeLike
February 23, 2012 at 2:46 pm
Interesting. Because my impression so far is that the mystique of “solid work, no reference to importance/impact” has produced nothing particularly different from the usual journals in the fields I tend to follow.
Maybe this is because these scientists haven’t latched onto PLoSONE as an alternative to an array of what might be considered methodological or dump journals?
LikeLike
February 23, 2012 at 3:05 pm
“…….the quality of work [in PLoS ONE] is not bad, …….very mundane stuff in my field.”
My experience with PLoS ONE can be stated similarly; out of more than 1600 references in my Endnote bibliography library, only one is from that journal. I haven’t come across many examples that are important enough to cite.
LikeLike
February 24, 2012 at 11:29 am
I think it seemed my comment was that the WT language was welcome- meaning “paper published in X does not equal paper of quality Y, so don’t use X as shorthand for Y”
I think this is clear.
LikeLike
March 7, 2012 at 1:56 pm
[…] unhealthy fixation on publishing in these journals, and this is a frequently discussed topic by Drug Monkey and Bjorn Brembs, among others. I am relieved it’s over. It degenerated into an irrational […]
LikeLike