More on "shitasse" journals

January 30, 2012

A bit of a Twittstorm was raised today by Jonathan Eisen (@phylogenomics) who posted a (Modest?) proposal that we should respond to Elsevier’s naughtiness over the Research Works Act by

stop helping promote articles published in Elsevier journals.  Don’t blog about papers in Elsevier journals.  Don’t tweet about them.  Don’t use Elsevier papers for journal clubs.  In essence, ignore them – consider them dead – make them invisible.  Not completely of course.  Any work should be considered a contribution to science or math or whatever your field is.  But there are LOTS and LOTS of things to do with your time.   

This is in strikeout because Dr. J. Eisen thought better of his snark. As I suggest above, perhaps he was making a Modest Proposal. At any rate, the original twittstorm was fanned by YHN.

My recent readers may have come to the conclusion that I am one of those Open Access wackanuts. I am not. I am, or have been, rather a skeptic of the more….excessively fervent OpenScienceELEV3NTY111!!!111!! types.

It is my position that the TruBeliever Acolytes of OpenAccess need a firm hand now and again to bring them back down to reality. Think of the responsibility churches have to keep the snake-handlers and speakers-of-tongues within reason. The way regular Christian folks need to point out that Pat Robertson is, in fact, pretty insane with his God’s revenge on the Hurricane-Belt stuff. You take my point.

Anyway, there is one core point that needs to be explicated in this because it segued from (J.) Eisen’s flipping of the bird to the overall notion that it is okay to dismiss / ignore research papers on the basis of where they are published. Some Twitt going by @lightsam1 is of the PhysioProffian opinion that there exist “shitasse” journals in which we will never find anything of any worth, scientifically, so there is every excuse to simply ignore them.

This is unscholarly in the extreme. The argument, in my view, is exactly the flip side of lauding papers that are published in GlamourMags as if they are something special. They are not.

In either case. The science is the thing. There should be no substitution, ever, for making a scientific judgment about the merits of a given paper based on association.

This was the essence of my objection to (J.) Eisen’s original post. No matter how pissed you may be at the publisher, it is not right to overlook the best, most relevant papers. It is not. Similarly, it is not right to overlook the best (or first, or most comprehensive) citation because it was not in a sufficiently Glamourous journal. This runs counter to good scholarship in academic science. I will entertain the debate over priority, if it is the best scholarship to cite the first report that touches on an issue simply because it appeared first. I happen to think an excess of this is a very large part of Glamour problem but…ok. It is always okay to have differences of opinion over what is the “best” or “most comprehensive” or “most elegant” demonstration of a much-replicated effect. Fine. That we can debate.

However. The notion that you are citing (or not citing) a paper based on where it is published is always wrong.

Consistent with what I was saying in a prior post, there was an excuse in the Pre-PubMed era to focus on a subset of the available journals because humans weren’t capable of keeping up with everything. In the PubMed era, however? No excuse whatsoever. Online databases and search engines provide readily available, simple and reasonably* comprehensive mechanisms to sort the literature.

If anything, you should be almost unaware of journal identity and IF and perceived “status” these days. It just doesn’t have any scholarly value.

*remember, not everything relevant to your work is indexed by your favorite search database. Who know when some odd economics paper might be really cool to cite, eh?

In the previous post on journal publishing, I observed that sub-sub-specialty journals were an anachronism of the era prior to the establishment of nearly comprehensive search engines and databases like PubMed. In that era, dividing the monthly output of scientific papers into journals made sense. First of all, it would be pretty hard to pick up a monthly issue of “The Omnibus Journal of Biomedical Science”. Second, it would be unduly laborious (and paper cut-y) to keep flipping around from some index or TOC to the abstracts you wanted to scan. So there were certain physical realities driving journal specialization.

Not to mention the fact that across the decades from 1886 to 1996 (PubMed established) there was a gradual and sustained addition of sub-specialty societies, narrower and narrower subfields of interest and an all around expansion of academic science. This came with a desire for yet another group of scientists to have selection of the studies they most wanted to read into a smaller number of journals.

I am not privy to all the details of the history of journals in academic science. Not even close. But what I do know is that a publisher such as Elsevier has a metric boatload of small sub-specialty journals at present time. Many of which are tied to an academic society. They continue to launch NEW ones. (Phew, I’m already link exhausted- Google “Official Journal Elsevier” and see what you get. The list is enormous.)

It is, or has been, in the interests of both Elsevier and the academic society to continue this arrangement. Occasionally societies will switch publishers. For example, Neuropsychopharmacology jumped from Elsevier to Nature Publishing Group in recent memory**. Occasionally you’ll be looking at the online site for a journal and notice a truncation in the archive..and have to Google around to figure out who used to publish the journal. Nevertheless, it is clear that Elsevier thinks these arrangements are good ones. Presumably because they get good return from libraries when they bundle a bunch of journals into a fixed price menu.

[Sidebar: This is a bit of a fly in the ointment, btw. One thing I do laud the publishers for is when they’ve taken effort to PDF all of their back catalog…back to vol 1, issue 1 in the dark ages in some cases. When there’s a shift in the publisher that took place prior to the online age it seems to me that their motivation for putting up a back catalog for a journal they no longer publish is not very high.]

What do the societies get in return?

I am, shall we say*, somewhat informed about moves by at least two society level journals to switch their default member subscription from print to online. The response seemed to be overwhelming approval and lack of opting-for-print amongst the memberships. No surprise, almost all of us are complete and total converts to the benefits of online access to journal articles and personal PDF archives on our computers. Yes, even the rapidly emeritizing cohort. Still, it is nice to see the data, so to speak. Nice to see that if a society stops sending print issues to clog up faculty bookshelves collecting dust, nobody objects.

But……ego. Somehow I bet the existing societies would get their backs up a little bit if there was a suggestion that they simply give up their journal. Neuropolarbear asked what could be done about the assy position being taken by some publishers on the Research Works Act issue. This is the one trying to reverse the law demanding the deposit of all NIH funded papers into PubMedCentral (in peer-reviewed, accepted, manuscript form).

One thing we could do is to demand our society journals stop working with the jerky publishers.

This thought is what brought up all the above blathering. It is very likely that each and every small journal couldn’t make it on their own. Well, duh, of course not. As noted by the irrepressible Comrade Physioproffe

From what I understand, the other issue moneywise is that big publishers like Elsevier force institutions to pay subscription fees for shitteasse journals that no one reads by bundling them with their flagship journals. Those journals wouldn’t even exist if they had to survive on their own submission/publication fees.

But if all we’re talking about is a sort of virtual journal…why can’t some other umbrella journal publisher just kind of take up the slack? Why couldn’t a PLoS ONE type of outfit agree to provide all the publishing services and put some sort of tag on the article to group by academic society?

*christ that was priggish, wasn’t it?

**fascinating. In the case of Neuropsychopharmacology, the entire back catalog was transferred over to NPG so if you click on an article that your print copy insists was published by Elsevier, boom, you end up at NPG.

The Rock Talk blog of the NIH’s Office of Extramural research notes that a report (pdf) overviewing the responses to a prior RFI has just been released.

A few things that I’ve picked up on first skim.

Vote leveraging. They had 219 responses. That’s IT. 5% were from NIH staffe and 20% from official representatives so really that is only 164 comments from normal folk like us. If you responded (and I did) you get to be 0.6% of the response. Coolio. This is why I continue to urge you to participate in such things- apathy means that your efforts count for a lot more than just your own little opinion. You represent a much larger fraction.

Diversity. This particular RFI did not ask specifically about diversity to my recollection. But many people brought it up, no doubt because the Ginther et al 2011 report was published before comments were due. There were also lots of comments about the challenges faced by women who head laboratories.

Staff Scientists / nonPI stable careers. As you know, a topic on which I have opinions, DearReader.

Suppose something like this were made available for career Ph.D. scientists as essentially a fellowship. Without any requirement for a professorial appointment and minimal actual research component. The important point being that it is applied for, awarded to and evaluated for renewal by the career scientist with every expectation that this is a career award. There would be details of course. You’d have to have a host lab at most times- but allow for transition if one lab loses grant support or something. Nice and easy for the supported career scientist to find a new lab, don’t you think? “Hey, PI Smith, I have my salary supported and I’d like to come play in your lab…” would go over quite nicely. Progress could be evaluated just as with any other award, keeping the pressure on for the individual to publish

Well, this new report says:

There was much support among individual commenters to create permanent career staff scientists positions. They saw this as a way for all parties to reap the benefits of training support provided by NIH. Institution commenters where divided, some taking a cautious approach to the idea of utilizing staff scientist in the lab, citing possible adverse effects including potential loss of innovative ideas (currently provided by graduates) and the reduction in project budgets to cover the salaries for these positions.

and ends up with the “NIH Action Recommendation” of

Provide grant mechanisms and change the funding policy to increase project budgets to support the costs associated with permanent staff.

Emphasis added. I will be fascinated to see how closely the eventual initiative on this hews to my proposal. Fascinated.

I mean seriously. The original post over on @mbeisen’s It is NOT junk blog drew out a Elsevier flack Tom Reller who had a thing or two to say about the Research Works Act being peddled by New York CongressCritter Carolyn Maloney. In that last linked comment, Mr. Reller claimed:

When I say peer review process, I think you’re hearing just peer review. Peer review is exactly what you say it is, volunteer work in the name of academic science, and it never would have dawned upon me to have to say this, but of course we know peer reviewers aren’t paid. And we know that we, and our journals need them. If you ever reviewed for one of our journals, thank you, and we truly hope you continue to do so. I know all about the issues related to how difficult it is to find quality peer reviewers. You’re often paid by the government, and it is time consuming, we get it.

All we’re saying is that there is a peer review process, and there is more to peer review than just the peer reviewers themselves. Our systems track it, our publishers support it, then there’s what we do to print and preserve the outcome. Michael granted as much.

We call final published articles private sector information products because we (the private sector) have added value, as before mentioned, to the articles.

I find my self searching for this “added value”. By listening and reading what the publishers and their representatives have to say, perhaps we can find some clues to this mysterious value that they add to the scientific paper.

One Graham Taylor, flack for the UK Publishers Association, has an opinion bit up in the Guardian (in response to something written by Mike Taylor). Mr. Taylor opines:

We need a flow of accessible funds through the scholarly communication system to finance what we do. Hitherto these funds have flowed through academic library budgets, the “old” subscription model, which Dr Taylor describes as “a useful service in pre-internet days”. In future they will likely flow from research funding agencies (and a few charities and foundations) looking to enable open access.

Well, yes and no. We do need to continue to finance the distribution of scientific papers / scientific information. For sure. What we don’t need is Mr G. Taylor’s “we”. The publishers are a classic middle man, standing between the producer and consumer. Sometimes that brings value, sometimes it brings blood sucking leechery that adds cost but no value to the system. The real question at hand is determining whether the traditional publishers are starting to veer from “adds value” into “leeching off the system”.

Publishers pursue the goal of universal access through whatever means are practically available.

HAHAHAHAH! Where “practically available” means “We, the publishing middle man get paid.” Right? If their goal was universal access they’d be whittling down their profits to the unmentionable, cutting salaries down toward the public sector salaries, etc. Ever seen the publisher representatives in a typical editorial board meeting? Yeah, you can pick them out by the nice clothes. Ahem.

It is widely acknowledged that there is not an access problem for researchers based in universities, research institutes or the corporate sector.

Absolutely and completely false. The smaller institutions, colleges, etc do not have anywhere near the access to the breadth of journals available. And oftentimes they are forced to make tough choices because of journal “bundle” options offered by, e.g., Elsevier. Companies large and small have to decide how to fund securing the literature- some do it on a one-off basis because it is cheaper. Many folks at smaller biotechs hose off the local University in one way or another- through their spouses employment, through their old access from when they were a postdoc, friends..or just stopping by the University campus and using the wifi. Either way, people are having to cobble together their “access” to the entirety of the academic literature. With variable success.

Pub Med Central changed all that. All the subsequent research was available to anyone, from community college to small private college to biotech startup. All right there. Easy, peasy from the desktop.

Public funds have not paid for the peer-reviewed articles that are based on research supported by agencies such as the National Institutes of Health (NIH). They have only paid for the research itself and whatever reports the researchers are required to submit to the agency.

Another falsehood, wrapped up with a disingenuous misdirecting belittlement. “Only” for the research? ONLY????? These publishers seem universally unaware that the VAST, VAST majority of the value of an academic article is the bloody research. The damn content. That is what has value. The fancy layout? That’s nice and all but we can do without that. The science is the thing. Trying to dismiss this as a minor contribution is…well…..that takes some serious chutzpah.

There is another, more serious misdirection here. The public does in fact “pay for” the entire article at present. By way of library access fees and the other ways that articles are obtained. Money is fungible so one way or another, the public contribution in NIH grants, public University support, student tuition and probably even the health care system (hospitals need access, yo) pays for these articles.

Public funds are already paying the middle man “for the peer reviewed article”. The question is, once again, can we make this process more efficient (yet equally effective in distributing science) by eliminating the middle man?

The journal article based on the research has been the subject of significant extra investment that must somehow be recovered if scholarly communication as we know it is to survive.

I am still trying to get my head around this claim because it is the cornerstone of the publishers’ argument. What “significant extra investment”? The reviewers themselves work for free*. For real journals, with academic, real-scientist editorial staff, the editors are maybe paid a little and the administrative person who works in the Editor in Chief’s office gets paid as well. Sure. But this is penny ante stuff. There are not 5 to 10 full time people with the sole job of being editor. Suppose the NIH decided to run it’s own journals…think they couldn’t just take over the salary lines? Of course they could.

This brings me to the online peer reviewing software systems, such as ScholarOne Manuscripts or Editorial Manager. Sure, maybe some publishers have developed their products in house. But there are lots of options. These are not functions that we require the big publishers to provide either.

So what do the publisher’s add to the “peer review” of journal articles? I really don’t know. Keep in mind that I categorically reject the notion that we need GlamourMag style professional editors to get this job done.

It often appears to me that their only unique role in this process, given modern internet technologies and tools, is to figure out ways to keep sucking money out of the system, but providing “value” that we really don’t need.

many do object to a mandate that appropriates their material without compensation.

Now this is just gobsmacking. Since the vast majority of their “material” consists of the academic research, writing and peer reviewing services that they get for FREE, it is really frigging rich for the publishers to cry about appropriating material without compensation. I mean do they even listen to themselves? Ever?

Finally, the dinosaur sinking in the tar pit part:

Such journals are on the whole by their very nature tailored and adapted to the needs and interests of specific research communities. This is a complex and nuanced system that needs time to adapt to new methodologies.

Dedicated journals are an anachronism held over from before the advent of PubMed, pure and simple. Partitioning scientific research into field journals of different emphasis and focus was necessary for humans to find the most relevant work. Now we have search engines and online databases. The function served by subspecialization of print journals is entirely unnecessary at present. If you need it for organizational means in the (virtual) editorial offices of a general “Journal of the NIH” or “Journal of all US Science funded by the Taxpayers”, there is no reason that we couldn’t have subsections partitioned off.

Dr [M] Taylor’s assumption that this can somehow all be routinely accommodated on a “service” basis is to misunderstand the nature of publishing. Publishers invest at their own risk and quality standards are essential to manage that risk. We need a market to organise such a high volume of transactions. Take that away and we would be left with a Stalinist nightmare.

No, you misunderstand that we don’t give a rat’s patootie about “the nature of publishing”. We care about the distribution of scientific results! Print or even online “Publishing” as an industry is irrelevant. We just need the lower-case-p version of publishing. We do NOT need a “market” to organize any such thing. Does the “market” get involved in the peer review of NIH grants? No? Then why do we need it to publish the results? We do not.

And dude. Seriously. The “Stalinist nightmare” bit? Get a grip.

*UPDATE: Neuromusic emphasized a very important point in a comment. Peer reviewers work at no charge to the publisher. Their time is not “free” however, since this is considered to be part and parcel of their job. Whether it be “service” of a hard money faculty member or as part of time allocated on grant awards, it is expected that scientists will contribute to the review of manuscripts. This raises an interesting point. Consulting fees in industry vary, of course, but for a fairly new Assistant Professor the expected rate in my fields of interest is about $100 / hr or $1,000 / day. If publishers are going to act graspy about their “compensation” then they are going to have to start budgeting at least a couple of hundred smacks per reviewer for each manuscript. Hey, look at the plus side- going to be a lot easier to get papers reviewed if the invitations come with a $200 carrot.

Commenter Adriana suggests something very interesting on a prior post addressing NIH grant funding policies, the change to only a single revision, etc.

Some of this seems counter productive. People that have spent maybe 15 or so years setting up model systems and generating hypotheses (paid for by NIH) are getting cut off right at the time their projects are starting to produce high yields. In a lot of cases the funds are diverted to new investigators with worse priority scores to set up more new models systems. Just when they get going they will probably get cut off too.

My first reaction was “whoa!” Bad optics, homes! Telling the grant funders that “well, give me 15 years of full funding and then see what I really produce!” sounds a bit like vapor ware. A confidence racket of some sort.

But I’ve also mentioned something about the trendlines of scientific careers. And sure, I’ve often seen PIs experience a huge surge in productivity during the middle career years after an initial interval of struggle. The grants come in, the postdocs come buzzing around and all of a sudden the lab is a serious player. It can take 5-8 years, easy. Several of my most salient examples match this sort of timeline. Fifteen years though? Fifteen years from the start of the lab to the acceleration phase?

That is a little more unusual to me. I”m not saying there are not people that have ups and downs throughout their careers, sure there are. I know plenty. So a downturn in year 10 is not necessarily predicting a lasting decline and yes, when grants come back the lab starts roaring again. But usually, this is not from a lab that has been in the doldrums for the prior 10 year interval. Usually, from what I see, that lab has had prior indications of the potential.

Let us leave specific predictors of timeline aside for a moment however. It doesn’t really matter if we’re talking early-mid or mid-mid career, some of the implications of the current NIH policy-making still apply.

I think I’ve tried a time or two to get into the “cry me a river” post for people like…..well, me. I don’t feel sorry for me. Okay, that’s a lie, I do feel very selfishly sorry for myself if I cannot obtain the grant funding that I desire and think that I, in some way, deserve.

“I saw the generation just prior to me hit the sigmoidal acceleration phase when I was a late postdoc and early faculty member. That is my model of expected value in the NIH-funded career path. If I do as well as they did in the early phase, dammit, I deserve my chance to shine. I’ve struggled to build not just a lab, not just a few projects but a scientific program. And now, just as I’m getting to the point where things should be relatively comfortable and highly productive..this. The NIH budget goes south, those extra grants are harder to come by, the renewals much, much less certain and I’m still struggling. Dammit.”

Sound familiar my mid-career friends?

Of course, our “struggle” much of the time is not the struggle just to stay alive. We have a grant or two to last us through the bad times. When things get really rough, heck, we can finally get around to writing up old papers to keep productivity smooth. We have endless amounts of technical preliminary data on which to base new applications, maybe even some pilot data that supports a hypothesis. We know a lot of folks on study sections personally and some of them are, at last, kinda junior to us. Junior enough that they might, gasp, actually see our applications as coming from “a well established and accomplished scientist”.

Things could be worse.

This is why my sympathies for people like me continue to be muted when it comes to different policy points over there in NIH land. Yes, I would for sure like to see the next generation ahead of me suffering the same conditions that I seem to experience. See the old guard be treated as harshly as everyone else in the granting system. You know what? It may be happening, much as we think the grass is greener. How will we know until after the fact? After all the data emerge in 8-15 years showing what happened in this time of stress.

Will the NIH system really experience investigator dropout in serious numbers? Will we come to find that the major effect was on the amount of funding over 10yrs of PI time, but not on the number of total investigators? Will we find that my generation, the mid-career types, pay a specific toll….as is being hypothesized by Adriana?

I will say that the NIH ignores the winy, selfish complaints of the current crop of mid-career investigators at their peril. These are the very people who are putting in the long hours on study sections reviewing grants. If they feel that ESI folks are getting too much in the way of assistance, regardless of whether they are or not on some objective measure, they are going to push back. And continue to punish ESI applications. Subconsciously perhaps, but it will be there. Actually, it has always been there.

Mid-career folks have probably always been tremendously to blame for the fact that every initiative of the NIH to help the newly independent secure grant funding has fallen short. Because NIH is in a reinforcing cycle of pecking order.

Driven by the nearly inescapable, hindbrain driven, emotional, fierce belief of mid career scientists that dammit, they deserve their chance for an explosive phase in their labs.

I don’t have any answers for the NIH.

I feel this way about my own laboratory and research program.

I bet a million dollars you do too.

And help keep his attention on this possible throw-away from the State of the Union address.

I was heartened by several observations from the US President that seemed to suggest he understands that investment in basic research (no, not just targeted development) was the key to sustained economic growth into the future. But you need to help keep him on task. And get Congress on board.

I noted a few months ago that a petition has been launched to collect signatures favoring minor increases in funding for the NIH. There are 3,931 people as of this writing.

Are you one of them? Have you passed the link around your lab, department or Uni? How about to your academic socities? Have you posted it on your Facebook and Twitter feeds?

Please do.