JAMA discourages pre-print deposition

An alert from the Twitters points us to the author guidelines for JAMA. It appears to be the instructions for all JAMA titles even though the below tweet refers specifically to JAMA Psychiatry.

Thread revival! Submitting to JAMA psychiatry and they discourage preprints and explicitly factor into accept/reject decision pic.twitter.com/Odv2RLyfja

— Claire Gillan (@clairegillanTCD) April 25, 2019

The critical bit:

Public dissemination of manuscripts prior to, simultaneous with, or following submission to this journal, such as posting the manuscript on preprint servers or other repositories, is discouraged, and will be considered in the evaluation of  manuscripts submitted for possible publication in this journal. The evaluation will involve making a determination of whether publication of the submitted manuscript will add meaningful new information to the medical literature or will be redundant with information already disseminated with the posting of the preprint. Authors should provide information about any preprint postings, including copies of the posted manuscript and a link to it, at the time of submission of the manuscript to this journal

JAMA is not keen on pre-prints. They threaten that they will not accept manuscripts for publication on the basis that it is now “redundant” with information “already disseminated”. The require that you send them ” Copies of all related or similar manuscripts and reports (ie, those containing substantially similar content or using the same or similar data) that have been previously published or posted, or are under consideration elsewhere must be provided at the time of manuscript submission “.

Good luck with that JAMA. I predict that they will fight the good fight for a little while but will eventually cave. As you know, I think that the NIH policy encouraging pre-prints was the watershed moment. The fight is over. It’s down to cleaning out pockets of doomed resistance. Nothing left but the shouting. Etc. NIH grant success is a huge driver of the behavior of USA biomedical researchers. Anything that is perceived as enhancing the chances of a grant being awarded is very likely to be adopted by majorities of applicants. This is particularly important to newer applicants because they have fewer pubs to rely upon and the need is urgent to show that they are making progress in their newly independent careers. So the noobs are establishing a pre-print habit/tradition for themselves that is likely to carry forward in their careers. It’s OVER. Preprints are here and will stay.

My prediction is that authors will start avoiding peer-reviewed publication venues that discourage (this JAMA policy is more like ‘prevent’) pre-print communication. The more prestigious journals can probably hold out for a little while. If authors perceive a potential career benefit by being accepted in a JAMA title that is higher than the value of pre-printing, fine, they may hesitate until they’ve gotten rejected. My bet is that on the main, we will evolve to a point where authors won’t put up with this. There are too may reasons, including establishment of scientific priority, that will push authors away from journals which oppose pre-prints.

Anyone else know of other publishers or journal titles that are aggressively opposing pre-prints?

Preprints and NIH Study Section Behavior

An interesting pre-print discussion emerged on Twitter today in the wake of an observation

Reality ✔️. Discovered this week while chairing NIH panel that 50% of panel had no idea what @biorxivpreprint is. Others were skeptical. @cziscience @mbeisen @pollyp1 @superkash and others – We need to work on this. Any ideas?

— Holly Ingraham (@hollyingrahamSF) March 10, 2019

that members of study sections apparently are not up to speed on the NIH policy encouraging the use of pre-prints and permitting them to be cited in NIH grant applications. The relevant Notice [NOT-OD-17-050] was issued in March of 2017 and it is long past time for most reviewers to be aware of what pre-prints are, what they are not and to understand that NIH has issued the above referenced Notice.

Now, the ensuring Twitscussion diverted off into several related topics but the part I find worth addressing is a tone that suggests that not only should NIH grant reviewers understand what a pre-print it, but that they should view them in some particular way. Typically this is expressed as outrage that reviewers do not view pre-prints favorably and essentially just like a published paper. On this I do not agree and will push a different agenda. NIH reviewers were not told how to view pre-prints in the context of grant review by the NIH as far as I know. Or, to the extent the NIH issued instructions, it was to essentially put pre-prints down below peer reviewed work.

The NIH agrees that interim research products offer lower quality information than peer-reviewed products. This policy is not intended to replace peer-review, nor peer-reviewed journals …

Further, the NIH is instructing awardees to explicitly state in preprints text that the work is not peer-reviewed.  These two practices should help reviewers easily identify interim products.  The NIH will offer explicit guidance to reviewers reminding them that interim research products are not peer-reviewed. Further, since interim products are new to so many biomedical disciplines, the NIH hopes that these conventions will become the norm for all interim products, and will help the media and the public understand that interim products have undergone less review than peer-reviewed articles.

https://grants.nih.gov/grants/guide/notice-files/NOT-OD-17-050.html

Given this, I would suggest that NIH reviewers are quite free to discount pre-prints entirely, to view them as preliminary data (and be grumpy about this as an effort to evade the page limits of the NIH application)…..or to treat them as fully equivalent to a peer reviewed paper because they disagree with the NIH’s tone / take on this. Reviewers get to decided. And as is typical, if reviewers on the same panel disagree they are free to hammer this disagreement out during the Discussion of applications.

I believe that pre-print fans should understand that they have to advocate and discuss their views on pre-prints and also understand that merely whinge about how reviewers must be violating the integrity of review or some such if they do not agree with the most fervent pre-print fans is not helpful. We advocate first by using pre-prints with regularity ourselves. We advocate next by taking advantage of the NIH policy and citing our pre-prints in our grant applications, identified as such. Then, if we happen to be invited to serve on study sections we can access a more direct lever, the Discussion of proposals. (Actually, just writing something in the critique about how it is admirable would be helpful as well. NIH seems to suggest in their Notice that perhaps this would go under Rigor.)

Preprint Ratio

We’re approximately one year into the NIH policy that encouraged deposition of manuscripts into preprint servers. My perception is that the number of labs taking the time to do so is steadily increasing.

It is rather slow compared to what I would have expected, going by the grant applications I have reviewed in the past year.

Nevertheless, preprint deposition is getting popular enough that the secondary questions are worth discussing.

How many preprints are too many?

Meaning, is there a ratio of preprints to now-published-in-Journalofology preprints that is of concern?

It is sort of like the way I once viewed listing conference abstracts on a CV. It’s all good if you can see a natural progression leading up to eventual publication of a paper. If there are a lot of conference presentations that never led to papers then this seems….worrisome?

So I’ve been thinking about how preprints may be similar. If one has a ton of preprints that never ever seen to get published, this may be an indication of certain traits. Bad traits. Inability to close type of traits.

So I have been thinking that one of the things guiding my preprint behavior is how many my lab has at a given time that have not advanced to publication yet. And maybe there are times when waiting to upload more preprints is advisable.

Thoughts, Dear Reader?

Responding to comments on pre-prints

almost tenured PI raised some interesting questions in a comment:

So you want me to submit a preprint so I can get comments that I have to spend time responding to? No thanks. I spend enough time responding to the comments of reviewers from real journals. I can’t imagine how much time I’d have to spend responding to comments that are public and immortalized online. No paper is perfect. How many comments does a paper need before it’s acceptable for publication? Where does it end? I do not need more work when it already feels like the bar to publishing a paper keeps rising and rising.

Ok, so first off we need to recognize that the null hypothesis right now has to be that there will not be extensive substantial commentary on the preprints. PubMed Central shut down its commenting scheme for lack of use. Journals have been trying out various systems for years (a decade or more?) without general success. The fear that each posted pre-print will attract a host of journal-club style comments is probably not well supported.

But lets suppose your preprint does get some critical comment*. Are you obliged to respond?

This ties into the uncertainty and disagreement over when to submit the preprint. At what stage of the process do you post it? Looking at the offerings on bioRxiv, I think many folks are very close to my position. Namely, we are waiting to submit a preprint until it is ready to go out for peer review at a journal.

So any comments it gets are being made in parallel with review and I can choose to address or not when I get the original decision back and need to revise the manuscript. Would the comments then somehow contaminate the primary review? Would the reviewers of a revision see the comments on the pre-print and demand you address those as well as the regular peer comments? Would an Editor? For now I seriously doubt this is a problem.

So, while I think there may be many reasons for people not to want to post their manuscripts as pre-prints, I don’t think the fear that this will be an extra dose of Reviewer #3 is well supported.

__
*I had one get a comment and we ended up including something in a revision to address it so, win-win.