Priority

February 19, 2016

I am working up a serious antipathy to the notion of scientific priority, spurred most recently by the #ASAPbio conference and the associated fervent promotion of pre-print deposit of scientific manuscripts.

In science, the concept of priority refers to the fact that we think of the first person to [think up, discover, demonstrate, support, prove, find, establish] something as somehow special and deserving of credit.

For example, the first paleontologist to show that this odd collection of fossils over here belonged to a species of Megatyrannoteethdeath* not previously known to us gets a lot of street cred for a new discovery.

Watson and Crick, similarly, are famed for working out the double helical structure of DNA** because they provided the scientific community with convincing amounts of rationale and evidence first.

Etc.

Typically the most special thing about the scientists being respected is that they got there first. Someone else could have stumbled across the right bits of fossil. Many someones were hotly trying to determine how DNA was structured and how it worked.

This is the case for much of modern bioscience. There are typically many someones that have at least thought about a given issue, problem or puzzle. Many who have spent more than just a tiny bit of thought on it. Sometimes multiple scientists (or scientific groups, typically) are independently working on a given idea, concept, biological system, puzzle or whathaveyou.

As in much of life, to the victor go the spoils. Meaning the Nobel prize in some cases. Meaning critical grant funding in other cases- funding that not only pays the salary of the scientists with priority but that goes to support their pursuit of other “first” discoveries. Remember in the Jurassic Park movies how the sober paleontology work was so desperately in need of research funds? That. In addition, the priority of a finding might dictate which junior scientists get Professorial rank jobs, the all-important credit for publication in a desired rank of scientific journal and ultimately the incremental accumulation of citations to that paper. Finally, if there ends up being a commercial value angle, the ones who have this priority may profit from that fact.

It’s all very American, right? Get there first, do something someone else has not done and you should profit from that accomplishment. yeeehaw***.

Problem is……****

The pursuit of priority holds back the progress of science in many ways. It keeps people from working on a topic because they figure that some other lab is way ahead of them and will beat them to the punch (science always can use a different take, no two labs come up with the exact same constellation of evidence). It unfairly keeps people from being able to get rewarded for their work (in a multi-year, multi-person, expensive pursuit of the same thing does it make sense that a 2 week difference in when a manuscript is submitted is all-critical to the credit?). It keeps people from collaborating or sharing their ideas lest someone else swoop in and score the credit by publishing first. It can fuel the inability to replicate findings (what if the group with priority was wrong and nobody else bothered to put the effort in because they couldn’t get enough credit?).

These are the things I am pondering as we rush forward with the idea that pre-publication manuscripts should be publicized in a pre-print archive. One of the universally promoted reasons for this need is, in fact, scientific priority. Which has a very, very large downside to it.
__
*I made that Genus up but if anyone wants to use it, feel free

**no, not for being dicks. that came later.

***NSFW

****NSFW

45 Responses to “Priority”

  1. AcademicLurker Says:

    The DNA story is an interesting case because, if I’m remembering correctly, the model was pretty under constrained* when W&C published it. It was 10 more years of work before it was considered “established”. It worked out OK for W&C, but had things gone differently, they might have embarrassed themselves by rushing into print early.

    *From reading Rosalind Franklin’s biography, it seems that she was opposed to under constrained model building in general, and when she was shown W&C’s structure her reaction was basically “It’s a nice model. Now how are they going to prove it?”

    Like

  2. Eskimo Says:

    DM, I don’t want to get all McCarthyist on you, but– the race for priority is just a part of competition and scarcity, which you seem comfortable with otherwise. What’s the alternative: central planning from above, and everybody get done with their projects whenever?

    Like

  3. banditokat Says:

    I like gin.

    Like

  4. drugmonkey Says:

    What makes you think I am comfortable with competition and scarcity in science?

    Like

  5. Eskimo Says:

    Because you explain them so well. Perhaps I am simply saying that you are not naive.

    Like

  6. dr24hours Says:

    The problem with despising scientific priority is that you’re actually despising a basic tenant of human nature: the desire for recognition.

    Good luck with that.

    Like

  7. drugmonkey Says:

    Because you explain them so well.

    It is always good to remember that advice on navigating systems as we find them is not the same as endorsing the operation of those systems.

    Good luck with that.

    It is like general socioeconomic disparity in the US. Saying that the gap between rich and poor is too extreme and should be closed is NOT in any way saying we need to go all harrison bergeron, despite what it is in the interest of the Oligarcy for you to believe. Similarly, we can reel back the excesses of the scientific priority chase while still motivating people to achieve.

    Like

  8. drugmonkey Says:

    Another way to put this is that it should be possible to survive and do a good job as an academic scientist without everyone trying to get the next Nobel prize.

    Like

  9. dr mho Says:

    you’re right- everyone should be special so no one is special….

    Like

  10. poke Says:

    Gosh. It’s hard for me to fathom what about pre-prints you find so threatening.

    Like

  11. Pleb Says:

    Why would a preprint server be of use to biologists? I agree with DM’s concerns, however am also perplexed by the reasoning in favor of such servers in the first place. May I ask, do physicists actually put ArXiv citations in their CVs, and do TT physics/maths position search committees consider pre-print papers as something of sufficient weight to warrant a job interview for a particular candidate over another one with an actual published paper? Publications in journals are the currency by which we’re all measured, rightly and wrongly and inclusive of all the problems with peer review and glam hopping, and so what use is it to have a paper in pre-pub? Is there an honorary period I have to wait before submitting to a journal? Fantastic, so add that time to the time it takes to get the paper accepted in an actual journal, and we’re looking at a process that is even more agonizing and drawn out then before. How does this help anyone? What if I’m working on the same thing as a competitor and I submit to a pre-pub, but they skip and go strait for the journal, and get the “publication” first? Who gets the real credit then?

    Like

  12. jmz4 Says:

    It’s part of the broader problem of how we think of science as the result of genius instead of sustained collaborative effort.
    So while I grant that our egos are a powerful motivating source, I don’t think funding agencies should encourage them by dangling awards for supposed novelty in our faces.
    I mean, it says it right there in the NIH applications. Explain how your work will be unique, special and paradigm-shifting. All this type of thinking encourages is nepotism and cannibalism.

    Like

  13. Rheophile Says:

    Pleb: I’m a (bio-)physicist, so let me try to take your questions one by one.

    Q: Why would a preprint server be of use to biologists?
    A: The same reason conference talks are useful – it gives you the ability to publicize your work, get feedback, and stake out turf. Allows you to communicate science faster. On a selfish level, it means you can turn a “submitted” line on your CV into “submitted, and you can check it yourself, and I stand by this publicly” – which is a heck of a lot stronger. And you can accumulate citations.

    Q:Do physicists actually put ArXiv citations in their CVs?
    A: Yes, absolutely.

    Q: Do search committees take this seriously?
    A: Sure – it shows productivity, and allows search committees to judge the work independently if they want. Obviously it’s not as good as a glam paper, but it helped me this year – I had two papers that hadn’t been accepted, but were accessible to search committees and on my CV. Plus,those papers now have a handful of citations even before they’re published – showing people care about the work.

    Q: Is there an honorary period I have to wait before submitting to a journal?
    A: I usually submit both at the same time. Other people submit the “preprint” on acceptance as a way to get the result out there to people without journal access.

    Q: What happens when one group preprints and another doesn’t?
    A: Because preprints are well-read, having the preprint out means people think about your result, cite it, and credit it. Often, if one group has a paper in submission but no preprint and a new result comes out, they will put out a preprint themselves. Credit accumulation is already fuzzy (big name vs little name, first submitted vs first published), so preprints don’t make things that much worse.

    Caveats: some of this is a lot more important for theoretical work, because you can see paper, response papers, papers building on new ideas, etc, all in a few months – faster than a typical referee cycle at some journals. And in physics, 100% of important journals I know take preprinted articles. Cell Press is doing a lot of damage with its vague policy about this.

    Like

  14. beerbrains Says:

    two things:

    I’m uncomfortable with general hatred of priority, because I do think competition drives progress*. I suppose it’s possible to imagine a plato’ian ideal of academia, where there is just enough competition, but also collegiality, collaboration, shared credit, etc. While this is certainly something to strive for, I think I’d rather have the competition, negatives and all, than a big rainbow unicorn party.

    Regarding pre print servers, I have general apathy — I don’t think it will destroy, nor will it revolutionize science. *Shrug*. That having been said, I follow deep learning research, where articles are often published on ArXive before a real journal. I can think of many occasions where I purposefully chose to read and cite the arXive version, not the journal version, because the arXive version is way more useful. I mean, a paper that actually has details, instead of the high level, over edited, glam description, is really nice to have available. In some cases I have found out about the paper bc of a glam publication, and then read the arxive version to get more details….
    So I do appreciate author formatted freely available versions of papers, I guess this way of using preprint journals is actually somewhat similar to what vosshall et. al. are championing. Other than the pppr business, that is.

    *caveat: I like competition. Like, I myself personally, am very motivated by competition. Maybe this makes me a bad scientist, but the constant competition is actually one of the things that draws me to stay in academia.

    Like

  15. Jonathan Badger Says:

    @Pleb
    I’ve worked in several different fields of science — as a traditional microbiologist in grad school, as a postdoc in a computer science department, as a computational genomicist at a soft money institute, and now as a computational biologist in the intramural NCI program. They all have very different ways of doing science, and things like preprints, alphabetical author lists, peer reviewed conference proceedings, and publishing books and monographs that are rare in some fields are completely normal in others. I think the problem is people who have only worked in one field think the rules of that field are universal. It’s like people who have only lived in one country.

    Like

  16. Newbie PI Says:

    Here is why I am NOT going to submit preprints unless it becomes a standard custom in my field (I just did a search and there is not even one preprint available in my area of research):

    1. It’s extra work. I would have to submit to a standard journal and to a preprint server, then field comments from anyone in the world who wants to review my preprint, plus respond to journal reviewer critiques. Yet the ultimate outcome is still the same – publication in a standard journal.

    2. Only peer-reviewed publications count in my promotions and tenure dossier. I can already list submitted publications in this document, so I doubt that a list of preprints would be more compelling than the list of papers submitted to a peer-reviewed journal.

    3. Someone above mentioned that preprints get citations, and may get citations even after the traditionally published version comes out. This a big problem for those of us who worry about h-index. H-index is a tenure criteria at my university. The expected h-index at each faculty level is clearly defined. Splitting my citations between a preprint and standard journal article would be terrible for this particular metric.

    4. I work in a basic science field. An extra couple of months between submission and publication of our work is not really a big deal, certainly not a life and death situation.

    5. I already submit our newest work in conference abstracts. I submit unpublished stuff because it means that I’m more likely to get a talk at the meeting. So the basic ideas are out there, without my competitors having access to our exact experiments before the work is ready for peer-reviewed submission.

    6. Something about the uber-famous people that are pushing this just doesn’t sit well with me. Maybe it will help them stake their claim faster, but it feels disingenuous that they are framing this as a revolution for science.

    Like


  17. > It’s extra work. I would have to submit to a standard journal and to a preprint server, then field comments from anyone in the world who wants to review my preprint, plus respond to journal reviewer critiques. Yet the ultimate outcome is still the same – publication in a standard journal.

    You don’t *have to* respond to comments from people who read your pre-print any more than you do comments from people who read your published journal paper. But the idea that one can ignore all comments on scientific work other than those from a small number of random anonymous reviewers is really harmful to science.

    Like

  18. mH Says:

    The basic outcome of the meeting discussion was that preprints will change “priority” discussions very little if at all (as much as the Cell Press rep wanted everyone to believe it was all about nasty little scoopers).

    Priority and credit in science, proportionate to how much money or fame is tied to the particular line of work, is a retroactive, cumulative, historical, negotiated, fraught game of Rashomon. No one is going to give a particular shit about dates in a preprint archive. It will remain the same messy arguing and history revising and influence peddling and politicking and review writing game that it has been for 500 years.

    Like

  19. drugmonkey Says:

    I would be happy to share peer review comments with you that focus on novelty of results. And impact which can be related to the degree of novel/replication. Preprints will be fuel for the fires of people who seem to think that absolute uniquity is required for every publication.

    Like


  20. […] I’m Excited! A Post Pre-Print-Posting-Powwow Post Common Antibiotics May Cause Delirium, Confusion And Hallucinations Priority […]

    Like

  21. Grumpy Says:

    As a physicist who routinely uses arxiv, I find all of the resistance to using preprint server really silly.

    The preprint server gives your paper more visibility and allows those at institutions without extensive journal subscriptions access. It also lets ppl at institutions with extensive subscriptions but who are too lazy to login to their library proxy have access. This is in addition to the benefit of spreading the word about your results earlier than if you waited til peer review. Not everyone in physics chooses to post before their paper has been accepted…Im not a fan of the wait-until-acceptance strategy but it is certainly better than never posting at all.

    The comment about it being extra work is absurd: you spend several people years on a quality study and you want to complain about a couple of hours of formatting to give your publication broader access and visibility?

    And for some of you, I think you are letting your dislike of some famous ppl who recently started pushing preprints cloud your judgment. Riffraff like myself and numerous other commenters on your blog have been suggesting this for years.

    Like

  22. datahound Says:

    Major takeaways for me as a participant in the ASAPBio meeting were that (1) ArXiv was started in large part as a way of democratizing communication in physics wherein everyone would have access to new ideas at the same time regarding of connections and participation/invitation to key meetings; (2) promotion committees etc. in physics, math, and some social science fields do use preprint depositions in sensible ways as noted by Rheophile; (3) priority discussions do occur in different contexts but preprint depositions are just one bit of data taken into consideration; (4) there remain some practical and many cultural issues to deal with in biology/biomedicine with respect to preprint servers but the benefits seem to me to very likely to greatly exceed the costs and problems.

    Other fields have many years of experience with preprint servers and we would do well to understand this experience rather than assuming that our instincts and preconceived notions outweigh the data. There are differences between fields but these a largely cultural and potentially changeable rather than intrinsic.

    One of my favorite comments from Paul Ginsparg who started arXiv in 1991 was that an early criticism was that it was too much to expect that most physicists could learn how to use an internet browser…

    Like

  23. drugmonkey Says:

    So…. “Trust us”?

    Like

  24. Newbie PI Says:

    Leslie Voshall has tweeted that the “most important voice” for preprints is a new 3rd year professor named James Fraser. The reason that he’s so important: he’s from UCSF and publishes glam papers!

    Yes, please create more work for all of us who publish in society journals with reasonable turnaround so that Leslie and James can lay their claim within their field while shmoozing Nature editors for the next two years.

    Like

  25. Jonathan Badger Says:

    @Grumpy
    That’s what I don’t get about a lot of anti-preprint, anti-OA and anti-anti-Glam criticism. Originally the typical argument was that people who wanted to change things were losers who couldn’t compete in the “normal” (by biomedical standards) scientific process. But when big names got involved suddenly the argument turns into “well, *of course* those elites want to change the rules — *they’ve* already proven themselves”. Classic special pleading.

    Like

  26. qaz Says:

    I have a question for all the pro-preprint arguers. How do you propose to handle the multiple-version problem? Papers change through peer review. (This is the purpose of peer-review. And for all the BS about “my paper didn’t change”, most papers change, sometimes in minor ways, sometimes dramatically, in peer review.) Furthermore, papers change through editing. (Sometimes because journals have these things called editors, who sometimes suggest language changes. And sometimes because authors catch errors when proof-reading.) So what happens when you put your paper in the preprint archive and then what gets published later is different?

    – Are preprints linked with final publications? How is this accomplished without adding an additional burden to authors. (Yet another database to keep up to date.)

    – How are you going to get journals to permit the final version into the archive? (We already have this fight with PubMedCentral, which is only legally allowed to have the final accepted version – not the edited or proof-read version.)

    Like

  27. AcademicLurker Says:

    @qaz: On the physics arxiv, the manuscripts are updated.

    Like

  28. my one cent Says:

    qaz,

    For arXiv, the author can update their article at any time. Past versions are preserved but the most up-to-date one is the version that is shown to the readers first. In practice this is not much work since the author just uploads a new file, and the system handles the indexing. Also, adding a connection to the final article is done by filling out a comment box, which is not much work. Keeping the article up to date is ultimately the author’s responsibility, of course, but I’ve found posting to arXiv is not much more work than posting something on a personal website.

    The issue of permissions is slightly trickier, and depends on the specific journal’s policies. Most journals do not want the final formatted article posted on arXiv, but for some you can modify the preprint to fix typos or correct errors caught in review.

    Like

  29. Jonathan Badger Says:

    So what happens when you put your paper in the preprint archive and then what gets published later is different?

    That’s actually the *idea*, qaz. The published paper is *supposed* to be different (and better) than the preprint based on comments. Yes, this depends on the field culture accepting that commenting on preprints is a worthwhile activity (which doesn’t seem to be the case so far in much of biology)

    Are preprints linked with final publications? How is this accomplished without adding an additional burden to authors.

    Typically yes. How hard is it to include a fricking link? Anybody dealing with sequence data has dealt with accession numbers for decades. It’s not that hard.

    How are you going to get journals to permit the final version into the archive? (We already have this fight with PubMedCentral, which is only legally allowed to have the final accepted version – not the edited or proof-read version.)

    You just force them to by ignoring their rules. The weird kowtowing to the whims of publishers (which stops some biologists from even self-archiving the final PDFs of their papers on their web sites via an”internal policeman” in their heads) needs to stop. Publishers aren’t in a great negotiating position these days. If they want to survive, *they* need to change.

    Like

  30. drugmonkey Says:

    JB- who was making that “losers who can’t compete” argument? Hmmmm?

    Like

  31. Jonathan Badger Says:

    Think back to 2005 or so when OA was a new thing. Basically every article about OA either in part or in whole promoted the idea that traditional publishing were the gatekeepers of science and that the OA movement was going to destroy science by allowing the publication of low quality things that would never get published before. Even today, you still read things like that in places like “Scholarly Kitchen” and other blogs connected to the publishing industry.

    Like

  32. drugmonkey Says:

    Yes. And which demographics of science were making the predictions?

    Like

  33. mH Says:

    qaz – you can’t update on bioRxiv, I think. the idea is to have versioning. forward & backward links. the preprint people and journal publishers all want this, so they will do it. bioRxiv already links when they can (semi-automated, but imperfect).

    Thanks for the sanity, Grumpy and Datahound. The teacup rattling has gotten kinda out of control.

    Like

  34. mH Says:

    I remember when I lived in Singapore they opened their first metro line that would have driverless trains. It took a few years to build.

    About a week before it opened, there was a concerned letter in the newspaper: “Without a driver, how will the train stop? I can’t believe no one’s thought of this!”

    The moral of the story: arXiv has been around forever and there is lots of information about how it works and how it is used by the physics community available. There are whitepapers and perspectives at http://asapbio.org/ about plans, possibilities, where this could go, and where no one really seems interested in it going. 95% of the reactionary responses to the idea of using preprints are addressed there. It makes me wonder how much people are actually interested in this topic and how much they want to just lash out at what a person (or type of person) they don’t like happens to be investing energy into.

    There are concerns. But it’s just not that big a deal.

    Like

  35. drugmonkey Says:

    It makes me wonder how much people are actually interested in this topic and how much they want to just lash out at what a person (or type of person) they don’t like happens to be investing energy into.

    ….speaking of ad hominem attacks.

    Like

  36. mH Says:

    You’re welcome.

    Like

  37. Jonathan Badger Says:

    @DM
    “demographics”? In what sense?

    Like

  38. drugmonkey Says:

    Your comment appears to conflate two entirely opposed sides, JB.

    Like

  39. Dusanbe Says:

    I’m starting a glam preprint repository called “Preternature”. We will reject 99% of all pr33ps sent our way. It’ll be like “Nature”, but with no peer-review and no revisions to get in the way of all the big, swinging, vertically ascending awesomeness.

    Like

  40. Grumpy Says:

    DM,
    You can nitpick at JB and other commemters all you want. But this all seems like a whole lot of wasted breath just to avoid posting your hard-earned papers on a free, well-organized, open access server.

    Like

  41. qaz Says:

    Grumpy – When a free, well-organized, open-access server appears that does not increase my time beyond the checking I already have to do on google scholar (which many colleagues use to find my work), scopus (which my university uses for promotion decisions), pubmed (which my funding demands), and the three other university-required databases (which are required by various bureaucratic components that don’t talk to each other), I will totally post all of my papers there happily. Oh yeah, and one that doesn’t preclude my ability to try to fight to get papers in “high-impact” journals (because for all the wishing that it weren’t so, the fact remains that papers published in higher-impact journals get read more, cited more, and drive internal promotion, grant scores leading to funding for me to do more research, and my students’ ability to succeed in their careers).

    Like

  42. JustThisGuyYouKnow Says:

    Qaz,

    You are describing bioarxiv, which only requires you to upload a PDF and is already indexed by google scholar. Almost all high impact journals accept preprints and there is evidence from physics that papers available as pre-prints are more highly cited that those that appear in publication for the first time. Even if that correlation is not causative, it’s very hard to imagine how a preprint could cost you citations.

    Like

  43. Grumpy Says:

    Qaz,

    I agree completely about the cluster$%&# of pub-indexing services that demand our attention. I hate pubmed (why the hell do they not index physical review journals?!). And I admit to ignoring requests for preprints on Mendeley, researchgate, etc cuz I don’t remember my login info and I don’t wanna deal with it.

    But the arxiv is not that. You won’t have to do anything because your trainees will be the ones to submit them. And you’ll never need to login or maintain anything–you can choose to maintain a profile but plenty of ppl don’t bother.

    The main effect you will see is when you Google your paper the first hit will be straight to a PDF of the latest version of the paper.

    I’ve heard there are some issues with ACS journals (eg JACS, Nano Letters) fussing about their papers being on arxiv. But all of the other top journals i deal with, including all of NPG, are OK with it.

    Like

  44. qaz Says:

    Why would my trainees be the ones to submit the paper to the archive? If a trainee of mine submitted a publication anywhere they would be in huge trouble. All lab output goes through me.

    Like

  45. jmz4 Says:

    http://www.sciencemag.org/news/2016/03/zika-virus-kills-developing-brain-cells

    Here’s an example of a pre-print showing up in press alongside published work. So maybe they are useful for establishing priority.

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: