Impact, in the Time of Corona

In an earlier post I touched on themes that are being kicked around the Science Twitters about how perhaps we should be easing up on the criteria for manuscript publication. It is probably most focused in the discussion of demanding additional experiments be conducted, something that is not possible for those who have shut down their laboratory operations for the Corona Virus Crisis.

I, of course, find all of this fascinating because I think in regular times, we need to be throttling back on such demands.

The reasons for such demands vary, of course. You can dress it up all you want with fine talk of “need to show the mechanism” and “need to present a complete story” and, most nebulously, “enhance the impact”. This is all nonsense. From the perspective of the peers who are doing the reviewing there are really only two perspectives.

1. Competition
2. Unregulated desire we all have to want to see more, more, more data if we find the topic of interest.

From the side of the journal itself, there is only one perspective and that is competitive advantage in the marketplace. The degree to which the editorial staff fall strictly on the side of the journal, strictly on the side of the peer scientists or some uncomfortable balance in between varies.

But as I’ve said before, I have had occasion to see academic editors in action and they all, at some point, get pressure to improve their impact factor. Often this is from the publisher. Sometimes, it is from the associated academic society which is grumpy about “their” journal slowly losing the cachet it used to have (real or imagined).

So, do standards having to do with the nitty-gritty of demands for more data that might be relevant to the Time of Corona slow/shut downs actually affect Impact? Is there a reason that a given journal should try to just hold on to business as usual? Or is there an argument that topicality is key, papers get cited for reasons not having to do with the extra conceits about “complete story” or “shows mechanism” and it would be better just to accept the papers if they seem to be of current interest in the field?

I’ve written at least one post in the past with the intent to:

encourage you to take a similar quantitative look at your own work if you should happen to be feeling down in the dumps after another insult directed at you by the system. This is not for external bragging, nobody gives a crap about the behind-the-curtain reality of JIF, h-index and the like. You aren’t going to convince anyone that your work is better just because it outpoints the JIF of a journal it didn’t get published in. …It’s not about that…This is about your internal dialogue and your Imposter Syndrome. If this helps, use it.

There is one thing I didn’t really explore in whingy section of that post, where I was arguing that the citations of several of my papers published elsewhere showed how stupid it was for the editors of the original venue to reject them. And it is relevant to the Time of Corona discussions.

I think a lot of my papers garner citations based on timing and topicality more than much else. For various reasons I tend to work in thinly populated sub-sub-areas where you would expect the relatively specific citations to arise. Another way to say this is that my papers are “not of general interest”, which is a subtext, or explicit reason, for many a rejection in the past. So the question is always: Will it take off?

That is, this thing that I’ve decided is of interest to me may be of interest to others in the near or distant future. If it’s in the distant future, you get to say you were ahead of the game. (This may not be all that comforting if disinterest in the now has prevented you from getting or sustaining your career. Remember that guy who was Nobel adjacent but had been driving a shuttle bus for years?) If it’s in the near future, you get to claim leadership or argue that the work you published showed others that they should get on this too. I still believe that the sort of short timeline that gets you within the JIF calculation window may be more a factor of happening to be slightly ahead of the others, rather than your papers stimulating them de novo, but you get to claim it anyway.

For any of these things does it matter that you showed mechanism or provided a complete story? Usually not. Usually it is the timing. You happened to publish first and the other authors coming along several months in your wake are forced to cite you. In the more distant, medium term then maybe do you start seeing citations of your paper from work that was truly motivated by it and depends on it. I’d say a 2+ year runway on that.

This citations, unfortunately, will come in just past the JIT window and don’t contribute to the journal’s desire to raise its impact.

I have a particular journal which I love to throw shade at because they reject my papers at a high rate and then those papers very frequently go on to beat their JIF. I.e., if they had accepted my work it would have been a net boost to their JIF….assuming the lower performing manuscripts that they did accept were rejected in favor of mine. But of course, the reason that their JIF continues to languish behind where the society and the publisher thinks it “should” be is that they are not good at predicting what will improve their JIF and what will not.

In short, their prediction of impact sucks.

Today’s musing were brought around by something slightly different which is that I happened to be reviewing a few papers that this journal did publish, in a topic domain reasonably close to mine, not particularly more “complete story” but, and I will full admit this, they do seem a little more “shows mechanism” sciency in a limited way in which my work could, I just find that particular approach to be pedantic and ultimately of lesser importance than broader strokes.

These papers are not outpointing mine. They are not being cited at rates that are significantly inflating the JIF of this journal. They are doing okay, I rush to admit. They are about the middle of the distribution for the journal and pacing some of my more middle ground offering in my whinge category. Nothing appears to be matching my handful of better ones though.

Why?

Well, one can speculate that we were on the earlier side of things. And the initial basic description (gasp) of certain facts was a higher demand item than would be a more quantitative (or otherwise sciencey-er) offering published much, much later.

One can also speculate that for imprecise reasons our work was of broader interest in the sense that we covered a few distinct sub-sub-sub field approaches (models, techniques, that sort of thing) instead of one, thereby broadening the reach of the single paper.

I think this is relevant to the Time of Corona and the slackening of demands for more data upon initial peer review. I just don’t think in the balance, it is a good idea for journals to hold the line. Far better to get topical stuff out there sooner rather than later. To try to ride the coming wave instead of playing catchup with “higher quality” work. Because for the level of journal I am talking about, they do not see the truly breathtakingly novel stuff. They just don’t. They see workmanlike good science. And if they don’t accept the paper, another journal will quite quickly.

And then the fish that got away will be racking up JIF points for that other journal.

This also applies to authors, btw. I mean sure, we are often subject to evaluation based on the journal identity and JIF rather than the actual citations to our papers. Why do you think I waste my time submitting work to this one? But there is another game at foot as well and that game does depend on individual paper citations. Which are benefited by getting that paper published and in front of people as quickly as possible. It’s not an easy calculation. But I think that in the Time of Corona you should probably shift your needle slightly in the “publish it now” direction.

Fixing the NIH, the perennial fight

In the wake of NIH’s announcement of their advice to reviewers to ignore any corona shutdown effects on the generation of preliminary data (blog post) there was a discussion on Science Twitter about fixing the NIH.

As per usual it eventually devolved into me shouting at people about how solutions that derive from their own personal interest are unlikely to be actual solutions. As is usually the case when I piss people off enough, they eventually do the full reveal, totally validating what I am saying. In this case, the person said in tweets that it is okay to ask people “who can afford it to tighten their belts for a year” and that they personally believe that they “have enough money” right now and therefore it would be okay if they happen to get a good score on another grant in the near future it would be just peachy if NIH skipped over it.

It is not news that PIs who feel that they have sufficient grant funding for the near future are totally okay not getting any more, for awhile. Yay for you that you feel all generous ONCE YOU HAVE BEEN SERVED.

This is what I’ve been saying in this particular kerfuffle and online for years. Yeah, we come first. Everything about our views on NIH and fixing perceived problems derives from our own personal selfish interests. It has limits, sure. We know we don’t need ALL of the NIH money. We just “need” what we want, we come first and anything that is left over can support our virtue-blasting prescriptions to help everyone else. Sometimes the “we” is a little broader than our own personal lab. Maybe it is people we feel are categorically like ourselves in some way. Maybe it is our friends. Maybe it is members of our field that we really like (work product-wise). It’s still essentially selfish.

But those other guys. The ones we don’t know. The veiled ‘Others’. Well it is totally okay that they be forced to take a haircut. After all, they can just “tighten their belt”.

It’s almost funny how impenetrable the cognitive defenses are. We NIH-dependent types cannot fathom that for every other person or category of person that we deem to have unwarranted funding or deem able to “tighten their belt”… there are just as many people pointing the finger at us for similar disadvantage. Those people have roughly the same arguments for why they “need” the money as we do (it is almost inevitably some version of “my lab will close down”). And in every case there are numerous somebodies out there pointing the finger and saying, cold-heartedly, “so what, fuck off, you aren’t needed here”.

I don’t know how you can get as far as some people have in this career and not grasp this. Unless you are extremely fortunate in the NIH grant getting game (and there are still some who are) then you have been told regularly, by a panel of your peers, that you and your work are not needed here. That you can take your series of ND grant outcomes and piss off. This should give you pause about your personal entitlement relative to a lot of other people. Relatedly, the NIH at one point started doing what I think they should have been doing since forever- considering the per-investigator success rate. Well, in a rolling 5 year interval some 87 thousand unique PIs submit RPG applications to the NIH and only 32 thousand are awarded grants. Sixty percent of applicants over five years are disappointed. That is peer review telling them they are unworthy.

Then we come to the chatter boxes and opinion havers about who is the Real Problem. We talk about this endlessly. In the current discussion it was the RICH MED SCHOOLS that were screwing everything up. For years we’ve had the discussions where the fingers are pointed at soft money job categories (blaming Universities for creating them and desiring to punish PIs for taking them), jobs at high overhead institutions (ditto) and perceived too-well-funded individuals. We’ve had the kerfuffle over riff-raff. The September 2019 Advisory Council for the Center for Scientific Review included one Councilor stating essentially that if someone hadn’t been funded after a few years of trying this proved they didn’t deserve funding ever. There was not general pushback from the rest of the panel, I will note. We’ve had allusions to a species of angry BSD who, upon getting a disappointing grant score, rails about “associate professors I’ve never heard of from East Jerkoff State University killing my grant” with the subtext that such individuals do not deserve any sort of NIH funding themselves. We have the discussion of “independence” and how those PIs who are clearly subordinate in BSD’s lab don’t deserve a fair chance because it is “really just the BSD trying to get more cash”.

I just don’t think we can move forward by ignoring math and funding facts, on the one hand, nor by attacking all those other people and suggesting they should bear the brunt of any pain because we have decided in our own personal wisdom that they “can afford” belt tightening.

The specific proposal on the table was bridge funding. A suggestion that NIH should pump out R56 awards to anyone who has a grant review criticized for a lack of preliminary data, tied to the notion that we cannot generate any more data from our closed labs in the Time of Corona. Of course this is zero change because this is exactly what the R56 program is for in good times and in bad. I guess this person wants a greater share of the NIH pie devoted to R56 right now. Which means fewer grants being awarded in full. Which is more or less in the space of putting in cuts to new awards (and existing awards) to prop up success rates which, again, is what NIH has done regularly in recent memory (aka my entire career as a PI). This is all fine. I actually support the idea of having this stop-gap.

What I don’t like about it is that it is totally random and hugely biased with respect to how cosy one is with Program. And like just about everyone, I’d love to have a PO throwing a R56 my way when I need it and I would resent them throwing one to someone I think undeserving when it is at the price of them not being able to pick up my payline+3%ile application. You can feel free to go RePORTERing for your favorite ICs decisions on R56 awards. You can see what the PI in question has been up to in terms of prior awards and make some inferences about their successes at peer review. Are these people uniquely special? Not at my favorite ICs they aren’t. There are tons of investigators just as worthy and just as struggling and, presumably just as “screwed” by peer review, that are not getting bridge funding. Maybe look into that before you think doubling or even quadrupling the number of R56 awards is any sort of general improvement.

Strategically, the NIH should do this, with great fanfare. They should claim they are quadrupling the number of R56 bridge awards and they should do so. They should yell that this is all so that people who can’t generate data during the Time of Corona aren’t disadvantaged. Because it will be years before anyone realizes that they are just going to the same old usual suspects. Entrenching the program biases that already exist. Disproportionately awarded to BIG MED SCHOOLS and all the above categories of Undeserving Bad People. Years before anyone who just loves this idea realizes that Peter was robbed to pay Paul.

Years before anyone cottons on to the fact that those with slightly more preliminary data, perhaps generated by their labs in the time everyone else was shut down, or who worked harder to submit more applications will still be advantaged.

This goes beyond opinionating at the NIH during times of crisis.

It speaks to how we, the peers doing the reviewing, are behaving at study section. Implicitly or explicitly, we bring our little opinions about who “deserves” funding. And it contaminates our review of merit. It makes us protect our own and run down the Others. Implicitly or explicitly, this is absolutely indisputable.

Now I would argue that the less that one examines and interrogates one’s own biases, the more likely one is to let them be expressed in grant scoring. The more one acknowledges the arbitrary and selfish nature of one’s preferences, the more likely one is to be aware of, and counter, them. And the more one doubles down to “prove” to someone else that one’s little biases are in fact right and just and entirely awesome….the more one is likely to let these matters contaminate one’s review.

Grant Review in the Time of Corona: NIH Says “No Slackening!”

Sally Amero, Ph.D., NIH’s Review Policy Officer
and Extramural Research Integrity Liaison Officer, has posted a new entry on the Open Mike blog addressing reviewer guidance in the Time of Corona. They have listed a number of things that are now supposed not to affect scoring. The list includes:

• Some key personnel on grant applications may be called up to serve in patient testing or patient care roles, diverting effort from the proposed research
• Feasibility of the proposed approach may be affected, for example if direct patient contact is required
• The environment may not be functional or accessible
• Additional human subjects protections may be in order, for example if the application was submitted prior to the viral outbreak
• Animal welfare may be affected, if institutions are closed temporarily
• Biohazards may include insufficient protections for research personnel
• Recruitment plans and inclusion plans may be delayed, if certain patient populations are affected by the viral outbreak
• Travel for key personnel or trainees to attend scientific conferences, meetings of consortium leadership, etc., may be postponed temporarily
• Curricula proposed in training grant applications may have to be converted to online formats temporarily
• Conferences proposed in R13/U13 applications may be cancelled or postponed.

Honestly, I’m not seeing how we are in a situation where this comes into the consideration. Nothing moves quickly enough with respect to grant proposals for future work. I mean, any applicants should be optimistic and act like everything will be normal status, for grants submitted this round for first possible funding, ah, NEXT APRIL. Grants received for review in the upcoming June/July study sections were for the most part received before this shutdown happened so likewise, there is no reason they would have had call to mention the Corona Crisis. That part is totally perplexing.

The next bit, however is a real punch in the gut.

We have also had many questions from applicants asking what they should do if they don’t have enough preliminary data for the application they had planned to submit. While it may not be the most popular answer, we always recommend that applicants submit the best application possible. If preliminary data is lacking, consider waiting to submit a stronger application for a later due date.

Aka “Screw you”.

I will admit this was entirely predictable.

There is no guarantee that grant review in the coming rounds will take Corona-related excuses seriously. And even if they do, this is still competition. A competition where if you’ve happened to be more productive than the next person, your chances are better. Are the preliminary data supportive? Is your productivity coming along? Well, the next PI looks fine and you look bad so…. so sorry, ND. Nobody can ever have confidence that where they are when they shut down for corona will ever be enough to get them their next bit of funding.

I don’t see any way for the NIH to navigate this. Sure, they could give out supplements to existing grants. But, that only benefits the currently funded. Bridge awards for those that had near-miss scores? Sure, but how many can they afford? What impact would this have on new grants? After all, the NIH shows no signs yet of shutting down receipt and review or of funding per Council round as normal. But if we are relying on this, then we are under huge pressure to keep submitting grants as normal. Which would be helped by new Preliminary Data. And more publications.

So we PIs are hugely, hugely still motivated to work as normal. To seek any excuse as to why our ongoing studies are absolutely essential. To keep valuable stuff going, by hook or by crook…. Among other reasons, WE DON’T KNOW THE END DATE!

I hate being right when it comes to my cynical views on how the NIH behaves. But it is very clear. They are positively encouraging the scofflaws to keep on working, to keep pressing their people to come to work and to tell their administration whatever is necessary to keep it rolling. The NIH is positively underlining the word Essential for our employees. If you don’t keep generating data, the lab’s chances of getting funded go down, relative to the labs that keep on working. Same thing for fellowships, trainees. That other person gunning for the rare K99 in your cohort is working so…..

Here’s the weird thing. These people at the NIH have to know that their exhortations to reviewers to do this, that or the other generally do not work. Look how the early stage / young investigator thing has played out across four or five decades. Look at the whole SABV initiative. Look at the remarks we’ve seen where grant reviewers refuse to accept that pre-prints are meaningful.

All they would have had to do is put in some meaningless pablum about how they were going to “remind reviewers that they should assume issues resulting from the coronavirus pandemic should not affect scores” and include Preliminary Data may not be as strong as in other times in the above bullet point list.

It is NOT “eating our seed corn”

The concept that we are “eating our seed corn” if we don’t do X, Y or Z to support junior scientists is completely misinformed, inapplicable and wrong.

This was super popular back when the ESI issues were being debated and the NIH was trying to justify giving special consideration to fund the applications of new comers to the system. I do happen to support continued efforts to help those who are on the short end of the NIH grant award stick, but this is mostly about the concept and how it leads to bad thinking.

“Eating our seed corn” has raised its misguided head in the Time of Corona as we are discussing University polices that have, apparently, started to slow walk new hires, pull back startup funds of recent hires, etc. There was even a little hint of graduate programs pulling offers of graduate stipends if the candidates had not responded to an offer yet, despite the deadline for response being in the future.

This is bad. Yes, it’s devastating for those individuals who are in the transition zones right now and are being denied opportunities that were in front of them. It’s devastating for departments and laboratories that were very much looking forward to securing new contributors. What it is NOT is “eating our seed corn”.

For those that have never so much as planted a food garden…. I am going to risk insulting your intelligence and to point out the obvious. “Seed corn” concerns were from a time of agriculture when a person hoping to grow a crop couldn’t just run down to the feed store and buy seeds whenever they wanted to. It comes from a time where you had to save your own seeds from the harvest time so that you could use them, about six months later after the winter snows had cleared, to grow next year’s crop(s). No problem right? Millenia of agriculture agrees- set aside enough seeds fro harvest to plant for next year. Easy peasy.

But…sometimes there wasn’t enough food to get through the winter. Seeds, of course, are also food. The corn kernels that we eat are those same seeds that can be planted to grow next year’s crops. And if you eat all your seeds to make it through this winter, you are going to have no corn crop next year. Or the year after that. or ever. Until someone takes pity on you and gives you some of their seed corn.

You can’t just make new seeds after you’ve eaten them.

New scientists are not like this. At all.

We CAN make new ones whenever we want, even if we’ve skipped several cycles. As I’ve noted in another context, if we have a department that literally closes it’s graduate program admissions for five years….they can start right back up in year six with essential zero headaches. The same professors suddenly forgot how to train graduate students? Please.

That’s because the proper analogy is more like acorns. Graduate student production is a perennial, not an annual, crop. If you have a big old oak tree on your property, it’s gonna grow acorns. Every year. We don’t chop it down to eat the tree when we get really hungry in the winter, right? It’s not edible. So next year, it’s gonna grow more acorns. And the cycle of health for that tree is really, really long. It doesn’t care if you ate 25% or 100% of the acorns it grew last year, it’s going to produce more next year. And the year after that. And after that.

If growing conditions are terrible, sure, many perennial agricultural producers may have low output that season. Some may even fail to produce anything edible that season at all. But as soon as the conditions return to favorable, that plant produces another crop. It takes a really, really bad set of conditions, sustained for years likely, to kill an oak tree. Short of devastating trauma, that is.

Sticking with the agricultural references, we are facing a water shortage and not a wildfire. We are not Little House on the Prairie where we have only ourselves on which to rely for seeds. We are most certainly not solely dependent on annual food crops. The enterprise of scientific research in the US is a perennial. It has persistence and resilience.

The ESI debate was no different. We were not then, and are not now, talking about the sort of existential emergency that is described by “eating our seed corn”. We are talking about priorities of how many plants and in what variety we can support, given a water supply that is rationed by external forces. We’re only getting so many acre-feet this year. And it looks to be less than we got last year.

The point is that we need to make rational, thinking choices about what we are going to prioritize and support. We should not panic, running all about screaming that every crop will be gone forever if we don’t water it just like it was watered last year.

Grant seeking in the Time of Corona

The NIH has been responding to the corona virus epidemic / crisis by shooting out various funding opportunity announcements to encourage new research on the issue. They will fund supplements, administrative and competing, as well as new grants and contracts. This is what NIH does in the face of perceived new health issues.

This response is perhaps more rapid than usual, but it is not very different from responses to other perceived crises such as the HIV/AIDS one, the SARS one, the Ebola one, the opioid one, etc. It’s not all that different from sudden political support for things such as the ongoing War on Cancer or the BRAINI scam.

As usual this sparks some minor debate in the ranks of the NIH funded science community. Is it some sort of outrage that individuals seek to create some sort of artificial, Frankenstein’s monster type of research program to respond to such funding opportunities? Is it distastefully mercenary? Will it just end up funding poorly considered, crap science?

Some seem to be arguing this line with respect to the corona virus crisis.

I am one who shakes my head ruefully and says “well, that’s how this system works”. On a tactical level my advice to grant funded PIs is to just say “well, two Aims for them, one Aim for me and let’s call it a day”.

Meaning sure, try to meet the intentions of the FOA by marrying what you already do to the interest of the day. Do some credible work on their interests – after all, at some level we do in fact work for NIH priorities which means national taxpayer priorities. We are fortunate to live in an investigator-initiated environment for the most part so is it really so terrible that once in awhile we’re sort of told what to do? I say no. Especially since we’re able to invent up the boundaries of what we’re being told to do. But this is also the opportunity to get some funding for what really interests you. To the tune of at least an Aim, probably more. How is that not a good thing?

I went through some of this as an observer and a participant PI during the HIV/AIDS version of this. Congress pushed a bunch of money at NIH for HIV/AIDS research and, the way I understand it, instructed NIH on who was going to be in charge of how much. Well, a lot of money ended up in the hands of NIDA. I can’t recall all the whys on that—those decisions were made before I was aware of this situation.

But, I very much was aware during a time when grants were supposed to come in with basically four groups or manipulations or what have you: Control, Immunodeficiency Virus Related, Drug Related, Virus + Drug. Another way to put it is: “How does Drug X affect pathogenicity in your immunodeficiency virus model?”

This is pretty specific but I think it generalizes to corona where there will be a lot of objection to people marrying The Real, Important, Critical Work on Corona Virus to Whatever They Happen To Do.

I didn’t think I was going to have an angle on corona virus at all. Shocked me to find that NIDA was actually out front. Why? Remember all that speculation back in the earlier days that Chinese men were perhaps more at risk than Chinese women due to cigarette smoking rates? And then there was some loose association of that with vaping in Scientific American (I think) speculation and boom, off to the races.

NIDA published one of the first FOA that I saw. NOT-DA-20-047 appeared March 19. Notice of Special Interest (NOSI) regarding the Availability of Administrative Supplements and Urgent Competitive Revisions for Research on the 2019 Novel Coronavirus

It’s a very broad one. Not just about smoking. They are ON it.

In order to rapidly improve our understanding of the risks, prevalence, and available control measures for 2019-nCoV in substance using or HIV-affected populations, NIDA is encouraging the submission of applications for Competitive Revisions to active grants to address the following research areas of interest:

Research to determine whether substance use (especially smoking tobacco or marijuana, vaping, opioids and other drug use) is a risk factor for the onset and progression of COVID-19.

Research on how HIV among persons who use substances may impact the onset and progression of COVID-19.

Research to understand system-level responses to COVID-19 prevention and risk mitigation in secure settings such as prisons and jails, with a particular emphasis on detainees with substance use disorder (SUD). For example:

Research to understand the respiratory effects of SARS-CoV-2 infection among individuals with substance use disorders (SUD); in particular those with nicotine, marijuana, opioid, and methamphetamine use disorders.

Research to understand how the respiratory effects of COVID-19 influences the rate of opioid overdoses both in pain patients as well as patients with an opioid use disorders and also to assess how it influences the outcomes for naloxone interventions for overdose reversal

Research to develop therapeutic approaches for comorbid SARS-CoV-2 infection and SUDs.

Research to evaluate drug-drug interaction of medications to treat SARS-CoV-2 and substances of abuse or medications to treat SUDs.

Research to understand system- or organizational-level responses to identify, prevent, or mitigate the impact of COVID-19 in service settings that serve vulnerable populations, including people who are homeless or unstably housed.

Research to understand and mitigate the impact of COVID-19 in methadone treatment programs and syringe exchange services.

Research on how potential overcrowding of emergency departments and health services will impact the treatment of opioid overdoses and of opioid use disorder

Research using ongoing studies to understand the broad impacts of COVID-19 (e.g., school closures, food insecurity, anxiety, social isolation, family loss) on neurodevelopment, substance use, substance use disorders, and access to addiction treatment.

COVID-19: Potential Implications for Individuals with Substance Use Disorders

is a webpage with more of their thinking on this.

So, is it terrible if I were to respond to this by firing the lab back up? By turning stones at my University until I found someone with a decent rodent-related set of expertises in corona viruses? Started plotting an attack on funding?

I am being ASKED to do so by the NIH. Encouraged to get in the game. And that means, you guessed it, putting the lab to work on this.

Are we the baddies?

Progress in the Time of Corona

One of the thorniest issues that we will face in the now, and in the coming months, is progress. Scientific progress, career progress, etc. I touched on this a few days ago. It has many dimensions. I may blog a lot about this, fair warning.

Several days (weeks?) ago, we had a few rounds on Twitter related to altering our peer review standards for manuscript evaluation and acceptance. It’s a pretty simple question for the day. Is the Time of Corona such that we need to alter this aspect of our professional scientific behavior? Why? To what end? What are the advantages and for whom? Are there downsides to doing so?

As a review, unneeded for most of my audience, scientific papers are the primary output, deliverable good, work product, etc of the academic scientist. Most pointedly, the academic scientist funded by the taxpayer. Published papers. To get a paper published in an academic journal, the scientists who did the work and wrote the paper submit it for consideration to a journal. Whereupon an editor at the journal decides either to reject it outright (colloquially a “desk reject”) or to send it to scientific peers (other academics who are likewise trying to get their papers published) for review. Typically 3 peers, although my most usual journals accept 2 as a minimum these days, and editors can use more if necessary. The peers examine the paper and make recommendations to the editor as to whether it should be accepted as is (rarely happens), rejected outright (fairly common) or reconsidered after the authors make some changes to the manuscript. This latter is a very usual outcome and I don’t think I’ve ever had a paper ultimately published that did not get there without making a lot of changes in response to what peers had to say about it.

Peer comments can range from identifying typographical errors to demanding that the authors conduct more experiments, occasionally running to the tune of hundreds of thousands of dollars in expense (counting staff time) and months to years of person-effort. These are all couched as “this is necessary before the authors should be allowed to publish this work”. Of course, assigned reviewers rarely agree in every particular and ultimately the editor has to make a call as to what is reasonable or unreasonable with respect to apparent demands from any particular reviewer.

But this brings us to the Time of Corona. We are, most of us, mostly or totally shut down. Our institutions do not want us, or our staff members, in the labs doing work as usual. Which means that conducting new research studies for a manuscript that we have submitted for review is something between impossible and very, very, very unlikely.

So. How should we, as professionals in a community, respond to this Time of Corona? Should we just push the pause button on scientific publication, just as we are pushing the pause button on scientific data generation? Ride it out? Refuse to alter our stance on whether more data are “required for publication” and just accept that we’re all going to have to wait for this to be over and for our labs to re-start?

This would be consistent with a stance that, first, our usual demands for more work are actually valid and second, that we should be taking this shutdown seriously, meaning accepting that THINGS ARE DIFFERENT now.

I am seeing, however, some sentiments that we should be altering our standards, specifically because of the lab shutdowns. That this is what is different, but that it is still essential to be able to publish whatever (?) manuscripts we have ready to submit.

This is fascinating to me. After all, I tend to believe that each and every manuscript I submit is ready to be accepted for publication. I don’t tend to do some sort of strategy of holding back data in hand, or nearly completed, so that in response to the inevitable demands for more, we can respond with “Yes, you reviewers were totally right and now we have included new experiments. Thank you for your brilliant suggestion!”. People do this. I may have done it once or twice but I don’t feel good about it. 🙂

I believe that when I am reviewing manuscripts, I try to minimize my demands for new data and more work. My review stance is to try to first understand what the reviewers are setting out to study, what they have presented data on, and what conclusions or claims they are trying to make. Any of the three can be adjusted if I think the manuscript falls short. They can more narrowly constrain their stated goals, they can add more data and/or they can alter their claims to meet the data they have presented. Any of those are perfectly valid responses in my view. It doesn’t have to be “more data are required no matter what”.

I may be on a rather extreme part of the distribution on this, I don’t know. But I do experience editors and reviewers who seem to ultimately behave in a similar way on both my manuscripts and those manuscripts to which I’ve contributed a review. So I think, that probably my fellow scientists that have my ~core skepticism about the necessity for peer review demands for more, more, more are probably not so exercised about this issue. It is more the folks who are steeped in the understanding that this is the way peer review of manuscripts should work, by default and in majority of cases, who are starting to whinge.

I’m kinda amused. I would be delighted if the Time of Corona made some of these Stockholm Syndrome victims within science think a little harder about the necessity of their culture of demands for more, more, more data no matter what.