Theological waccaloons win because they are powered by religious fervor and exhaust normal people
February 14, 2018
Some self-congratulatory meeting of the OpenAccess Illuminati* took place recently and a summary of takeaway points has been posted by Stephen Curry (the other one).
These people are exhausting. They just keep bleating away with their talking points and refuse entirely to ever address the clear problems with their plans.
Anonymous peer review exists for a reason.
To hear them tell it, the only reason is so hateful incompetent reviewers can prevent their sterling works of genius from being published right away.
This is not the reason for having anonymous peer review in science.
Their critics regularly bring up the reason we have anonymous peer review and the virtues of such an approach. The OA Illuminati refuse to address this. At best they will vaguely acknowledge their understanding of the issue and then hand wave about how it isn’t a problem just …um…because they say so.
It’s also weird that 80%+ of their supposed problems with peer review as we know it are attributable to their own participation in the Glamour Science game. Some of them also see problems with GlamHumping but they never connect the dots to see that Glamming is the driver of most of their supposed problems with peer review as currently practiced.
Which tells you a lot about how their real goals align with the ones that they talk about in public.
Edited to add:
Professor Curry weighed in on twitter to insist that the goal is not to force everyone to sign reviews. See, his plan allows people to opt out if they choose. This is probably even worse for the goal of getting an even-handed and honest review of scientific papers. And even more tellingly, is designing the experiment so that it cannot do anything other than provide evidence in support of their hypothesis. Neat trick.
Here’s how it will go down. People will sign their reviews when they have “nice, constructive” things to say about the paper. BSDs, who are already unassailable and are the ones self-righteously saying they sign all their reviews now, will continue to feel free to be dicks. And the people** who feel that attaching their name to their true opinion will still feel pressure. To not review, to soft-pedal and sign or to supply an unsigned but critical review. All of this is distorting.
Most importantly for the open-review fans, it will generate a record of signed reviews that seem wonderfully constructive or deserved (the Emperor’s, sorry BSDs, critical pants are very fine indeed) and a record of seemingly unconstructive critical unsigned reviews (which we can surely dismiss because they are anonymous cowards). So you see? It proves the theory! Open reviews are “better” and anonymous reviews are mean and unjustified. It’s a can’t-miss bet for these people.
The choice to not-review is significant. I know we all like to think that “obvious flaws” would occur to anyone reading a paper. That’s nonsense. Having been involved in manuscript and grant review for quite some time now I am here to tell you that the assigned reviewers (typically 3) all provide unique insight. Sometimes during grant review other panel members see other things the three assigned people missed and in manuscript review the AE or EIC see something. I’m sure you could do parallel sets of three reviewers and it would take quite a large sample before every single concern has been identified. Comparing this experience to the number of comments that are made in all of the various open-commenting systems (PubMed Commons commenting system was just shuttered for lack of general interest by the way) and we simply cannot believe claims that any reviewer can be omitted*** with no loss of function. Not to mention the fact that open commenting systems are just as subject to the above discussed opt-in problems as are signed official review systems of peer review.
__
*hosted at HHMI headquarters which I’m sure tells us nothing about the purpose
**this is never an all-or-none associated with reviewer traits. It will be a manuscript-by-manuscript choice process which makes it nearly impossible to assess the quelling and distorting effect this will have on high quality review of papers.
***yes, we never have an overwhelmingly large sample of reviewers. The point here is the systematic distortion.
DrugMonkey 
February 14, 2018 at 1:05 pm
Dude Ted, don’t worry. Preprints will save us all.
LikeLike
February 14, 2018 at 1:36 pm
How we review papers (anonymous peer review vs. publicly signed review?) is a different question from how we access them (OA vs. paywall); which is a different question from how we pay for the (for profit corporation vs. non-profit societies); which itself is a different question from how we apportion prestige (glam humping vs. simply counting papers? not sure any alternatives exist).
Having a wholistic VISION for how science SHOULD be done on each of these dimensions is fine in theory, but seems to be distinctly correlated with a theological approach.
LikeLike
February 14, 2018 at 1:45 pm
All I can say is thank god my sciencing days are over. If open peer review became A Thing, I would promptly stop reviewing papers. At least until #metoo makes all dudes everywhere totally ok with accepting criticism from women who are their inferiors and don’t belong in science and probably took some deserving man’s spot as an affirmative action hire and will probably quit when they have a baby and just need to get laid and oh my god can’t you take a joke, you should smile more.
LikeLike
February 14, 2018 at 1:47 pm
Right, becca, but the Venn overlap between OA and open-peer-review congregations is substantial. As is the overlap in their approach to dealing with quite obvious and clearly expressed detrimental effects of their fondest wishes to change science.
LikeLike
February 14, 2018 at 1:48 pm
At least until #metoo makes all dudes everywhere totally ok with accepting criticism from women who are their inferiors and don’t belong in science and probably took some deserving man’s spot as an affirmative action hire and will probably quit when they have a baby and just need to get laid and oh my god can’t you take a joke, you should smile more.
Zuska has not lost one iota of her ability to nail the issue to the wall I see…..
LikeLike
February 14, 2018 at 6:27 pm
Open peer review is already a thing in a number of journals (the Frontiers series of journals in particular). Somehow the dire predictions of how it would degenerate into bullying or petty tit-for-tat revenge reviewing haven’t materialized. People are talking about the idea as if it is just theoretical, and it isn’t. It clearly works okay. Whether it is really an improvement over anonymous peer review is unclear, but it isn’t as if anonymity is some perfect system that needs to be preserved at all costs.
LikeLike
February 14, 2018 at 8:53 pm
“Somehow the dire predictions of how it would degenerate into bullying or petty tit-for-tat revenge reviewing haven’t materialized”. Did you read DM’s post?
“It clearly works okay”, based on what? Your gut filling?
LikeLike
February 15, 2018 at 12:45 am
Somehow the dire predictions of how it would degenerate into bullying or petty tit-for-tat revenge reviewing haven’t materialized.
I am entirely unfamiliar with such predictions and am not addressing this variety of critique in the post. My “dire predictions” revolve around a loss of the virtues of anonymous peer review, i.e, honest opinion, minimally contaminated by career and social fears of revenge from angry authors or other fans of the work in question.
It clearly works okay.
One or two journals out of the scores that service a given subfield are hardly likely to give us a clean look at the question. The opt-in factor for Frontiers is tremendous. Heavy selection for people who are all inside the church already. But the recent meeting and other opinion piece type communications from the Illuminati of Open Science / Open Review suggest a much broader, universal agenda. And that is a far different matter.
Perhaps even more importantly, you cannot possible assess the distorting effects on peer reviewing that I am talking about here.
it isn’t as if anonymity is some perfect system that needs to be preserved at all costs.
Sure. But neither is anonymous peer review such a flawed system that it needs to be torn down.
LikeLike
February 15, 2018 at 5:02 am
Um, you are literally just making the dire predictions I said in different words – the idea that people would be afraid to give honest opinions when they actually are accountable for them due to fear of revenge. And looking at open peer review at Frontiers, it’s clear that people (including myself, hardly a BSD Glamhound) aren’t afraid to give their honest criticism. Yes, it’s hard to quantify the effects on peer review, but at least it’s possible because the data are there. They aren’t there for anonymous peer review.
LikeLike
February 15, 2018 at 6:50 am
The Frontiers peer review system is better characterised as “sieve review” than “open review”.
LikeLike
February 15, 2018 at 10:43 am
jb, allow me to be the second person on this thread (and thanks, Zuska, for making this point with much more style!) to say that if forced to sign my reviews, I would decline to review.
And looking at open peer review at Frontiers, it’s clear that people (including myself, hardly a BSD Glamhound) aren’t afraid to give their honest criticism.
I’ll re-affirm DM’s notion that this is a small opt-in club. As a (probably) male person, you perhaps are not listening to what Zuska is saying about the problems signed reviews have for women and non-white folk.
LikeLike
February 15, 2018 at 11:22 am
The problem with playing the privilege card isn’t that privilege doesn’t exist, but that it is a multifaceted concept that the typical hierarchy model doesn’t really capture well. Yes, I’m a white male. And that gives me a certain amount of privilege. But on the other hand, I’m 1) in a non tenure track position and 2) not an Ivy Leaguer either by education or by institution. Both those things make me rather unprivileged in the scientific world in many ways. For example, I recently requested access to some clinical data from an Ivy League school but was refused because being non-tenure track I’m unworthy — instead I’m supposed to find a tenured PI with access who will grant me “collaborator” status. So it certainly isn’t my great privileged status that makes me feel comfortable signing reviews (or posting on blogs using my real name).
LikeLike
February 15, 2018 at 11:42 am
JB, why is it so difficult for you to grasp an exceptionally simple point about the distorting effects of letting peer review be conducted only by people just like you?
LikeLike
February 15, 2018 at 11:51 am
I thought your argument was that signed reviews were the pet project of Glamhumping OA Illuminati who are probably HHMI fellows rolling in money. Such people are manifestly not “just like me”.
LikeLike
February 15, 2018 at 12:01 pm
JB, I wasn’t going to say anything, because this point seems obvious to me. When I posted things under my real name (back as a grad student), I didn’t worry about some jerk treating me like a second class scientist because I was too busy worrying about the violent rape and death threats that rained down on me when I showed competence in areas coded male.
LikeLike
February 15, 2018 at 12:24 pm
Theologically charismatic leaders always need willing congregations to chant “Amen” on cue, JB.
LikeLike
February 15, 2018 at 8:52 pm
I also decline to do reviews if required to sign them. Getting funding and papers through reviewers is tough enough without someone there hating your guts. If you have any of that clear evidence that this is not a problem, post it.
I am happy to review and give my honest opinion of your paper, so much that I do it for free. But I can’t control whether the authors like my comments. There are very soind reasons, as well as some bad ones, to need anonymity in reviews.
LikeLike
February 16, 2018 at 12:54 pm
The Frontiers review process is not open-review. It is something else. I like TL’s term “sieve review”. (For those who don’t know, the Frontiers process is that the reviewer is only named if the paper gets accepted and the paper is only accepted if the reviewer “signs off” (endorses) it. Reviews are completely anonymous during the process. If the paper is rejected, reviewers are not revealed. For a paper to be accepted, the two reviewers endorse it. I particularly like this process because as a reviewer, one rejects a paper by “withdrawing from review” – what that means is that your review stays with the paper and the editor can still see it, but you refuse to endorse the paper – your name is never revealed and you are not one of the two required endorsers. It’s a very good system.)
This process is important because this is very different from an open-review system. Also, don’t forget, the real problems (GlamHounds who call editors to fight reviewers) aren’t really publishing in Frontiers.
It is very important to note that there are three issues that are actually orthogonal to each other:
1. Open-access. Should a paper be available to the public or behind a paywall? In my experience, this is silly because essentially all papers are available after 1 year (because of funder requirements) and through contacting authors.
2. Glamour. PloS Biology and eLife are proof that the GlamHound problem is unrelated to the open-access problem. (And with Nature Communications and Cell Reports.)
2a. Working downward instead of upward. We send our paper to the fancy Glam journal, fight for a year, then try at Semi-Glam, fight for another year (or more), then finally give up and dump the paper in a workhorse journal. The delay to publication could be completely solved if all papers were published in a workhorse journal and moved “up” to SemiGlam and then to Glam based on metrics. Frontiers tried this for a while, but seem to have given up on it as not really viable.
3. Open-review. This is a third issue. I think there are very important reasons to maintain the anonymity of peer review, but there are processes that can adjust it and improve it. (q.v. Frontiers)
By the way, I really HATE the idea of publishing the reviews with the paper. This is apparently becoming popular. I find it stupid and wasteful. All it shows is how the food was cooked. What matters is the result and the discovery. The whole point of a paper is that it is a distillation, a readable communication. We don’t need to see the arguments or mistakes made along the way.
LikeLike
February 16, 2018 at 1:57 pm
but you refuse to endorse the paper – your name is never revealed and you are not one of the two required endorsers
….what? does the editor just keep seeking new reviews until two people sign off?
could be completely solved if all papers were published in a workhorse journal and moved “up” to SemiGlam and then to Glam based on metrics
lololo. no of course the GlamHounds won’t go for this. ow, my side.
I really HATE the idea of publishing the reviews with the paper
I do as well. People are just insane if they cannot understand that any paper can be absolutely eviscerated if someone cares to do so. the only conceivable outcome is that this facilitates political agendas’ and #allegedprofession idiots’ illegitimate attacks on science. at some point having to defend ever paper against all possible complaints strangles the progress of science. I draw the line somewhere just after the peer review process completes.
LikeLike
February 16, 2018 at 2:28 pm
“People are just insane if they cannot understand that any paper can be absolutely eviscerated if someone cares to do so”
So your idea is that papers that pass peer review are just reviewed by nice people who don’t do this? Not that some papers are flawed and others aren’t? If you are worried about “illegitimate attacks on science”, this postmodernist idea that any paper can be trashed is excellent ammo for them.
LikeLike
February 16, 2018 at 5:26 pm
“Open peer review” can mean many different things – I’ve found this taxonomy helpful: https://f1000research.com/articles/6-588/v2 and we tried to use it throughout the meeting, which focused mainly on open reports (publishing the content of peer reviews, with or without names). Signing peer reviews is obviously much more problematic for the good reasons discussed here.
Two points from qaz/DM:
“could be completely solved if all papers were published in a workhorse journal and moved “up” to SemiGlam and then to Glam based on metrics
lololo. no of course the GlamHounds won’t go for this. ow, my side.”
This is almost exactly the model in this proposal from HHMI: asapbio.org/digital-age
“I really HATE the idea of publishing the reviews with the paper… I find it stupid and wasteful. All it shows is how the food was cooked. ”
Personally I’d like to know it WAS cooked…
LikeLike
February 16, 2018 at 7:24 pm
….what? does the editor just keep seeking new reviews until two people sign off?
They can. When a reviewer doesn’t like a paper, one of three things happens. Editors can reject the paper or the authors can withdraw the paper or the can send it to another reviewer to see if the other reviewer will endorse it.
This is actually a good thing. It prevents the ability of a reviewer with an agenda to block a paper. Moreover, if you want to pass garbage on as a reviewer, then that’s your business. And I can ignore other things that you have endorsed.
lololo. no of course the GlamHounds won’t go for this. ow, my side.
(eye-roll) I didn’t say it would happen. I said that it was the solution to the problem.
Personally I’d like to know it WAS cooked…
I don’t care what the reviewers said. I care what the paper says. If it’s something important that I need to build off of, I had better be able to read the paper and make up my own mind. If it’s something that’s neat or fun or not in my wheelhouse, I’m happy to let science take the time to work out if it’s real or not.
LikeLike
February 19, 2018 at 10:55 pm
I am in favor of double blind anonymous peer review; I think a signed one where no one has anonymity is a bad idea, that would lend itself to all sorts of nasty politics and vendetta.
What I do not understand is why there seems to be resistance to peer review where the authors and their affiliations are also anonymous to the referees, in addition to anonymous refereeing. I think this would be particularly beneficial in glamour journals.
LikeLike
February 20, 2018 at 9:35 am
I have “resistance” because I don’t think it can be done on a systematic high-confidence basis. And because I think partial solutions that look like it is double blind, when it is only partially so, leads to identity-based distortions that are worse than what we get with single blind.
LikeLike
February 20, 2018 at 6:01 pm
In response to Qaz/DM:
1. Open access is not a given in all areas and funders. Just because it happens in your workhorse journals doesn’t mean you can assume that experience applies to everyone in every subdiscipline in the biomedical/biological sciences. Not everyone is funded by big NIH/NSF grants and so their contributions aren’t open access by funder’s choice.
2. Reviewer comments and the reviewer process are SUPREMELY helpful for training ECR/postdocs/predocs. I haven’t seen any discussion of harm, and am not sure why something that has no harm to you but a massive benefit to ECRs/trainees is something you would poo-poo.
LikeLike
February 20, 2018 at 6:28 pm
If you have not seen discussion of harms 1) you are not paying attention and 2) this further reinforces my main point which is that the waccaloons never include the full discussion in their little self-congratulatory conclaves.
LikeLike
February 28, 2018 at 9:33 pm
On a related note, preprints are also a total crock and screw over new or non-famous PIs.
1. So you want me to submit a preprint so I can get comments that I have to spend time responding to? No thanks. I spend enough time responding to the comments of reviewers from real journals. I can’t imagine how much time I’d have to spend responding to comments that are public and immortalized online. No paper is perfect. How many comments does a paper need before it’s acceptable for publication? Where does it end? I do not need more work when it already feels like the bar to publishing a paper keeps rising and rising.
2. But isn’t it immoral to deny the world my work while I wait for it to be published? I publish most often in a society journal that generally provides reviews within a month. The editor then makes a decision within a couple days after we resubmit and it’s published online the same day as acceptance. The people who care most about the work have either seen it at a meeting or they are the reviewers. The world will survive for two months while my paper is being reviewed and revised. Then it’s freely available on Pubmed after a year, or I’ll send it immediately to anyone who emails me or requests it on Researchgate. Open access is not a real issue in my opinion.
3. Don’t I want to stake my claim immediately as the original discoverer of our Nobel Prize-winning findings? This might actually work for the HHMI folks. But, really, a pre-print from my lab would mean absolutely nothing if an HHMI Prof published the same thing a year later in Nature. Who do you think is going to get credit? get cited? So, if anything, submitting a preprint is more likely to get me scooped by a more famous and better-resourced lab.
4. Don’t I want to list my preprints on my grant applications? No! I have plenty of real, published papers to show my productivity. Having my preliminary data published has backfired on me in more than one grant application because the reviewers say we’re being incremental. It’s often better to show the exciting, unpublished prelim data in the application itself. (I might make an exception here for a brand spanking new prof who hasn’t published any independent stuff yet)
5. There is an HHMI professor who Tweeted that a Cell editor contacted her after reading her pre-print and asked her to submit to their journal. Of course, it sailed right through review and became a Cell paper. So doesn’t this mean that we should all be submitting preprints? Barf.
LikeLike
March 2, 2018 at 8:39 am
Amen to the above @amosttenured.
I agree with DM that this is at the core a cry of rage from the no-longer-glitterati
LikeLike
March 2, 2018 at 10:29 am
I’m not sure that is what I said MB. I see it more as people who have tired of the game they created OR that think they now deserve to have a more extensive privilege within the game.
LikeLike
March 2, 2018 at 11:20 am
The Frontiers process still has problems. I put a paper through the wringer (that it deserved) once. It took 3 rounds to get it to an acceptable place. I was ok with the final product, but not ok with revealing my name to them, based on the 3 previous rounds of exchanges.
I’ll leave my other favorite gem from review here. I was a bystander to this: “Since year 200C, the identical experimental model, XYZ KO, using identical experimental methods, has been exploited by the ABC lab in multiple papers: (list omitted) … Apparently, the ABC lab would like to claim more space in J. Neuroscience this year (201D), based on the same model, same brain region, identical techniques, and miniature advancements in the field (current manuscript).”
This went on and on and on and on. The reviewer clearly had a chip on the shoulder the size of, say, Montana. My and the other review were standard-fare: good paper that presents a major result, some things to change, accept. Editorial decision: reject.
This is a big part of why I no longer submit there.
LikeLike
March 2, 2018 at 1:43 pm
J Neuro is just turning into the ultimate sadness. JIF has sunk firmly below 6 and they still act like they are a mid-teens JIF journal. The Editorial staff and reviewers also still have conceits about what minority of the breadth of work presented at the Annual Meeting “belongs in” the Journal. Is this a Society journal or not? If it is, it should reflect the interests of the Society- all of it.
Combine this with a pronounced inability of the new Editorial staff to break the insider club of acceptance and you can see where this spiral goes downward uninterrupted.
LikeLike
March 2, 2018 at 2:13 pm
[…] almost tenured PI raised some interesting questions in a comment: […]
LikeLike
March 3, 2018 at 5:46 pm
“I know we all like to think that “obvious flaws” would occur to anyone reading a paper. That’s nonsense. Having been involved in manuscript and grant review for quite some time now I am here to tell you that the assigned reviewers (typically 3) all provide unique insight. … and we simply cannot believe claims that any reviewer can be omitted*** with no loss of function. Not to mention the fact that open commenting systems are just as subject to the above discussed opt-in problems as are signed official review systems of peer review.”
I wonder how this Nature paper didn’t get any serious public feedback during the last 4 years:
https://pubpeer.com/publications/E35BC0AE39F51EAB685A7725189DAB
LikeLike