Bias at work
April 12, 2016
A piece in Vox summarizes a study from Nextions showing that lawyers are more critical of a brief written by an African-American.
I immediately thought of scientific manuscript review and the not-unusual request to have a revision “thoroughly edited by a native English speaker”. My confirmation bias suggests that this is way more common when the first author has an apparently Asian surname.
It would be interesting to see a similar balanced test for scientific writing and review, wouldn’t it?
My second thought was…. Ginther. Is this not another one of the thousand cuts contributing to African-American PIs’ lower success rates and need to revise the proposal extra times? Seems as though it might be.
DrugMonkey 
April 12, 2016 at 3:40 pm
I am Asian American and I _often_ request for a manuscript to be reviewed by member of the authorship team whose first language is English, or by a professional editing service. Nothing annoys me more greatly than a manuscript submission that is rife with errors of grammar, improper use of articles, errors of tense, etc. Of all the things that could be easily checked off of a pre-submission checklist, “make sure the manuscript is written properly” is one of the easiest things to correct.
LikeLike
April 12, 2016 at 6:43 pm
“It would be interesting to see a similar balanced test for scientific writing and review, wouldn’t it?”
Yes, given how crazy reviewers on study section get about typos (as evidenced by vociferous commenters on this blog), I could see this being a germane to the Ginther report.
Another interesting report from that same Nextations place notes the insidious effect of (presumed) affirmative action hiring and the taint it leaves on applicants. I could see that being relevant to a minority working at a prestigious institution and not getting the full benefit of the institution due to a presumption of affirmative action hiring.
Still, no error bars or statistics in the Vox article or the original study release…
kinda shady.
LikeLike
April 13, 2016 at 8:52 am
An argument for double-blind review?
LikeLike
April 13, 2016 at 9:38 am
Recommending professional editing is appropriate, but specifically a native speaker is not.
LikeLike
April 13, 2016 at 11:05 am
Are we not to *supposed* to point out terrible grammar in manuscripts? Would it be better for things to get published with bad grammar and/or incorrect word usage? I recently reviewed a paper where the authors confused the verb “to shed” with “to shred”. As in some cases the sentence actually would make (factually incorrect) but plausible sense, this isn’t being merely pedantic.
LikeLike
April 13, 2016 at 11:49 am
“An argument for double-blind review?”
Maybe. But reviewers’ comments about the manuscript needing the attention of an English editor are usually justified (in my experience). Which means that it would be easy for blinded reviewers to ascertain that the authors are foreigners and possibly even localize them to a specific region (Asia, Eastern Europe, …)
Now, if there is bias even when the English is perfect, that’s another story. But I am not convinced that this sort of bias is pervasive. A study needs to be done. Give two randomly chosen groups of scientists the same paper to review. One group gets a paper with Asian names as authors, the other gets Euro/American names. Then see if the scores differ. This can actually be done objectively because most journals ask for some sort of ranking on the Accept – Minor Changes – Major Changes – Reject scale.
Sounds like the sort of experiment the editor-in-chief of a major Neuroscience journal might consider doing…
LikeLike
April 13, 2016 at 1:15 pm
“Are we not to *supposed* to point out terrible grammar in manuscripts? Would it be better for things to get published with bad grammar and/or incorrect word usage?”
Whoosh.
Yes you should point out the shit that is wrong or needs clarification/rephrasing but saying “please have someone who is a native English Speaker edit this” is making (often racially biased) implications about the author(s) native language.
I actually caught myself almost about to do this once and yep, the authors were Chinese. I modified the review to get rid of such statements while leaving in the gist that they needed to take care of the issues.
LikeLike
April 13, 2016 at 1:52 pm
I worked for a time at a biotech firm in Montreal where the working language was French (not my native language). Needless to say, I made quite a few grammar mistakes when I made reports in French, and people could certainly tell it wasn’t my native language. It’s pretty obvious that certain manuscripts aren’t written by native speakers — there’s no real “implication” to be made.
LikeLike
April 13, 2016 at 2:38 pm
Ask yourself what exactly is supposed to be the added benefit of using a statement like this, as opposed to a more neutral one?
If in fact they are not native speakers, they will understand that was the root of the grammar issues. As non native speakers, they are highly attuned and sensitive to that fact, just as you say you were about French. Pointing it out adds exactly nothing except the opportunity for bias.
LikeLike
April 13, 2016 at 2:42 pm
Just make the proposal review process as blind as possible. Hide the personnel’s names and institutions. Let the reviewers judge ONLY the idea/hypothesis, its significance and the proposed approach. After the proposals are reviewed and ranked, let the Program Officer validate (pass/fail only) the environment, facilities and PI’s credentials.
But, of course, nobody is going to entertain such an outrageous idea. Panel members love to play their little funding politics, based on various biases.
LikeLike
April 13, 2016 at 3:10 pm
An argument for double-blind review?
Arguments for double-blind review are the same as arguments for magic unicorn leprechaun gold funding for the NIH.
Are we not to *supposed* to point out terrible grammar in manuscripts?
Sure, but in this context the question is whether such errors interfere with the ability to judge the science itself in an unbiased manner. I bet that it does. Didn’t we just do a thing on people who say that too many typos make them assume the science is sloppy too?
reviewers’ comments about the manuscript needing the attention of an English editor are usually justified (in my experience).
If I had a $25,000 NIH Grant module for every time a reviewer found the two typos in my grant app or manuscript and said “this thing is riddled with errors and needs a comprehensive proofreading”, I would be a happy PI. My recollection is that non-native English in manuscript review is not terribly dissimilar, particularly when you figure that the ticky-tack small stuff is really in the hands of the copy editor. My threshold is whether or not the phrasing interferes with understanding.
But I am not convinced that this sort of bias is pervasive. A study needs to be done.
Yeah, well that Nextions outfit did one. I see no reason that our null for scientific manuscript review isn’t that there is bias, needs to be proved otherwise.
LikeLike
April 13, 2016 at 3:25 pm
DM’s past post is relevant to double-blind review:
http://drugmonkey.scientopia.org/2012/04/18/double-blind-paper-and-grant-review-cant-work/
I see no problem though. Proposals should consist only of the following sections: (1) Idea/hypothesis, plus a review of the present state of the knowledge. (2) Significance. (3) Proposed approach.
That’s all. These three components are enough to judge if the proposal makes sense and should be funded, and they do not disclose anything that would allow for reliably identifying the grantee. Well, if underdogs get mistaken for some big guys, all the power to them.
LikeLike
April 13, 2016 at 3:43 pm
Unless the journal does not use a copy editor, I do not point out minor typos. I will ask the authors to revise places that are difficult to understand. This is sometimes due to poor grammar, but more often due to a lack of precision and clarity in the writing. If the entire manuscript was poorly written, I can understand recommending a manuscript be reviewed by a professional editor. However, requesting that the manuscript be reviewed by a “native speaker” is presumptuous and wholly inappropriate.
LikeLike
April 13, 2016 at 4:20 pm
“I see no problem though. Proposals should consist only of the following sections: (1) Idea/hypothesis, plus a review of the present state of the knowledge. (2) Significance. (3) Proposed approach.”
-Feasibility(e.g. research history) and institutional environment are big parts of the score though, right? Would it be it be a good idea to offload those judgements to a PO?
LikeLike
April 13, 2016 at 4:25 pm
I’ve had the “native English speaker” review twice from ~ 100 papers. The last time, I wrote back saying that although I’m a Liverpudlian, I still consider myself a native English speaker. It’s not like I write in scouse (Liverpool dialect)! I’ve always assumed that such comments come from Ivy League Grammar Nazi reviewers just to make the recipients feel inferior. In the UK at least snobbishness was still the rule in science when I started my career. Best choice I ever made was coming to North America where people judge me on what I say, not by the accent with which I say it.
LikeLike
April 14, 2016 at 2:07 am
Anonymouse, so anyone would be able to propose approaches using any technique and equipment? It would not be possible or appropriate to find out if they actually have the equipment, or if they have already spent the multiple years needed to find-tune the methods.
I agree with jmz4, feasibility is a big deal.
LikeLike
April 14, 2016 at 8:32 am
‘ey, bacillus, ay think yous should tertally submit yer next paper in scouse!
LikeLike
April 14, 2016 at 10:45 am
JL- maybe there could be two stage review. Ideas first, feasibility afterwards.
LikeLike
April 14, 2016 at 10:46 am
And, warming to the theme, maybe it could be JIT-style where you cough up the feasibility prelim data only after the Big Idea looks like it is in contention.
LikeLike
April 14, 2016 at 1:16 pm
@ grumble. Ta la, irrle get a lorra laffs if i doz it like tha
LikeLike
April 14, 2016 at 7:25 pm
“Feasibility(e.g. research history) and institutional environment are big parts of the score though, right? Would it be it be a good idea to offload those judgements to a PO?”
Why not? It’s a pass/fail thing. Either somebody has the necessary facilities, or not. As for the soft factors, well, that’s the point – I want to hide these things from the reviewers to eliminate the biases.
LikeLike
April 17, 2016 at 11:31 am
It’s a fair thing to say if the paper comes from a country where the language it is written in is rare. Also, with modern economics, printing technology and fast turn around there are no copy editors.
Less than ten errors Eli will point out. More, and at a minimum a resubmission is needed
LikeLike