On the perceptions of well intentioned people regarding the NIH’s little diversity whoopsie

August 26, 2011

BikeMonkey Guest Post
Believe you me, it does not escape my attention that instead of working on my grant that is due in approx one month’s time, I am talking about the Ginther et al report. No, I am not obligated to say jack squatte about it. These little distractions are optional. As is the mentoring “tax” that the senior author of that report, Raynard Kington, discussed. Likewise participation in the well-intentioned “enhance the diversity” efforts of our Universities and professional societies. Yet…here we are.
The DM has been taking a few whacks at what appears to be the reasonably well-intentioned musings of one Michael Eisen. I am fascinated by the latter’s defensive comment:

But I’m shocked at how many people leapt to the immediate conclusion that the peer review system penalizes applications from black PIs when we know that black scientists face all sorts of other obstacles that both discourage them from entering the field in the first place and make it more difficult for them once they are here. I just felt it was pretty naive on the NIHs part to expect anything different – as if they thought the things they were doing to promote the careers of black scientists had actually solved all the problems they face. And then to look at the data and cry racism is just making the problem even worse by both discouraging black scientists from joining the field and making it harder for them to recruit people once there here.

leapt to the immediate conclusion“. “cry racism“. Yes, perhaps I should reconsider the “well-intentioned” bit. These are stock in trade phrasings of anti-affirmative action people.

Dr. Eisen has gone astray. If you actually read what the Ginther report chose to look at for their analyses, and barring that read the Tabak and Collins executive summary, it is entirely obvious that these descriptions are inaccurate. The authors went to great lengths to try to address some of the more obvious questions that were only secondarily or tertiarily linked to the race of the PI. Even in the proposals for future study and review, Tabak and Collins bent over backward to cover the “disparities in education and research opportunities” that might be the true causal agent.
So why is it Dr. Eisen’s perception that the NIH went jumping to racial conclusions?

So, if I don’t believe peer review is pervasively racist, but I believe the data in the paper, I have to believe instead that NIH study sections find that grant applications from black scientists are – on average – marginally less impressive than those of their comparably experienced and accomplished white colleagues.

Because his posed theoretical is likely not so theoretical. He is highly invested in the denial of racism in the peer review process. I’d like to point out to him that this may possible conflict with another of his stated goals:

Rather than discouraging aspiring black scientists by portraying a field filled with insurmountable obstacles, we should emphasize that biomedical science offers them the opportunity to be judged not by the color of their skin, but by the content of their R01 application.

If there is one item that is way up there on the list of things that discourage black scientists (right behind the data that illustrate the disparity) it is the reflexive denial by non-blacks that there could possibly be any racism at work. For those who have made their peace detente with the underlying, long established, subtle and hard to reverse discrimination realities, I’d hazard that this denial is in fact much more of a discouraging stimulus.
Look, I’m not suggesting that the NIH needs to only pursue implicit racial bias in the minds of the study section members. The many goals and hypotheses that are on the table are admirable and have a chance of payoff. But they should not shy away from the possibility that there is bias at the level of peer review either.

37 Responses to “On the perceptions of well intentioned people regarding the NIH’s little diversity whoopsie”

  1. Your comment is awaiting moderation.

    I just felt that it was important not to just emphasize that there are black PIs whose applications get not just grudging acceptance but actual enthusiasm from reviewers to counter the the negative side of the data which I think gives the false impression of a peer review process whose decks are stacked against black PIs.

    In other news, we have a black president. Therefore, racism is dead.


  2. anon Says:

    I’m not pulling up the article right now, but I remember their metric was grant acceptance rather than priority scores. Using priority scores could get at some of the actual racism concerns. If African American applicant are more likely to have lower scored grants accepted over their own, that could point to a major internal NIH problem (I doubt this would be seen).
    If the individual reviewers’ scores are farther apart for African American applicants, that could be a sign that individual reviewers are biased (not saying conscious bias) against African American applicants and that’s enough to drop a significant number below funding levels. If this is the case, one might even be able to look at specific reviewers to see how much they do or do not diverge from co-reviewers in a study section depending on applicant race.


  3. Spiny Norman Says:

    Remember that Eisen is HHMI and that he may be able to see only what’s happening at the top. The article indicates that from his perspective, below that it’s turtles, all the way down.


  4. bikemonkey Says:

    the post by Dr. Eisen, and one a day or so ago from DM on this blog, address the scores. It’s kind of the point….


  5. becca Says:

    Ok, Yeah. I when I read Eisen I couldn’t tell if he didn’t read the actual report, and therefore didn’t know many of those things were controlled for, or if he ignored the things that were controlled for and wanted… something else (not well-defined) to be factored in.
    I mean, it’s not like there *aren’t* highpoints of the report. If you’re a hispanic or an asian from an English speaking country, yay?
    But the highpoint of the report is NOT what Eisen seems to think.


  6. bikemonkey Says:

    becca, I think the “highpoint” of the report is determined by large extent what the reader brings in with himself. Everyone is going to be motivated to point the finger elsewhere and go confirmation-biasing for support. Who knows? Maybe the bigots commenting over at the Chronicle for Higher Ed, Salon and elsewhere are correct, the black applicants just suck.


  7. Maybe the bigots commenting over at the Chronicle for Higher Ed, Salon and elsewhere are correct, the black applicants just suck.

    Is that the same as saying that the black applicants’ educational and scientific experiences–starting from when they were schoolkids–have been relatively impoverished compared to the white applicants, and leads to a difference in application quality?


  8. I’m sorry you have chosen to retract the “well intentioned” label – that was completely unwarranted.
    I stand by my assertion that people leapt to the conclusion that racism in peer review underlies the results presented by Ginter et al – in my blog post I cited specific examples from the Boston Globe editorial page and two members of Congress. I also listened to an NPR story in which Raynard Kington attributed this to unconscious bias in peer review. I wasn’t making this up.
    As I said clearly in my post, I am a strong supporter of efforts to promote minority involvement and success in science. And it is precisely because of this that I see unsupported claims that peer review is biased as a major problem. The major problem we face in these efforts is that way too few black students enter science. Why this is true is undoubtedly complicated. I’m sure poor science education plays a major role. But it’s my experience in over a decade of trying to recruit a more diverse graduate student population, that black students interested in science are turned off by seeing a field that has so few black faces in it. And it certainly can’t help if they think, on top of that, that they won’t be treated fairly in peer review if they choose to become a researcher.
    I strongly support the NIHs proposal to investigate this issue directly (though I think the experiments are more complicated than they’ve proposed). But I think it was irresponsible for people to highlight the possibility that peer review is racist before they do not have direct evidence that it is the case. Maybe they’ll find it – I don’t discount the possibility, though I would be exceptionally dismayed. But all of my experience in peer review suggests that they will have to look elsewhere.


  9. becca Says:

    Michael Eisen- let’s just say, for the sake of argument, that 100% of the difference in scores was attributable to bias. Heck, in theory, it could have been 110%- the black applicants could have been better than the white applicants, for any given score range.
    If so, is it ethical to continue to try to convince black students to go into the profession? Is it ethical to lie to them about the fairness of the system if that’s the only way to get them into it (and eventually make the system more fair)?
    I don’t actually have a definitive answer to those questions, for my own moral compass. But I’m curious how you see it.


  10. I believe the most important thing is to provide people with accurate information – and that there are very few situations (this isn’t one of them) where it’s ok to do something under false pretenses. If I knew that there were bias in peer review – even 1% – I would feel morally obligated to tell people about it so they could make the decision on their own with full information.
    And, I am not suggesting we bury the data or shy away from doing the experiments to figure out if there’s bias in peer review. But it was incredibly irresponsible of the authors to publish this paper in a high-profile venue and then going around insinuating that it is the result of biased peer review without doing the experiments to test their hypothesis.


  11. BikeMonkey Says:

    I disagree entirely that one had to have 100% proof positive of racial bias before you can consider the possibility. Not least of which is that until neuroscience takes several quantum leaps forward, we can never truly *know* the (sub)conscious mind of the reviewer.
    Setting an impossible burden of proof is reminiscent of some rather unpleasant denialist behavior.
    I will agree with you that the media pursued the racial bias angle to the exclusion of a balanced portrayal. That is what they do, and I condemn them for it. It is unfair, however, to paint the NIH with the media story. Even with Kington’s interview, how do you know of the NPR folks pull-quoted? How do you know he didn’t spend 60 min on the complicated picture and the extracted the quote tha made *their* story for them?


  12. I like the way you every time you disagree with me on a point you add some ad hominem quip about how my point of view reminds you of some racist meme. Could you please cut that shit out?
    At no point did I say that one had to have 100% proof positive racial bias before considering the possibility. I said that they should have done the experiments they proposed before making a deal about data that only hints at the possibility. If someone submitted an experimental paper with data that hinted at a very significant conclusion, but they hadn’t yet done the key experiment to nail it down, the reviewers would have said “do the experiment and call us when it’s done”. That would I think they should have done.
    (I would have said Science should have held them to a higher standard, but after the arsenic paper I’m no longer sure they have any standards)


  13. Jadehawk Says:

    I don’t think it’s correct to say that black students don’t already assume there’s bias against them, and that a story in the news about bias would actually be making something look more negative to them.
    In my experiences, many members of minorities (in the sociological sense) tend to have had so many experiences with bias against them that they come to assume it as a default. So, as bikemonkey said, it’s not likely that a news story about bias is going to make a difference, but people denying the possibility/likelihood of bias likely might, because such denial would mean if bias exists, then it will continue existing unchallenged.


  14. CoR Says:

    @Michael, I couldn’t agree more re: the current standards of Science. I think however that you are taking the third reviewer stance by suggesting another series of experiments before publication, and in this case, I would suggest there is an overall benefit to the system by exposing any iota of racial disparity, wherever its origination. Certainly some brown people will be dismayed and discouraged from research careers if they think their chances are lessened in the peer review stage. As a female, non-tenured, TT researcher with 2 very young children, I personally have gone thru ‘What’s the point?’ phases in light of research that shows TT women with young children are (~)27% less likely than men (and men with children) to get tenure. I’m also currently preparing a fellowship application that, in the past 2 years, has been awarded to ~20-30% women. So the numbers are depressing. But I also contend that this information is power — I can do my part to support people that are in the minority, I can bring these stats up with people in positions of power that might not be aware of the research. If someone were to have discouraged the release of this information in lieu of MOAR research on the topic, and in fear that it would be discouraging to me, then I would feel not only discouraged, but also completely patronized.


  15. @CoR – i HATE that third reviewer – I was just having an argument with them…
    I agree with you completely. I believe more strongly than anyone in data and its free and rapid release. My point wasn’t that they shouldn’t have published the data, rather that I don’t that the authors have encouraged the media to run with an interpretation (that peer review is biased) that the data don’t establish is true in the face of other possibilities. And that if they wanted to make the case for biased review, they needed to do other experiments.


  16. Can we pivot the topic here? We can disagree about what the causes of the differential success are, but I think we all agree that it’s really bad. To me the biggest problem is how few black scientists there are. Despite considerable effort on my university’s part to encourage minority undergrads to pursue careers in research, the number of black applicants to our graduate programs remains frustratingly small. The number of black graduate students in the sciences is smaller than the number in law school and med school – so there is clearly something relatively unappealing about careers in research. This is why I worry most about additional things that discourage students inclined to join the field from doing so, and think we need to do a much better job of highlighting successes and opportunities. But I also think it’s clear that the things we do – summer research programs, etc… – aren’t working or aren’t enough and that we need new ideas.


  17. Bashir has a post up regarding racial disparities in success rates at NIH and NSF. If NSF’s version of peer review manages to be almost fair, why can’t NIH’s?
    While I think the Ginther report speaks for itself, Bashir’s data supports the idea that there is a systematic (perhaps unconscious) bias in the NIH’s processes. The alternative is that peer review is bias-free and that only in biomedical science are black scientists are less talented than everyone else.


  18. Difference between NSF and NIH is interesting.
    I don’t have nearly as much experience with NSF as I do with NIH, but the process itself isn’t all that different. What is different is that the NSF proposals have a “broader impact” section that often reveals the PIs race, and highlights how they will promote diversity in their field – something that the NSF strongly promotes and, in my experience, its reviewers strongly support.
    So I wonder whether the NIH might, somewhat counterintuitively, increase the diversity of its grantees by revealing their race of applicants. It’s something the NIH should consider in the experiments they’re proposing – in addition to blinding some review panels to the PIs identity, they should also reveal PIs race explicitly to different panels. Given how strongly committed most people in science I know are to promoting diversity, and given the data from the NSF, it’s possible black PIs would do better when their race is known.


  19. whimple Says:

    The alternative is that peer review is bias-free and that only in biomedical science are black scientists are less talented than everyone else.
    That’s true so long as “talented” means “put forward the hypotheses the majority non-black reviewers want tested”. Another possibility is that the black applicants are putting forth hypotheses that are different, but just as worthy. To me this is the about the reason why you’d want diversity, otherwise if it’s just the same old science, who really cares what the race of the proposer is?


  20. DrugMonkey Says:

    The reason, whimple, is that if the mysterious combination that leads to the best science is distributed randomly with respect to race, sex, or any other characteristic, systemic bias on those characteristics will hinder science


  21. whimple Says:

    The “best” science is (by definition of ‘fundable’) that most homogenously acceptable to the review panels. If we do analysis that reveals transgendered PI’s (for example) get scored more poorly, it doesn’t mean there is necessarily systemic bias against the transgendered, just that with a different perspective on their life experiences, the transgendered don’t write homogenously acceptable proposals. In other words, the reviewers have a bias against a difference in perspective, not a difference in race. That’s a much worse problem than garden-variety racism would be, from the standpoint of scientific progress.


  22. I find it difficult to see how being transgendered or black or while or Albanian or anything influences the way you think about stem cells. DrugMonkey’s view of the problem is right – the problem is that the probability of being a great scientist and race are not inherently coupled in any way – so any bias in the system by definition reduces the probability of funding the best science.
    I’m also not sure I totally agree that getting funded requires toeing the line. Yes, there is a bias against the most out-there kind of ideas, and it’s amazing how many of the biggest advances in science were rejected as grant proposals. But the worst scoring proposals, in my experience, are the ones that are completely boring – that offer no hope of discovering anything new.


  23. whimple Says:

    How do you think being black influences the way you think about priorities in health-related research priorities? Answer: unless you’re black, you have absolutely no clue. I bet the stem cell grants from blacks do just as well as the stem cell grants from everybody else.


  24. I find it difficult to see how being transgendered or black or while or Albanian or anything influences the way you think about stem cells.

    Le sigh.


  25. whimple – as tabak and collins say in their response to the ginter article they looked at the success of black PIs across different research areas and found that the problem was systemic – which is not consistent with your hypothesis
    and DrugMonkey – i don’t see how sherley counters whimple’s point since, although he didn’t get tenure at MIT, he is actually fairly well-funded, having received a $500,000/year NIH pioneer award


  26. whimple Says:

    M.E. – you mean this?
    We have also explored the possibility that underrepresented applicants traditionally apply for support in scientific fields that have lower success rates. Our preliminary analyses indicate that this is not the case; however, more in-depth study is needed.
    Field choice is not quite the same thing as scientific perspective. The question I raise is whether within a given field, the black investigators have a different perspective on which issues are worth investigating and what the best approach for those investigations would be.


  27. whimple: My perspective on my work is heavily influenced by my life experiences – though not in any easy to categorize way – and I assume this is true of everyone. So, yes, I can see how black PIs might have different perspectives on all sorts of things. My original point was that I would expect this effect to be less race-specific in a more technical field like stem cells, but now that I think about it more, I’m not so sure. If it were true that there were some common characteristics to the ways black PIs approached problems that were different from the way white PIs approached problems (which, given the way race pervades American society is not a crazy idea), then it could produce a systematic tendency for white reviewers (who are obviously the majority of reviewers) to give lower scores to black PIs, not because they’re black per se, but because their ideas and approaches don’t make as much sense to them. And I’m sure, to some extent, that this is true.


  28. abda Says:

    I just read the Science paper and read over some of the commentary, and I have to say that my interpretation of their data was similar to Eisen’s: peer review is likely *NOT* the major problem, but rather the black applicants’ other scientific background seemed to be driving their poor success rate. Even after correcting for publications etc, there must be many other variables that make it harder for blacks (but NOT hispanics, asians, et al) to succeed in winning grants. To me, one of the strongest pieces of data was the need for multiple submissions by black PIs and the fact that many failed to resubmit grants. This seems like a key problem that might be addressable: black PIs must be getting very discouraged after the initial review, and may have other external influences that prevent resubmission. Maybe their chairmen are less supportive? Maybe they aren’t getting the same startup packages to support the lab during the first grant submission? Are the best students, techs, and postdocs steered toward or away from their labs? These are some of my first guesses about other LIKELY problems black PIs may encounter, and which seem more likely to account for the 10% difference than the peer review itself.


  29. BikeMonkey Says:

    Interesting how you can read through the laundry list of factors used to test the contribution of PI characteristics other than race, grasp that the relationship still held and yet hold firm to your belief that surely, surely some unstated other factor simply must be the cause. Instead of review bias, I mean.
    Also fascinating how you pick two aspects of the revision issue, blow straight past the one that would be linked to bias (the *need* to revise) and start ‘splainin away on the one that points the blame at the applicant.


  30. Gruffi Gummi Says:

    Why aren’t Asians (who definitely are not lily-white WASPs) not affected by the alleged racial bias?


  31. Gruffi Gummi Says:

    Oops, my grammar sucks (perhaps because I am not a native-speaking, lily-white WASP). But the question remains valid: why are Asians not affected?


  32. abda Says:

    Asians were affected, but this was driven largely by non-native speakers (actually I think they stratified the non-US trained, and assumed these Asians were non-native speakers).
    I didn’t “blow by” the initial low success rate, I mentioned that along with the failure to resubmit when I said “one of the strongest pieces of data was the need for multiple submissions by black PIs and the fact that many failed to resubmit grants”. I don’t think we know enough about why black PIs have a lower initial success rate, but in general the first time success rate is pretty low for everyone, and overall success means you have to be prepared to resubmit. But black PIs don’t. Don’t you agree that this is a well-defined, possibly addressable problem for black PIs??? Then I speculated about the possible underlying issues that I think may account for a bigger part of the 10% gap. Maybe one way to encourage resubmission by black PIs is to increase “bridge” funding such as R56. Any other ideas?
    Actually, I would guess that racial bias does arise during the peer review, not overtly but in more subtle ways such as how people’s CVs are interpreted (e.g., if an applicant went to Morehouse or another HBCU this may not get counted as strongly as someone who graduated from Georgetown). But until NIH does its follow-on study comparing “de-identified” applications to standard applications, I don’t think we know how big the effect of bias during peer review is. I bet it isn’t that big. I bet other issues such as getting lowballed during recruitment and having grad students steered towards other labs are bigger problems. And not having departmental support (=$ to keep going) for resubmission. One of the reasons I think this is because these are more face-to-face decisions, and I think that in general, mostly white dept. chairs may be less supportive of their scant black faculty.
    Not sure how you got to “blame the applicant” from my comment, or if you are just mad about me disagreeing with you about what is driving the 10% gap.


  33. bikemonkey Says:

    They did stratify Asian by domestic and foreign PHD granting institution but I don’t believe there was any attempt to determine language status abda? It is, of course, only one obvious inference. There are other hypotheses, not as directly related to the ability to write in English.
    I didn’t “blow by” the initial low success rate, I mentioned that
    Yes, you mentioned it but then focused all your remarks elsewhere. As you are here. I just find it very fascinating to see how hard people are working to find ways to question the notion of systematic bias in review. And reacting in high dudgeon that the official NIH commentary might admit the possibility of such a thing.
    Don’t you agree that this is a well-defined, possibly addressable problem for black PIs?
    Yes, I certainly do.
    Maybe one way to encourage resubmission by black PIs is to increase “bridge” funding such as R56. Any other ideas?
    Increase all and sundry ways that Program picks up grants out of the strict order of priority score. The analyses in Ginther (and the Tabak/Collins commentary) seem to claim that for a given score, the Program behavior is similar, indeed the Supplementary figure S1 supports this notion. But if there *is* bias at the point of review then what Program should be doing is picking up Black applicants’ proposals preferentially.
    I bet it isn’t that big. I bet other issues such as getting lowballed during recruitment and having grad students steered towards other labs are bigger problems….and I think that in general, mostly white dept. chairs may be less supportive of their scant black faculty.
    While I agree that there are very likely many small insults having a big cumulative effect, I think all of us with opinions are just making shit up, absent hard data. I am wary of people or institutions who seem very keen to wall of a particular subset of the available hypotheses for action / nonaction. Particularly when an institution moves to wall off a direction that would point the finger at themselves/demand a specific set of remedies to ride off tilting at problems that really are the domain of someone else. If the NIH points the finger at those theoretically racist and unsupportive department chairs in the majority-white Universities without examining their own clay feet, this would be a very bad thing to do.
    Scientific peers who think of themselves and their study section buddies as “surely not racist!” and therefore wall off that set of hypotheses are similarly problematic.
    or if you are just mad about me disagreeing with you
    I assure you that my emotive valence toward your remarks is far from “mad”. You would be closer to the mark with “amused”.


  34. Spiny Norman Says:

    “The “best” science is (by definition of ‘fundable’) that most homogenously acceptable to the review panels.”
    I call bullshit on that, too. Believing that requires a real misunderstanding of study section dynamics.
    In an environment of sub-10% paylines, the FUNDABLE science is the science that can get at least two of the typical three reviewers to go out of their way to advocate for your grant when it’s under discussion.
    In short: your proposal has to be good enough that two or three other people will put their own ass on the line for it, and do so in preference to the other applications that they’ve read.


  35. Drinkycro Says:

    I don’t know how I found this old thread, but this is some of the fucking stupidest stuff I’ve read on the internet: liberal whites arguing with other liberal whites over who’s the most liberal white on the science blogosphere.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: