A reaction to the racial disparity in NIH grant success rates

in which a picture is worth a thousand words….

but you need to go read the post by Hermitage.
Also, here’s a link to the Chronicle bit on the disparity in undergraduate scholarship awards mentioned in the post at The Hermitage.

White students make up 62 percent of full-time students enrolled in four-year colleges but receive 76 percent of institutional merit scholarships; and white students are 40 percent more likely to receive private scholarships than minority students are.

And the OER Rock Talk blog thread on the racial disparity continues…

On the perceptions of well intentioned people regarding the NIH’s little diversity whoopsie

BikeMonkey Guest PostBelieve you me, it does not escape my attention that instead of working on my grant that is due in approx one month’s time, I am talking about the Ginther et al report. No, I am not obligated to say jack squatte about it. These little distractions are optional. As is the mentoring “tax” that the senior author of that report, Raynard Kington, discussed. Likewise participation in the well-intentioned “enhance the diversity” efforts of our Universities and professional societies. Yet…here we are.
The DM has been taking a few whacks at what appears to be the reasonably well-intentioned musings of one Michael Eisen. I am fascinated by the latter’s defensive comment:

But I’m shocked at how many people leapt to the immediate conclusion that the peer review system penalizes applications from black PIs when we know that black scientists face all sorts of other obstacles that both discourage them from entering the field in the first place and make it more difficult for them once they are here. I just felt it was pretty naive on the NIHs part to expect anything different – as if they thought the things they were doing to promote the careers of black scientists had actually solved all the problems they face. And then to look at the data and cry racism is just making the problem even worse by both discouraging black scientists from joining the field and making it harder for them to recruit people once there here.

“leapt to the immediate conclusion“. “cry racism“. Yes, perhaps I should reconsider the “well-intentioned” bit. These are stock in trade phrasings of anti-affirmative action people.

Your Grant in Review: Productivity on Prior Awards

A commenter named ES over at writedit’s blog asked:

I am wondering if I can conduct unfunded research projects using my current RO1 support since I have more experience and more publications on these unfunded projects. In contrast, testing our hypothesis proposed in my funded RO1 is practically not promising. If I decided to do something else which was not originally proposed in my funded RO1, how can I renew my funded RO1 or submit the report down the road.

and expands upon the theme:

the issue here is that my unfunded and funded project are total irrelevant. It is almost impossible to convert my unfunded project to something which is even close to the funded project. Can I do whatever Science takes me to with the unfunded project, and submit the unfunded project as new applications later, instead of renewal of my RO1? Will this cause a bad record for my academic career?

There was also a related query from Saheli in that thread:

2. Should the renewal reflect a substantial continuity to the current grant?
3. What the ingredients of a successful renewal?

The question from ES led to some startlingly bad advice.

Another white-hot discussion of NIH grant review over at Rock Talk

Whew, I can feel the keyboards screaming in agony as the OUTRAGED PI’s hammer away at their comments.
The Rock Talk blog entry is supposed to be about What Can You Do When Two Reviewers Contradict Each Other. Now admittedly, the blog advice is, well, laughable.

In this situation we encourage you to use your best judgment. Take a look at all of the reviewer’s comments and criterion scores* and the scientific review officer’s summary of the discussion and then make a decision on how best to proceed from there. If the summary statement is unclear, you can always contact your program officer for clarification.

ahahahahaa, no wonder people are pissed about that non-answer to the question they themselves have posed in the blog entry!

NIH want to train Early Career Reviewers

This is fantastic.

…CSR is piloting a new program that we call the early career reviewer, where we will take complete novice reviewers, people who have not reviewed for NIH before, very early in their career, probably new investigators.

More thoughts on the matter from Your Humble Narrator and Prof-Like Substance.

Things NIH Program Officers probably shouldn’t say to applicants

I have blogged, now and again, about the ~four-tiered nature of the selection process for NIH grant awards. These consist of initial peer-review, Program Officer expression of programmatic interests, the National Advisory Council (of peer scientists) for each Institute or Center of the NIH and the Director of each IC. We tend to clump the latter three into “Program”, since they really all do express the interests of a particular IC whereas the initial peer review process focuses so strongly on the quality of the science and the strength of the overall proposal.
Unlike the apparent position of many fellow scientists, I have no problem with this multi-tiered selection process nor, more pointedly, do I have a problem when Program interests override or overturn the strict priority order / priority score that comes out of initial peer review.
I have even mentioned now and again that I think a little more transparency from Program on how they select applications outside of the review order would go a long way to damping down the whinging.
A bit of Program Officer transparency has emerged, however, that gives me pause.

Your Grant in Review: Appealing the review outcome

A recent Notice from the NIH (NOT-OD-11-064) indicates that there is a need to standardize and refine the appeal process.
Here’s what struck me on seeing this Notice pop up: I bet there has been a massive uptick in the rate of appeals since the sunsetting of the A2 and the threats to rigorously weed out thinly concealed revisions as “new” submissions.
One viewpoint on the wisdom of appealing the scoring of your grant proposal that is very common is captured in this comment over at the NIGMS blog:

Based on everything I have read about the appeals process on various Web pages of the NIH and Institute Web sites, it seems like you’d have to be extremely foolish and poorly informed to bother appealing.

NIGMS Director Berg responded:

Lake Wobegon effect in NIH grant review

All the Investigators are strong….and the Environments are above-average.
The “Investigator” and “Environment” criteria have been an explicit part of NIH grant review since forever, and have been given approximately equal weight with Approach, Significance and Innovation.
The blurbs in the official NIH notice on the current scheme read:

Investigator(s). Are the PD/PIs, collaborators, and other researchers well suited to the project? If Early Stage Investigators or New Investigators, do they have appropriate experience and training? If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)? If the project is collaborative or multi-PD/PI, do the investigators have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project?
…
Environment. Will the scientific environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the scientific environment, subject populations, or collaborative arrangements?

I always had the distinct impression these were essentially throwaway criteria because they were almost always rated very highly. Sometimes the “Investigator” criterion would be a place to cap on the more-junior career status or lack of productivity but for the most part it was treated very politely.
Sally Rockey has recently posted the verification of this impression on the OER blog.

Another fascinating proposal on how to fix the NIH

Really, first there was the diatribe of one D. Noonan that kept giving and giving and giving. And seriously, people, you need to comment and remain engaged. Some clown wants to publish his rant in

…ASBMB Today, the monthly news magazine for the American Society for Biochemistry and Molecular Biology (www.asbmb.org/asbmbtoday).

Rumor has it that this is being passed around as if it represents the will of the damn people, or some such nonsense. And these diatribes from left field have a way of being used to support existing agendas at the NIH. Your voices of reason need to be heard so make sure to comment over at Rock Talk.
Oh, and go read Odyssey’s comment on mid-late career folks who simply will not listen to anyone about the quality or importance of their research programs.
Meet you back here after the jump…

Special consideration on NIH applications for “Unfunded Experienced Investigators”?

A lengthy comment on the new blog of the head of the Office of Extramural Research at the NIH is an absolute goldmine. One of the specific proposals is that the NIH should now have a special initiative to fund grants for investigators who have previously held NIH awards but do not currently hold NIH funding.

Create funding vehicles that emphasize funding of smaller research operations…As done with the New Investigators, create a category of “Unfunded Established Investigators” and fund this category in the 25-30% range.

No offense but this bloody well already exists.
1) When they were using the R56 Bridge mechanism liberally, I seemed to notice that it was going preferentially to established investigators. You know, the long time buddies of Program staff who just happened to be hitting a dry spot.
2) When the squeeze came down around 2006 or so, I had conversations (Hi Ed!) with more than one PO in which they made it overwhelmingly clear they were all about “saving” their established investigators who were (allegedly) “going to have to close their labs”. My point that these folks had hard money jobs with tenure and were not at the same risk as newly minted Assistant Professors fell on deaf ears, I can tell you. As did my points about so what if they can’t compete, what about those of us who ‘should’ be just hitting our stride who fully deserve the multiple awards if we can successfully compete for fundable scores.
3) We’ve talked several times about the psychology of the study section and the way people are biased in ways that do not follow the strictest assessment of the merits of an application. It is a human endeavor and it is simply ignorant to pretend human judges are ‘unbiased’, or ever could be. One of the biases is that experienced investigators are more like the reviewers and therefore the reviewer sympathies are going to be with their generational peers, not the n00bs.
So no, I don’t favor a creation of a special category of help for experienced investigators who run out of funding. Yes, even though that would in theory be to my own benefit some day.

Jeremy Berg Demystifies the Funding Decision Process at NIGMS

A new post from Director Jeremy Berg of NIGMS gives an overview of the process by which his Institute makes final funding decisions. The part about the gray zone decisions was particularly interesting:

For each application, the responsible program director presents the scientific topic as well as factors such as whether the applicant is an ESI or new investigator, how much other support the applicant has (particularly if the application represents the only support available to the investigator), whether the Council has given us specific advice on the application, whether the scientific area is perceived to be particularly exciting, and how much other research we already support in the general area of the application. The other members of the unit listen to these presentations, and the group then produces a prioritized list of applications.

Emphasis added. Note that? Program staff have to be strong advocates for your application during the selection process that occurs in the dim twilight of the gray / pickup zone. This means you have to write an application that trips their triggers and that they can understand. This is almost as important as speaking clearly to the people who are reviewing your grant at the study section level. This is also why I point out that schmoozing Program staff at scientific meetings is your opportunity to advocate in a general way for the things that you feel are important in science, not just your specific proposal at the current time. You want to educate them and bring them around to your way of seeing things as a gut-feeling or belief. If they understand the arguments even before they see your specific application, they are going to be more equipped, more interested and therefore a better advocate for your proposal.

Is the NIH trying to get rid of smaller laboratories?

In a recent comment thread Comrade PhysioProf revisited a claim or insinuation he’s made before.

These changes–like almost all of the enhancing peer review changes–are really designed to hasten attrition of substantial numbers of marginal PIs/insitutions that were attracted into the system during the budget doubling of the late 90s/early 00s. These people/institutions need to be flushed from the system so that success rates can normalize.
Of course, this kind of plan cannot be openly sold to Congress, so it is packaged as “enhancing peer review” and “funding grants sooner”, but it is really all about making it easier for quality investigators to get their grants and more difficult for marginal investigators. If you were the kind of investigator who could never get a grant funded until you had been beaten to an A2 by study section, or who needed 25 full pages of preliminary data to convince a study section to fund your boring/incremental science, now you’re never gonna get a grant and you’re gonna be flushed.

He’s talking about me, so naturally I take umbrage at this assertion.

Initial outcome of limiting NIH apps to a single revision?

The latest Peer Review Notes issue [pdf] from the Center for Scientific Review of the US National Institutes of Health reports some initial data on their move to eliminate the second round of revision of grant applications.
Personally, I thought this was very likely only a partial fix to the problem. As I’ve discussed, I was no fan of the way that available rounds of revision led study sections to refuse to get serious about reviewing apps until they had returned from at least one round of review. So I think it is a good idea to try to break this particular time-wasting bit of study section culture.

What about when Program does not interfere with the initial priority score of NIH applications?

NIGMS has released their annual update on the review outcome for NIH R01 applications directed their way for potential funding (see bottom of this post for prior Fiscal Year links). The most salient figure is the histogram of percentile ranks arising from the initial review, identified by whether they were selected for funding or not.
As far as I’ve ever seen, NIGMS is the only NIH Institute or Center that does this. As you can see from the figure, one of the most interesting features here is that we can identify how many “skips” and “exceptions” are in their pool of applications.
Skips refer to grant applications which appeared to score well within the apparent (or published) payline and did not get funded for one reason or other. Exceptions refer to those applications which did not score within the apparent payline but were selected for funding anyway. The latter are substantially more common than the former, of course. We’ve talked about these exceptions (i.e., “pickups“) before.

Where to put your Preliminary Data in the new 12-page NIH grant application

One of the biggest changes that has emerged in my grant writing since the NIH R01 application was dropped from 25 to 12 pages (for the meat of the proposal) and restructured has been the way that I present my preliminary data. A recent exchange with @superkash reminded me that I’ve never brought this up on the blog and as usual, I’m curious as to how my readers are approaching this situation.
Under the old format, there was a specific section for Preliminary Data (or a Progress Report for competing continuation applications). In fact it was one of the three major ones (Background and Significance and Research Plan were the other two). With the restructuring, this evaporated. The three major sections are now Significance, Innovation and Approach.
So where are the preliminary data supposed to go?