Jocelyn Kaiser at ScienceInsider has obtained data on PI numbers from the NIH.

NIH PIs Graphic

Nice.

I think this graph should be pinned up right next to Sally Rockey’s desk. It is absolutely essential to any attempts to understand and fix grant application success rates and submission churning.

UPDATE 03/12/14: I should have noted that this graph depicts PIs who hold R01-equivalent grants (R01, R23, R29, R37 with ARRA excluded). The Science piece has this to say about the differential from RPG:

NIH shared these data for two sets of grants: research project grants (RPGs), which include all research grants, and R01 equivalents, a slightly smaller category that includes the bread-and-butter R01 grants that support most independent labs.

NIH-PIs-RPG-R01eqBut if you read carefully, they’ve posted the excel files for both the R01-equivalents and RPG datasets. Woo-hoo! Let’s get to graphing, shall we? There is nothing like a good comparison graph to make summary language a little more useful. Don’t you think? I know I do….

A “slightly smaller category” eh? Well, I spy some trends in this direct comparison. Let’s try another way to look at it. How about we express the difference between the number of RPG and R01-equivalent numbers to see how many folks have been supported on non-R01/equivalent Research Project Grants over the years…
NIHPI-RPGdifferentialWell I’ll be hornswaggled. All this invention of DP-this and RC-that and RL-whatsit and all the various U-mechs and P01 (Center components seem to be excluded) in recent years seemingly has had an effect. Sure, the number of R01 equivalent PIs only slightly drifted down from the end of the doubling until now (relieved briefly by the stimulus). So those in NIH land could say “Look, we’re not sacrificing R01s, our BreadNButter(TM) Mech!”. But in the context of the growth of nonR01 RPG projects, well….hmmm.

While I’m getting all irate about the pathetic non-response to the Ginther report, I have been neglecting to think about the intramural research at NIH.

From Biochemme Belle:

The takeaway message from the report of Ginther and colleagues (2011) on Race, Ethnicity and NIH Research Awards can be summed up by this passage from the end of the article:

Applications from black and Asian investigators were significantly less likely to receive R01 funding compared with whites for grants submitted once or twice. For grants submitted three or more times, we found no significant difference in award probability between blacks and whites; however, Asians remained almost 4 percentage points less likely to receive an R01 award (P < .05). Together, these data indicate that black and Asian investigators are less likely to be awarded an R01 on the first or second attempt, blacks and Hispanics are less likely to resubmit a revised application, and black investigators that do resubmit have to do so more often to receive an award.

Recall that these data reflect applications received for Fiscal Years 2000 to 2006.

Interestingly, we were just discussing the most recent funding data from the NIH with a particular focus on the triaged applications. A comment on the Rock Talk blog of the OER at NIH was key.

I received a table of data covering A0 R01s received between FY 2010 and FY2012 (ARRA funds and solicited applications were excluded). Overall at NIH, 2.3% of new R01s that were “not scored” as A0s were funded as A1s (range at different ICs was 0.0% to 8.4%), and 8.7% of renewals that were unscored as A0s were funded as A1s (range 0.0% to 25.7%).

I noted the following for a key distinction between new and competing-continuation applications.

The mean and selected ICs I checked tell the same tale, i.e., that Type 2 apps have a much better shot at getting funded after triage on the A0. NIDA is actually pretty extreme from what I can tell- 2.8% versus 15.2%. So if there is a difference in the A1 resubmission rate for Type 1 and Type 2 (and I bet Type 2 apps that get triaged on A0 are much more likely to be amended and resubmitted) apps, the above analysis doesn’t move the relative disadvantage around all that much. However for NIAAA the Type 1 and Type 2 numbers are closer- 4.7% versus 9.8%. So for NIAAA supplicants, a halving of the resubmission rate for Type 1 might bring the odds for Type 1 and Type 2 much closer.

So look. If you were going to try to really screw over some category of investigators you would make sure they were more likely to be triaged and then make it really unlikely that a triaged application could be revised into the fundable range. You could stoke this by giving an extra boost to triaged applications that had already been funded for a prior interval….because your process has already screened your target population to decrease representation in the first place. It’s a feed-forward acceleration.

What else could you do? Oh yes. About those revisions, poorer chances on the first 1-2 attempts and the need for Asian and black PIs to submit more often to get funded. Hey I know, you could prevent everybody from submitting too many revised versions of the grant! That would provide another amplification of the screening procedure.

So yeah. The NIH halved the number of permitted revisions to previously unfunded applications for those submitted after January 25, 2009.

Think we’re ever going to see an extension of the Ginther analysis to applications submitted from FY2007 onward? I mean, we’re seeing evidence in this time of pronounced budgetary grimness that the NIH is slipping on its rather overt efforts to keep early stage investigator success rates similar to experienced investigators’ and to keep women’s success rates similar to mens’.

The odds are good that the plight of African-American and possibly even Asian/Asian-American applicants to the NIH has gotten even worse than it was for Fiscal Years 2000-2006.

NIH Blames the Victim

January 16, 2014

Just look at this text from RFA-RM-13-017:

The overarching goal of the Diversity Program Consortium is to enhance the diversity of well-trained biomedical research scientists who can successfully compete for NIH research funding and/or otherwise contribute to the NIH-funded workforce. The BUILD and NRMN initiatives are not intended to support replication or expansion of existing programs at applicant institutions (for example, simply increasing the number of participants in current NIH-funded research training or mentoring programs would not be responsive to this funding announcement).

The three forgoing major initiatives share one thing in common: Make the black PIs better in the future.

The disparity we’ve been talking about? That is clearly all the fault of the current black PIs….they just aren’t up to snuff.

Specifics? also revealing

 

Goals for the NRMN include the following:

  • Working with the Diversity Program Consortium to establish core competencies and hallmarks of success at each stage of biomedical research careers (i.e., undergraduate, graduate, postdoctoral, early career faculty).

  • Developing standards and metrics for effective face-to-face and online mentoring.

  • Connecting students, postdoctoral fellows, and faculty in the biomedical research workforce with experienced mentors, including those with NIH funding, both in person and through online networks.

  • Developing innovative strategies for mentoring and testing efficacy of these approaches.

  • Active outreach is expected to be required to draw mentees into the network who otherwise would have limited access to research mentors.

  • Developing innovative and novel methods to teach effective mentoring skills and providing training to individuals who participate as mentors in the NRMN.

  • Providing professional development activities (grant writing seminars, mock study sections, etc.) and biomedical research career “survival” strategies, and/or facilitating participation in existing development opportunities outside the NRMN.

  • Enhancing mentee access to information and perceptions about biomedical research careers and funding opportunities at the NIH and increasing understanding of the requirements and strategies for success in biomedical careers through mentorship.

  • Creating effective networking opportunities for students, postdoctoral fellows, and early career faculty from diverse backgrounds with the larger biomedical research community.

  • Enhancing ability of mentees to attain NIH funding.

To my eye, only one of these comes even slightly close to recognizing that there are biases in the NIH system that work unfairly against underrepresented PIs.

Jeremy Berg made a comment

If you look at the data in the Ginther report, the biggest difference for African-American applicants is the percentage of “not discussed” applications. For African-Americans, 691/1149 =60.0% of the applications were not discussed whereas for Whites, 23,437/58,124 =40% were not discussed (see supplementary material to the paper). The actual funding curves (funding probability as a function of priority score) are quite similar (Supplementary Figure S1). If applications are not discussed, program has very little ability to make a case for funding, even if this were to be deemed good policy.

that irritated me because it sounds like yet another version of the feigned-helpless response of the NIH on this topic. It also made me take a look at some numbers and bench race my proposal that the NIH should, right away, simply pick up enough applications from African American PIs to equalize success rates. Just as they have so clearly done, historically, for Early Stage Investigators and very likely done for woman PIs.

Here’s the S1 figure from Ginther et al, 2011:
Ginther-S1

[In the below analysis I am eyeballing the probabilities for illustration’s sake. If I’m off by a point or two this is immaterial to the the overall thrust of the argument.]

My knee jerk response to Berg’s comment is that there are plenty of African-American PI’s applications available for pickup. As in, far more than would be required to make up the aggregate success rate discrepancy (which was about 10% in award probability). So talking about the triage rate is a distraction (but see below for more on that).

There is a risk here of falling into the Privilege-Thinking, i.e. that we cannot possible countenance any redress of discrimination that, gasp, puts the previously underrepresented group above the well represented groups even by the smallest smidge. But looking at Supplementary Fig1 from Gither, and keeping in mind that the African American PI application number is only 2% of the White applications, we can figure out that a substantial effect on African American PI’s award probability would cause only an imperceptible change in that for White PI applications. And there’s an amazing sweetener….merit.

Looking at the award probability graph from S1 of Ginther, we note that there are some 15% of the African-American PI’s grants scoring in the 175 bin (old scoring method, youngsters) that were not funded. About 55-56% of all ethnic/racial category grants in the next higher (worse) scoring bin were funded. So if Program picks up more of the better scoring applications from African American PIs (175 bin) at the expense of the worse scoring applications of White PIs (200 bin), we have actually ENHANCED MERIT of the total population of funded grants. Right? Win/Win.

So if we were to follow my suggestion, what would be the relative impact? Well thanks to the 2% ratio of African-American to White PI apps, it works like this:

Take the 175 scoring bin in which about 88% of white PIs and 85% of AA PIs were successful. Take a round number of 1,000 apps in that scoring bin (for didactic purposes, also ignoring the other ethnicities) and you get a 980/20 White/African-AmericanPI ratio of apps. In that 175 bin we’d need 3 more African-American PI apps funded to get to 100%. In the next higher (worse) scoring bin (200 score), about 56% of White PI apps were funded. Taking three from this bin and awarding three more AA PI awards in the next better scoring bin would plunge the White PI award probability from 56% to 55.7%. Whoa, belt up cowboy.

Moving down the curve with the same logic, we find in the 200 score bin that there are about 9 AA PI applications needed to put the 200 score bin to 100%. Looking down to the next worse scoring bin (225) and pulling these 9 apps from white PIs we end up changing the award probability for these apps from 22% to ..wait for it….. 20.8%.

And so on.

(And actually, the percentage changes would be smaller in reality because there is typically not a flat distribution across these bins and there are very likely more applications in each worse-scoring bin compared to the next better-scoring bin. I assumed 1,000 in each bin for my example.)

Another way to look at this issue is to take Berg’s triage numbers from above. To move to 40% triage rate for the African-AmericanPI applications, we need to shift 20% (230 applications) into the discussed pile. This represents a whopping 0.4% of the White PI apps being shifted onto the triage pile to keep the numbers discussed the same.

These are entirely trivial numbers in terms of the “hit” to the chances of White PIs and yet you could easily equalize the success rate or award probability for African-American PIs.

It is even more astounding that this could be done by picking up African-American PI applications that scored better than the White PI applications that would go unfunded to make up the difference.

Tell me how this is not a no-brainer for the NIH?

As you know I am distinctly unimpressed with the NIH’s response to the Ginther report which identified a disparity in the success rate of African-American PIs when submitting grant applications to the NIH.

The NIH response (i.e., where they have placed their hard money investment in change) has been to blame pipeline issues. The efforts are directed at getting more African-American trainees into the pipeline and, somehow, training them better. The subtext here is twofold.

First, it argues that the problem is that the existing African-American PIs submitting to the NIH just kinda suck. They are deserving of lower success rates! Clearly. Otherwise, the NIH would not be looking in the direction of getting new ones. Right? Right.

Second, it argues that there is no actual bias in the review of applications. Nothing to see here. No reason to ask about review bias or anything. No reason to ask whether the system needs to be revamped, right now, to lead to better outcome.

A journalist has been poking around a bit. The most interesting bits involve Collins’ and Tabak’s initial response to Ginther and the current feigned-helplessness tack that is being followed.

From Paul Basken in the Chronicle of Higher Education:

Regarding the possibility of bias in its own handling of grant applications, the NIH has taken some initial steps, including giving its top leaders bias-awareness training. But a project promised by the NIH’s director, Francis S. Collins, to directly test for bias in the agency’s grant-evaluation systems has stalled, with officials stymied by the legal and scientific challenges of crafting such an experiment.

“The design of the studies has proven to be difficult,” said Richard K. Nakamura, director of the Center for Scientific Review, the NIH division that handles incoming grant applications.

Hmmm. “difficult”, eh? Unlike making scientific advances, hey, that stuff is easy. This, however, just stumps us.

Dr. Collins, in his immediate response to the Ginther study, promised to conduct pilot experiments in which NIH grant-review panels were given identical applications, one using existing protocols and another in which any possible clue to the applicant’s race—such as name or academic institution—had been removed.

“The well-described and insidious possibility of unconscious bias must be assessed,” Dr. Collins and his deputy, Lawrence A. Tabak, wrote at the time.

Oh yes, I remember this editorial distinctly. It seemed very well-intentioned. Good optics. Did we forget that the head of the NIH is a political appointment with all that that entails? I didn’t.

The NIH, however, is still working on the problem, Mr. Nakamura said. It hopes to soon begin taking applications from researchers willing to carry out such a study of possible biases in NIH grant approvals, and the NIH also recently gave Molly Carnes, a professor of medicine, psychiatry, and industrial and systems engineering at the University of Wisconsin at Madison, a grant to conduct her own investigation of the matter, Mr. Nakamura said.

The legal challenges include a requirement that applicants get a full airing of their submission, he said. The scientific challenges include figuring out ways to get an unvarnished assessment from a review panel whose members traditionally expect to know anyone qualified in the field, he said.

What a freaking joke. Applicants have to get a full airing and will have to opt-in, eh? Funny, I don’t recall ever being asked to opt-in to any of the non-traditional review mechanisms that the CSR uses. These include phone-only reviews, video-conference reviews and online chat-room reviews. Heck, they don’t even so much as disclose that this is what happened to your application! So the idea that it is a “legal” hurdle that is solved by applicants volunteering for their little test is clearly bogus.

Second, the notion that a pilot study would prevent “full airing” is nonsense. I see very few alternatives other than taking the same pool of applications and putting them through regular review as the control condition and then trying to do a bias-decreasing review as the experimental condition. The NIH is perfectly free to use the normal, control review as the official review. See? No difference in the “full airing”.

I totally agree it will be scientifically difficult to try to set up PI blind review but hey, since we already have so many geniuses calling for blinded review anyway…this is well worth the effort.

But “blind” review is not the only way to go here. How’s about simply mixing up the review panels a bit? Bring in a panel that is heavy in precisely those individuals who have struggled with lower success rates- based on PI characteristics, University characteristics, training characteristics, etc. See if that changes anything. Take a “normal” panel and provide them with extensive instruction on the Ginther data. Etc. Use your imagination people, this is not hard.

Disappointingly, the CHE piece contains not one single bit of investigation into the real question of interest. Why is this any different from any other area of perceived disparity between interests and study section outcome at the NIH? From topic domain to PI characteristics (sex and relative age) to University characteristics (like aggregate NIH funding, geography, Congressional district, University type/rank, etc) the NIH is full willing to use Program prerogative to redress the imbalance. They do so by funding grants out of order and, sometimes, by setting up funding mechanisms that limit who can compete for the grants.

2013-FundingByCareerStageIn the recent case of young/recently transitioned investigators they have trumpeted the disparity loudly, hamfistedly and brazenly “corrected” the study section disparity with special paylines and out of order pickups that amount to an affirmative action quota system [PDF].
All with exceptionally poor descriptions of exactly why they need to do so, save “we’re eating out seed corn” and similar platitudes. All without any attempt to address the root problem of why study sections return poorer scores for early stage investigators. All without proving bias, describing the nature of the bias and without clearly demonstrating the feared outcome of any such bias.

“Eating our seed corn” is a nice catch phrase but it is essentially meaningless. Especially when there are always more freshly trained PHD scientist eager and ready to step up. Why would we care if a generation is “lost” to science? The existing greybeards can always be replaced by whatever fresh faces are immediately available, after all. And there was very little crying about the “lost” GenerationX scientists, remember. Actually, none, outside of GenerationX itself.

The point being, the NIH did not wait for overwhelming proof of nefarious bias. They just acted very directly to put a quota system in place. Although, as we’ve seen in recent data this has slipped a bit in the past two Fiscal Years, the point remains.

Why, you might ask yourself, are they not doing the same in response to Ginther?

Our longtime blog commenter dsks is always insightful. This time, the proposal is such a doozy that it is worth dragging up as a new post.

… just make it official and block all triaged applications from subsequent resubmission. Maybe then use the extra reviewer time and money to bring back the A2, perhaps restricting it to A1 proposals that come in under ~30%ile or something.

Hell, I think any proposal that consistently scores better than 20%ile should be allowed to be resubmitted ad infinitum until it gets funded. Having to completely restructure a proposal because it couldn’t quite make the last yard over what is accepted to be a rather arbitrary pay-line is insane.

On first blush that first one sounds pretty good. Not so sure about the endless queuing of an above payline, below 20%ile grant, personally. (I mean, isn’t this where Program steps in and just picks it up already?)

This reminds me of something, though. Unlike in times past, the applicant now has some information on just how strong the rejection really was because of the criterion scores. This gives some specific quantification in contrast to only being able to parse the language of the review.

One would hope that there would be some correlation between the criterion scores and the choice of the PI to resubmit. As in, if you get 4s and 5s on Approach or Significance, maybe it is worth it. 7s and 8s mean you really better not bother.

In case my comment never makes it out of moderation at RockTalk….

Interesting to contrast your Big Data and BRAINI approaches with your one for diversity. Try switching those around…”establish a forum..blah, blah…in partnership…blah, blah..to engage” in Big Data. Can’t you hear the outraged howling about what a joke of an effort that would be? It is embarrassing that the NIH has chosen to kick the can down the road and hide behind fake-helplessness when it comes to enhancing diversity. In the case of BRAINI, BigData and yes, discrimination against a particular class of PI applicants (the young) the NIH fixes things with hard money- awards for research projects. Why does it draw back when it comes to fixing the inequality of grant awards identified in Ginther?

When you face up to the reasons why you are in full cry and issuing real, R01 NGA solutions for the dismal plight of ESIs and doing nothing similar for underrepresented PIs then you will understand why the Ginther report found what it did.

ESIs continue, at least six years on, to benefit from payline breaks and pickups. You trumpet this behavior as a wonderful thing. Why are you not doing the same to redress the discrimination against underrepresented PIs? How is it different?

The Ginther bombshell dropped in August of 2011. There has been plenty of time to put in real, effective fixes. The numbers are such that the NIH would have had to fund mere handfuls of new grants to ensure success rate parity. And they could still do all the can-kicking, ineffectual hand waving stuff as well.

And what about you, o transitioning scientists complaining about an “unfair” NIH system stacked against the young? Is your complaint really about fairness? Or is it really about your own personal success?

If it is a principled stand, you should be name dropping Ginther as often as you do the fabled “42 years before first R01” stat.

From the description in Nature.

On 5 December, agency director Francis Collins told an advisory committee that the NIH should consider supporting more individual researchers, as opposed to research proposals as it does now — an idea inspired in part by the success of the high-stakes Pioneer awards handed out by the NIH’s Common Fund.

Pioneer awards are described as follows:

The NIH Pioneer Award initiative complements NIH’s traditional, investigator-initiated grant programs by supporting individual scientists of exceptional creativity who propose pioneering and possibly transforming approaches to addressing major biomedical or behavioral challenges that have the potential to produce an unusually high impact on a broad area of biomedical or behavioral research. To be considered pioneering, the proposed research must reflect substantially different scientific directions from those already being pursued in the investigator’s research program or elsewhere.

Another report I saw on this quoted Francis Collins as referring to “superstars”.

I’m unimpressed by this whole business. By referring to “superstars”, the HHMI approach and the Pioneers program NIH Director Collins makes it clear that he is talking about picking a very limited number of winners. At best each IC will get one? Maybe? So this will not do very much to help with the large bulk of NIH supported (and those desiring future support) investigators who feel that the job of securing grant money is taking away from their ability to do great science. This will not be some wholesale conversion of the NIH from project-based proposals to person/lab support. That’s my prediction anyway.

And as such, this reflects no real change. The primary concern of those opposed to this would be that it cordons off a part of the NIH pot in a place that they cannot try to reach it. If these selected superstars have the money based on their genius, then your project cannot be funded by those dollars.

Moving slightly down the road, the selection of superstars also means that the vast majority of us know that we have no shot at those funds in any case.

But here’s the thing that leaves me unimpressed.

This whole line of attack is nothing but a recognition that the superstars have to grub for grant money in the trenches now, but that they never had to do so in the past.

The NIH system has been a hybrid system that incorporates both project-based and people-based approaches. The latter is not formal, but it is reality. Once upon a time if you had a fairly healthy scientific pulse, you could renew your core grant (which rapidly evolved into a lab-based funding reality, no matter what was on the page every 5 years for competing renewal) for 25+ years. “I just applied for money when I needed it” said a colleague to me within the last two years. These people could also pull in additional grants for just about whatever half-decent additional project struck their fancy. In nearly all ways that count, many, many of our respective subfield luminaries (not superstars, I’m talking the top 20-30%) in the past three decades enjoyed defacto person-based funding.

Because of this, there was a pool of money the rest of the plebes, and the noobs, could not realistically access. In theory, sure. But in practice, no.

The current Collins trial-balloon will very likely only turn back the clock a tiny bit. It will be incredibly unfair on paper, but in reality it is no less fair than what was going on during the 80s and 90s and yes, well into the 00s.

The sad part is that it is unlikely to work. The genius superstars are still doing okay when it comes to funding. And of them, there will be many who fail to produce the genius, superstar, pioneering breakthrough innovations that Francis Collins is intimating they will all produce. There will be many of them that, without Collins’ intervention, will indeed make amazing breakthroughs. Many of both categories that might perhaps be awarded grants under this new expansion of the Pioneers program would still manage to win an equivalent amount of project-based funding in the absence of Collins’ plan going through.

I’m just not a big believer in making bets on who is going to revolutionize science and give them all the grant money. I believe a more distributed, less directed, individual investigator initiated approach is the demonstrated success model. When we try to pick a few winners we do less well at creating innovation.

So my suggestion is to figure out a way to relieve far more of the extramural research team from the current tyranny of the grant game. Not just a handpicked few but many. 30%? 50%? More.

All of us are spending far too much time on grants. Spending far too much time on creative thinking about data and what-ifs for yet another application, instead of following up on those great ideas. Many, many of us just-folks in the system would do a lot better if we were able to “just apply for a new grant when we needed it”. The scientific product would be much better and the cost-ratio would be improved.

Streamline the process for more of the NIH extramural force and guess what? The “superstars” will also be relieved! They will likewise get to spend more time thinking about innovation and, since they are superstars (right?) their innovation will be amazing.

My best proposal for how they should do this is easy because it uses an existing mechanism. They could start this process….tomorrow.

My proposal for making the system more person-based and less subject to the vagaries of review is to expand the R37/MERIT program. This is the program that awards an occasional highly-meritorious competing award an extended non-competing interval. So instead of having to think about renewal in 5 yrs, you have 10. There is still noncompeting review and rumor has it that some ICs have been willing to cancel R37s midstream for lack of production. Rumor also has it that many ICs take an extra hard look during year 5. But regardless, the structure is there.

A five year proposal that is now given 10 years? That should make almost any PI feel a lot more free to pursue blind alleys and risky new directions.

An article in the CHE by Paul Basken was brought to my attention because of the comments of Francis Collins regarding an emphasis on the “people, not projects” side of the equation. But something else drew my eye, way down the page, because I hadn’t heard of it before.

One panel member, Shirley M. Tilghman, a molecular biologist who is a former president of Princeton University, said one way to clear NIH resources for younger researchers would be a grant that would pay senior researchers to wind down their labs and distribute their resources to others in return for a commitment to seek no more NIH money.

She referred to it as a “terminal grant,” though conceded a different term would likely be necessary to make it more palatable.

The Howard Hughes Medical Institute has a similar program, in which it phases out grantees over a five-year period. The program is too new for a deep analysis, though it appears well received by scientists, said Robert T. Tjian, president of Hughes. It’s “a graceful and productive way for scientists to plan their future involvement in research and teaching as they approach the end of a natural cycle in a scientific life,” said Mr. Tjian, a professor of biochemistry and molecular biology at the University of California at Berkeley.

Wow. Seems okay on the face of it. I see a fair number of people grumbling about how they are going to retire but they still keep putting in the proposals. From a psychological perspective it might work to have them commit to an end date five years away, rather than, saying “Pack it in RIGHT NOW”. Over the next five years maybe that would have a net effect? Seems worth a try, maybe?

I do wonder how this could possibly work in the NIH system on a practical basis. I mean, how can you hold the PI to his or her commitment to stop submitting any grants? How can you keep them from being a significant Investigator on a project for which they are not the PI? The University submits the grants, after all.

But if we suppose it *can* work, is this the best solution? Wouldn’t it be better to just stop funding them? To stop extending any Programmatic pickups to PIs over a certain age? Or to, say, throw down a policy refusing any applications for anything beyond year 20 of a given project? Wouldn’t this, in the end, get rid of more people than offering all of them a Parachute Grant?

And if the plan is to “wind down” a person’s career…..doesn’t this totally fly in the face of the formal structure of the NIH, i.e. that the grant is based on a project, not a person? Are we talking a reverse K99/R00 that starts off with an independent research phase and then ends up as an emeritus fellowship that pays the salary and nothing else for a few years*? Or perhaps we’re talking a project that has to be taken over by a younger PI in years 2-5?

I doubt this will get much traction but if it does, it will be fascinating to see all the proposals for how it should work. I’m sure a few of you will have a go at it in the comments…..
__
*Paying the salary of an Emeritus Professor to sort of wander around the Department helping out has some resonance with my proposal for Staff Scientist Fellowships, I note. I am not entirely dismissing it as valueless.

Defunding the NIH

December 4, 2013

A article in the Pacific Standard magazine by Michael White provides an update on my prior post on The NIH Un-Doubling. The primary point in that post was a graph published in 2007 in

Heinig SJ, Krakower JY, Dickler HB, Korn D. Sustaining the engine of U.S. biomedical discovery. N Engl J Med. 2007 Sep 6;357(10):1042-7. [Publisher Link]

which presented the NIH budget allocations in dollar amounts adjusted for inflation* (expressed in 1998 dollars). The “undoubling” part reflected the 2007 allocation and 2008 Bush administration request in comparison with a trendline established from the early 1970s until the beginning of the doubling. It’s worth revisiting the graph from that article
Heinig07-NIHbudget-trend.jpeg.jpg

Figure 1. NIH Appropriations (Adjusted for Inflation in Biomedical Research) from 1965 through 2007, the President’s Request for 2008, and Projected Historical Trends through 2010.
All values have been adjusted according to the Biomedical Research and Development Price Index on the basis of a standard set of relevant goods and services (with 1998 as the base year). The trend line indicates average real annual growth between fiscal years 1971 and 1998 (3.34%), with projected growth (dashed line) at the same rate. The red square indicates the president’s proposed NIH budget for fiscal year 2008, also adjusted for inflation in biomedical research.

because the updated one, below, only starts in 1990.

NIHBudget-MAW-edit-497x400This new article How We’re Unintentionally Defunding the NIH provides the update, now represented in 2011 dollars. I’m not immediately seeing whether Michael White made this graph himself or sourced it from somewhere else but he does cite a Congressional Research Services report by John F. Sargent Jr which is worth a read.

This is fascinating. We’ve discussed historical funding trends and success rates under NIH extramural grant awards in the past. One post I wrote is highly pertinent:


The red trace depicts success rates from 1962 to 2008 for R01 equivalents (R01, R23, R29, R37). Note that they are not broken down by experienced/new investigators status, nor are new applications distinguished from competing continuation applications*. The blue line shows total number of applications reviewed…which may or may not be of interest to you. [update 7/12/12: I forgot to mention that the data in the 60s are listed as “estimated” success rates.]

The bottom line here is that looking at the actual numbers can be handy when playing the latest round of “We had it tougher than you did” at the w(h)ine and cheese hour after departmental seminar…Things are worse than they’ve ever been and these dismal patterns have bee sustained for much longer. … Anyone who tries to tell you they had it as hard or harder at any time in the past versus now is high as a kite. Period.

One key takeaway from this new graph is a consideration for those who insist that the NIH doubling interval was a poisoned gift. There are those that claim that our current woes are because research Universities and Medical Schools built up tremendous amounts of new infrastructure and personnel during the doubling, with the expectation that that rate of NIH budget escalation would continue. The thinking is that we experienced a bubble and the only reason we have problems now (during this extended interval of budget flatlining and therefore slipping purchasing power**) with dismal success rates. Too many mouths at the trough, is the way I put the situation, even if I don’t specifically blame the doubling interval for this.

This new graph makes it very clear that we have not just returned to the 3.3% growth trendline for the NIH budget. We have fallen off that line. Furthermore, the stimulus funding and the modest increases the Obama Administration have bruited as an initial budget offering are insufficient to change this divergence. It is absolutely clear that the NIH purchasing power is shrinking. Shrinking below the trends established from 1971 to 1998.

This is not a contraction relative to the doubling interval anymore! We’re way beyond that. We look to be as far below the historical trendline as we were above the line at the peak (end) of the doubling interval. We’re something on the order of $8-$10 Billion in the hole, something around 75% of where the historical trendline would have taken us. That seems like a lot of money until you realize

__
*from here:
RPGsuccessbyYear.png
source

**using BRDPI (Biomedical Research and Development Price Index)

Dr Strangely Strange returns us to the usual conundrum.

What are the key issues that NIH could easily address to make the system more fair, inclusive and to encourage better science (other than throwing more money at us). …Is there something else that we can all agree on that it would make a difference?

I had my usual, highly cynical albeit informed, response:

if the online discussions tell us anything it is that every single person insists that the “obvious”, “just”, “rational”, “fair”, etc solution to the problems of the NIH are whatever just so conveniently happen to suit their own situation or imagined near-future situation.

But here’s what I think we can agree on. We need to shrink the number of people with their hands out for NIH funds. Shrink the number of people being supported as professional scientists. And by “number”, this includes the notion of fractional people, i.e., those who only spend part of their time being paid by federal grant dollars.

The question is…who?

Who gets chopped?

Who is either kicked out of the system or prevented from entering the system in the first place*?

This is where we disagree. Fervently. It is an obvious truth that everyone starts with a very simple and universal principle on who should be shelled out of the NIH-supported system.

“Not Me”.

What I want to suggest for today’s futile exercise in getting the readership to follow their plans ALL the way down is this. Go on RePORTER. Search out some key words that are nice and broad or if you are under a smallish IC just search the whole I or C.

Run through that list and pick out something like 20% of the PIs that you would vote permanently off the island. Find 20% of your peers that you would ace without any detrimental impact on the broader scientific subfield of your interest.

I’d be interested if you come to any general set of criteria for deciding, how you did that. So maybe drop us a comment.

__
*This is a topic for another day but the “painless” solution of turning off the PhD tap is only painless if you forget senior undergraduate you when you were deciding what you really wanted to do was to go to graduate school and earn your PhD.

this comment is bylined from Francis Collins and Sally Rockey:

There have been concerns expressed that NIH is not doing anything to limit the number of Ph.D.s being produced. It’s important to remember that NIH does not control graduate enrollments. We are, however, firmly committed to the premise that bioscience Ph.D.s provide invaluable contributions to a whole variety of fields. Furthermore, there is no definitive evidence that Ph.D. production exceeds current employment opportunities.
Emphasis added.
This is why we can’t have nice things. Because the two most important administrative persons at the NIH are fully committed to pretending that they do not understand that their enterprise depends on the exploitation of doctoral “trainees”. They simply cannot acknowledge that a constantly turning-over stream of doctoral students and postdoctoral fellows generates more work for less money and therefore is essential to the system as they know it.
Yes. It is true that PhD holding individuals have vanishingly low unemployment rates in these times of economic downturn. It is also true that even graduate students (under the NIH extramural umbrella*)  make far, far more than minimum wage.
But we need to have a discussion about timeline, the years spent in various uncertain “trainee” job descriptions, the hours of work and the carrots that are being extended, never to be eaten. We also have to have a discussion about what job people enter graduate school thinking they are in training for. And what their chances are of obtaining those jobs. And what balance of “cheap labor” versus “training” is really being accomplished under the broad auspices of the NIH extramural system of support for science.
We also need to have a discussion about motivations of individual behavior under the increasingly competitive system and whether it has reached a tipping point in which the labor exploitation is no longer worth it.
If competition leads to faking and fraud, this costs an awful lot of NIH money when other people have to unpack why their experiments based on a prior finding are not working. If the competition leads to secretive, noncollaborative, scoop-laden dicing for Glamour publications, at some point this is costing the NIH money and progress. Duplication of work, if motivated only by secrecy, costs the NIH money. It costs progress.
If people feel betrayed by the system, they may just…..slack. Phone it in. Spend all day playing Candy Crush instead of working because they have to just put in their 3-5 years of postdoctoral work before schlepping off to an industry job. Or trying to be a NIH SRO or Program Officer. Or Glamour Mag editor. Or whatever your “alt career” du jour happens to be.
Leadership is not just keeping the leaking ship on course, Drs Collins and Rockey.
Sometimes it requires patching some holes, painting the hull and tuning up a balky engine room.
Sometimes it requires thinking harder about the before-the-mast swabbies who are keeping your vessel afloat.
*most of the broadly-defined biomedical programs

From the Science Careers section, Michael Price reports on a recent National Academies of Science symposium on the NIH foofraw about Biomedical career trajectories. The NAS, you will recall, is a society of very elite and highly established scientists in the US. It will not surprise you one bit to learn that they cannot fathom making changes in our system of research labor to benefit the peons anymore than the NIH can:

First issued in June 2012, the working group’s report made a controversial proposal: that funding should gradually be moved away from R01 grants and toward new NIH training grants in an effort to decouple graduate student and postdoc stipends. But responses to this proposal were tepid at the June [Advisory Committee to the Director (ACD) of the National Institutes of Health (NIH)] meeting where the proposals were first presented. Such a move would reduce the number of graduate students and postdocs available to principal investigators (PIs), and make trainees more expensive to hire, some ACD members argued. That would reduce PIs’ autonomy and encumber the research enterprise. “One wants to be sure that the principal investigators, who are supposed to be doing the research, continue to have enough flexibility to be able to support the research they want to do,” offered biologist Robert Horvitz of the Massachusetts Institute of Technology in Cambridge.

Reduce the number of easily exploitable laborers and/or make them more expensive. Presumably by forcing PIs to conduct more of their work with a more-permanent workforce (at any degree level). Permanent employees* which have that nasty tendency to gain seniority and consequently cost more money compared with the constantly turning-over grad student and postdoc labor pool.

And reduce our autonomy to hire foreign workers to further suppress wages and expectations for the domestic PhD pool. (Individual and Institutional postdoctoral and graduate “training” fellowships from the NIH currently only extend to US citizens. So I imagine PIs are assuming a shift to more fellowships would “reduce PIs’ autonomy” to hire foreign PhDs.)

the Price article continues:

When the ACD convened in December to discuss implementing the working group’s recommendations, this one had vanished from the agenda. The discussions at the December meeting avoided controversial issues, centering on whether, in an era in which only a small minority of scientists can realistically expect academic research careers, universities were adequately training students for a range of careers beyond the tenure track.

So it isn’t just the NAS Greybearded and BlueHaired contingent. This is the NIH response to their own working group.

Pass the buck.

Really strong work there, NIH.

Anything better from the NAS meeting?

In contrast to the measured discussion at December’s ACD meeting, the attendees of last week’s NAS meeting—mostly researchers who have studied the academic labor market—were critical of the status quo, arguing that keeping things the way they are would be disastrous for the scientific workforce.

m’kay…..and…..

There aren’t enough permanent jobs in academia for the vast majority of science graduates—and yet little has been done to curtail the production of doctorates, Ginther argues. “Employment has been stagnant, but Ph.D. production has been zooming,” Ginther said.

Ginther? Remember her? Wonder how NIH is coming along on the R01 funding disparity issue? HAAHA, I crack myself up.

Anyway…is anyone at NAS or the ACD discussing how we need to shut down the PhD firehose in addition to functionally restricting the import of foreign labor? hell no….

At December’s ACD meeting, the discussion focused on tweaking graduate programs to better prepare students for jobs outside academia, and several ACD members pointed to the relatively low unemployment numbers among science Ph.D.s as reassurance about trainees’ professional prospects.

Oh, but the scuttlebutt. That’s a brightspot, right?

None of the presenters at last week’s meeting put forth any radical suggestions for how to overhaul the academic training system, but the tenor of the discussions was far more critical of established practices than the discussions heard at NIH in December 2012. After Ginther’s presentation, this reporter overheard a chat between two meeting attendees. One suggested that science professors cannot in good conscience encourage their students to pursue a Ph.D.,

Sigh. No “radical suggestions”, eh? So basically there is no real difference from the ACD meeting. Ok, so one overheard conversation is snarky….but this does not a “tenor” make. How do you know the ACD folks didn’t also say such things outside of the formal presentations and the journalist just didn’t happen to be there to eavesdrop? Lots of people are saying this, they just aren’t saying it very loud, from a big platform or in large numbers. When you start seeing the premier graduate training programs in a subarea of science trumpeting their 30% or 50% reductions in admissions, instead of the record increases**, then we’ll be making some strides on the “tenor”.

Remember though, the NIH is taking all this stuff very, very seriously.

the ACD moved forward with most of the working group’s other recommendations, including proposals that would: establish a new funding program to explore how to better train grad students and postdocs for nonacademic careers; require trainees funded by NIH to have an individual development plan; encourage institutions to limit time-to-graduation for graduate students to 5 years; encourage institutions to track the career outcomes of their graduates; and encourage NIH study sections to look favorably upon grant proposals from teams that include staff scientists

Right.

1) Nonacademic careers in science are also drying up. This is the ultimate in buck-passing and feigned ignorance of what time it is on the street.
2) IDPs? Are you kidding? What good does it do to lay out specifically “I’d like to take these steps to become a tenure-track faculty” when there are STILL no jobs and no research funding for those who manage to land them? IDPs are the very definition of rearranging deck chairs.
3) I totally support faster time to PhD awards for the individual. However on a broad basis, this just accelerates the problem by letting local departments up their throughput of newly minted PhDs. Worthless goal if it is not combined with throttling back on the number of PhD students being trained overall.
4) Making training departments track outcomes is good but..to what end? So that prospective graduate students will somehow make better choices? Ha. And last I checked, when PhD programs are criticized for job outcome they start waving their hands furiously and shout about the intervening postdoctoral years and how it is in no way their fault or influence that determines tenure-track achievement of their graduates.
5) “encourage” study sections? Yeah, just like the NIH has been encouraging study sections to treat tenure-track traditional hire Assistant Professors better. Since the early 80s at the least and all to no avail. As we know, the only way the NIH could make any strides on that problem was with affirmative action style quotas for younger PIs.

Tilghman, who headed the working group and I think has been around the NIH for a few rodeos before, is not impressed:

Yet, the working group’s chair, former Princeton University president Shirley Tilghman, told Science Careers that she couldn’t “help but go back to [her] cynicism” so long as NIH merely “encouraged” many of these measures.

Where “cynicism” is code for “understanding that NIH has no intention whatsoever in changing and is merely engaging in their usual Kabuki theater to blunt the fangs of any Congressional staff that may happen to get a wild hair over any of this career stuff”.

Score me as “cynical” too.

[ h/t: DJMH ]
__
*and yeah. It sucks to have a 5-year grant funding cycle and try to match that on to supporting permanent employees. I get that this is not easy. I deal with this myself, you know. My convenience doesn’t excuse systematic labor exploitation, though.

**Dude I can’t even. Bragging about record admits for several recent years now, followed finally this year by some attempt to figure out if the participating faculty can actually afford to take on graduate students. FFS.

Interesting exchange on the twitts today with someone who is intimating that the process of selecting peers to serve as grant reviewers on NIH study sections requires some transparency and fixing.

As my longer term Readers are aware, my main objection along these lines is that I think Assistant Professors should not be excluded and that the purge urged on by Toni Scarpa back some years ago was misguided. I will also venture that I think it is ridiculous that the peer review pool is limited to those Professorial rank people who have already won funding from the NIH (for the most part). If really pressed, I’ve been know to suggest that it is even unfair that the more senior postdoc types who have not yet won a faculty-level appointment cannot review grants.

Other than that, I am generally down with the official mandates to seek ethnic/racial, gender and geographic representation on panels. My personal experience has been that the SROs do a pretty good job at this. Also, because of these factors, I have found that the types of institutions represented spans the range pretty well..small mostly teaching profs, big Research Uni profs, research insitutes of various sizes, public Unis, private Unis, Med Schools and academic departments.

So it is with some confusion that I read someone asserting that there is a problem with who is selected.

My query of the day, therefore, is to ask you if you know of people who seek to serve on study section but cannot seem to land an invite. Alternately, do you know of categories of investigators that are routinely overlooked?