Jeremy Berg has a new President’s Message up at ASBMB Today. It looks into a topic of substantial interest to me, i.e., the fate of investigators funded by the NIH. This contrasts with our more-usual focus on the fate of applications.

With that said, the analysis does place the impact of the sequester in relatively sharp focus: There were about a thousand fewer investigators funded by these mechanisms in FY13 compared with FY12. This represents more than six times the number of investigators who lost this funding from FY11 to FY12 and a 3.8 percent drop in the R-mechanism-funded investigator cohort.

another tidbit addresses the usual claim from NIHlandia that R-mechs and R01s in particular are always prioritized.

In her post, Rockey notes that the total funding for all research project grants, or RPGs, dropped from $15.92 billion in FY12 to $14.92 billion in FY13, a decrease of 6.3 percent. The total funding going to the R series awards that I examined (which makes up about 85 percent of the RPG pool) dropped by 8.9 percent.

What accounts for this difference? U01 awards comprise the largest remaining portion of the RPG pool…The funds devoted to U01 awards remained essentially constant from FY12 to FY13 at $1.57 billion.

Go Read the whole thing.

This type of analysis really needs more attention at the NIH level. They’ve come a looooong way in recent years in terms of their willingness to focus on what they are actually doing in terms of applications, funding, etc. This is in no small part due to the efforts of Jeremy Berg, who used to be the Director of NIGMS. But tracking the fate of applications only goes so far, particularly when it is assessed only on a 1-2 year basis.

The demand on the NIH budget is related to the pool of PIs seeking funding. This pool is considerably less elastic than the submission of grant applications. PIs don’t submit grant applications endlessly for fun, you know. They seek a certain level of funding. Once they reach that, they tend to stop submitting applications. A lot of the increase in application churn over the past decade or so has to do with the relative stability of funding. When odds of continuing an ongoing project are high, a large number of PIs can just submit one or two apps every 5 years and all is well. Uncertainty is what makes her submit each and every year.

Similarly, when a PI is out of funding completely, the number of applications from this lab will rise dramatically….right up until one of them hits.

I argue that if solutions to the application churn and the funding uncertainty (which decreases overall productivity of the NIH enterprise) are to be found, they will depend on a clear understanding of the dynamics of the PI population.

Berg has identified two years in which the PI turnover is very different. How do these numbers compare with historical trends? Which is the unusual one? Or is this the expected range?

Can we see the 1,000 PI loss as a temporary situation or a permanent fix? It is an open question as to how many sequential years without NIH funding will affect the PI. Do these individuals tend to regain funding in 2, 3 or 4 years’ time? Do they tend to go away and never come back? More usefully, what proportion of the lost investigators will follow these fates?

The same questions arise for the other factoids Berg mentions. The R00 transition to other funding would seem to be incredibly important to know. But a one year gap seems hardly worth discussing. This can easily happen under the current conditions. But if they are not getting funded after 2 or maybe 3 years after the R00 expires? This is of greater impact.

Still, a welcome first step, Dr. Berg. Let’s hope Sally Rockey is listening.

Berg2014IntramuralChartJeremy Berg has a new column up at ASBMB Today which examines the distribution of NIH intramural funding. Among other things, he notes that you can play along at home via searching RePORTER using the ZIA activity code (i.e., in place of R01, R21, etc). At first blush you might think “WOWZA!”. The intramural lab is pretty dang flush. If you think about the direct costs of an extramural R01 grant – the full modular is only $250K per year. So you would need three awards (ok, the third one could be an R21) just to clear the first bin. But there are interesting caveats sprinkled throughout Berg’s comments and in the first comment to the piece. Note the “Total Costs”? Well, apparently there is an indirect costs rate within the IRPs and Berg comments that it is so variable that it is hard to issue anything similar to a negotiated extramural IDC rate for the entire NIH Intramural program. The comment from an ex-IRP investigator points to more issues. There may be some shared costs inserted into a given PI’s apparent budget that this PI has no control over. Whether this is part of the overhead or an overhead-like cost….or maybe a cost shard across one IC’s IRP…who knows?

We also don’t know what a given PI has to pay for out of his or her ZIA allocation. What are animal housing costs like? Are they subsidized for certain ICs’ IRPs? For certain labs? Who is a PI and who is a staff scientist of some sort within the IRPs? Do these status’ differ? Are they comparable to extramural lab operations? I know for certain sure that people who are more or less the equivalent of an extramural Assistant/Associate Professor in a soft money job category exist within the NIH IRPs without being considered a PI with their own ZIA allocation. So that means that a “PI” on the chart that Berg presents may in fact be equivalent to 2-3 PIs out here in extramural land. (And yes, I understand that some of the larger extramural labs similarly have several people who would otherwise be heading their own lab all subsumed within the grants awarded to one BigCheez PI.)

With that said, however, the IRP is supposed to be special. As Berg notes

The IRP mission statement asserts that the IRP should “conduct distinct, high-impact laboratory, clinical, and population-based research” and that it should support research that “cannot be readily funded or accomplished in traditional academia.”

So by one way of looking at it, we shouldn’t be comparing the IRP scientists to ourselves. They should be different.

Even if we think of IRP investigators as not much different from ourselves, I’m having difficulty making any sense of these numbers. It is nice to see them, but it is really hard to compare to what is going on with extramural grant funding.

Perhaps of greater value is the analysis Berg presents for whether NIH’s intramural research is feeling their fair share of the budgetary pain.

In 2003, when I became an NIH institute director, the overall NIH appropriation was $26.74 billion, while the overall intramural program consumed $2.56 billion, or 9.6 percent. In fiscal 2013, the overall NIH appropriation was $29.15 billion, and the intramural share had grown to $3.26 billion, or 11.2 percent.
 
Some of this growth is because of ongoing intramural activities, such as those involving the NIH Clinical Center, where, like at other hospitals, costs are very hard to contain below rates of inflation, or because of new activities, such as the NIH Chemical Genomics Center. The IRP is particularly expensive in terms of taxpayer dollars, because it is difficult to leverage the federal support to the IRP with funds from other sources as occurs in the extramural community.

So I guess that would be “no”. No the IRP, in aggregate, is not sharing the pain of the flatlined budget. There is no doubt that some of the individual components of the various IRPs are. It is inevitable. Previously flush budgets no doubt being reduced. Senior folk being pushed out. Mid and lower level employees being cashiered. I’m sure there are counter examples. But as a whole, it is clear that the IRP is being protected, inevitably at the expense of R-mech extramural awards.

 

 

New Grant Snooping

February 4, 2014

As usual, I like to keep and eye on RePORTER and SILK to see what the various ICs of my own dearest interest are up to with regard to grants that were supposed to fund Dec 1, 2013. Per usual, there was no budget and the more conservative ICs wait around to do anything. Some of the less-conservative ones do tend to start funding new grant awards in December and Jan so there is always something to see on SILK.

I noticed something interesting. NIAID has 44 new R01s listed that were on the A1 revision and 19 that were funded on the “first” submission. RePORTER notes that 30 funded in Dec, 12 of these funded in Jan and  17 on or after 2/1/2014 (not sure if I miscounted totals on SILK or RePORTER hasn’t caught up or what).

My ICs of dearest concern are still waiting, only a bare handful of new R01s are listed.

NCI has 36 new R01 apps funded on A1, 21 on the A0. DK is running 15/13.

Scanning down the rest of the list of ICs, it looks like DK is about as close to even as it gets and that a 2:1 ratio of A1 to A0 being funded is not too far off the mean.

 

I still think we’d be a lot better off if something like 2/3rd of grants were awarded on first submission and the A1s were only about a third.

Jeremy Berg made a comment

If you look at the data in the Ginther report, the biggest difference for African-American applicants is the percentage of “not discussed” applications. For African-Americans, 691/1149 =60.0% of the applications were not discussed whereas for Whites, 23,437/58,124 =40% were not discussed (see supplementary material to the paper). The actual funding curves (funding probability as a function of priority score) are quite similar (Supplementary Figure S1). If applications are not discussed, program has very little ability to make a case for funding, even if this were to be deemed good policy.

that irritated me because it sounds like yet another version of the feigned-helpless response of the NIH on this topic. It also made me take a look at some numbers and bench race my proposal that the NIH should, right away, simply pick up enough applications from African American PIs to equalize success rates. Just as they have so clearly done, historically, for Early Stage Investigators and very likely done for woman PIs.

Here’s the S1 figure from Ginther et al, 2011:
Ginther-S1

[In the below analysis I am eyeballing the probabilities for illustration’s sake. If I’m off by a point or two this is immaterial to the the overall thrust of the argument.]

My knee jerk response to Berg’s comment is that there are plenty of African-American PI’s applications available for pickup. As in, far more than would be required to make up the aggregate success rate discrepancy (which was about 10% in award probability). So talking about the triage rate is a distraction (but see below for more on that).

There is a risk here of falling into the Privilege-Thinking, i.e. that we cannot possible countenance any redress of discrimination that, gasp, puts the previously underrepresented group above the well represented groups even by the smallest smidge. But looking at Supplementary Fig1 from Gither, and keeping in mind that the African American PI application number is only 2% of the White applications, we can figure out that a substantial effect on African American PI’s award probability would cause only an imperceptible change in that for White PI applications. And there’s an amazing sweetener….merit.

Looking at the award probability graph from S1 of Ginther, we note that there are some 15% of the African-American PI’s grants scoring in the 175 bin (old scoring method, youngsters) that were not funded. About 55-56% of all ethnic/racial category grants in the next higher (worse) scoring bin were funded. So if Program picks up more of the better scoring applications from African American PIs (175 bin) at the expense of the worse scoring applications of White PIs (200 bin), we have actually ENHANCED MERIT of the total population of funded grants. Right? Win/Win.

So if we were to follow my suggestion, what would be the relative impact? Well thanks to the 2% ratio of African-American to White PI apps, it works like this:

Take the 175 scoring bin in which about 88% of white PIs and 85% of AA PIs were successful. Take a round number of 1,000 apps in that scoring bin (for didactic purposes, also ignoring the other ethnicities) and you get a 980/20 White/African-AmericanPI ratio of apps. In that 175 bin we’d need 3 more African-American PI apps funded to get to 100%. In the next higher (worse) scoring bin (200 score), about 56% of White PI apps were funded. Taking three from this bin and awarding three more AA PI awards in the next better scoring bin would plunge the White PI award probability from 56% to 55.7%. Whoa, belt up cowboy.

Moving down the curve with the same logic, we find in the 200 score bin that there are about 9 AA PI applications needed to put the 200 score bin to 100%. Looking down to the next worse scoring bin (225) and pulling these 9 apps from white PIs we end up changing the award probability for these apps from 22% to ..wait for it….. 20.8%.

And so on.

(And actually, the percentage changes would be smaller in reality because there is typically not a flat distribution across these bins and there are very likely more applications in each worse-scoring bin compared to the next better-scoring bin. I assumed 1,000 in each bin for my example.)

Another way to look at this issue is to take Berg’s triage numbers from above. To move to 40% triage rate for the African-AmericanPI applications, we need to shift 20% (230 applications) into the discussed pile. This represents a whopping 0.4% of the White PI apps being shifted onto the triage pile to keep the numbers discussed the same.

These are entirely trivial numbers in terms of the “hit” to the chances of White PIs and yet you could easily equalize the success rate or award probability for African-American PIs.

It is even more astounding that this could be done by picking up African-American PI applications that scored better than the White PI applications that would go unfunded to make up the difference.

Tell me how this is not a no-brainer for the NIH?

A post over at Rock Talk blog describes some recent funding data from the NIH. The takeaway message is that every thing is down. Fewer grants awarded, fewer percentages of the applications being funded. Not exactly news to my audience. However, head over to the NIH data book for some interesting tidbits.

2013-FundingByCareerStageFirst up, my oldest soapbox, the new investigator. As you can see, up to FY2006 the PI who had not previously had any NIH funding faced a steeper hurdle to get a new grant (Type 1) funding compared to established investigators. This was despite the “New Investigator” checkbox at the top of the application and the fact that reviewers were instructed to give such applications a break. And they did in my experience….just not enough to actually get them funded. Study section discussion that ended with “…but this investigator is new and highly promising so that’s why I’m giving it such a good score…[insert clearly unfundable post-discussion score]” were not uncommon during my term of appointed service. So round about FY2007 the prior NIH Director, Zerhouni, put in place an affirmative action system to fund newly-transitioned independent investigators. There’s a great description in this Science news bit [PDF]. You can see the result below.

Interestingly, this will to maintain success rates of the inexperienced PIs at levels similar to the experienced PIs has evaporated for FY2011 and FY2013. See title.

2013-FundingBySexofPINext, the slightly more subtle case of women PIs. This will be a two-grapher. First, the overall Research Project Grant success rate broken down by PI sex. As you can see, up through FY2002 there was a disparity which disappeared in the subsequent years. Miracle? Hell no. I guarantee you there has been some placing of the affirmative action fingers on the scale for the sex disparity as well. Fortunately, the elastic hasn’t snapped back in the past two FYs as it has for inexperienced investigators. But I’m keeping a suspicious eye on it, as should you. Notice how women trickle along juuuuust a little bit behind men? Interesting, isn’t it, how the disparity is never actually reversed? You know, because if whomever was previously advantaged even slipped back to disadvantaged (instead of merely equal) the whole world would end.

2013-FundingBySexandTypeR01Moving along, we downshift to R01-equivalent grants so as to perform the analysis of new proposals versus competing continuation (aka, “renewal”) applications. There are mechanisms included in the “RPG” grouping that cannot be continued so this is necessary. What we find is that the disparity for woman PIs in continuing their R01/equivalent grants has been maintained all along. New grants have been level in recent years. There is a halfway decent bet that this may be down to the graybeard factor. This hypothesis depends on the idea that the longer a given R01 has been continued, the higher the success rate for each subsequent renewal. These data also show that a goodly amount of the sex disparity up through FY2002 was addressed at the renewal stage. Not all of it. But clearly gains were made. This kind of selectivity suggests the heavy hand of affirmative action quota filling to me.

This is why I am pro-quota and totally in support of the heavy hand of Program in redressing study section biases, btw. Over time, it is the one thing that helps. Awareness, upgrading women’s representation on study section (see the early 1970s)…New Investigator checkboxes and ESI initiatives* all fail. Quota-making works.

__
*In that Science bit I link it says:

Told about the quotas, study sections began “punishing the young investigators with bad scores,” says Zerhouni. That is, a previous slight gap in review scores for new grant applications from firsttime and seasoned investigators widened in 2007 and 2008, Berg says. It revealed a bias against new investigators, Zerhouni says.

Our longtime blog commenter dsks is always insightful. This time, the proposal is such a doozy that it is worth dragging up as a new post.

… just make it official and block all triaged applications from subsequent resubmission. Maybe then use the extra reviewer time and money to bring back the A2, perhaps restricting it to A1 proposals that come in under ~30%ile or something.

Hell, I think any proposal that consistently scores better than 20%ile should be allowed to be resubmitted ad infinitum until it gets funded. Having to completely restructure a proposal because it couldn’t quite make the last yard over what is accepted to be a rather arbitrary pay-line is insane.

On first blush that first one sounds pretty good. Not so sure about the endless queuing of an above payline, below 20%ile grant, personally. (I mean, isn’t this where Program steps in and just picks it up already?)

This reminds me of something, though. Unlike in times past, the applicant now has some information on just how strong the rejection really was because of the criterion scores. This gives some specific quantification in contrast to only being able to parse the language of the review.

One would hope that there would be some correlation between the criterion scores and the choice of the PI to resubmit. As in, if you get 4s and 5s on Approach or Significance, maybe it is worth it. 7s and 8s mean you really better not bother.

A December 18 post on the Rock Talk blog issued an update on the funding rate situation for grant applications submitted to the NIH. The data provide

…an early snapshot on success rates for 2013 competing research project grant (RPG) applications and awards.

We received 49,581 competing RPG applications at NIH in fiscal year 2013, slightly declining compared to last year (51,313 applications in FY2012).

… In FY2013 we made 8,310 competing RPG awards, 722 fewer than in FY2012. This puts the overall research project grant (RPG) success rate at 16.8%, a decline from the 17.6% reported in FY2012. One might have expected a bigger drop in the success rates since we made about 8% fewer competing awards this year, but the reduction in the number of applications explains part of it.

emphasis added, as if I need to do so.

See this graph for a recent historical trend on success rates and application submission numbers. With respect to the latter, you can see that the small decrease to 49,581 is not hugely significant. We’ll have to wait for a few more years to be convinced of any trend. Success rates are at an all-time low. This is rather unsurprising to anyone of you that has been paying attention to doing at the NIH and is a result of the long trend toward Defunding the NIH.

Of greater interest in the Rock Talk post was a comment made in response to a query about the fate of initially-triaged applications. A Deborah Frank wrote:

A few months ago, I emailed Rock Talk to ask the same question as Mr. Doherty’s question #3. My query was routed to the Freedom of Information Act Office, and a few months later I received a table of data covering A0 R01s received between FY 2010 and FY2012 (ARRA funds and solicited applications were excluded). Overall at NIH, 2.3% of new R01s that were “not scored” as A0s were funded as A1s (range at different ICs was 0.0% to 8.4%), and 8.7% of renewals that were unscored as A0s were funded as A1s (range 0.0% to 25.7%). These data have at least two limitations. First, funding decisions made in 2013 were not included, so the actual success rates are likely a bit higher. Second, the table does not indicate how many of the unscored A0s were resubmitted.

The NIH data miner / blog team then posted a link to an excel spreadsheet with the relevant numbers for ICs, divided by Type 1 (new) and Type 2 (renewal, aka competing continuation) applications. The spreadsheet notes that this analysis is for unsolicited (i.e., non-RFA) applications and that since the FY2013 funding data were not complete when these were generated (7/15/2013), it is possible that some A0 submitted in this interval may still be funded.

Now, this is not precisely the same as the usual success rate numbers because of

  • the aforementioned exclusions
  • the way A0 and A1 submitted in the same FY are counted as one application in success rate calculation
  • the fact that if an A1 is not submitted it isn’t (cannot be) counted in success rate

Nevertheless, keeping these details in mind it is hard to escape noticing that one is facing steep odds to get a triaged A0 Type 1 proposal funded. On the face of it, anyway. And I have to tell you, Dear Reader, that this is consistent with my personal experience. I can’t recall ever getting a triaged application to the funded level on the next submission. In fact I’m hard pressed to recall getting a triaged A0 funded as an A2 when that was still possible.

Yet I continue to revise them. Not entirely sure why, looking at these data.

Moving along, it is really disappointing that the NIH didn’t go ahead and put all the relevant numbers in their spreadsheet. The thing that PIs really want to know is still terribly obscured by this selected analysis. NIDA, for example, lists 394 unscored Type 1 applications of which 11 (2.8%) were eventually funded. But unlike the now-disappeared CSR FY2004 databook analysis (see here, here for reference to it), they have failed to provide the number of applications that were initially triaged that the PI actually resubmitted as A1! If only half of the triaged applications were amended and resubmitted, then the odds go to 5.6%.

Is this difference relevant to PI decision making? I don’t know for sure but I suspect it would be. It is also relevant to understanding the different success rate for initially-triaged Type 1 and Type 2 applications. The mean and selected ICs I checked tell the same tale, i.e., that Type 2 apps have a much better shot at getting funded after triage on the A0. NIDA is actually pretty extreme from what I can tell- 2.8% versus 15.2%. So if there is a difference in the A1 resubmission rate for Type 1 and Type 2 (and I bet Type 2 apps that get triaged on A0 are much more likely to be amended and resubmitted) apps, the above analysis doesn’t move the relative disadvantage around all that much. However for NIAAA the Type 1 and Type 2 numbers are closer- 4.7% versus 9.8%. So for NIAAA supplicants, a halving of the resubmission rate for Type 1 might bring the odds for Type 1 and Type 2 much closer.

Do these data change my approach? They probably should. However, there is a factor of submission dates here. For any given round, new applications are submitted one month and then amended applications are due the next month. So if you are a few weeks away from the second deadline and considering whether to resubmit an application or not….there is no “new” application that you could submit right now. You have to wait for the next round. So if you are feeling grant pressure..what else are you going to do? Take the low odds or take the guarantee of zero odds?

Final note. I continue to believe, until NIH demonstrates my error very clearly, that considerable numbers of “A0” submissions are really a reworking of ideas that have been previously reviewed. I also believe that these “A0” submissions are disproportionally likely to be funded due to the prior submission/review rounds. Whether this is due to improved grant crafting, additional preliminary data, better approaches, gradual convincing of a study section or Program is not critical here, I’d say all these contribute. If I am correct, then there is value in continuing to work the steps by resubmitting a triaged A0.
__
Additional Reading:

NIH Historical Success Rates Explain Current Attitudes

More data on historical success rates for NIH grants

Old Boys’ Network Favors Men’s Continuing Grants?

A query on the Twitts today:

reminded me of this post. It originally went up 12 July, 2012.

This reality is echoed in personal anecdote. If I look across my grant submissions within a particular part of the lab over the years, I am more or less proposing the same scope of work in each R01. I started submitting grants within the first few years of the modular budgeting era and was matching my proposals to what could be accomplished within the $250K limit. Time marched on…but it took me a long time to cotton on to the purchasing power issue. I just squeezed and tried to compensate by proposing new projects. Because of the considerably reduced hit rate, I’ve taken to doing traditional budgeting lately. And, what do you know? It comes in at about $375K. Same scope as I used to fit within the $250 limit.


You are probably aware, DearReader, of the concept of inflation. This means that the amount of money that you pay today for a good or service is higher than the amount of money that you paid yesterday.

On average.

So for example, this US inflation calculator tells me that the purchasing power of $12,000 in 1972 has the purchasing power of $65,975.60 in 2012. This is a convenient set of figures if, for example, you are shooting the breeze with a senior faculty member* who started his or her Assistant Professor appointment in the early 70s. You may want to grapple with pay on even terms. Naturally, not every good or service has the same inflation rate and this is just one model/estimator. Jeans may cost less and houses may cost more. etc.

Moving along, we come to the discussion of NIH Grants. In the past I’ve posted the analysis that shows that the doubling of the NIH budget was rapidly un-doubled and fell back on the historical trend line. [see update suggesting we are now defunding the NIH] That analysis depended on the Biomedical Research and Development Price Index or BRDPI. This brings us to an interest in the purchasing power of the full modular R01. “Modular” refers to the specification of the budget for most NIH grant types in units of $25,000 in direct costs. These are the “modules”.

There has been a cap of $250,000 per year in direct costs since the 6/1/1999 initiation of this structure, if I have that right. You can ask for more money per year but then you revert to a line-item type budget (called “traditional budgeting”). The modular cap has not changed and, I assert, this limit affects the vast majority of NIH R01 proposals since there is high motivation (or has been, I may have touched on reasons for future changes before) to adhere to the modular grant structure. Overall, I do like the notion of the modular budgeting procedures because it keeps reviewers from ticky-tacking a bunch of irrelevancies about grants when they should focus on the science.

However, the use of a limit like this brings up the unpleasant inevitability of inflation.

Comrade PhysioProf has been noting that the real purchasing power of the R01 has been dropping due to inflation in the context of postdoctoral fellow demands for ever increasing salaries. He’s not alone in noticing. I offer today, a graphical depiction pulled from data provided by the NIH Office of Budget on the BRDPI.
I”ve taken their table of yearly adjustments and used those to calculate the increase necessary to keep pace with inflation (black bars) and the decrement in purchasing power (red bars). The starting point was the 2001 fiscal year (and the BRDPI spreadsheet is older so the 2011 BRDPI adjustment is predicted, rather than actual). As you can see, a full modular $250,000 year in 2011 has 69% of the purchasing power of that same award in 2001.

For those looking at the increasing numbers of applications being submitted presented in the prior post, you must include some understanding of this inflationary pressure in your thinking.

The second thing we’ve found here is the target number to restore spending parity.

In simple terms, we should now be advocating for an increase to $350,000 as the new modular cap.

__
*Particularly handy when said senior (or emeritized, retired) faculty members are members of one’s own family. just sayin.

From the description in Nature.

On 5 December, agency director Francis Collins told an advisory committee that the NIH should consider supporting more individual researchers, as opposed to research proposals as it does now — an idea inspired in part by the success of the high-stakes Pioneer awards handed out by the NIH’s Common Fund.

Pioneer awards are described as follows:

The NIH Pioneer Award initiative complements NIH’s traditional, investigator-initiated grant programs by supporting individual scientists of exceptional creativity who propose pioneering and possibly transforming approaches to addressing major biomedical or behavioral challenges that have the potential to produce an unusually high impact on a broad area of biomedical or behavioral research. To be considered pioneering, the proposed research must reflect substantially different scientific directions from those already being pursued in the investigator’s research program or elsewhere.

Another report I saw on this quoted Francis Collins as referring to “superstars”.

I’m unimpressed by this whole business. By referring to “superstars”, the HHMI approach and the Pioneers program NIH Director Collins makes it clear that he is talking about picking a very limited number of winners. At best each IC will get one? Maybe? So this will not do very much to help with the large bulk of NIH supported (and those desiring future support) investigators who feel that the job of securing grant money is taking away from their ability to do great science. This will not be some wholesale conversion of the NIH from project-based proposals to person/lab support. That’s my prediction anyway.

And as such, this reflects no real change. The primary concern of those opposed to this would be that it cordons off a part of the NIH pot in a place that they cannot try to reach it. If these selected superstars have the money based on their genius, then your project cannot be funded by those dollars.

Moving slightly down the road, the selection of superstars also means that the vast majority of us know that we have no shot at those funds in any case.

But here’s the thing that leaves me unimpressed.

This whole line of attack is nothing but a recognition that the superstars have to grub for grant money in the trenches now, but that they never had to do so in the past.

The NIH system has been a hybrid system that incorporates both project-based and people-based approaches. The latter is not formal, but it is reality. Once upon a time if you had a fairly healthy scientific pulse, you could renew your core grant (which rapidly evolved into a lab-based funding reality, no matter what was on the page every 5 years for competing renewal) for 25+ years. “I just applied for money when I needed it” said a colleague to me within the last two years. These people could also pull in additional grants for just about whatever half-decent additional project struck their fancy. In nearly all ways that count, many, many of our respective subfield luminaries (not superstars, I’m talking the top 20-30%) in the past three decades enjoyed defacto person-based funding.

Because of this, there was a pool of money the rest of the plebes, and the noobs, could not realistically access. In theory, sure. But in practice, no.

The current Collins trial-balloon will very likely only turn back the clock a tiny bit. It will be incredibly unfair on paper, but in reality it is no less fair than what was going on during the 80s and 90s and yes, well into the 00s.

The sad part is that it is unlikely to work. The genius superstars are still doing okay when it comes to funding. And of them, there will be many who fail to produce the genius, superstar, pioneering breakthrough innovations that Francis Collins is intimating they will all produce. There will be many of them that, without Collins’ intervention, will indeed make amazing breakthroughs. Many of both categories that might perhaps be awarded grants under this new expansion of the Pioneers program would still manage to win an equivalent amount of project-based funding in the absence of Collins’ plan going through.

I’m just not a big believer in making bets on who is going to revolutionize science and give them all the grant money. I believe a more distributed, less directed, individual investigator initiated approach is the demonstrated success model. When we try to pick a few winners we do less well at creating innovation.

So my suggestion is to figure out a way to relieve far more of the extramural research team from the current tyranny of the grant game. Not just a handpicked few but many. 30%? 50%? More.

All of us are spending far too much time on grants. Spending far too much time on creative thinking about data and what-ifs for yet another application, instead of following up on those great ideas. Many, many of us just-folks in the system would do a lot better if we were able to “just apply for a new grant when we needed it”. The scientific product would be much better and the cost-ratio would be improved.

Streamline the process for more of the NIH extramural force and guess what? The “superstars” will also be relieved! They will likewise get to spend more time thinking about innovation and, since they are superstars (right?) their innovation will be amazing.

My best proposal for how they should do this is easy because it uses an existing mechanism. They could start this process….tomorrow.

My proposal for making the system more person-based and less subject to the vagaries of review is to expand the R37/MERIT program. This is the program that awards an occasional highly-meritorious competing award an extended non-competing interval. So instead of having to think about renewal in 5 yrs, you have 10. There is still noncompeting review and rumor has it that some ICs have been willing to cancel R37s midstream for lack of production. Rumor also has it that many ICs take an extra hard look during year 5. But regardless, the structure is there.

A five year proposal that is now given 10 years? That should make almost any PI feel a lot more free to pursue blind alleys and risky new directions.

An article in the CHE by Paul Basken was brought to my attention because of the comments of Francis Collins regarding an emphasis on the “people, not projects” side of the equation. But something else drew my eye, way down the page, because I hadn’t heard of it before.

One panel member, Shirley M. Tilghman, a molecular biologist who is a former president of Princeton University, said one way to clear NIH resources for younger researchers would be a grant that would pay senior researchers to wind down their labs and distribute their resources to others in return for a commitment to seek no more NIH money.

She referred to it as a “terminal grant,” though conceded a different term would likely be necessary to make it more palatable.

The Howard Hughes Medical Institute has a similar program, in which it phases out grantees over a five-year period. The program is too new for a deep analysis, though it appears well received by scientists, said Robert T. Tjian, president of Hughes. It’s “a graceful and productive way for scientists to plan their future involvement in research and teaching as they approach the end of a natural cycle in a scientific life,” said Mr. Tjian, a professor of biochemistry and molecular biology at the University of California at Berkeley.

Wow. Seems okay on the face of it. I see a fair number of people grumbling about how they are going to retire but they still keep putting in the proposals. From a psychological perspective it might work to have them commit to an end date five years away, rather than, saying “Pack it in RIGHT NOW”. Over the next five years maybe that would have a net effect? Seems worth a try, maybe?

I do wonder how this could possibly work in the NIH system on a practical basis. I mean, how can you hold the PI to his or her commitment to stop submitting any grants? How can you keep them from being a significant Investigator on a project for which they are not the PI? The University submits the grants, after all.

But if we suppose it *can* work, is this the best solution? Wouldn’t it be better to just stop funding them? To stop extending any Programmatic pickups to PIs over a certain age? Or to, say, throw down a policy refusing any applications for anything beyond year 20 of a given project? Wouldn’t this, in the end, get rid of more people than offering all of them a Parachute Grant?

And if the plan is to “wind down” a person’s career…..doesn’t this totally fly in the face of the formal structure of the NIH, i.e. that the grant is based on a project, not a person? Are we talking a reverse K99/R00 that starts off with an independent research phase and then ends up as an emeritus fellowship that pays the salary and nothing else for a few years*? Or perhaps we’re talking a project that has to be taken over by a younger PI in years 2-5?

I doubt this will get much traction but if it does, it will be fascinating to see all the proposals for how it should work. I’m sure a few of you will have a go at it in the comments…..
__
*Paying the salary of an Emeritus Professor to sort of wander around the Department helping out has some resonance with my proposal for Staff Scientist Fellowships, I note. I am not entirely dismissing it as valueless.

Defunding the NIH

December 4, 2013

A article in the Pacific Standard magazine by Michael White provides an update on my prior post on The NIH Un-Doubling. The primary point in that post was a graph published in 2007 in

Heinig SJ, Krakower JY, Dickler HB, Korn D. Sustaining the engine of U.S. biomedical discovery. N Engl J Med. 2007 Sep 6;357(10):1042-7. [Publisher Link]

which presented the NIH budget allocations in dollar amounts adjusted for inflation* (expressed in 1998 dollars). The “undoubling” part reflected the 2007 allocation and 2008 Bush administration request in comparison with a trendline established from the early 1970s until the beginning of the doubling. It’s worth revisiting the graph from that article
Heinig07-NIHbudget-trend.jpeg.jpg

Figure 1. NIH Appropriations (Adjusted for Inflation in Biomedical Research) from 1965 through 2007, the President’s Request for 2008, and Projected Historical Trends through 2010.
All values have been adjusted according to the Biomedical Research and Development Price Index on the basis of a standard set of relevant goods and services (with 1998 as the base year). The trend line indicates average real annual growth between fiscal years 1971 and 1998 (3.34%), with projected growth (dashed line) at the same rate. The red square indicates the president’s proposed NIH budget for fiscal year 2008, also adjusted for inflation in biomedical research.

because the updated one, below, only starts in 1990.

NIHBudget-MAW-edit-497x400This new article How We’re Unintentionally Defunding the NIH provides the update, now represented in 2011 dollars. I’m not immediately seeing whether Michael White made this graph himself or sourced it from somewhere else but he does cite a Congressional Research Services report by John F. Sargent Jr which is worth a read.

This is fascinating. We’ve discussed historical funding trends and success rates under NIH extramural grant awards in the past. One post I wrote is highly pertinent:


The red trace depicts success rates from 1962 to 2008 for R01 equivalents (R01, R23, R29, R37). Note that they are not broken down by experienced/new investigators status, nor are new applications distinguished from competing continuation applications*. The blue line shows total number of applications reviewed…which may or may not be of interest to you. [update 7/12/12: I forgot to mention that the data in the 60s are listed as “estimated” success rates.]

The bottom line here is that looking at the actual numbers can be handy when playing the latest round of “We had it tougher than you did” at the w(h)ine and cheese hour after departmental seminar…Things are worse than they’ve ever been and these dismal patterns have bee sustained for much longer. … Anyone who tries to tell you they had it as hard or harder at any time in the past versus now is high as a kite. Period.

One key takeaway from this new graph is a consideration for those who insist that the NIH doubling interval was a poisoned gift. There are those that claim that our current woes are because research Universities and Medical Schools built up tremendous amounts of new infrastructure and personnel during the doubling, with the expectation that that rate of NIH budget escalation would continue. The thinking is that we experienced a bubble and the only reason we have problems now (during this extended interval of budget flatlining and therefore slipping purchasing power**) with dismal success rates. Too many mouths at the trough, is the way I put the situation, even if I don’t specifically blame the doubling interval for this.

This new graph makes it very clear that we have not just returned to the 3.3% growth trendline for the NIH budget. We have fallen off that line. Furthermore, the stimulus funding and the modest increases the Obama Administration have bruited as an initial budget offering are insufficient to change this divergence. It is absolutely clear that the NIH purchasing power is shrinking. Shrinking below the trends established from 1971 to 1998.

This is not a contraction relative to the doubling interval anymore! We’re way beyond that. We look to be as far below the historical trendline as we were above the line at the peak (end) of the doubling interval. We’re something on the order of $8-$10 Billion in the hole, something around 75% of where the historical trendline would have taken us. That seems like a lot of money until you realize

__
*from here:
RPGsuccessbyYear.png
source

**using BRDPI (Biomedical Research and Development Price Index)

Continuing

November 20, 2013

It will be interesting to watch our favorite NIH Institutes’ behavior with regard to starting grants for this round. Traditionally, many ICs are conservative with the Dec 1 start dates when Congress has us under Continuing Resolution instead of a real appropriation. The NIH ICs wait until late Jan-early Mar in hopes that Congress will act.

It is pretty clear, however, that CR is the best we can hope for until at least Feb of 2015 with a new Congress in place.

The possible upside is that the NIH ICs will just go ahead and roll out grants for Dec 1 under the realization that budget levels are predictable at current amounts.

An article in the CHE raises the spectre of the NIH limiting the number of grant applications that a given University may submit.

At a time of dwindling federal budgets, the National Institutes of Health is considering one sure-fire way to raise record-low grant-approval rates: Have researchers apply for fewer grants.

According to how it was written this thinking is due to comments from Sally Rockey, head of Extramural Research at the NIH.

One idea getting some internal study, said Sally J. Rockey, the NIH’s deputy director for extramural research, is to press universities—or perhaps even force them—to simply submit fewer grant applications.

“We have to think about it as a community, how we control demand,” Ms. Rockey told attendees at a conference held here by the Association of Public and Land-Grant Universities. “Because writing applications, submitting applications, and reviewing applications is extraordinarily costly to the community.”

although the article backtracks a bit…

Either way, the NIH is not looking to push anything on universities that they don’t want, Ms. Rockey said. “We have to have a conversation together about how to do all this,” she said.

It tends to leave you with the impression the NIH is actively considering this as an option.

So Rockey clarified matters on her blog.

I have seen the very recent report and follow-on discussions that NIH is considering asking institutions to limit grant applications as a way to control demand. Let me present the facts. You may remember the dialogue we had back in October 2011 on how NIH should manage science in fiscally challenging times. The option of limiting applications was raised at that time but was discarded at the outset and we are not pursuing it now.

also…

The discussion of how to manage NIH funds that we had in October 2011 was engaging and informative, and did result in changes in policy. … The community offered lots of other ideas as well that we may decide to consider sometime in the future, but at the moment limiting applications by institution is not one of them.

Seems pretty clear.

Dr Strangely Strange returns us to the usual conundrum.

What are the key issues that NIH could easily address to make the system more fair, inclusive and to encourage better science (other than throwing more money at us). …Is there something else that we can all agree on that it would make a difference?

I had my usual, highly cynical albeit informed, response:

if the online discussions tell us anything it is that every single person insists that the “obvious”, “just”, “rational”, “fair”, etc solution to the problems of the NIH are whatever just so conveniently happen to suit their own situation or imagined near-future situation.

But here’s what I think we can agree on. We need to shrink the number of people with their hands out for NIH funds. Shrink the number of people being supported as professional scientists. And by “number”, this includes the notion of fractional people, i.e., those who only spend part of their time being paid by federal grant dollars.

The question is…who?

Who gets chopped?

Who is either kicked out of the system or prevented from entering the system in the first place*?

This is where we disagree. Fervently. It is an obvious truth that everyone starts with a very simple and universal principle on who should be shelled out of the NIH-supported system.

“Not Me”.

What I want to suggest for today’s futile exercise in getting the readership to follow their plans ALL the way down is this. Go on RePORTER. Search out some key words that are nice and broad or if you are under a smallish IC just search the whole I or C.

Run through that list and pick out something like 20% of the PIs that you would vote permanently off the island. Find 20% of your peers that you would ace without any detrimental impact on the broader scientific subfield of your interest.

I’d be interested if you come to any general set of criteria for deciding, how you did that. So maybe drop us a comment.

__
*This is a topic for another day but the “painless” solution of turning off the PhD tap is only painless if you forget senior undergraduate you when you were deciding what you really wanted to do was to go to graduate school and earn your PhD.

Elevating a comment from Chris:

IMPORTANT UPDATE: My study section (originally scheduled for Oct 1-2, then cancelled until Feb) is now back on! I just heard from our chair that we will be holding an online meeting in the next few days (thanks for the advance notice). According to our chair, the “significant pressure” put on the NIH from the extramural community has led them to reconsider their decision to cancel everything and bring things back online. I have no authority on this, but would assume that this will happen across the board. So take heart, all may not be lost for this round…

I am likewise hearing rumor that the CSR is reconsidering what they are going to do.

Stay tuned folks, this ride ain’t over yet.

Updated: ps, your comments at Rock Talk can’t help but be viewed as part of the “significant pressure”. Go to it.

Update 2: