Is a prior interval without funding irrelevant to grant review?

A comment from williams on a prior post intrigues me. S/he asserts:

My opinion is that reviewers should assess the scientific merit of a grant in itself (impact/significance/ PI’s scientific potential). Period. If an investigator has had an interval of no support that is irrelevant to the science she/he is proposing to do at this time in point.

In theory, I would agree. In theory.

The trouble is that this sort of assumes “all else equal”. And I just don’t see anything other than vanishingly rare circumstances meeting this standard.

Now to sidetrack for a little bit, the discussion was mired down by the term “failed grant” which to me means an interval of funding that has resulted in some less-than-productive outcome. arzey had said

You can almost always learn more about how to do an experiment from a failed experiment than from a successful one. Or to paraphrase: all happy grants/PI’s are alike, every unhappy grant/PI has lots to teach us.

The commenter williams, however, seems to be obsessed with failed grant applications, i.e., those which do not win funding. A different matter, in my view and somewhat less interesting. Of course we should learn from our applications that fell short. With the comment I started with, we perhaps reach a happy middle ground.

The Investigator component of grant review looms large. Despite the fact that formally speaking the NIH award is to a University and that PIs and other staff could be swapped around willy-nilly in theory, the participating investigators are a huge factor. A large part of the assessment of Investigators is the publication record of the participating staff. Not just mere publications, either, but oftentimes an assessment of how publications fit in with the prior and ongoing research support. If you have relatively little research support, well then reviewers are going to extend more latitude for a publication record that is less than expected. If you have a huge amount of funding, these expectations can rise.

Now let us acknowledge that williams is correct that grant review is supposed to focus on the qualities of the current plan. A good grant score on a new application (i.e., not a continuation of an existing project) is not supposed to be an award for doing well in a prior interval of funding. It is supposed to be an informed prediction that the current proposal will be successful if funded. Likewise, an excellent grant application should not be penalized for apparent deficits in the past work of the participating investigators under a different award.

Herein lies the trouble. Most people assume that past performance of an investigator is a good predictor of future performance, leaving the specifics of a given scientific plan aside. So there is an overwhelming tendency to view a wildly successful prior interval of funding (aka, “track record”) as a good predictor that the current application will result in similar success…..just because of the PI. Similarly, there is a tendency to view a lack of success in a prior interval as a prediction that the current application will not go well.

Favorably disposed reviewers will be looking for some reason to excuse what looks like dismal past performance, I hasten to reassure you. In fact, I have recommended before that if there is some apparent deficit in your own track record, you do what you can to provide an explanation in the application. Subtly. The advocate reviewer needs something to work with. I have personally seen all sorts of explanations, not excuses, explanations in grants. They can go down pretty well. Everything from trailing spouse issues to health conditions to child bearing to local weather disaster to local institutional screwage. If the reviewers like your proposal, they are looking to come up with reasons why your past interval of suboptimal performance does not hold as a predictor of your future performance leading the project.

All this does, however, is confirm the validity of the more general assumption that past performance predicts future performance. Right?

So how would we view an interval without funding? Here we are talking a mid to late career investigator who has had prior support, say, and then has gone for some serious interval (say 2+ years) without a grant. Should the Investigator criterion take a hit? Is this any different from going too long without any scientific publication? Does it, at some level, say something about an investigator if s/he is unable to keep the lab funded*?

I think it has to be a consideration. It is as much a part of the assessment of Investigators as any other traits and credits, in my view.
__
* to be clear I am talking relative to expectation. There are going to be some job categories for which continual funding is not an expectation. And some for which it is something more than an expectation. Let us not get distracted yelling about subfield differences and job types.