Your Grant in Review: Productivity on Prior Awards

A commenter named ES over at writedit’s blog asked:

I am wondering if I can conduct unfunded research projects using my current RO1 support since I have more experience and more publications on these unfunded projects. In contrast, testing our hypothesis proposed in my funded RO1 is practically not promising. If I decided to do something else which was not originally proposed in my funded RO1, how can I renew my funded RO1 or submit the report down the road.

and expands upon the theme:

the issue here is that my unfunded and funded project are total irrelevant. It is almost impossible to convert my unfunded project to something which is even close to the funded project. Can I do whatever Science takes me to with the unfunded project, and submit the unfunded project as new applications later, instead of renewal of my RO1? Will this cause a bad record for my academic career?

There was also a related query from Saheli in that thread:

2. Should the renewal reflect a substantial continuity to the current grant?
3. What the ingredients of a successful renewal?

The question from ES led to some startlingly bad advice.

writedit: “you run the risk of not being funded again (& not just in your attempt to renew the R01).”
cc: “i would not change anything without discussing w/project officer, who should want to see the best science come forward.”
The real howler, was in this one from writedit:

I would hope at the very least that PIs who spend their grant $ on other projects have presented them to their peers and received some feedback on feasibility, suggested tweaks, etc. … the refinements a PI might not make on his/her own without going through the proposal process. Funds are too tight and the margin of error too small (in terms of available funds for re-dos), methinks, to wing it on one’s own.

Although CPP and BM are all over it in the comments, I thought I had a contribution.
The core question here is how will the reviewers of your current grant application view your record of accomplishment on your prior grant awards.
I submit to you that in most cases there is a cascade of preferences that is approximately ordered as follows.

1) Did you knock our socks off? This could be amazing ELEVENTY type findings, GlamourPub record (whether “expected” for your lab or not), unbelievably revolutionary advances, etc. If you have a record* of this, nobody is going to think twice about what your Aims may have been. Probably won’t even give a hoot whether your work is a close match** to the funding IC, for that matter.

2) Were you productive? Even if you didn’t WOW the world, if you’ve pumped out a respectable number of papers that have some discernible impact on a scientific field, you are in good shape. The more, the merrier. If you look “fabulously productive” and have contributed all kinds of interesting new science on the strength of your award(s), this is going to go down like gangbusters with the review panels. At this level of accomplishment you’d probably be safest at least be doing stuff that is vaguely in line with the IC that has funded your work. But even so, and related to ES’s query, better to have produced papers on topics far afield, than not to have published anything at all due to your original Aims being a dry well.

3) Were you productive in addressing your overall goals? This is an important distinction from the Specific Aims. It is not necessary, in my view, that you hew closely to Aims first dreamed up 7 years prior to the conclusion of the actual study. But if you have moderate, or disappointing, productivity it is probably next most-helpful that you have published work related to the overall theme of the project. What was the big idea? What was mentioned in the first three sentences of your Specific Aims page? If you have published work related to this broad picture, that’s good.

4) Did you address your original Specific Aims? Now true, this can be a big obsession of certain reviewers. Not saying it isn’t a good idea to have papers that you can connect clearly to your prior Aims. This is particularly important for a competitive renewal application. I place this far down the list however, to emphasize my usual theme: A grant is not a contract. It is quite natural in the course of actual science that you will change your approaches and priorities for experiments. Maybe you’ve been beaten to the punch. Maybe your ongoing studies tell you that your original predictions were bad and you need to go in a whole new direction. Maybe the field as a whole has moved on. With that said, however, you will make it easier on your advocate reviewers if you have addressed your Aims. And since by that I mean with published papers, this is where you need to think hard about your tactical decisions. You might want to squeeze a drop out of a dry well to meet the “addressed Aims” criterion but maybe that money, effort and time would be better spent on a new direction which will lead to three pubs instead of one? I’d go with the latter, personally.

Circling back to writedit’s absurd comment, I want to emphasize that deciding tactics is the normal daily business of the scientist who heads a research group and manages research funds. Where the most useful, interesting and productive science lies is on you to decide, PI. You don’t require “the proposal process” or the input of formal peer review to determine how to do your science! I mean sure, we all may seek advice from colleagues about planned experiments and all. But there is nothing magical about proposal review in a NIH study section that will be able to help you make ongoing tactical decisions within the scope of conducting the actual work related to your funded 5yr plan.

With respect to the conflation of your ability to get a favorable review of a competing continuation and new projects, I emphasize for ES’s question that these are very different questions. It is absolutely ok to fail to submit a competing continuation for one project that went badly in preference to submitting a new project. This will not lead to a “bad record for (your) academic career”. It is preferable, yes, to renew some of your projects. This is because of some old school thinking and grant award experiences. But times have changed and the thinking needs to change with it. In terms of priorities, this comes down the list behind, you guessed it, productivity.

The bottom line is, you should worry first about the publications and your substantive contribution to a subfield of science. Be a scientist. Don’t march in lock step to a plan laid out in a grant proposal if it is not leading you anywhere useful.
__
*one shot wonder GlamourPub may not be enough but an amazing bit of scientific advance may very well be the ticket.
**of course this works best for the basic research stuff. The more basic you are, the better you can argue that starting with Aims in National Institute X’s wheelhouse led inexorably to findings more relevant to National Institute Y.