Your Grant in Review: Productivity on Prior Awards

August 26, 2011

A commenter named ES over at writedit’s blog asked:

I am wondering if I can conduct unfunded research projects using my current RO1 support since I have more experience and more publications on these unfunded projects. In contrast, testing our hypothesis proposed in my funded RO1 is practically not promising. If I decided to do something else which was not originally proposed in my funded RO1, how can I renew my funded RO1 or submit the report down the road.

and expands upon the theme:

the issue here is that my unfunded and funded project are total irrelevant. It is almost impossible to convert my unfunded project to something which is even close to the funded project. Can I do whatever Science takes me to with the unfunded project, and submit the unfunded project as new applications later, instead of renewal of my RO1? Will this cause a bad record for my academic career?

There was also a related query from Saheli in that thread:

2. Should the renewal reflect a substantial continuity to the current grant?
3. What the ingredients of a successful renewal?

The question from ES led to some startlingly bad advice.

writedit: “you run the risk of not being funded again (& not just in your attempt to renew the R01).”
cc: “i would not change anything without discussing w/project officer, who should want to see the best science come forward.”
The real howler, was in this one from writedit:

I would hope at the very least that PIs who spend their grant $ on other projects have presented them to their peers and received some feedback on feasibility, suggested tweaks, etc. … the refinements a PI might not make on his/her own without going through the proposal process. Funds are too tight and the margin of error too small (in terms of available funds for re-dos), methinks, to wing it on one’s own.

Although CPP and BM are all over it in the comments, I thought I had a contribution.
The core question here is how will the reviewers of your current grant application view your record of accomplishment on your prior grant awards.
I submit to you that in most cases there is a cascade of preferences that is approximately ordered as follows.

1) Did you knock our socks off? This could be amazing ELEVENTY type findings, GlamourPub record (whether “expected” for your lab or not), unbelievably revolutionary advances, etc. If you have a record* of this, nobody is going to think twice about what your Aims may have been. Probably won’t even give a hoot whether your work is a close match** to the funding IC, for that matter.

2) Were you productive? Even if you didn’t WOW the world, if you’ve pumped out a respectable number of papers that have some discernible impact on a scientific field, you are in good shape. The more, the merrier. If you look “fabulously productive” and have contributed all kinds of interesting new science on the strength of your award(s), this is going to go down like gangbusters with the review panels. At this level of accomplishment you’d probably be safest at least be doing stuff that is vaguely in line with the IC that has funded your work. But even so, and related to ES’s query, better to have produced papers on topics far afield, than not to have published anything at all due to your original Aims being a dry well.

3) Were you productive in addressing your overall goals? This is an important distinction from the Specific Aims. It is not necessary, in my view, that you hew closely to Aims first dreamed up 7 years prior to the conclusion of the actual study. But if you have moderate, or disappointing, productivity it is probably next most-helpful that you have published work related to the overall theme of the project. What was the big idea? What was mentioned in the first three sentences of your Specific Aims page? If you have published work related to this broad picture, that’s good.

4) Did you address your original Specific Aims? Now true, this can be a big obsession of certain reviewers. Not saying it isn’t a good idea to have papers that you can connect clearly to your prior Aims. This is particularly important for a competitive renewal application. I place this far down the list however, to emphasize my usual theme: A grant is not a contract. It is quite natural in the course of actual science that you will change your approaches and priorities for experiments. Maybe you’ve been beaten to the punch. Maybe your ongoing studies tell you that your original predictions were bad and you need to go in a whole new direction. Maybe the field as a whole has moved on. With that said, however, you will make it easier on your advocate reviewers if you have addressed your Aims. And since by that I mean with published papers, this is where you need to think hard about your tactical decisions. You might want to squeeze a drop out of a dry well to meet the “addressed Aims” criterion but maybe that money, effort and time would be better spent on a new direction which will lead to three pubs instead of one? I’d go with the latter, personally.

Circling back to writedit’s absurd comment, I want to emphasize that deciding tactics is the normal daily business of the scientist who heads a research group and manages research funds. Where the most useful, interesting and productive science lies is on you to decide, PI. You don’t require “the proposal process” or the input of formal peer review to determine how to do your science! I mean sure, we all may seek advice from colleagues about planned experiments and all. But there is nothing magical about proposal review in a NIH study section that will be able to help you make ongoing tactical decisions within the scope of conducting the actual work related to your funded 5yr plan.

With respect to the conflation of your ability to get a favorable review of a competing continuation and new projects, I emphasize for ES’s question that these are very different questions. It is absolutely ok to fail to submit a competing continuation for one project that went badly in preference to submitting a new project. This will not lead to a “bad record for (your) academic career”. It is preferable, yes, to renew some of your projects. This is because of some old school thinking and grant award experiences. But times have changed and the thinking needs to change with it. In terms of priorities, this comes down the list behind, you guessed it, productivity.

The bottom line is, you should worry first about the publications and your substantive contribution to a subfield of science. Be a scientist. Don’t march in lock step to a plan laid out in a grant proposal if it is not leading you anywhere useful.
*one shot wonder GlamourPub may not be enough but an amazing bit of scientific advance may very well be the ticket.
**of course this works best for the basic research stuff. The more basic you are, the better you can argue that starting with Aims in National Institute X’s wheelhouse led inexorably to findings more relevant to National Institute Y.

10 Responses to “Your Grant in Review: Productivity on Prior Awards”

  1. I dunno what’s got into Writedit. Maybe too many years being an academic bureaucrat has addled the fucken brain.


  2. DrugMonkey Says:

    Be nice, now. Look, this just highlights the fact that different viewpoints on the grant game can result in different interpretations of how to succeed.


  3. If I were a n00b looking for guidance, I would favor the viewpoints of those individuals who write grants, obtain grants, review grants, pursue the science, publish papers, and renew grants, over those of academic bureaucrats who observe such activities but do not engage in them. Bad advice needs to be debunked in no uncertain terms.
    The idea that you should contact your Program Officer to discuss potential divergence from your proposed specific aims is downright dangerous.


  4. The idea that you should contact your Program Officer to discuss potential divergence from your proposed specific aims is downright dangerous.

    To amplify: it is like pulling up to a cop all drunke offe your fucken asse leaving the parking lot of the local tavern and asking her if she thinks you’re too drunk to drive. At the end of the day, academic bureaucrat grant administrators like writedit represent the interests of the institution, not those of individual PIs. It is important to keep this in mind.


  5. DrugMonkey Says:

    umm…interesting analogy.
    but yes, the job of the PI is to show the Program Officer why what she has accomplished is important and mission critical, not to rely on the PO to tell her what to do in advance.


  6. hui Says:

    Hi, longtime reader delurking here.
    Is the logical extension of the idea that “a grant is not a contract” to ask whether we need to propose experimental plans at all? That is, if the assertion that the successful recipient of an R01 should be able to figure out how best to spend the money, and study section provides little to no additional input with regards to experimental direction, why even bother having study section read proposals? Might not their time be better spent reading one page CVs which include a short paragraph on research directions, then ranking those?
    Or was that the idea behind the great Proposal Shortening? Not to mention, people can rank one page CVs from anywhere – leading to “virtual” study section being more practical than travel. Hm.
    At this point, why not have a lottery? Maybe your probability of winning the lottery could be weighted by factors that suggest a researcher would be better able to spend it (eg institution, prior publications, reputation)?
    Or is that essentially the current system? Except that the weights are biased and given disproportionate influence.
    Please note that I ask these questions with absolutely no snarkiness. I ask as an early stage postdoc who may have just finally put the big picture together.


  7. abda Says:

    @ hui,
    I think the study sections DO evaluate the research plans and the R01 for all of the scientific points that you normally think about. So, for winning the grant, you have to at least show that you have a well proposed and planned experimental design. The reviewers will have seen that you can think through the whole process, and have confidence in your ability to plan. But, in carrying out the experiments, you may have to shift focus. Most reviewers acknowledge this as a likely outcome in 4-5 years, since the whole field will move in ways that are unpredictable. In fact, if you just stuck with your original plans despite the field moving forward, that would be a killer flaw.
    Regarding the lottery system, unfortunately with the payline going below 10%, we really do have a lottery system between the top grants (after all, there really is no difference between the 3rd best and 4th best grants in a study section). That sucks, but you have to get yourself into the lottery, which is a small pool of maybe 10 grants per study section–so write great grants and do fantastic work. It still matters.


  8. In my experience, R grants are pretty open-end, while U grants are much more milestone-driven. The funding mechanism matters.


  9. […] visited these themes before in a prior post. I think I covered most of the issues but in a slightly different […]


  10. […] have frequently hit the productivity zone identified in NIGMS data…and had my competing revisions criticized severely for lack of […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: