Interpreting Grantsmanship Advice, Take Eleventy
September 22, 2008
Your local Office of Grants and Contracts staff only get a look at one part of the grant puzzle and are therefore just as bad as everyone else in accounting for career stage when providing grant advice. This thought struck me recently when I was speaking with a colleague who is preparing to submit her first NIH R01 application. Now the degree of involvement of your Grants & Contracts people varies quite a bit from place to place so YMMV. In this particular case we’re talking about the type of person who assists the PIs with the final assembly of the grant and therefore has some experience with respect to the parts which are not the core scientific parts (e.g., the biosketches, the environment and equipment descriptions, the Vertebrate Animals or Human Subjects, etc.). A person who in their Pre-Award role can answer Institutional information questions, provide guidance on how this particular University handles particular details, etc. The type of person who might helpfully tell the newbie on her first grant to take a look at the BigCheez’s sections and use those, or that might say it looks okay if the n00b does this on her own. You might be inclined to take this person’s word since they are so experienced in seeing what gets funded and what gets triaged at your institution.
Bad Idea.
Permit me a little diversion into additional background. The NIH has announced additional specifics on the action phase of the Review-of-Peer-Review business that we’ve been watching emerge over the past few years. PhysioProf noted in his coverage that the plan to decrease the length of the NIH grant application from 25 pages to 12 pages will land disproportionately hard on the newer and less experienced investigators.
I have said it before, and I’ll say it again, shortening the R01 to 12 pages is going to differentially harm new investigators, who basically need to overwhelm a study section with preliminary data. It is thus absolutely essential that new investigator R01 applications be reviewed in their own clusters.
His comment with respect to clustering refers to the following proposal in the email that was sent (I presume to current PIs).
Review Like Applications Together: NIH is establishing an Early Stage Investigator (ESI) designation. In 2009, NIH will evaluate clustering ESI applications for review.
I went looking for the definition of ESI because I expected that it would be a redefinition for a reason. Sure enough, it is not the same thing as a traditional “New Investigator”. Nope. According to the Aug 2008 OER Nexus:
we are focusing our attention on encouraging earlier transitions into independent research careers by developing programs that focus specifically on early stage investigators (those researchers that are within a set number of years of their terminal degree)…By shifting new investigator incentives to those at earlier career stages, we hope to shorten the prolonged periods of training.
So this clustering of review is important but isn’t going to do anything about current newly-independent investigators. By the way.
To navigate back towards the point, these initiatives remind us that the newer and less tried investigator (NI or ESI or Assistant Prof or however you want to define her) is at a significant disadvantage in the NIH funding system. The most fundamental problem starts with the study section, as I have been know to observe a time or two. The study section does not treat new and established investigators similarly, period. Giving grantsmanship advice on anything from the minor details to the Specific Aims that does not recognize this fact is bad advice.
My close colleague has been observing a few things. Such as the fact that her BigCheez does not appear to include anything like a complete Vertebrate Animals section, does not even remotely estimate the number of mice to be used in the breeding and fails to include certain key components like say the method of euthanasia. It is also the case that this BigCheez does not present a traditional layout with Preliminary Data under one section, Background in another and Research Plans in a third. Detailed methodology? Hah! “Appropriate scope”? Each Aim is about 3 normal R01s…. so this BigCheez may be an extreme example of someone who gets away with unbelievably non-canonical gran preparation on the strength of his rather impressive list of CNS publications each year. Nevertheless, the principles generalize. My colleague knows full well that if she tried to submit a grant structured like the BigCheez it would not. go. well. at. all.
How would the Contracts and Grants guy know this, however? All these staffers see is the fantabulous grant success of the BigCheez types and the struggles of the other people. And they suggest in ways big and small that the problem with the junior people is that they just need to write like Dr. Big. Wrong.
To be clear, junior people need to hew as closely as they can to canonical grant preparation. To take the biosketch, Abstract, equipment and subject protection sections as seriously as they possibly can. Structure the proposal in the maddeningly stultifying order and include the interpretation of results handwaving you may hate (Hi PP!). Not because the reviewers really care that much about this stuff (beyond their duty on the subject protections, of course) but because it is StockCritique bait. An easy way for the reviewer who is on the fence to leap off to the bad side. Dr. Big can get way with things that you cannot.
Facilities and Equipment? Dr. Big uses the same generic description of Teh Massive Lab for everything, no? You need to make it clear what stuff is the property of your own lab, what is shared equipment (and the basis for “rights” to use it), how much space is yours, etc. This is a very active and useful section for dealing with the inevitable “Independence” issues.
Biosketch? Generally not going to be a problem but if you do happen to have more on your CV than will fit the Biosketch page limits, you’d better make sure to tailor what you put in there to the proposal in question. CheezDoodle might be a little more lax about making sure that the pubs that are most relevant to the current proposal are customized into each application.
Subjects Protections? A place to show that you’ve thought longer and harder than the average bear, know the official standards and subfield practices. It is okay to slide a bit of “general Methods” in here as well, IME, to save pages (these sections do not count in the page limits).
Detailed Plans? Including what the money will be spent on (yes, even in modular, no not with specific line items), how long thing are expected to take to accomplish, what experiments will be conducted…
This is one of the areas in which the successful junior investigator writes her way into funding- by preparing such a technically superior application that the reviewer is practically forced to come to grips with the science. Make it easier to deal with the science than to nit-pick, is my advice.
General Writing? Do we even need to discuss why first impressions matter? Maybe BigCheez can pack in a bunch of densely written text because she’s too lazy to continue to edit/refine the presentation. Less tried investigators cannot do this.
I think you take my point so I won’t go on (right now anyway). There are many places to seek grantsmanship advice and I advise using most of them. Some investigators are lucky enough to have staffers such as writedit around (although most are poor imitations of the gold standard, I have little doubt). Others work in departments with highly tuned mentoring or grant-assisting programs. Seek the advice, please do, but with a clear head. Always, always, always…remember that it is key to understand from whence the experience and advice derives my friends. Account for any blindness that may have developed with respect to the relative career status of the PI.
September 22, 2008 at 4:13 pm
Great post. I agree.
Reading this counts as work or ‘training’ for me, right? So it’s okay that I’m not actually getting anything, um, done right now?
LikeLike
September 22, 2008 at 5:00 pm
‘those researchers that are within a set number of years of their terminal degree……So this clustering of review is important but isn’t going to do anything about current newly-independent investigators. By the way.’
As someone who missed the K99 and K22 boat by about 1 year… this makes me kind of want to throw up in the trash can beneath my desk. It is also really bad for people who have had a non-traditional career path- which added time between terminal degree and acceptance of first job…
Also- very good points about the level of detail in all sections for jr. people. Reviewers find and use any weakness in a new investigator- when with the bigcheeze they are more likely just to say- oh, surely he KNOWS what he is doing….
LikeLike
September 22, 2008 at 5:12 pm
The people in Grants & Contracts don’t know jack diddly fucking shit about the aspects of an R01 that are most important to success in peer review. In fact, their degree of relevant knowledge is exactly inversely proportional to the importance of a particular aspect of the application to getting a good score.
LikeLike
September 22, 2008 at 5:42 pm
msphd: of course. reading this blog counts as ‘working’.
drdrA: this is one of those things that tries virtue. when it looks as if the system finally gets it and starts making things easier for those behind you it is maddening. someone on another thread was asking about the midcareer investigator who gets neither the current NewInvestigator sympathy nor the GreyBeard boost. this leads to some “hey, get in line you punks!” thinking. I would not be surprised if this explains much of why we are in the current scenario of (imho) imbalanced review- “they did it to me so I’m going to do it to the next bunch, it is only fair”.
physioprof: they are not all complete idiots and many are, ime, very helpful with certain aspects of grant preparation. in theory, these functionaries can and should have sample facilities, equipment, vertebrate animals and human subjects’ sections from the university to distribute to all first-time writers. a good thing on the face of it.
LikeLike
September 22, 2008 at 7:38 pm
So, does anyone want to read a certain new investigator’s R01 application that is to be submitted for Oct deadline? Just to make sure all these key points have been hit…
Seriously though, fantabulous advice. I’ve been getting advice (and readings) from the local big cheeses. Its quite remarkable how dissimilar my grant is structured from their massively dense 15 yrs of work-type grants. Of course, they’ve said the same thing. Keep the scope tight as a button and the methods detailed and avoid the stock-critiques at all costs.
LikeLike
September 22, 2008 at 8:39 pm
I’ve had a front seat to the mid-career issues as well- via my spouse and the first renewal limbo you describe- and that’s not pleasant either.
LikeLike
September 23, 2008 at 8:58 am
@2 above: “As someone who missed the K99 and K22 boat by about 1 year… this makes me kind of want to throw up in the trash can beneath my desk. It is also really bad for people who have had a non-traditional career path- which added time between terminal degree and acceptance of first job…”
I fall into this category as well (non-traditional track) so I am sensitive to the definition of what a new investigator is (just missed the K99 cutoff). But life moves on and, fortunately, I just Iost “new investigator” status because I was recently awarded an R01 ( fantastic!) However, I was sobered up by a BigCheez who reminded me that the new investigator status is only good for that first R01 grant.
So, as a new investigator, perhaps you “only” need to bring your A game. But after that first R01, you better bring your A+ game because you are in the big leagues and competition is insanely TOUGH. I am already thinking about that next grant….
LikeLike
September 23, 2008 at 10:24 am
I just Iost “new investigator” status because I was recently awarded an R01 ( fantastic!)
I suppose you can take advantage of this situation by having multiple R01s in the hopper when you still have New Investigator status?
LikeLike
September 23, 2008 at 1:51 pm
“As someone who missed the K99 and K22 boat by about 1 year… this makes me kind of want to throw up in the trash can beneath my desk.”
I had similar feelings 6 months into my TT position when they touted the “Pathway to Independence” program as a mechanism to help new investigators…and then made it clear that I didn’t qualify as a new investigator.
LikeLike
September 23, 2008 at 1:51 pm
I’ve had a front seat to the mid-career issues as well- via my spouse and the first renewal limbo you describe- and that’s not pleasant either.
So, as a new investigator, perhaps you “only” need to bring your A game. But after that first R01, you better bring your A+ game because you are in the big leagues and competition is insanely TOUGH. I am already thinking about that next grant….
I understand what you are saying, believe me. I’m in what I guess would be termed early mid-career so you do the math. I got my start in the relative good times which meant that my second stage came along during the belt tightening era. I know what you all are talking about.
And still. Even with the recent wave of New Investigator pickups by program I am far, far, far happier to be facing the not-inconsequential worries that come with the mid-career zone than I would be to be trying to get an appointment or first grant. I really believe the deck is stacked that severely against those without “track record”.
Think of the arguments about being able to run a research program, demonstrating independence, etc that are faced by brand-new investigators. The requirement for preliminary data to be competitive for R01. By the time you’ve had a few years of PIdom under your belt, these criticisms melt away (not entirely but significantly). All you have to do is prove you have been productive and that you still have an idea or two.
The problem that I think really agonizes mid career people is the prospect of losing significant momentum that has been built up so laboriously. Staff and equipment and other resources were hard to acquire, you’ve finally gotten everything in your lab humming and……gak, loss of a grant renewal or inability to get that R01 you really, really should have gotten means…lab crash. Dropping the very postdocs and techs that are so precious to you. Losing resources or equipment time or something. Going back almost to square 1. It fucking HURTS! I know.
but still. You have a backlog of data you can scrounge into a couple of papers. You have a job that permits you to keep writing grants and enough preliminary data to support R01 apps. You have prior pubs and prior funding history. You have evidence of having trained postdocs, even if you can’t afford any right now. etc.
The brand New Investigator does not have such advantages.
LikeLike
September 23, 2008 at 2:42 pm
I think this may be very field-dependent. My own experience with submitting my first grants occurred right after paylines plummeted, and I felt that the two different study sections that reviewed the first two R01s I was awarded were extremely fair and did not stack the deck against me as someone without “track record”.
LikeLike
September 23, 2008 at 3:06 pm
I felt that the two different study sections that reviewed the first two R01s I was awarded were extremely fair and did not stack the deck against me as someone without “track record”.
I don’t think you can ever know this from the outside based on personal outcome. We have NI applications getting fundable scores in my study section. A stacked deck does not mean that nobody can possibly succeed. It is a question of how good you have to be and how good the others have to be to achieve a similar score.
I am perfectly willing to believe that you are so Teh Awesomest that your applications would have been funded regardless. And you may have obtained a 100 on all your apps, I don’t know. But suppose you scored a 125 when *objectively you were a 110- you couldn’t possibly care any less since you got your money. Suppose you scored a 140 when you were objectively a 125 and didn’t get your money. Different story entirely.
But by all means I am willing to entertain the notion that study sections vary tremendously in their fairness. Of course one way to get at this would be to examine their review statistics individually. Something I’m sure the CSR does which does not make it into the generalized summaries that get shown on powerpoint presentations from the brass… the generalized stats, however, support my position that there is a bias. The official noises that we must DoSomething about younger investigators- ditto. the Program pickups to “fix” study section behavior…
*there is no such thing of course so this will almost always have to be in the realm of unsatisfying group stats and subjective impressions from reviewers…
LikeLike
September 23, 2008 at 4:23 pm
Good point.
However, I will say that in the Investigator section of the summary statements for my first two R01s, the reviewers all rated me as an “outstanding” investigator “highly qualified” to carry out the proposed studies, while clearly acknowledging my “unproven/no PI track record” by referring to my “outstanding productivity as a post-doc”.
I’ll say it for you: It’s possible they said that shit and still dinged me on priority score.
LikeLike
September 23, 2008 at 4:59 pm
I’ll say it for you: It’s possible they said that shit and still dinged me on priority score.
It is hard for me to convey the degree to which I was astonished the first couple of times I heard the warm fuzzies about how great the New Investigator was bookended with a clearly out-of-funding score recommendation. That is why I rant, my friend.
LikeLike
September 24, 2008 at 6:01 pm
“The people in Grants & Contracts don’t know jack diddly fucking shit about the aspects of an R01 that are most important to success in peer review. In fact, their degree of relevant knowledge is exactly inversely proportional to the importance of a particular aspect of the application to getting a good score.”
-Ok those might be fighting words. I am a research admin, with years at CSR at NIH under my belt, but I have the common sense not to compare and R01 of an established investigator and apply it to the new investigator. Sometimes I think the role of the pre award person is to make sure everything is administratively perfect so it doesn’t get booted out, and to follow every damn guideline.
There have been times I have seen incomplete sections (such as human subjects) and I bring this up to an investigator, and the new investigators tend to be so stubborn at times they just ignore it. I have even seen curse words and words of condemnation in introductions, I feel it’s my role to say something, but just submit what the PI wants.
LikeLike