Reprint: Open Access Grantsmanship

October 12, 2010

This originally appeared on October 17, 2007.



BikeMonkey Re-Post
I was reading one of the summaries of the CSR Peer Review open house roundtable things, from the “Neuroscience” one. The thing that struck me was that over 50% of participants had never had a grant triaged.
Now the first thing that comes to mind is the people that NIH usually drags in to give them an “authoritative” opinion on various topics of interest. The opinions are most frequently sought from research luminaries, heads of institutions and society officialdom, i.e. (very) senior scientists.

These PIs are utterly unrepresentative of the pool of NIH applicants (and potential applicants).

In this particular case, however, there is a possible alternate explanation which is that triage is not in fact the norm for “good” scientists and those of us getting streamlined with any regularity are just writing bad grants. Or are bad scientists. etc. This is the sort of thing that keeps junior (and not so junior) scientists awake at night. It would be nice to have some current and specific information on triage rates across the NIH. Trouble is, this is not a data set that is easily obtained. The specific questions of the day being, “What proportion of scientists (as opposed to applications) applying to the NIH are triaged? Have these numbers changed over time? What do the per-PI triage rate distributions look like?”.

I was discussing this a little bit with a colleague and came to the realization that this is the sort of information we don’t even get a clear bead on with our usual collegial chit chat. Mostly because it is hard to keep track in a bunch of casual conversations how many grants someone has put in, how many times they’ve bitched about a triage, etc. I realize also because it is ever so slightly taboo to really ask someone these sorts of things. I, for one, wouldn’t feel comfortable asking “So, how many grants have you put in and how many triages?” The few (two?) times someone has asked me for similar specifics on how many grants I’ve put in, I recall having a slight negative reaction. Like “Hey, that’s private dude!“. There’s another issue which I realized after quashing the first sentiment which is that even I don’t keep track of the numbers very well. I know this sounds strange to someone in year 2 of grant writing but after a while…

So, for today’s lesson and in the spirit of OpenScience, I’ve bothered to pull my grant submission data from Commons for your entertainment and derision. Be kind. YMMV, of course. And naturally, this only counts the stuff where I have been the PI for the submission.

I’ve been putting proposals in since early 2000. I count up 20 Type 1 and 2 Type 2 applications submitted. Yikes, has it really been that many? In that list I have 8 A1s and 2 A2s. Fifteen of the applications were scored and 7 were streamlined. Three were funded so far. (I will note that this is not sufficient in soft-money land, of course and the answer to “how?” is that I’ve acquired at least an equivalent portfolio through sub-components, pilots and the like.)

Of course, the question of individual “success rate” beyond simple definitional purposes is complicated. I’ve abandoned my Type 2 after two triages so there would theoretically have been a third chance that I’ve chosen not to take. I’ve lost interest in pursuing a particular line with two grants because a competitor in the field (and now at least two I note) has gotten funded to do very similar stuff. I have at least three that are in active revision mode, although these will trickle across several rounds as I get to them. I also have a few more that are somewhat promising after the -01 or A1 review but are a bit down my priority list. Never gone, of course. Especially the ideas. Nevertheless it is hard to determine what the denominator should be. As the DM is fond of pointing out, if you haven’t revised you haven’t really submitted an application.

I note something a little more subtle which is that I have, for the most part, taken at least two lines of attack on a given research area of interest to me. This means that one of the two usually gets abandoned if the other is funded or sidelined if the other has a more promising score. Sometimes a sidelined one later gets dropped because my own work has moved on, the field indicates different directions or a competitor gets a related grant funded.

What other fun things does this sort of review reveal?

  • So far, I’ve never had a grant triaged in an SEP review even though a couple of 250-280 range scores show that it was a close thing.
  • In the salad days of the early noughties a 170 was a 19%-ile and fundable, in this most recent round a 28.6%-ile and not even close.
  • I have 5 scores in the 160-175 range over the years. I might encourage people to view this range as a good indicator that you are being taken seriously as a scientist, you have good ideas and can write a grant. It may be a matter of luck to improve from this range. For example it is hard to show where my one 120/1.6% scored application is this dramatically/categorically different, say, from my 160-170 scored ones.
  • Of the 7 applications that have been reviewed in the most-frequent study section, I have 4 triages as well as a personal best score. This is relevant to theories that “that particular study section hates me”.
  • None of my proposals funded to date have gotten there after an initial triage. This stat is contaminated by the reduced chance that I will have revised a triaged proposal, of course.

So, nothing too surprising here since I had a pretty decent seat-of-the-pants recollection of how I’ve been doing. I think the most interesting thing is the fate of triaged applications. Mostly because a very common question from new applicants is “Should I abandon a triaged application?”. My default response to this is “No way”, mostly because of the way revisions are treated; DM has similar attitudes posted here and there. Also because of a sort of back-of-the-head suspicion that we’ve had applications triaged in our section which then are revised into highly competitive applications. But are they? Again, this is an area where some hard CSR data could be useful. The anecdote of me suggests at this point that perhaps it is wise to abandon a triaged application. This counters my gut feeling but the data are what they are.

The link regarding the 50% never-triaged also suggests that 23% of respondents had a previously triaged grant eventually win funding.


Looking again at Commons, it would appear that I have submitted four additional grants with myself as the PI since writing this- one new, one A1 and two A2. All were scored and two have been funded. I have also put in roughly a grants’ worth of effort on about 5 additional submissions over the time since the original post.

Advertisements

No Responses Yet to “Reprint: Open Access Grantsmanship”

  1. Dorothea Says:

    Thanks for this! It is hugely helpful for librarians and grant administrators working with applicants.

    Like

  2. BikeMonkey Says:

    Where do you see this applying to the situations that you run across with applicants? What kinds of advice do they expect from you?

    Like

  3. Malone Says:

    DM dude, 3 of 20 is awesome in the ways I am thinking. I got to know dude, at what number did you get your first grant? 1 of 1, 1 of 5,..? I also got to know if you had an independent position when you started writing in 1990s? 20 applications is a serious number and I am asking these questions because I want to know how the hell you keep motivated to keep writing.

    Like

  4. BikeMonkey Says:

    If memory holds, I landed my first fundable score on the third proposal I submitted.

    Like

  5. BikeMonkey Says:

    MY motivation?

    Desire, Fear, Obligation, Anger…

    In no particular order.

    My appointment was independent, formally speaking.

    Like

  6. Malone Says:

    This is spooky, I am feeling “Desire, Fear, Obligation, Anger”. Right now “Anger” is toping the list. My fear is from fear of failure and the fear that this excruciating process will chew me and spit out with no motivations left to pursue the desire to excel.

    This post connects with me in odd ways and contents which aren’t the primary intentions and focus of this post.

    Like

  7. Malone Says:

    And the “Obligation” right now is slowing down my productivity. All my efforts started with a pure ambition, but the progress is pushing my process into a vicious cycle.

    Hell, this sounds like a bad poem.

    Like

  8. BikeMonkey Says:

    This post connects with me in odd ways and contents which aren’t the primary intentions and focus of this post.

    Which is….? 🙂

    The fear of failing as a grant-supported scientist;

    The anger at the idiotic reviewer #3 who kills your fantastic proposal;

    The obligation to your hiring department, peers, lab staff and trainees;

    The desire to work on that pressing scientific question.

    These are all part and parcel of the job. Everyone complains about their lack of success and how the granting process is unfair. To truly assess that, however, it is necessary to ask how hard someone is working at the job.

    Twenty or thirty R01 applications sounds like a lot to someone who has just completed his or her second one. It gets easier. And it has been a fairly long period of time. But you still have to be able to grind these babies out with some regularity.

    Or maybe not. Maybe I am unusual in the number of applications that I’ve submitted to get to the level of minimal success I have enjoyed. But at least you can see what I would say to some investigator who is complaining about the system. And if s/he had submitted only 2-3 apps and concluded that it all sucks, well, I’d suggest trying a little harder. I don’t see any reason to assume that anyone can beat the odds. And the odds are more or less in line with the NIH success rate. So if that number is 20%, well you need to put in at least 5 apps to have confidence. If it is 10%, then 10 apps.

    If you are throwing in the towel before submitting these sorts of numbers (depending of course on the concurrent funding environment) you are asserting that you are something quite special as a scientist. There is probably little justification for that.

    Like


  9. I have averaged two type 1 or type 2 R01 applications per year over my independent career, and have had ~25% of these funded

    Like

  10. Malone Says:

    Hey DM:

    Thanking you for seriously jolting me back to reality. I have been having hard time comprehending what was going on, I am nothing special as you said.

    When I read your response at October 12, 2010 at 2:24 pm, I thought you said in so many words to quit being a baby and grow up (actually in my head I thought you said, “stfu and quit being a pussy”).

    I had a glimpse of what I used to be just a year or two ago. Somehow over these two agonizing years, I have started feeling self pity, and feeling my efforts should already be paying up or why my peers who aren’t that great with work are growing up while I am stuck. I understand that I am cherry picking points to compare to feel the self pity.

    I felt like somebody jolted me back to the enthusiasm and vigor I had trying new exciting things when I read your response (may be that is just my interpretation). I hope this will stay with me, I will quit being a baby, stfu and work even harder.

    -M

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: