Should we be funding projects or laboratories?
September 30, 2008
I’ve remarked before about a curious duality of the NIH system of support for research conducted in your local University or research institute. A duality that is annoying for grant reviewers. Well, at least one grant reviewer. Your Humble Narrator.
The duality is this. The NIH granting mechanisms are supposed to be focused on the project that is being proposed. What is most important are the specific experimental domains detailed, fairly specifically, in the application. All else, including the investigative team, their CVs and track records, the available resources and even the preliminary work are supposed to influence scoring of an application only to a limited and focused extent. Namely the degree to which the bits of extra evidence support the notion that the proposed project and experiments will be conducted successfully.
In many cases, as I have been known to observe, the extra bits such as the track record of the investigator become the prime mover of scoring decisions because the project is unfocused or proposed in such a way that success seems unlikely…save for the track record of the PI, of course.
What I find irritating as a reviewer is not the program-based or the laboratory-based focus per se. I can review a proposal either way, frankly. It isn’t rocket brain surgical science. What really chaps my hide is the duality. The way that sometimes other reviewers will stick to the project-based approach and sometimes venture to a laboratory-based review. When two reviewers on the same application take different approaches things get annoying because nobody explicitly discusses the duality at work. Not to mention that despite their overt rules, CSR for damnsure sees this going on often enough that it is engaging in willing maintenance of this nonsense, if you ask me.
Now it is true that I find the laboratory-based proposals really irritating to read as well. They are usually a nightmarish conglomerate of several different directions and methods based on what they find interesting at the moment (senior investigators) or a Bucky-Beaver bushytailed 15yr+ outline of the GreatProblems they play to solve (junior investigators). Either way, annoying.
So what say your Dear Reader? If you were designing a support mechanism what would you adopt? Funding based on “The PI seems good and works on interesting stuff, give ’em a pile of money to play”? or the often-stultifying and laughably discordant-with-actual-science project-based award system? What are the pros and cons? And are you simply selecting the option that is best for you at your current stage?
DrugMonkey 
September 30, 2008 at 8:02 am
There should be a balance of funding mechanisms that focus on one or the other. What that exact balance ought to be, I’m not sure. How about 25% of funds devoted to investigator-based mechanisms and 75% of funds devoted to project-based mechanisms?
LikeLike
September 30, 2008 at 8:52 am
I’ve come to the conclusion that there are different types of spending, and there should be different types of funding. If our department spent a year letting us work on building cool technology x properly, we could then (sort of simultaneously) solve ten different projects that used it. Instead, we work on ten projects that have no common underlying technology and we never develop past the point of playing catchup.
The funding should sometimes be “we know that if we spend an additional 100k on this lab for this project, they’ll solve 1M worth of problems with it later.”
Of course, it is also essential that specific problems are solved at some point. The key is balance. It’s also an iterated interaction problem – the investment is only worthwhile if the funding agency is likely to fund this lab more than once, something that isn’t always the case.
LikeLike
September 30, 2008 at 8:59 am
The duality may not be as stark as suggested above. I think the reality is that in all cases, it is the investigator being funded, not the project. This makes sense because if you fund the investigator, you know what you’re getting for your money: the investigator. If you fund the project, you have no idea what you’re getting for your money, because the project may or may not work, or the investigator might use the money for something else. Accordingly, we could say the real funding criterion is this:
We will fund: “people who work on interesting projects and have the ability to get interesting results from those projects”
So if you have a strong track record, that’s evidence that you qualify. If you don’t have a strong track record, because you’re new, then your ability to put together a cogent project proposal can be viewed as evidence that you also qualify. In other words, the project proposal is actually a comment on the nature of investigator, rather than a project proposal per se.
LikeLike
September 30, 2008 at 10:13 am
Maybe my field is different, or NSF is different, or I’m just supremely naive, but I don’t think that my field is as focused on the investigator. It undoubtedly plays some role, but even when NSF was closer to 30% funding (which is a high rate for us), big names were not guaranteed continuous funding. (I often heard my big name advisor gripe about this when he was in a dry spell). I hear a lot of talk come out of NSF about “portfolio balancing”.
Incidentally, I believe Canada has a scenario closer to what PP is suggesting. Every PI is guaranteed a small grant to fund their lab, but I also know of people who obtained more than that amount, or additional grants, so they must have additional funding for specific ideas…at least this is through NSERC, don’t know if they have a separate biomedical agency that acts differently.
LikeLike
September 30, 2008 at 10:58 am
The set-up River Tam describes sounds like what would work best in a coherant system. Small grants to keep labs running are incredibly valuble; both for scientists and for the science (we all know how much reinventing the wheel is done when there is not some continuity in labs- it’s best to minimize that).
That said, I can also see the argument for dramatically (politically) agenda-driven grants. Or maybe they should be “prizes” for solving particular problems? I wonder how big the prize/grant has to be to get scientists to try to think like business people and focus on deliverables. My scientific-culture socialized mind says “science just doesn’t work like that” but I don’t have a truly good reason as to why it doesn’t. Certainly there are ample facinating fundamental biological questions that can be paired with practical problems to be solved.
To be blunt and cynical (which I, of course, excel at) we have no frickin clue how to fund science to get the best science. I’m none too sure anyone really knows how to identify the best science (the “I knows it when I sees it” argument is so incredibly shady*).
So it would seem to make sense to have some diversity in funding options to achieve the diverse aims we tend to have for science.
*On the other hand, for the sake of honesty and something vaguely resembling humility, I should note that it is entirely possible that it’s just me who has no frickin clue how to identify the best science.
PS- DM, formatting is looking wonky again
LikeLike
September 30, 2008 at 11:08 am
We should be funding projects BUT, oftentimes, the proposed work has already been done, just not published. Are we funding the proposed work or are we going through the motions of being critical reviewers?
LikeLike
September 30, 2008 at 11:10 am
Yeah, I’d love to know more about the details of the Canadian system. Any Canadians here? (I mean real Canadians, not Americans wishing really hard right now that they were Canadians.
LikeLike
September 30, 2008 at 11:23 am
There are a number of funding agencies that are entirely based on funding investigators per se rather than specific projects – the Howard Hughes Medical Institute in the U.S.A and the Max Planck Institutes in Germany are the most prominent examples. Other examples may include NIH intramural and Cancer Research U.K. . Has anybody ever compared the research outputs of these versus “classical” NIH project funding to determine what produces the best science?
LikeLike
September 30, 2008 at 11:39 am
Has anybody ever compared the research outputs of these versus “classical” NIH project funding to determine what produces the best science?
The HHMI labs with which I am familiar are no slouches when it comes to acquiring multiple NIH R01s. So this is hard to test. There is no doubt that these labs can do lots of extra great stuff because of the non-dedicated cash. Also that the HHMI funding is not really accounted for when determining how fantastically great the NIH-grant related productivity is. But then again, HHMI funding should probably be viewed simply as part of the institutional support which IS a legitimate criterion. the rich get richer, unfortunately…
still, reference to exactly such labs is what puts me on the path of thinking it would be nice if the NIH itself had some (extramural) laboratory-based mechanisms of support. actually they do, these include things like the K05, MERIT extensions could be viewed this way, and their pickups of less-than-meritorious proposals from very senior investigators in recent years can be seen this way too. If you read the language for the R56 Bridge mech they have an explicit desire to keep labs alive with essentially laboratory-based funding if the lab is about to crater. Certain labs, anyway.
LikeLike
September 30, 2008 at 12:09 pm
You need a web-form or email for “Dear DM” letters (possible topics for posts)
LikeLike
September 30, 2008 at 12:15 pm
You need a web-form or email for “Dear DM” letters (possible topics for posts)
I have one, my email is listed in the contact tab in the header- most Sb blogs have similar. it’s drugmnky at the googly one.
also, you may always feel free to make requests in the comments. that would have the benefit of perhaps trolling some of the other bloggers who focus on similar issues into writing their own posts or lengthy comment in response.
what did you want to talk about?
LikeLike
September 30, 2008 at 12:30 pm
Dear DM,
We’ve all seen Bad Emails from Clueless Students. What is the best way to compose a “Dear Supercool Scientist, please send me your awesome transgenic mice” letter?
-a CluelessStudent who wishes to keep their Cluelessness a secret from the wider scientific community
LikeLike
September 30, 2008 at 1:05 pm
My vote goes for funding projects.
However, I do agree that there are PIs who deserve grants for investigation into new cool areas based almost solely on their track record of visionary insight and practical success. So maybe we can set aside a small % of the budget for a separate mechanism (BSD01) or even a separate institute (NIBSD)wherein the aims and reviewer guidelines are suitably adjusted. So maybe you can have the desired duality in funding without the undesirable duality in review?
LikeLike
September 30, 2008 at 1:09 pm
Sorry–I just realized that’s pretty much what ComradePP said.
LikeLike
September 30, 2008 at 1:53 pm
Allocation of money based on project versus investigator should be based partly on whether the proposed project is more basic research or translational/clinical research.
If I am proposing to give 3 different new drugs to groups of monkeys, measure the effects on their physiology and behavior over time, and then study the monkey brains at the end of the study using standard techniques–then the project criterion wins out. This presumes I have some previous expertise in this sort of monkey-based drug analysis, my experimental design is solid, and also that the study section agrees that it is important to test out these 3 drugs in monkeys in the first place.
On the other hand, If I am proposing to study the novel role of Cool Protein X in mechanisms of learning and memory, then a track record of making pioneering new discoveries counts for more. In this case, chances are very good that whatever crap I propose in my RO1 (using hypothesis-driven impeccable logic, with lots and lots of details and preliminary data to support my case–or not) won’t actually be what I end up doing. A track record of making big discoveries implies that if the research takes a sharp left turn, the investigator will be able to take advantage of the situation. If big-ass investigator is working in an area that the study section deems important, then the reviewers conclude that chances are good he/she will discover something worthwhile, if funded.
Of course, the problem with this demarcation is that it suggests that the big-ass hotshots should be the main people doing the basic research–and also that the small town grocers are incapable of thinking on their feet. Being a smallish grocer myself, I don’t actually believe this. But I can see how this sort of thinking influences the allocation of resources
LikeLike
September 30, 2008 at 1:54 pm
Becca, one of my agents suggests the following short and to the point.
I suppose depending on your relative stature in the field you may need to add an introductory line indicating who you are working with, your training stage, etc, just for politeness sake…
LikeLike
September 30, 2008 at 1:58 pm
That is totally wrong. You don’t offer to collaborate with someone in relation to a published reagent just to induce them to send you the reagent. If you *want* to collaborate, that’s a different story.
You politely request the reagent, and state that you will, of course, acknowledge their generosity in providing the published reagent to you in any publications that result from its use.
LikeLike
September 30, 2008 at 5:41 pm
Yes, the trouble is that transgenic mice (particularly tissue-specific, knockout, ultra-cool transgenic mice) are decidedly non-trival to generate in the first place. So I was wondering about whether the etiquette for requesting them differed from the etiquette for something like a plasmid. I think I asked “wrong”, but it looks like the sainted Dr. BigShotte might be sending me mice anyway (I would have been happy with frozen bone marrow- mice plural, [as in, a breeding pair], were beyond what I was hoping for!).
However, it can’t hurt to know how to do it in the future.
LikeLike
September 30, 2008 at 5:50 pm
You politely request the reagent, and state that you will, of course, acknowledge their generosity in providing the published reagent to you in any publications that result from its use.
And if they say no go see what journal they published in and whether that journal has a policy on sharing reagents/mice. If it does, send the no email to the editor in chief and ask for clarification of the policy on sharing published reagents. I have used this strategy previously and it works, these rules exist for a reason you know. It may not make you any friends though, so consider that as well.
LikeLike
September 30, 2008 at 7:04 pm
Funding the lab/investigator rather than the project seems like a way for the rich to get richer and the good old boys to keep their club small.
LikeLike
September 30, 2008 at 8:31 pm
Funding the lab/investigator rather than the project seems like a way for the rich to get richer and the good old boys to keep their club small.
Yep. This is the risk and some (such as I) might argue that despite the best project-based intentions of the overt NIH system, the covert investigator-based system has some OldBoyz/Girlz aspects to it.
LikeLike
October 1, 2008 at 1:58 am
Funding the investigator basically doesn’t make much sense to me, personally. If the investigator is so awesome that we can throw money at them no matter what, then surely they will be able to put together a nice innovative, coherent, and hypothesis-driven R01 application. If they are too lazy and/or too busy showing their BigAss off at conferences and talks all over the world, then they can give the task of putting the grant together to some of the YoungAndBrilliant postdocs from their lab. Why, if they are amazingly productive and clearly capable of writing a C/N/S level paper, would they not be able to write an R01 app. Give me a break! Funding labs, like many have said before me, just contributes to the rich getting richer phenomenon and creates a huge resource gap between the CelebrityLabz and the rest of us. This is, IMHO, deleterious for science because it creates a lot of resentment in the lower ranks and consequently results in loss of a lot of talent to industry and other careers. To me it is kind of like feudalism vs. democracy battle, and democracy (with all its problems) should prevail.
LikeLike
October 2, 2008 at 12:55 pm
There are a number of funding agencies that are entirely based on funding investigators per se rather than specific projects – the Howard Hughes Medical Institute in the U.S.A and the Max Planck Institutes in Germany are the most prominent examples.
In addition to DM’s points, there’s a difference between supporting a stable of hand-picked superstars and maintaining a country-wide research system. The former is always (or maybe not?) going to outperform the latter on a per-capita basis.
LikeLike
October 2, 2008 at 1:18 pm
I want to sound in on the side of funding the investigator rather than the project (since most people seem to be on the project side). I think there are two important advantages to funding investigators rather than projects.
First, and foremost, it’s a LOT easier to identify successful investigators than likely-to-succeed projects. We can just look at the last five-year track-record and say “good work, here’s another five years”. Second, the real science, the breakthrough stuff that most of us [I think!] are trying to do doesn’t work by knowing what the answer is going to be before we do it. The real cool stuff occurs when we are given the freedom to pursue directions. One of the biggest complaints about the NIH funding system is how conservative it is. It is almost impossible to break into a new field. Once you get labeled as a “computational” person or a “pharmacology” person or “someone who works on amygdala”, it is almost impossible to shift to do anything else. I know people who have published multiple papers in experimental physiology from a brain structure for over a decade who still can’t get funded to do experimental physiology in that structure. (I even know one National Academy member who had trouble with this – took her a decade to get credibility to switch fields.)
Now, I’m not suggesting that we keep funding bigger labs more and more. I’m suggesting that every lab that has been minimally successful over the last five years should get a basic R01-size grant. How many labs could run succesfully on one R01? (Most, I believe.) Imagine if any successful lab could get one R01 by just sending in the set of papers published in the previous five years. Then, if that lab wants more than that one R01, they can go through the hell (and it is hell, as we all know) of writing 25 pages convincing some idiot on study-section [and I’ve been that idiot, so I know] that you really do know what your doing and it really is a good project. I’d be perfectly happy with having a (say 10%) funding mechanism for funding projects, but that should be over-and-above (and separate!) from the laboratory funding.
LikeLike
October 2, 2008 at 1:53 pm
and under your approach qaz, how do you identify the successful New Investigator? On postdoctoral track record?
with respect to single R01 funding, this may work for hard money investigators. not so much for soft money. although that’s perhaps a decent trade-off for their general lack of other time-consuming responsibilities like teaching.
I am not sure how your system gets away from the conservatism, either. unless you dismantle the individual IC structure of NIH. but then you end up with potentially a worse problem which would be the tyranny of the majority. a very much worse problem IMO. In fact, your approach would worsen the problem would it not? because review panels would hew even more closely to “their” people and exclude interlopers they don’t know yet..
LikeLike
October 2, 2008 at 4:37 pm
My goal here is to make some baseline money a given for scientists making good progress and to remove (or at least reduce) the burden of grant writing, which I think hurts science badly (by costing valuable time and effort).
The first R01 could either be a special category or (more reasonably) could be based on one’s track record after using two to three years of one’s startup money. That second concept is very much what happens to most investigators anyway. It takes 3-5 years to have enough preliminary data to get your first R01. Those 3-5 years are generally covered by startup costs. I would argue that we could have a separate category for first R01s, saying “let this person into the laboratory cycle”.
The problem of conservatism is in the don’t want to fund new projects, not really in the who’s-in-the-old-boy-network (my experience crossing fields has been that most fields are very willing to welcome new people, just not to fund their projects). What I’ve seen on study sections (all ad-hoc so far, but on multiple R01, NRSA, and center-grant sections) is that people don’t trust new people to do their work. I keep hearing “this is a stellar investigator who does great X work who is now trying to do Y. Unfortunately, stellar investigator has no track record in Y…” I’m honestly not too worried about the “letting people in” once they’ve been doing the work. The problem with funding seems more to be about “I don’t believe you can do that”. That’s why it’s so much easier to break into a new field publishing papers than getting grants. Because once it’s been done, they can’t really argue that it can’t be done.
I envision three categories: (1) first R01, which takes into account plans, projects, preliminary data, etc. (2) continuation R01, which only looks at the last five years. and (3) a let-me-back-in category for people who’ve fallen off the track. Category 1 would be not-to-difficult, but something that should be doable for any reasonable A1-scientist. Category 2 should be a given unless something really goes wrong. Category 3 is the catch-all so we don’t really lose anyone willing to put the effort in.
From discussion with my senior colleagues, I suspect that this is how NIH used to work – your first R01 was hard but doable, and then continuations were pretty much a given as long as you kept up good work. It was a rare person that had two R01s.
Finally, in response to soft money, that’s another issue. And I agree there’s no good solution. But, I would argue that making scientists pay their own way through soft-money is a crime in itself. (I have a friend who has a tenured soft-money position. ! So they can’t fire her, but they don’t have to pay her either?) Institutions should be paying investigators for their work. Institutions have become dependent on NIH overhead. (I know of two different institutions where the solution to their budget problem is to hire 200 new faculty, each of whom would bring in $2M in grants…) But that’s another rant. (:
LikeLike