Independence
September 25, 2007
It is a StockCritique of grant review and promotion/tenure review alike.
The concern is related to the tendency we have to assume that the most senior person involved in a research collaboration is the one “really” calling the shots. The one providing the most sophisticated intellectual ideas and creativity. The one in charge. The assumption in the “independence”critique is that the person criticized may not have what it takes to succeed or excel scientifically “on their own” and is thus not worthy of promotion or the stewardship of a major grant award. Is this a valid criterion?
I’m wrestling with this issue for a number of reasons but let us focus on one domain for didactic purposes. I, as well as several other colleagues who review grants, have noticed a seemingly sharp uptick in the number of applications coming in from PIs who are more “transitioning” than “transitioned”. PIs whose job titles might be something other than “Assistant Professor” and ones who are still in or around the same laboratory or research group in which they have done a big chunk of postdoctoral work. In extreme cases the PI might still be titled “Postdoc” or have trained in the same place essentially since graduate school!
Readers of this blog might conclude that this trend, which I’ve been noticing for at least the past 3-4 rounds, delights me. And to the extent that it represents a recognition of the problems with junior scientist’s making the career transition to independence this does appear a positive step. To the extent that it opens up artificial barriers blocking the next generation of scientists- great.
The slightly more cynical view expressed by colleagues and, admittedly, myself is that this trend has been motivated by IC Program behavior both in capping per-PI award levels and in promoting grant success for New Investigators. In other words that the well-established PIs with very large research groups are thinking that grants for which they would otherwise be the PI will now be more successful with some convenient patsy long-term postdoc at the helm. The science, however, is just the same old stuff of the established group and PI.
The question for this post, however, is the same regardless of the motivation for a not-very-independent (*by appearance) PI to submit an application. Namely, why does it matter whether someone is “independent” or not? Why does this consideration figure so heavily into the “Investigator” section of the grant critique and in consideration for tenure? Isn’t the goal that “the best possible science” get accomplished? If we have evolved into large groups of postdocs and semi-career research scientists laboring under a BigCheez PI doesn’t this beg the question? Why does it matter who is the titular PI of a given project?
—
*the question of how we demonstrate (as applicants or job/promotion seekers) or determine (as reviewers and search/promotion committee members) scientific independence is perhaps a topic for another lengthy discussion.
DrugMonkey 
September 25, 2007 at 12:01 pm
If we have evolved into large groups of postdocs and semi-career research scientists laboring under a BigCheez PI doesn’t this beg the question?
I would say that we have evolved a long way towards that — but attitudes and evaluation criteria have not evolved to accomodate the new structure. There is no “semi-career scientist” position, though there are lots and lots of de facto semi-career scientists.
LikeLike
September 25, 2007 at 12:49 pm
My use of “semi” was intended to connote the same thing as your use of “de facto” I expect.
“attitudes and evaluation criteria have not evolved to accomodate the new structure”
Do I take it your answer to the main question is that traditional quibbling about “independence” can be accommodated with a new sensibility about long term “just doin’ the science” careers?
LikeLike
September 25, 2007 at 12:58 pm
One thing to consider is the problem faced by the mid-career, pre-tenure and even pre-Full Professor regarding “independence”, especially with the expectation that collaborating PIs get on the papers with even the smallest of contributions. It really inhibits the science if these more junior people have to be careful about collaborating with more senior faculty because they will not be seen as “independent” down the road.
LikeLike
September 25, 2007 at 2:51 pm
DM: yeah, pretty much. I think there’s room and a need for those long term “just doin’ the science” careers. Not everyone wants to be a BigCheez, but the system currently tries to force everyone to compete for BigCheez slots.
There arises the question of who will pay the long term labrats — if it’s entirely up to PIs, many will opt for traditional postdocs, thinking they can squeeze more work out of ’em that way. If institutional funding gets involved, it could mean a pretty drastic change from the way things are done now. (I am, of course, a complete tool and missed my chance to sit down, think through these issues and submit a proposal to the recent “call for suggestions” thingy…) There was an interesting discussion of ways to pay for such positions here.
Also, it occurs to me that, although it makes work for reviewers, the “convenient patsy postdoc” should be identifiable in most cases by comparing the current (patsy) grant with earlier grants and publications from the same lab.
LikeLike
September 25, 2007 at 3:41 pm
“There is no ‘semi-career scientist’ position, though there are lots and lots of de facto semi-career scientists.”
In my insitution we have “associate research scientist” and “research scientist” positions that “career scientist” positions. These people exist under the umbrella of some tenure-track or tenured faculty member, but they are supported in the submission of their own R01s. They vest into pensions and are in every respect full-time permanent employees of our institution. Their jobs are at the mercy of funding, but other than tenured faculty, whose aren’t?
As far as the usefulness of considering “independence” as a relevant characteristic of a PI in terms of funding an R01, I believe it is thought of in at least two ways.
First, if someone is not “independent”, then it mis perceived as indicating that they can probably still do their science even if they don’t get the grant. A truly independent asst. prof can’t do their science if they don’t get the grant.
Second, someone who has demonstrated “independence” is thought of as having demonstrated the ability to be creative and get shit done without relying on someone else for ideas and/or motivation.
You know, one thing that I think gets short shrift in all of this discussion of peer review is the benefit to the scientific enterprise of the review process that is completely independent of the funding decisions that it is the basis for. Reviewing lots of grants and exposing oneself to all of these raw, relatively untested scientific ideas is an intellectual benefit to the reviewer. So is discussing all of this raw science with other bright people.
LikeLike
September 26, 2007 at 4:57 am
No way should NIH money for non-independent scientists come out of the same pool as money for independent scientists. If the project proposed is really worth doing, the independent scientist who’s really in charge can do it or fund it — that’s their job. If the non-independent scientist really has what it takes, let them prove it by getting an independent position. If the institution for which they work really believes in them, let that institution demonstrate that by giving them an independent position. R01’s from non-independent people are a scam: an attempt by institutions to double-dip into the gravy pot. Everyone knows this, and that’s why lack of independence is, correctly, a very severe critique.
LikeLike
September 26, 2007 at 5:20 am
“Reviewing lots of grants and exposing oneself to all of these raw, relatively untested scientific ideas is an intellectual benefit to the reviewer. So is discussing all of this raw science with other bright people.”
Careful, some might read this as you having serious COI problems….:-)
But yes, study section can function in many ways like a scientific retreat- the “Speed Dating” version of it anyway.
LikeLike
September 26, 2007 at 5:27 am
my dear whimple, I understand the perspective. the fact that I have a reflexive bent this way myself was the reason for the post. what I was trying to ask is why do you believe this?
not all work “really worth doing” does get funded, btw.
but to get down to specifics, why does the scientist who “has what it takes” have to “prove it”, how does a demonstration of independence help us predict or judge the science? is a brand new asst prof moving halfway across the country to set up a new lab whilst gearing up the teaching preps really going to be more productive than the same person who remained in the womb of the prior giant lab? remember, we’re talking 5yr funding interval max…or are we?
LikeLike
September 26, 2007 at 7:19 am
Why does a scientist have to prove they have what it takes? You’re kidding right? Grants are assessed on who is going to do the science and where the science is going to be done, in addition to what the science actually is. Independence is required because without independence, there is no authority to truly commit to the project.
A separate consideration is that as one of the few (only?) renewable, long-term, individual NIH awards, the R01 functions in a career development capacity. Awarding an R01 is a statement that the awardee is for real and that long-term productivity (and financial support for that productivity) is predicted. In short, an R01 implies a long-term relationship between the NIH and the awardee. There is more involved than a simple five-year cash-for-science contract.
LikeLike
September 26, 2007 at 8:00 pm
“Independence is required because without independence, there is no authority to truly commit to the project.”
ok. so the “authority to commit to the project” is the criterion. or “a” criterion. your assertion is incorrect, of course, because in many scenarios the BigBoss wants the junior person to ‘truly commit’ to the project. take it off his/her hands so to speak. then we get to the question of whether the science will be truly the child of the non-independent person or will be (unduly) influenced by the BigBoss. we can ask why this matters.
we can next ask if any apparently independent asst prof who has a bunch of senior colleagues in collaborations and sitting tenure judgment is likely to be unduly influenced. also why it is a BigDeal in review that a junior person have a bunch of appropriately senior people as “collaborators” on their projects. this starts bleeding into the question of how to assess “independence”, of course. sure, the mere fact of being hired halfway across the country is a good clue. but it is by no means a perfect predictor.
“career development capacity”: It isn’t supposed to function this way. The NIH system is supposed to be project-based, not program based. “cash for science” as you put it. Of course in reality it is a mixture and quite unevenly (and stupidly) applied if I do say so. A debate of value for its own sake but a bit of a red herring here. There is no reason the NIH cannot have a “long-term relationship” with a non-independent scientist should one come to the conclusion that there is nothing wrong with non-independence.
LikeLike
September 27, 2007 at 2:53 am
“should one come to the conclusion that there is nothing wrong with non-independence.”
I think the idea is that the more independent scientists you have in the system, the more creative and potentially groundbreaking the science is in the aggregate. At least in theory, science is not like ditch-digging, where you can dig the most ditches if you have the bare minimum number of decision-makers to keep control of the maximum number of dudes with shovels. The theory is that science is like art or literature–it moves forward most rapidly if you have the greatest possible number of people each thinking totally for themselves and given the opportunity to realize their vision without any top-down control.
An interesting point of comparison is the Japanese system, which has relatively few independent scientists and a huge number of dudes with shovels. Whether it really is more effective or not, the US system is explicitly designed to allow for the greatest possible number of independent scientists.
Coincidentally, I have just been helping am associate research scientist who is in the lab of a BigCheezDoodle with his own R01 application. He is in quite a pickle: The study section has already made it clear that they really need to see evidence of independence: promotion to research scientist from associate research scientist, as well as institutional committment of his own space. At the institutional level, he is being told that they will neither promote him nor commit space to him unless and until the R01 is funded. I feel bad for him.
LikeLike
September 27, 2007 at 9:12 am
He is in quite a pickle…
A rather common pickle, in my experience. The ad-hoc solution hereabouts seems to be a letter from a department head to the effect that, should the RO1 be funded, a promotion and lab space will be forthcoming. I’ve never seen it work, presumably because the study sections don’t believe a word of it. Perhaps it would be better to fund the grant and follow up, with heavy duty repercussions for the department if they don’t cough up the promotion and space.
LikeLike
September 27, 2007 at 5:02 pm
A rather common pickle as Bill says. I have seen it “work” and rather repeatedly. This is by no means to say that everyone can pull it off, just that I have seen first grants get funded under similar scenarios.
Physioprof: Who in the heck on the panel is sufficiently familiar with your institution to know that “research scientist” is a meaningful promotion from “associate research scientist”? I mean geez, these institutions have a veritable salad of terms meaning different things. Quick quiz, what’s the difference between “assistant research scientist” and “assistant project scientist”? What is an “adjunct assistant professor”…when there is no other primary appointment? Does “Assistant Professor of Neuroscience” mean something different from “Assistant Professor”? Which is “better”? Is “Instructor” independent? grant writing?
Okay, back to Dr. Pickle’s scenario. Is this the A1 of a first-ever submission or is this a repeated critique on multiple submissions? I’ve questioned people on why they make these comments about space and independence when they sort-of like the app and I kid you not they thought it was a help to the applicant in demanding stuff from the chair/dean! Supposedly this sometimes works. In your case, obviously not. The point being if this was what was going on, it is by all means worth Dr. Pickle resubmitting. I’d also suggest (again, assuming this is not a continued and repeated criticism) that you just have to give them some decent answer. Say it directly that space is provided after award and while this is not ideal this is a growing reality for transitioning scientists. Alternately, one might approach it a bit obliquely and try to indicate that institutional space is awarded to everyone based on “need” (glossing over the fact that a tenure track appt might be defined as “needing” certain minimal space, heh).
You also say you feel “bad” for Dr. Pickle. Presumably you feel that s/he is indeed an independent scientist worthy of funding even if it doesn’t appear to be the case by the usual “on paper” metrics?
LikeLike
September 28, 2007 at 7:29 am
“Who in the heck on the panel is sufficiently familiar with your institution to know that ‘research scientist’ is a meaningful promotion from ‘associate research scientist’?”
Beats me. This is what Dr. Pickle told me.
“Is this the A1 of a first-ever submission or is this a repeated critique on multiple submissions?”
Neither. It is an R01 first submission, but it is based on an R21 submission that was scored, but cannot be resubmitted because the IC ended their R21 program.
“You also say you feel ‘bad’ for Dr. Pickle. Presumably you feel that s/he is indeed an independent scientist worthy of funding even if it doesn’t appear to be the case by the usual ‘on paper’ metrics?”
Non sequitur. I don’t have any opinion on whether Dr. Pickle “is indeed an independent scientist worthy of funding”.
I feel bad for him because he seems like a nice guy who is trying to get funded and there are structural obstacles in his way. And even if I thought he and his science totally sucked, I might still feel bad for him.
LikeLike
October 3, 2007 at 8:46 am
“Neither. It is an R01 first submission, but it is based on an R21 submission that was scored, but cannot be resubmitted because the IC ended their R21 program.”
OK, so if I have this right, Dr. Pickle has a sum total of one Summary Statement for an initial submission that the reviewers had every reason to expect could be revised. Pretty standard scenario. And a pretty standard response, I might note. This is something I go all soapbox on with the Dr. Pickles I run across. The best thing you can do IMO is try to communicate the process to Dr. Pickle. That until you’ve put in a revision you don’t really know where you stand. And it may take several revised grants to really know what the study section will “insist” on. Second, I try to communicate some points made here in which I tried to point out that one often does not require a full, convincing response. Often reviewers are willing to credit what might appear to be a partial answer or non-answer to the applicant.
As a reviewer if I see the first submission, presented straight up, from a Dr. Pickle, well I’m going to comment on independence issues like as not. However, when an applicant comes back with a good response with respect to how s/he’s really been running this corner of the lab for years, detailing supervisory responsibilities, connect the pub dots to say what s/he thinks is their sphere of independent work, mentions the contribution of this award to their career development (If awarded I will be appointed..), etc, well, I can then work with that.
LikeLike
January 21, 2008 at 6:17 am
[…] of scientific “independence”. In a prior post I tried to grapple with the question of why we care about independence. Now I want to get into how the postdoc/senior research associate / (very) junior faculty […]
LikeLike
May 27, 2016 at 8:54 am
[…] crediting amongst members of a research team or super-group. This reminded me of some thoughts I originally posted Sept 25, […]
LikeLike
May 27, 2016 at 9:32 am
[…] I addressed Independence of a NIH Grant PI to some extent way back in 2007, reposted in […]
LikeLike