Thought of the Day
August 29, 2013
What “best predicts” the success of a junior scientist is handing her a laboratory and R01 level funding.
The notion that past publication record predicts anything independently from these two factors is arrant nonsense.
DrugMonkey 
August 29, 2013 at 7:04 am
I suspect it predicts it a little. But probably an order of magnitude less important than the funding, yes.
LikeLike
August 29, 2013 at 7:12 am
I think what DM means is, given a lab and $1.2M, what can a junior scientist do? The outcome of that “experiment” would be the best indicator of how she would do career-wise.
LikeLike
August 29, 2013 at 7:27 am
A good department that wants to have successful junior faculty needs to do much more than give a lab and money. The real predictors of success are the things that happen after you fulfill basic needs like a room to work in, equipment and resources.
LikeLike
August 29, 2013 at 7:29 am
OF COURSE those with labs and funding will have better success than those without. Why did you bring it up? Did someone tell you that pubs predict success in the lab-less and money-less?
LikeLike
August 29, 2013 at 7:31 am
I meant this in the contrast with using pedigree and various publication counting measures to figure out who will “succeed” and who will obviously fail at Assistant Professor.
The one-way experiment and self-fulfilling nature of such beliefs reinforces confidence where it should not be placed.
Lots of people with stellar CVs end up cratering but those with shitty ones never participate in the experiment.
LikeLike
August 29, 2013 at 7:44 am
I think timing and opportunity have a lot to do with it. Given the right circumstances (hot research area, important discipline-spanning questions etc), enough cash and the right people working with them, most new Asst Profs could make things happen.
It also depends on what your definition of success is? Glamor pubs? More funding? Successful students?
LikeLike
August 29, 2013 at 7:46 am
Okay, so where’s the data?
LikeLike
August 29, 2013 at 8:04 am
It’ll come to you in a minute neuroecology
LikeLike
August 29, 2013 at 8:05 am
Physical health
Work ethic / willingness to commit
Passion for science
Ability to come up with new ideas
Resilience to failure
Department/university environment
Previous publication/funding record (insomuch as it influences likelihood of future publication/funding)
Writing ability (independent from previous advisors)
Creativity
Ability to synthesize the literature efficiently
Ability to manage time and distractions
Personal commitments/interests outside of work
Level of support from significant other and other family/friends
Competitors
Luck
Social skills
How much time they spend on blogs and twitter
Availability of funding in general
Level of integrity (whether willing to sacrifice integrity for more output)
Etc.
LikeLike
August 29, 2013 at 8:11 am
Savviness in “working the system”
Ability to give compelling talks (and thus attract attention to your lab)
Ability to collaborate effectively
Ability to work effectively with students and postdocs
Ability to attract students and postdocs who possess relevant skills
Openness to ideas from others
Recognizing your strengths (and that nobody excels at everything)
LikeLike
August 29, 2013 at 8:19 am
But wait a minute….I thought it was the consensus around here that the secret was using a particular referencing style….right?
LikeLike
August 29, 2013 at 8:19 am
Ability to prioritize projects.
Teaching load.
Ability to prepare teaching materials efficiently.
Ability to grade student assignments efficiently.
Ability to deal with ambiguity.
How you deal with conflicts.
LikeLike
August 29, 2013 at 8:32 am
Nice list, BioDataSci. Enough to scare off any wanna-be PI! đŸ™‚ But irrelevant if we’re trying to find a few concrete factors that we can use to decide who to hire for an open faculty slot or who to give that first big grant to. I have to disagree somewhat with DM, though.
While publication record isn’t an end-all, be-all, I think it is an important consideration. For example, I’ve been reviewing grants and have a certain R15 application in my pile. It’s from a young PI who trained in glamor labs but got very few publications, even fewer first author. The PI is now struggling, with no corresponding author publications in 6 years, even though a good number of undergrads, masters students and PhD students have been through the lab. Clearly the publication record wasn’t enough to get this person a job at a “top tier” research institution, and the lack of publications is becoming a self-fulfilling prophecy.
So, although having ELEVENTY!! CNS papers from your PhD work in a glamor lab doesn’t guarantee that you’ll be glamorous yourself, if your record does not reflect the environment you’ve been in, it’s a bad sign.
LikeLike
August 29, 2013 at 8:44 am
Good luck trying to find a few concrete factors that you can use! đŸ™‚ Next you’re going to tell me I have to come up with a short list of objective measures you can use to decide whom to marry! đŸ™‚
When it comes down to it, you’ll have some sense based on what you can measure (objectively or intuitively). But there are all sorts of influential factors that you can’t now and never will be able to measure and/or account for in aggregate.
LikeLike
August 29, 2013 at 9:08 am
The PI is now struggling, with no corresponding author publications in 6 years
That’s pretty bad. Especially if funded reasonably well.
LikeLike
August 29, 2013 at 9:37 am
Don’t be ridiculous, BDI, everyone knows that when choosing a life partner you look for someone who is A) good looking and B) wealthy. End of story.
LikeLike
August 29, 2013 at 11:05 am
Chris I’m confused as to where you disagree with me. Does this person have major funding or not? AREA suggests the local research resources and collaborations may not be the very best. Depending on what sort of AREA institution it is, the job might have a huge teaching load and minimal expectation to start a big publishing stream.
So it could very well be that, as I am arguing, the circumstances of the Asst Prof appointment/job itself are the most important factor.
LikeLike
August 29, 2013 at 11:27 am
This person does not have major funding. My suggestion is that underperformance during a PhD and postdoc in very successful labs seems to have been a reasonable predictor of underperformance as a PI, at least in this case. Clearly, this institution does not have the resources or support system available at a “top tier” place, but the PI has had many students (including MS and PhD students) in the lab for close to 6 years with adequate institutional support for the research they’re doing and still has no independent publication.
While having a fabulous publication record from the top labs doesn’t guarantee success (this is generally just an expected outcome of having been in a top lab), a poor publication record from top labs is a huge red flag and could be used as a predictor of future success.
I would never try to argue that this is ALWAYs the case – you could end up on the dead-end project and struggle in your PhD or postdoc. But I would hope it’s not often that this happens in BOTH the PhD and postdoc (if it does, you’ve clearly got some issues with picking research projects).
So, I agree with your statement in that a strong track record as a trainee doesn’t ensure future success, but suggest that a weak track record as a trainee could be a reasonable indicator that a career as an independent PI might not be the best choice, regardless of the amount of money that is thrown at the PI.
LikeLike
August 29, 2013 at 11:42 am
This person does not have major funding
Clearly, this institution does not have the resources or support system available at a “top tier” place
but the PI has had many students (including MS and PhD students) in the lab for close to 6 years
If this is not a “top tier” place, then perhaps the students are no good either. You can’t get blood from a stone pal.
LikeLike
August 29, 2013 at 11:59 am
I would like to see some data but anecdotally (at least from the 10-15 folks I know well) this may be true. I don’t think the data should be hard to collect though — perhaps look at the folks with say the Burroughs-Wellcome, Searle, Klingenstein, Whitehall, McKnight, NIH New Innovator awards (the R00 sample sizes may be small) etc. On the other hand, some of the Janelia junior fellows (and perhaps a couple of group leaders as well) are not faring so well.
But then again, what’s the measure of success? Glamour pubs? Society level pubs? Pubs, period?
LikeLike
August 29, 2013 at 12:16 pm
While having a fabulous publication record from the top labs doesn’t guarantee success (this is generally just an expected outcome of having been in a top lab), a poor publication record from top labs is a huge red flag and could be used as a predictor of future success.
This goes the other way as well. A fabulous publication record by a post-doc in a non-top lab is a very good predictor of success.
LikeLike
August 29, 2013 at 1:11 pm
Dave – I really don’t think I’m being too harsh here. I’m not expecting this person to have multiple publications per year or be publishing in top society journals. But if you’ve had multiple people defend Masters’ theses based on their work in your lab and if you have more than one senior graduate student who has been working in the lab for several years and you can’t cobble anything together into a little communication for a lower tier journal based on all of that work, you’re doing something wrong.
CPP – yes, that. Or if your work brought fame and glory to your previously unknown assistant professor PI. I know several people who did that and went on to be very successful.
LikeLike
August 29, 2013 at 1:19 pm
Chris’ story is interesting. I recently pointed out a new hire in Duke Chemistry who comes from two glamour labs but is without a single first-author publication save a review and an “in preparation”.
I was roundly criticized for being so focused on publication record and pedigree but i have to wonder, what was the basis of her hiring then? This supposedly amazing proposal or interview? All i could see was a lab full of students being told to write up their papers looking back and asking, ‘well, why don’t you write up yours?’.
LikeLike
August 29, 2013 at 3:07 pm
…and you can’t cobble anything together into a little communication for a lower tier journal based on all of that work, you’re doing something wrong
Yeh, I know. It is a bit odd.
I recently pointed out a new hire in Duke Chemistry who comes from two glamour labs but is without a single first-author publication save a review and an “in preparation”.
Great link. See you going at it there “Wolfie” LOL LOL. I swear I have heard people like Michael Eisen say that they regularly put post-docs into TT jobs without any papers, but I could be wrong. Others here would be able to confirm.
LikeLike
August 29, 2013 at 3:33 pm
No one gives a rat’s ass about “productivity” if it is not measured in dollars. BioDataSci’s list should be:
1) get more funding
2) there is no 2)
All else being equal, we’re going with someone who wrote a successful K99/R00.
LikeLike
August 29, 2013 at 4:12 pm
“1) get more funding
2) there is no 2)”
Making DM’s point
LikeLike
August 29, 2013 at 7:44 pm
Chris: Does this person have no papers or no papers as corresponding author? is it possible that he/she is letting the students be corresponding authors? Whether this is or not a good idea or kind behavior is another issue, of course.
BioDataSci: I like your list. Would add “citizenship coupled with job location”, which you may have included in “luck”. The vast majority of awards and special funding options are available only to people with certain combinations of citizenship and job location (American citizens in the US, European union citizens in Europe, Klingons in Qo’noS, etc.).
LikeLike
August 30, 2013 at 4:38 am
@BDS, I like the list, but a lot of the things on it overlap.
For example:
– Ability to come up with new ideas
– Creativity
– Passion for science
Same goes for these (all communication skills):
– Writing ability
– Ability to synthesize the literature efficiently
– Ability to give compelling talks
– Previous publication/funding record
And these (all people skills):
– Social skills
– Savviness in “working the system”
– Ability to work effectively with students and postdocs
– Ability to attract students and postdocs who possess relevant skills
– Openness to ideas from others
– Ability to collaborate effectively
Same again for (all work/life balance):
– Ability to manage time and distractions
– How much time they spend on blogs and twitter
– Level of support from significant other and other family/friends
– Personal commitments/interests outside of work
And (self identity):
– Work ethic / willingness to commit
– Level of integrity
– Resilience to failure
– Recognizing your strengths
And (external forces):
– Department/university environment
– Availability of funding in general
– Competitors
That leaves physical health and luck, which don’t fit into the above groups.
As such, the list of things you actually need to survive in this game, is actually quite short:
1) Do you have “the spark”?
2) Can you communicate?
3) Do you have people skills?
4) Do you have your work/life balance shit together?
5) Are you ethically/morally solid?
6) What’s it like out there?
7) Are you healthy and lucky?
and #8) Can you keep doing/being all this for another 3 decades or more?
Still a very scary list, even in reduced form!
LikeLike
August 30, 2013 at 7:29 am
Juan – the person has one publication from their time as a PI, it is a multi-PI collaborative paper, and this person is a middle author, which leads me to believe that their contribution was less that that of the other labs.
I don’t mean to harp on this one person, nor do I feel comfortable naming names as examples of people whose publication records are not what we might think they should be. I will note, however, that the postdoc advisor of the person Bad Wolf mentions is a good example of someone whose graduate work contributed significantly to making their assistant prof graduate PI famous and has since gone on to a strong career of their own. (Justin Du Bois. I’m OK naming names if we’re talking nice about the person.)
LikeLike
August 30, 2013 at 8:03 am
Neuropop- not exactly my point, no. My point is that most postdocs with half a scientific pulse will do good stuff with a lab and a chance. Fantastic eleventy people who never luck into a lab and some decent funding will not.
The lab and money is the critical factor. In the vast majority of cases, gaming out narrow differences on the CV is not the critical factor.
LikeLike
August 30, 2013 at 8:14 am
I’m surprized no one said lack of children.
LikeLike
August 30, 2013 at 8:42 am
But, surely previous publication record predicts funding success which in turn predicts overall success?
Or do your study sections not say “nice CV you got there but first show us you have a functioning lab and we might consider throwing some money your way”. Because mine does, and I’d love switching to yours if that’s the case.
LikeLike
August 30, 2013 at 9:13 am
If past success shouldn’t be used as an indicator of future success, how then are we supposed to choose from the 200 applicants? Random chance? A lottery?
I would argue that those without a good track record (pubs/money) are much less likely to be successful as an independent PI. Of course, good track record guarantees nothing, but at least it is a practical measure by which to choose.
I also agree with the notion that candidates from non-famous labs that had glamour mag publications should be at the top of the list.
LikeLike
August 30, 2013 at 10:28 am
previous publication record predicts funding success
In study sections that I am familiar with, pubs from the noob Asst Prof’s own lab and strong preliminary data are more important than a record of accomplishment as a postdoc.
those without a good track record (pubs/money) are much less likely to be successful as an independent PI.
And I would argue that the strongest prediction is to see what a noob Asst Prof pulls off with a lab and an R01 budget for a few years.
candidates from non-famous labs that had glamour mag publications should be at the top of the list.
What does glamour have to do with it?
LikeLike
August 30, 2013 at 1:59 pm
What does glamour have to do with it?
Good question. But the answer is increasingly….EVERYTHING!!!!
LikeLike
August 30, 2013 at 5:51 pm
DM: I agree with you that anyone with a scientific pulse with R01 level funding will do fine. My point was that there might be objective data lying around to validate this. Since startup packages vary widely, this cohort might be a good measure.
LikeLike
August 30, 2013 at 11:26 pm
DM this only shows how radically underfunded academia is in the USA: there are scads of people who would be doing good science if only they had the money.
LikeLike
September 1, 2013 at 8:59 am
I agree with you that anyone with a scientific pulse with R01 level funding will do fine.
This is demonstrably untrue. There are plenty of people who have apparently strong scientific pulses who get tenure-track faculty jobbes with nice start-up packages who flame out.
LikeLike
September 3, 2013 at 9:12 am
DM: I agree with you that anyone with a scientific pulse with R01 level funding will do fine.
Not quite the point I was trying to make. One of the big points I was trying to make was the self-fulfilling nature of any gatekeeping criteria. So if you think, say, only postdocs with Glamour pubs can be successful and only give them the jobs and R01s, it ignores the many people who would also have been successful without such accolades as a postdoc.
I was, to a lesser extent, suggesting that by the time we get to people who have successfully landed a job somewhere, fancy U or not, they are very highly selected for those who can succeed given the money. There will always be some who flame out as PP says, but I doubt the traditional measures of “past performance” predict that very well either.
I don’t happen to know that many who have flamed out but other-life-shit is a factor. Also I’ve seen dogged, unsuccessful pursuit of Glamour kill labs. this may be a subcategory of what I see as failure to achieve escape velocity, which includes ppl who do not seem to grasp the funding….urgency….that I underline on the blog. Then there are the people who decide this life really just isn’t for them after gaining a modest success level (i.e., some grant, some papers, looking good).
None of these factors are predictable from training or publication history. IMO.
LikeLike
September 3, 2013 at 11:53 am
[…] Food parasite probe hindered by budget deficiencies Velvet Ants Are Not Ant Mimics Assumptions, Agenda, and How the Conversation Really Goes Down (excellent) Overselling the microbiome award: VIB press release saying “Intestinal flora determines health of obese people” (sometimes blogging works!) Thought of the Day (also see this) […]
LikeLike