CVs and Faculty Searches

October 31, 2007

Young Female Scientist has another interesting post up, one concerning the use of CVs in the evaluation of applicants for academic PI positions. Her basic point is that an applicant’s publication record is, at best, an incomplete basis for predicting future success as a PI and, at worst, a misleading basis.

This is absolutely correct. But let’s unpack this conclusion, as I think it will be informative for trainees to get a better sense of what search committees think as they look at CVs and publication records.

YFS writes:

Your publication record is influenced by many factors. As a junior person (by that I mean, student or postdoc), you are not the person who chooses what gets written up, which figures are included, what journal it is submitted to. Even if you write the entire paper yourself, usually someone else has to weigh in before it goes out.

This is true, but incomplete. As I pointed out in yesterday’s post Manuscripts and Mentors, PIs are lazy. If you present a PI with a paper as a fait accompli, you will have a much greater influence on the content of the paper and the journal it is submitted to than otherwise.

So I would argue that, while yes, it is correct to say that a list of publications is a description of your contributions, it is not an accurate or complete picture of a person’s skills, accomplishments, or aptitude. It definitely does not describe whether you will be a good professor.

This is also true, but also incomplete. Publications are the currency of science. As a future PI, your success will be determined almost wholly by the publications that emanate from your lab. The best predictor of future success is past performance.

I know plenty of people with high impact papers who contributed only a fraction of the data to their own papers, AND cannot accurately describe, much less teach, the methods their labmates and collaborators used for the figures they contributed.

This is definitely true, and one of the goals of a faculty search committee is to determine the particular role played by the first author of a paper in a high-impact journal. Relevant sources of insight into this question come from letters of reference, from the job talk/chalk talk, and from conversations with the applicant.

However, I have a feeling that the reason this issue is key for search committees is different than YFS suspects. Search committees don’t care one iota whether the applicant “contributed only a fraction of the data” per se.

Future success as a PI does not rely on contributing data as the fruits of the PI’s own efforts at the bench. Future success as a PI relies on being able to guide the experimental efforts of others. Yes, this requires enough technical knowledge and understanding to design and troubleshoot experiments. But this knowledge doesn’t have to come from having actually performed such experiments as a post-doc. So long as a search committee is convinced that an applicant has sufficient expertise to write about, talk about, and guide others in the methodological requirements of the applicant’s proposed research program, it is completely irrelevant what experiments the applicant performed with her own hands as a post-doc.

I also know plenty of people whose work is top-notch, but it’s not in high impact journals because their senior, tenured PI doesn’t believe it matters where you publish, or more accurately, they know it doesn’t matter where they publish and they don’t care where you publish.

Or they just have no clue, or worse, no interest, in how to get a paper into a high impact journal. It’s quite a bit different than getting a paper into a ‘specialist’ journal.

Yes, these factors can influence where a post-doc publishes. They are external factors that are, looked upon narrowly, outside the control of the post-doc. In yesterday’s post Manuscripts and Mentors, I discussed some ways post-docs can influence these matters. And, as I also pointed out, at some point a post-doc has to decide whether their mentor is capable of effectively preparing them for, and launching them on, an independent career. If not, it is essential to find a mentor who is.

While there is often a lot of attention paid to whether a middle author actually contributed anything significant, little attention is paid to whether a first author deserved that slot.

There’s an assumption that goes along with first authorship that often is not deserved.

Search committees don’t care about whether first authorship is “deserved”. What they care about is whether it is a predictor of future success as a PI. There is one indubitable conclusion from the fact of a first authorship of a paper in a high-impact journal. This post-doc managed, by whatever means necessary and given the various factors in her environment many of which may have been outside her control, to have achieved exactly what is conventionally considered to be the best available predictor of their future success.

So, regardless of whether first authorship of a paper in a high-impact journal is “deserved”, achieving it shows that the post-doc is smart enough to know what is expected and resourceful enough to get it. This is, in large part, exactly what is needed for success as a PI: clear knowledge of what is expected and the resourcefulness to get it.

Brand-new PIs have even less control than post-docs of a variety of powerful factors in their environment that have a huge influence on their success: the talents of the trainees that join their nascent labs, the vagaries of grantsmanship and peer review, the provision of sufficient space and start-up resources by their departments, protection from excessive teaching and administrative duties, etc.

Nobody cares at all about any of that in assessing PIs for promotion and tenure. All they care about is how successful has the PI been in publishing and funding her science.

When you use something superficial, like a list of publications, to evaluate something bigger, like a faculty candidate, you’re bound to miss good candidates and you risk getting nothing but attractive-looking mediocrity.

In predicting whether someone who has never been a PI before is going to end up being a good one, there is no method of evaluation that can avoid false negatives and false positives. As a PI, the candidate is going to have to figure out how to effectively do a lot of things she never had to do as a post-doc and has received no training for: write large research grants, manage a lab full of personnel, really sell her research program to her field and subfield, garner invites to speak at meetings, garner invites to deliver departmental seminars at other institutions, etc.

But there is one thing that the PI is going to have to be able to effectively do that she has had the opportunity to do as a post-doc: play a lead role–first or senior author–in getting publications in high-impact journals, by whatever means necessary and given the various factors in her environment many of which may have been outside her control. And that is why success at this by a post-doc is considered a very good predictor of future success as a PI by search committees.

About these ads

25 Responses to “CVs and Faculty Searches”

  1. JSinger Says:

    So, regardless of whether first authorship of a paper in a high-impact journal is “deserved”, achieving it shows that the post-doc is smart enough to know what is expected and resourceful enough to get it.

    The “resourcefulness” (frequently some combination of whining and intimidation) that pushes one person ahead of another in zero-sum squabbling over authorship doesn’t seem especially transferable to the next level. Manipulating editors that way is a lot harder, although it certainly happens.

    Like

  2. whimple Says:

    I don’t think whining and intimidation works on study sections either.

    Like

  3. bikemonkey Says:

    JSinger I beg to differ. Manipulating editors can be an art form and a PI I know(and this PI has an objective record of C/N/S pubs that is tops in the world) raises it to a savant-like ability. Now of course, some of this is circular in that threats to publish in a competing journal only work when said threat is highly credible. but versions of this work everywhere that an editor will actually read your letter or email or take your call.

    but anyway, I think PhysioProf was really referring to the “git ‘er done” general thing. This is really hard to communicate without coming across as totally cynical, as “dirtying” the process, as almost encouraging cheating/science fraud and the like. But I think most younger and not-so-younger PIs have this gut feeling that what sets them (and their good postdocs) apart from the rest is the ability to just plain make it happen. To pull that submittable manuscript together. To hit that grant deadline. etc. We’ve all seen that “smartest student in the grad program” who never goes on to greatness because of a lack in the “git ‘er done” phenotype…

    Like

  4. drugmonkey Says:

    “… some trainees (and even PIs) exhibit experimental Brownian motion. They take little experimental steps left-and-right, back-and-forth, but never move decisively towards answering the key question. This protects them from the fear of possibly finding out that they have been largely wasting their time, but gets them no closer to a possibly exciting result. The trainee begins to confuse an illusory productivity of performing lots of experimental procedures to the real productivity of answering lots of scientific question.”

    Whole subfields do this. The system of making funding dependent on “productivity” is part of the problem. The more grant dependent one’s work becomes, the less one is able to “fail”. And the more each individual PI construes workman like progress in churning out a paper per year per grant as the target to smooth additional funding, the more the whole field moves this way.

    It matters not if your subfield “target” is the C/N/S paper either if the goal is “what gets us into those journals” rather than “what is the best avenue to repeatable, extendable and relevant science”.

    Like

  5. physioprof Says:

    I am an excitement junkie. I only want to do experiments that have the possibility of generating truly new information, unexpected outcomes, or novel techniques.

    Like

  6. whimple Says:

    physioprof Says:
    I am an excitement junkie. I only want to do experiments that have the possibility of generating truly new information, unexpected outcomes, or novel techniques.

    Yeah, me too, but unfortunately this isn’t what the NIH is looking for, at least not in my case. Mastering the disconnect between the proposable and the should-be-doneable is a tricky skill.

    Like

  7. physioprof Says:

    “Yeah, me too, but unfortunately this isn’t what the NIH is looking for, at least not in my case.”

    This is exactly what the EUREKA program is all about. Did you submit this round? If not, I suspect that they will have another round next year.

    Like

  8. whimple Says:

    My Institutes don’t participate in EUREKA (only 4 Institutes are involved). Also I’m not happy with the long odds and low payout of EUREKA. Frankly, I’m suspicious that the EUREKA concept is actually bogus and that only the most senior (already well-funded) will get these awards. I would, of course, be happy to be proved wrong about this.

    Like

  9. tva Says:

    Physioprof or anybody else, how do you think a search committe will view candidates that might not have C/S/N papers, but have bridge grants like K99s, K01, or Burroughs Welcome grants (currently only available to MD holders). In many cases postdocs with these grants have a good publication record (i.e Neuron, Nature Neuroscience, PNAS etc) but not neccessarily C/S/N. Does a record of funding offset C/S/N papers?

    Like

  10. neurolover Says:

    The Burroughs Welcome career awards are _not_ limited to MDs.

    Like

  11. drugmonkey Says:

    If I’m not mistaken, however, the local institution gates who may apply in a given year from that location. It is not inconceivable that local policies may introduce some additional criteria (?)

    Like

  12. tva Says:

    Following the introduction of K99/R00 Burroughs have changed their policy.Here is the quote from the website.Yes, the institution does decide who gets nominated. So, does a search committee care if somebody has received funding as a postdoc, do they put any value to i?

    “The Career Awards for Medical Scientists (CAMS) is the result of the reformulation of the Career Awards in the Biomedical Sciences (CABS) program, which was instituted by the Burroughs Wellcome Fund in 1995 and ran through the 2006 award year. In response to the NIH’s K99/R00 Pathway to Independence awards, the BWF has shifted focus to address the on-going problem of increasing the number of physician scientists and keeping them in research.”

    Like

  13. physioprof Says:

    “So, does a search committee care if somebody has received funding as a postdoc, do they put any value to it?”

    Yes, it has some influence, as it demonstrates that the applicant can write a fundable proposal. It is not nearly as important as the publication record.

    Like

  14. drugmonkey Says:

    tva, you are hopefully thinking in terms of parts of the puzzle, rather than categorical milestones that are going to guarantee a job. Neuron and Nat Neurosci are quite sufficient for some jobs and depending on subfield. Possibly more than sufficient. Other places, such as according to Physioprof, CNS may be essential.

    Funding history can be more important in some places and less important in others. Fellowships are a kind of expected value but the BWF was always something special. I would hope the K99/R00 is also going to be viewed as an indicator of above-and-beyond.

    The point is to burnish your CV as best you can in all of the relevant dimensions. Despite what YFS seems to think, you will find that often the substance comes along with the credential, so to speak. The person who has bothered to seek out the F32 funding instead of just taking the easy Institutional TG slot IS an improved scientist. In my view just the process of putting in one (unfunded) app is likely to make you a better independent scientist, all things equal.

    Like

  15. physioprof Says:

    “I would hope the K99/R00 is also going to be viewed as an indicator of above-and-beyond.”

    It definitely is. I am on a search committee now, and one of our applicants has a weak publication record as a post-doc. Because this applicant has a K99/R00, there will at least be some discussion of their application. Otherwise, it wouldn’t have even been close.

    Like

  16. whimple Says:

    I’d say your best bet is to have TWO high-quality first author research papers. Not CNS (unless one just happens), but still in premium journals. That indicates that you probably didn’t fluke first authorship on a CNS paper and that you have the ability to follow-up /extend your own work. Sprinkle in some middle authorships and a review and you’re in business. A K99/R00 is great, but nothing is going to beat publication lines on a CV.

    Like

  17. physioprof Says:

    Whimple is correct. Candidates with nothing but a single CNS first-authorship require more “digging” to see what they’re all about, especially if their post-doc was >2 years. What I consider most predictive of future success is 4-5 years of post-doctoral training with at least one CNS first-authorship and also first-authorship of two or more papers in solid journals. In the neuroscience business, “solid” means anything at the level of Journal of Neuroscience and above. Reviews as a post-doc don’t mean diddly, as having these is solely a function of whether the post-doctoral mentor is asked to write a review and whether the mentor happens to gift that post-doc with the review.

    Like

  18. Neuro-conservative Says:

    I am surprised that a K99/R00 is not weighted more heavily (at least by PhysioProf). Aren’t institutions eager to recruit people who are already bringing in $$ (not huge bucks, but 3 years including overhead)? At my institution that would definitely get more weight than one more paper (even in a solid journal), especially since the candidate presumably had a respectable publication track record in order to get the K. It also shows a different (and critical) skill set relative to paper-writing. I suppose it is possible, however, that Physio’s other applicants are clearly world-class (>1 C/N/S + >1 J Neurosci/Neuron, for example) — but then they should have applied for K99′s!

    Like

  19. physioprof Says:

    “Aren’t institutions eager to recruit people who are already bringing in $$ (not huge bucks, but 3 years including overhead)?”

    Maybe some care about the money per se. Institutions like mine have a long-term view in which the money from a K99/R00 is itself completely irrelevantb. The only relevance of such an award for us is its predictive value for long-term productivity–papers and grants.

    Like

  20. Neuro-conservative Says:

    I suspect your institution has a much larger endowment than mine ;)

    Like

  21. drugmonkey Says:

    “In the neuroscience business, “solid” means anything at the level of Journal of Neuroscience and above.”

    With all due respect you are completely full of crap on this. I mention this for the reader mostly, not because I think I’ll change your views on this much.

    The Society for Neuroscience, perhaps you’ve heard of it, is having a meeting this week. If you happen to attend, take a stroll around the grounds outside of your usual “area”, read some poster abstracts, look at some session titles, you know, familiarize yourself with the tremendous breadth of what is “neuroscience”. Then, if you can’t figure it out from this, sit down with your laptop (free wifi is working great at the conv cent so far) and do some judicious PubMed-ing on topics that you see that are outside your domain.

    The point is this. There are many domains of neuroscience that are very well populated and active that are not going to support a good quality postdoc inevitably ending up with one first-author CNS and 2 J Neurosci and above papers in 4-5 years as a default expectation. Second point is that the promising scientists in these subfields ARE getting jobs, grants and launching careers. Not as much as I’d like to see, sure. But they are most assuredly getting “in the neuroscience business” by any objective measure. That is, any measure other than “neuroscience is what I say it is”.

    I should also clarify a related point for the reader. By no means take my recognition of how C/N/S pubs are viewed by “most people” as an endorsement on my part that this represents a legitimate scientific measure of accomplishment. I most assuredly do not so endorse and in fact quite the contrary.

    Like

  22. physioprof Says:

    “With all due respect you are completely full of crap on this.”

    Well, I do agree that my statement was too broad, and based solely on my experiences in basic highly reductionistic cellular, molecular, and physiological neuroscience.

    “There are many domains of neuroscience that are very well populated and active that are not going to support a good quality postdoc inevitably ending up with one first-author CNS and 2 J Neurosci and above papers in 4-5 years as a default expectation.”

    Do we want to get specific with this? What are these domains? I’m not doubting you; I really want to know. This could spark a fascinating discussion worthy of a post on its own.

    And just to be clear, what I was saying was not that a good post-doc will inevitably end up with first-author CNS and JNeurosci papers in 4-5 years as a default expectation. I am aware that few good post-docs achieve this, and it requires a lot of luck. Nevertheless, these are the ones that are competitive for jobs in basic science departments at “high-prestige” universities. I am not saying that awesome science doesn’t go on that is published in more specialized journals and that awesome scientists don’t reside at non-”high-prestige” universities.

    I did fall in the trap of taking a very general tone, when what I am really referring to is a more specific context. So it’s good that you have called me on that. It is very important in these kind of discussions not only to be clear on context, but also to distinguish descriptive from prescriptive commentary. Some of this shit ain’t pretty, but it is real.

    Like

  23. drugmonkey Says:

    “Do we want to get specific with this? What are these domains? I’m not doubting you; I really want to know. This could spark a fascinating discussion worthy of a post on its own.”

    sure, and the full thing may have to wait for awhile. But since you ask, I can list a few. Almost anything epidemiological having to do with determining the scope and character of a given mental health condition. What could be more basic than telling us what we should be studying and with what degree of urgency? Basic cognitive/behavioral science, meaning studies in either humans or animals looking at behavior for its own sake and trying to interpret brain function. if it makes you any more comfortable that this is a useful area think how much we determined about trichromatic vision with psychophysics prior to understanding retinal cells. behavioral pharmacology- drug abuse, basic pharm/behavior or even therapy. despite all the aggressive talk about “rational drug design” of various stripes, this is to date bullshit. creating drugs is not the problem, determining what they do in an intact organism is the bottle neck. the list goes on…

    my point is not that research in these areas cannot be assisted by what is currently considered “hot” technical approaches. Just that the relative contribution to anything I think of as important is clearly on the side of some of these more established domains. And I say this as a very firm believer in the value of basic science.

    my point is also that there are no generalizable concepts of what “good science” is that rule in or out any particular types of approaches. “hotness”, rarity, technical difficulty, etc are just that, they do not convey anything about quality, efficiency, “rightness” or scientific truth.

    fortunately, our overall “system” of inquiry and support recognizes this which is why there are still scientists laboring away with less-than-sexy technical approaches that are funded and contributing to knowledge and public health problems alike. the system does not, however, recognize contributions in a way that relates to any of these general scientific goals and that pisses me off. Hence the response to your comment.

    I see no reason to sit back and let people claim, unexamined, the usual circular claims that C/N/S = “the best (only legitimate) possible science”.

    Like

  24. drugmonkey Says:

    “It is very important in these kind of discussions not only to be clear on context, but also to distinguish descriptive from prescriptive commentary. Some of this shit ain’t pretty, but it is real.”

    …and I am right there with you on this one. By all means I recommend trainees and developing scientists to recognize the priority of sensational and novel over quality, replicability and utility of a scientific approach. but if one wants to do it the hard way, all hope is not lost…

    Like


  25. [...] 15, 2008 In the context of tenure-track faculty job searches we’ve previously discussed CVs, the job talk, and the chalk talk. Now let’s talk about a sometimes underappreciated aspect [...]

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 40 other followers

%d bloggers like this: