CVs and Faculty Searches

Young Female Scientist has another interesting post up, one concerning the use of CVs in the evaluation of applicants for academic PI positions. Her basic point is that an applicant’s publication record is, at best, an incomplete basis for predicting future success as a PI and, at worst, a misleading basis.

This is absolutely correct. But let’s unpack this conclusion, as I think it will be informative for trainees to get a better sense of what search committees think as they look at CVs and publication records.

YFS writes:

Your publication record is influenced by many factors. As a junior person (by that I mean, student or postdoc), you are not the person who chooses what gets written up, which figures are included, what journal it is submitted to. Even if you write the entire paper yourself, usually someone else has to weigh in before it goes out.

This is true, but incomplete. As I pointed out in yesterday’s post Manuscripts and Mentors, PIs are lazy. If you present a PI with a paper as a fait accompli, you will have a much greater influence on the content of the paper and the journal it is submitted to than otherwise.

So I would argue that, while yes, it is correct to say that a list of publications is a description of your contributions, it is not an accurate or complete picture of a person’s skills, accomplishments, or aptitude. It definitely does not describe whether you will be a good professor.

This is also true, but also incomplete. Publications are the currency of science. As a future PI, your success will be determined almost wholly by the publications that emanate from your lab. The best predictor of future success is past performance.

I know plenty of people with high impact papers who contributed only a fraction of the data to their own papers, AND cannot accurately describe, much less teach, the methods their labmates and collaborators used for the figures they contributed.

This is definitely true, and one of the goals of a faculty search committee is to determine the particular role played by the first author of a paper in a high-impact journal. Relevant sources of insight into this question come from letters of reference, from the job talk/chalk talk, and from conversations with the applicant.

However, I have a feeling that the reason this issue is key for search committees is different than YFS suspects. Search committees don’t care one iota whether the applicant “contributed only a fraction of the data” per se.

Future success as a PI does not rely on contributing data as the fruits of the PI’s own efforts at the bench. Future success as a PI relies on being able to guide the experimental efforts of others. Yes, this requires enough technical knowledge and understanding to design and troubleshoot experiments. But this knowledge doesn’t have to come from having actually performed such experiments as a post-doc. So long as a search committee is convinced that an applicant has sufficient expertise to write about, talk about, and guide others in the methodological requirements of the applicant’s proposed research program, it is completely irrelevant what experiments the applicant performed with her own hands as a post-doc.

I also know plenty of people whose work is top-notch, but it’s not in high impact journals because their senior, tenured PI doesn’t believe it matters where you publish, or more accurately, they know it doesn’t matter where they publish and they don’t care where you publish.

Or they just have no clue, or worse, no interest, in how to get a paper into a high impact journal. It’s quite a bit different than getting a paper into a ‘specialist’ journal.

Yes, these factors can influence where a post-doc publishes. They are external factors that are, looked upon narrowly, outside the control of the post-doc. In yesterday’s post Manuscripts and Mentors, I discussed some ways post-docs can influence these matters. And, as I also pointed out, at some point a post-doc has to decide whether their mentor is capable of effectively preparing them for, and launching them on, an independent career. If not, it is essential to find a mentor who is.

While there is often a lot of attention paid to whether a middle author actually contributed anything significant, little attention is paid to whether a first author deserved that slot.

There’s an assumption that goes along with first authorship that often is not deserved.

Search committees don’t care about whether first authorship is “deserved”. What they care about is whether it is a predictor of future success as a PI. There is one indubitable conclusion from the fact of a first authorship of a paper in a high-impact journal. This post-doc managed, by whatever means necessary and given the various factors in her environment many of which may have been outside her control, to have achieved exactly what is conventionally considered to be the best available predictor of their future success.

So, regardless of whether first authorship of a paper in a high-impact journal is “deserved”, achieving it shows that the post-doc is smart enough to know what is expected and resourceful enough to get it. This is, in large part, exactly what is needed for success as a PI: clear knowledge of what is expected and the resourcefulness to get it.

Brand-new PIs have even less control than post-docs of a variety of powerful factors in their environment that have a huge influence on their success: the talents of the trainees that join their nascent labs, the vagaries of grantsmanship and peer review, the provision of sufficient space and start-up resources by their departments, protection from excessive teaching and administrative duties, etc.

Nobody cares at all about any of that in assessing PIs for promotion and tenure. All they care about is how successful has the PI been in publishing and funding her science.

When you use something superficial, like a list of publications, to evaluate something bigger, like a faculty candidate, you’re bound to miss good candidates and you risk getting nothing but attractive-looking mediocrity.

In predicting whether someone who has never been a PI before is going to end up being a good one, there is no method of evaluation that can avoid false negatives and false positives. As a PI, the candidate is going to have to figure out how to effectively do a lot of things she never had to do as a post-doc and has received no training for: write large research grants, manage a lab full of personnel, really sell her research program to her field and subfield, garner invites to speak at meetings, garner invites to deliver departmental seminars at other institutions, etc.

But there is one thing that the PI is going to have to be able to effectively do that she has had the opportunity to do as a post-doc: play a lead role–first or senior author–in getting publications in high-impact journals, by whatever means necessary and given the various factors in her environment many of which may have been outside her control. And that is why success at this by a post-doc is considered a very good predictor of future success as a PI by search committees.