Dibs, PhysioProf.

Has anyone else heard this surgeon guy with the “professional coach” book he’s shilling?

“The coaching model [is] what you think of with athletes and singers, who have someone who coaches them all the way through their career, even if they’re one of the best in the world. But violinists and surgeons — at least in our theory of how we’re supposed to do it — we don’t. You go to medical school, you go to Juilliard, and you graduate. You get a degree, you get in your 10,000 hours of practice, and then some cream [is] supposed to rise to the top.

“But I was really struck by how different these models are and tried to understand it … I had a fascinating discussion with Itzhak Perlman, the great violinist, and I said, ‘Why don’t violinists have coaches, but singers do?’ And he said, ‘I don’t know, but I think it’s a mistake.’

“He said he had a coach his entire career. In his case, his wife …

Do we need coaches, Professoriat? At the lectern or at the bench, okay, Atul Gawande covered that. But what about the rest of it?

Reviewing manuscripts or grants. Writing them. Advising trainees. Collaborating.

Would we benefit from professional coaching?

The NIH Grant system already funds "people, not projects"

First PhysioProf baited me with this ridiculous commentary in Nature from John P. A. Ioannidis. Then Genomic Repairman took a cast and then PiT threw a big box of chumsicle into the water.

What could I do, DearReader? What could I possible do but chime in?

Before I get into this, a minor caveat to my remarks. I do like the fact that Ioannidis proposed a number of different fixes and suggested a pilot-study type of approach. I’ve made similar proposals for trying new ideas on this very blog. However. Proposals for change should be based on a clear and honest recognition of the present reality.

Ioannidis:

Leading thinkers and experimenters worthy of unconditional support could be identified through peer assessment of their work and credentials. Appraisals of project-based proposals already take a scientist’s merit into account, but they typically give less weight to it than to the project plan. Peer assessment does not work well for early-career scientists, who have a short track record. But for those more established in their field, a career trajectory offers a wealth of information. By contrast, an isolated project is only a snapshot.

This is a not uncommon suggestion. Our old friend Noonan was proposing this just a little while back.

It is based in large part upon this false belief that the NIH system actually works as formally designed. It does not. Or at least, not so far as it is supposed to be a strictly project-based (i.e., proposal-based) system of grant funding. Ioannidis deploys “typically” in a context that makes it clear that he equates this with “nearly exclusively”. This is total nonsense.

Admittedly, I had an experience in the very first months of my independent career that emphasized the person-based nature of the NIH funding system. In spades, with emphasis and I will admit quite shockingly. I mean, I had grasped the substantial table-slanting toward the established investigator already as a late-postdoc. But this was a whole ‘nother bit of whammo.

So in my case I have been aware of this for some time. I didn’t understand how it really worked, insidiously and no doubt unconsciously for the most part, until my first study section meeting. That’s all it took.

I have related my anecdotes regarding this before. The first time, I think my jaw was literally hanging open in disbelief. It happened in my presence many times after and I have heard similar anecdotes from friends on other study sections. I have no reason to think that “our” end of the NIH world is so startlingly unique. It is encapsulated by the review of a grant, typically a competing continuation but sometimes a new proposal, from a very well respected and established scientist. The reviewer(s) get to the end of a rambling critique in which it is made emphatically clear that the proposal is jam-packed with stuff, confusing, devoid of logical design, consideration of alternatives, hypothesis testing and generally full of StockCritique Bait common to that particular section. In short, a crappy proposal that would be kicked unceremoniously to the curb, were it anyone less legendary. Then the reviewer finishes by saying “But I know Professor Grey Fox’s lab is going to knock our socks off with great stuff because she has such a fantastic track record of unbelievable contributions. Post-discussion score, 1”.

The perception that the NIH system is in part a Person- or Program-based funding system is reinforced by experiences with the Programmatic pickup behavior. When times started getting grim I had personal conversations with POs in which they stated nakedly that keeping “their established investigators” in grant funding was a high priority…they were deaf to my observations that new investigators who were not able to launch were also a high priority and a better long term investment.

When you are are considering a competing review, on study section, you get to see the Summary Statement and scores for the original proposal. Sometimes, that original Summary Statement makes it clear that the proposal sucked, was lucky to get a marginal score in review and Program picked up that dog turd anyway! Then, to add insult to injury, sometimes the current competing app is just as bad as the original one, the critiques you make are similar to many criticisms made by that earlier group of reviewers…and you later find out that Program picked up this dog turd too!!!

GAaaahhhh!

Unless it is ME, of course. Dude, I totally have NIH proposals under review these days and I am no longer a NoobProf. I mean, what am I saying here? Nevermind that stuff. If you are a reviewer out there looking over my proposals and notice any StockCritique Bait, feel free to ignore any deficiencies in my proposal writing. In favor of the fact that you know me and I’ve managed to publish a paper or two. And for the most part been “productive” on my prior awards. C’mon now. I need some of that People-based funding love! (Oh, and for any of my POs that are reading, I was KIDDING about that dog turd stuff- friends?)

Returning the NoobProfs who are now screaming in dismay back to earth, let me note that despite this fact, the Project-based part of the NIH system does work too. Good proposals from less known investigators get funded all the time. Just go to RePORTER and look up some Noobs in your subfield. Some of them get funded. From my study section experiences it became clear that in many of these cases the Noob got funded by proposing something excellent, not just because she happened to have postdoc’ed with ol’ Horace Grizzler or Grey Fox. Great proposals from young (and not so young) investigators who are not tied cosily into the system get funded.

So when you hear a guy like Ioannidis implying that the system needs novel introduction of “person-based” grant funding, realize that we already have a balanced system. It is not exclusively person-based, nor is it exclusively proposal-based. There is a mixture.

Naturally, people’s assessment of the current “balance” is greatly influenced by their perceptions (and misperceptions*) of their own status within the system and their predictions about how to make it easier for them to get the grant money that they deserve. I have a pretty short fuse for such nakedly self-serving myopia.

__
*How do they know they would be the one judged to have a track record that merits some sort of BSD-based funding?

40% of grad students get PI jobs in Academia?

A new post up at RockTalk has an interesting figure.

Figure 1. UCSF graduate student career preferences. Courtesy of C.N. Furman (sic) et al., CBE Life Sciences Education, 2011.

One would not assume that 40% of graduate students are going on to PI level appointments in academia.

Oh WAAAAAIIIITTTTT.

This is the fraction of ~3rd year graduate students that say they want an academic, PI type career.

That makes more sense. Unfortunately the paper is one long tilt at the straw man that everyone wants/expects doctoral students to all end up in Academia with PI level jobs. Nobody really believes that anymore…..do they? The paper exhorts local institutions and national funding bodies (Hi, NIH) to get on board with the notion of branching career outcomes as all being “successful”. Ok, good enough.

but guys. Really. The question here is whether those that end up wanting a PI job can secure one.

Right?

All else is just handwaving. These authors would have done a better service by tracking down UCSF grads from 5, 10, 15 and 20 years ago and see what jobs they were actually able to obtain.
__
Fuhrmann et al, Improving graduate education to support a branching career pipeline: recommendations based on a survey of doctoral students in the basic biomedical sciences. CBE Life Sci Educ. 2011 Fall;10(3):239-49. [PDF, PubMed]

Expansion of Multi PI/PD policies for NIH Grants

The NIH policy change to allow multiple individuals to serve as the Principal Investigator (or Program Director for BigMech Awards) has been a whopping and unqualified success from my perspective. There are several situations that were addressed by this change, most importantly the extension of credit to a more-junior scientist. Since the PI reputation and track record was always such a big deal for the review of grants, it was often the case that the BiggestCheez was listed as the PI. Regardless of the amount of involvement of this individual. It was not rare to have very senior investigators allocating less than 10% effort as “PI” when another faculty member (perhaps even Associate Professor level) was allocating much more, say 15-30% effort, and clearly running the show for that particular project. For no official credit.

There was also a much harder to assess impact on collaborative science. In theory, there might be collaborative projects that did not get proposed precisely because neither partner was willing to do it without PI credit. Perhaps this resulted in the inefficiencies of two projects being funded when one would have sufficed? Perhaps it resulted in the collaboration never happening.

At any rate, I viewed the initial round of Multi-PI grants with interest. The first round or two featured a lot of discussion. Of the Leadership Plan. Of the need for a Multi-PI versus single PI application for that particular project. After that? Barely a peep. Nobody seemed to give it much thought at all. I’ve seen relatively recent summary statments where the fact that it is a MultiPI application is barely even discussed by the reviewers!

From my point of view this has become a non-issue for grant review and my advice, accordingly, is for people to go for it* if they think it fits their plans. I am curious as to whether my Readers have any experiences on either side of the review process to share?

The NIH, perhaps reacting to the reality that peer review panels don’t give a hoot about Multi-PI leadership plans and rationales, has decided to let local institutions alter the Single/Multi designation of a project without competitive review. I will remind you that they already permit changes in the research team, including the PI, at the whim of the applicant institution (to whom the grant is actually awarded, of course). So this is not a major new step. Just an…..alignment of policies.

The recent Notice (NOT-OD-11-118) describes how the NIH will permit additional flexibility in the use of the Multi-PI/PD option.

After several years of experience with the multiple-PD/PI model, NIH has determined that there are legitimate circumstances under which it would be in the best interest of an active project to change either from a multiple-PD/PI model to a single-PD/PI model, or from a single-PD/PI model to a multiple-PD/PI model, and that peer review of the new leadership team and Leadership Plan may not be essential in these cases… a request to change an active award from a single-PD/PI to a multiple-PD/PI model, or from a multiple-PD/PI model to a single-PD/PI model, must be made by the grantee organization and should be based on the scientific needs of the project. Justifications based on administrative convenience will not be considered. If the arrangements proposed by the grantee, including the qualifications of any proposed replacement or addition, are not acceptable to the NIH awarding IC the grant may be suspended or terminated.

and since you will be wondering about the newbs…

A New Investigator who is added as a PD/PI on a substantial NIH independent research award after initial peer review will not lose their New Investigator status.

This latter makes for some very interesting grant strategy indeed. A noob could, in theory, be writing grants in collaboration with a more-senior PI to be submitted with that person as solo PI. With the understanding that if it gets funded, they will wait a year or two and then slip JuniorMint (who actually wrote the thing) into the Multi-PI slot. Then the Noob could be busily submitting her own ESI/NI qualified grants, all the while enjoying de facto major grant funding for which she will eventually get at least partial PI credit.

Hmmm…..

__
*I don’t think this is a good idea for Noob Faculty before they’ve landed their first major award, however.

PA! Huh. What's it good for? ….absolutely nothin! Say it again, PA!

A query to the blog is a very typical reaction of those new (and not so new) to the NIH Grant game. As you will see in my answer. First, the question:

I’m now confused about this whole Program Announcement thing. The PO said that my application would be judged normally, just as part of whatever else the study section was reviewing, and that there was no special money set aside for the PA. If that’s the case, what’s the point of the PA in the first place? I had been under the impression it would be judged with other grants responding to the PA, but apparently that’s not true.

Help?

It is the RFA that generally routes applications into a dedicated, special emphasis panel type study section for review. For those Funding Opportunity Announcements (FOA) there is indeed a set aside pool of money and generally just a single receipt date. The PAS (Program Announcement with Set Aside Funds) also has dedicated funding, generally for the first round of submissions, then it converts to a regular old Program Announcement (PA) type FOA. Applications submitted for a PAR (Program Announcement with special receipt, referral and/or review considerations) is reviewed by a special panel, generally within an IC.

Regular old PAs are open for 3 years and generally use the standard receipt dates. In the ICs of my greatest experience they tend to be renewed and thus may represent essentially permanent PAs for much of your grant writing life. As per the reader query, the applications are reviewed in standard, CSR study sections with the appropriate domains of coverage and expertise. Alongside those applications that use the generic, mechanism based FOA. I would argue that you would only use the latter if you had to. Again, in the ICs of my greatest experience the PAs can be incredibly broad. Take “PA-10-268 Neuroscience Research on Drug Abuse (R01)” as an example. If your IC of interest has such broad topic PAs…you might as well use them.

Now as the reader question intimates, there is no overtly special benefit to your chances of getting funded. And there may be no benefit at all. Hard to tell. Because of course this sort of business only matters* when Program is considering the grey zone pickup funding. Is there a slant or a formula for how many approximately equivalently scored grants they will select under one of their PAs versus the generic parent FOA? I would suspect so, else why have such things? But I can’t say for sure. Maybe it is just make work for Program staff….to lay out their priorities. Or maybe it is a defensive excuse for those rare cases when they decide to stiff a grant that came in under the payline “Sorry PI Squirrel, it didn’t fit any of our Programmatic Interests…don’t you read the PAs?”.

The bottom line here for those new to the system is not to get all that excited when language in a PA seems directed at your research program. It isn’t *that* good of a bennie. But you might as well have some idea what is in the PAs and respond to them when you can. Because you just never know when it might help.
__
*assuming you have a modicum of sense and are not submitting stuff that is clearly not going to be of interest under the generic R01 parent FOA.

Muffy would never hurt a fly

[h/t: jekka]

Innocence

As you are probably aware there was a lot of hoopla from the lefty libby dirty hippies in the US and kibitzing OldEuro types on social media because the State of Georgia killed this guy. The reasons are pretty well captured in the accompanying article

Davis has repeatedly said he did not kill MacPhail, and seven out of nine witnesses who gave evidence at his trial in 1991 have recanted or changed their testimony.

No murder weapon was ever found, no DNA evidence or fingerprints tie him to the crime, and other witnesses have since said the murder was committed by another man — a state’s witness who testified against him.

This is not hard to grasp.

I have donated to the Innocence Project because I believe in this part of their mission statement.

The Innocence Project’s groundbreaking use of DNA technology to free innocent people has provided irrefutable proof that wrongful convictions are not isolated or rare events but instead arise from systemic defects. Now an independent nonprofit organization closely affiliated with Cardozo School of Law at Yeshiva University, the Innocence Project’s mission is nothing less than to free the staggering numbers of innocent people who remain incarcerated and to bring substantive reform to the system responsible for their unjust imprisonment.

I did not do so because I oppose capital punishment. As it happens, another unhappy soul was also executed recently, this time in the State of Texas.

Texas executed Lawrence Brewer, a white supremacist who was unquestionably guilty of the gruesome dragging death slaying of a black man in 1998.

I intentionally linked to the leftie-libby DFH argument that these are morally and ethically the same events because I disagree. here’s his crux:

The death of James Byrd Jr. — the black man who was tied to the back of a pickup truck in Jasper, Texas and dragged to his death — is shocking to recall, almost 15 years later. His murder is almost unimaginably cruel; it is impossible to read the details without being overcome with anger and revulsion. Yet this is what James Byrd’s sister had to say on the eve of Lawrence Brewer’s execution: “If I saw him face to face, I’d tell him I forgive him for what he did. Otherwise I’d be like him.”

I pay exactly as much attention to victims’ pleas for mercy as I do to their pleas for vengeance. The reason we have a rule of law in the first place is that justice and punishment have to come from a reasonably detached (blind lady justice?), societal point of view. Remember Dukakis and his famous flail on the question of what he would do if his wife were raped? I think Kerry managed to ass that one up too. The real answer Dems should espouse is my answer.

“I’d want to go to work on the homes here with a pair of pliers and a blowtorch. Are you fucking kidding? Anyone would. And given half a chance I damn well would. But there is no place for that sort of gutter, BronzeAge revenge-of-the-powerful jurisprudence in a just society. And THAT is why I support the rule of law.”

But in a democratic society we also meander towards approximations via what is, at root, barely managed democracy. The will of the people, so to speak. And the will of this person is that we, as a society and after due process, execute a guy like Lawrence Brewer. And this asshole too. People like this. maybe this gang of assholes.

But I also think our crime solving and crime convicting systems suck and are tremendously error prone. And have incredibly naked and thoroughly established racial and socio-economic biases.

So I donate to a project that wants to improve that. Even if they do, at root, have goals that are at odds with my support of capital punishment as a valid societal option.

Striking a country-specific tone in your response to grant criticism?

Cath raised an interesting point:

There have been some unbloggably hilarious emails pinging around between the various PIs, including much discussion about country- and funding agency-specific conventions regarding the tone of such responses (a PI who trained overseas feels that Canadians are far too polite and passive in the face of bad reviews, and may well be correct).

This was partially in response to my post and, presumably, my advice not to be combative in your response to criticism of your NIH grant.

I am now imagining that the British funding agencies are more tolerant of a Prime Minister’s questions type of response to criticism.

“Would the study section agree, Madam SRO, that the previous review was riddled with errors of FACT? and furthermore that the assigned reviewers were incompetent by reason of poor preparation? and that the bias inherent in the cozy relationship of Reviewer #3 with my primary scientific competitor renders the prior set of critiques invalid?”

[hear, hear…mutter, mutter, stomp, stomp booo!]

Don't tense up

If you’ve been going through a run of disappointing grant reviews punctuated by nasty Third Reviewer comments, you tend to tense up.

Your next proposals are stiff…and jam packed with what is supposed to be ammunition to ward off the criticisms you’ve been receiving lately. Excessive citation of the lit to defend your hypotheses…and buffer concentrations. Review paper level exposition of your logical chain. Kitchen sink of preliminary data. Exhaustive detail of your alternate approaches.

The trouble is, then your grant is wall to wall text and nearly unreadable.

Also, all that nitpicky stuff? Sometimes it is just post hoc justification by reviewers who don’t like the whole thing for reasons only tangentially related to the nits they are picking.

So your defensive crouch isn’t actually helping. If you hook the reviewer hard with your big picture stuff they will often put up with a lot of seeming StockCritique bait.

Ponder

My grant writing episodes are always punctuated by days in which nothing gets accomplished and days of nearly obscene productivity and progress.

It is never clear to me why I cannot have all of the latter.

Blogrolling: zwitterionique

Mostly because these cracked me up:

Happy New Year:

A while ago I decided that I wasn’t going to let my status of post-doc keep me from amassing a dark army of minions to do my scientific bidding, so I started advising UROP students.

Training Environment:

Some people are jealous, some wonder aloud if our PI realizes that I’m running my own lab out of my bay and accuse me of taking on PI-like traits (not knowing where things are on my own bench or not remembering whether I’ve told a student to do something or clearly remembering a result from eight months ago, but not what they showed me yesterday or sending cryptic experimental ideas at odd hours of the night).

UROP Students are not Stupid:

They are inexperienced – and you are providing training and experience – but they are not dumb and will not stay to assist you in taking over the world if they don’t see what they are going to get out of the deal. Even though the benefits of working for me are blindingly obvious, I make sure that I remind my minions often of how their servitude is beneficial to them.

Good and bad study section sleuthing

On the one hand I’m delighted. Someone at writedit’s complains that s/he got a grant reviewed in a section that doesn’t get many grants funded at a given IC. I’m happy because it shows that this applicant is thinking strategically about the appropriate study section.

I have concerns, though. The rest of the comment seems to be blaming the IC for not being interested in the topic focus of the study section.

Hold on.

Without knowledge of the number of apps with assignment to the IC that are being reviewed in a given section, we know little. Maybe there were only five apps and three of them got funded. Maybe that other study section passed along six funded apps….but is nearly captive to the IC and reviewed 70 applications assigned to them. Better grant numbers but worse *odds* for the applicant.

I just looked at new grants for two certain ICs that arrived there through three roughly similar study sections. Two of the sections had reviewed the same number of recently funded grants for one IC1, the third was 0. Considering the other IC2, the latter reviewed about the same number of funded apps as the other two sections had funded at IC1. One of those batted zero and one sent perhaps a sixth as many to this second IC2.

So. We know we have two relatively captive sections that hand out fundable scores to IC1 and IC2 respectively. And we have a third section which reviews for both and hands out fundable scores for both.

But this is slim evidence….because of the base rate. Now I happen to know that the assignment of apps to two of the sections is also highly IC dependent….but not exclusively so. The remaining section gets mixed application assignment to IC1, IC2 and even an IC3 (substantial) and ICs4,5,6 (a handful each). (This is a very general and longitudinal/historical knowledge, btw.)

So if the mixed-assignment and the nearly-captive sections are getting the same number of apps funded at IC1…it is the *mixed* section that looks like the better bet to me. Because I assume they have fewer IC1 apps on their docket.

Let’s look at this another, bigoted way. Suppose one IC was legit, perhaps NIMH, and one was NCCAM. Would you rather your NIMH app was up against 89 other NIMH apps or up against 44 NIMH apps and 45 NCCAM apps?

OTOH what if yours was the NCCAM app? Would you rather be in a section that was practically guaranteed to hand out fundable scores to *some* applications for that IC? Or in one that could, in theory, blank that IC entirely if the apps were all worse than the top ones for a different IC?

To get back to the original comment, the point here is that you need a lot more information before you conclude a given study section is a deadend for your favorite IC. Also to realize that it may not reflect IC disinterest in the topic domain of the section that you favor.

Overselling the candidate on a letter of recommendation

brooksphd is pondering a letter of recommendation

“X just applied and she listed you as a reference!”
…
But this feels nice! And scary – is there an added layer of responsibility on both sides of this equation now?

There is, and I observed that one should avoid overselling the candidate in making one’s comments. To this brooksphd replied:

that’s the issue I’m thinking about. Did I, could I, would I maybe oversell (or undersell) someone? Really, would it be bad to now ‘oversell’ someone? to really emphasize their fit because you can write a better letter. Is it common practice? Same as under selling someone is an easy, “I certainly consider this candidate above average. Hir fit in your lab is good. S/he reads the literature and makes solutions at the correct concentration accurately…”

Now, I recognize it is common practice to oversell and I seek ways to include a lot of confidence in the letters that I write for people. I put the best possible spin on my estimation of their talents and I may occasionally neglect to mention the odd deficit that I have observed.

But you have to keep it within reason.

I’ve had at least one experience in the past where I took someone into the lab at least partially on the strength of a recommendation letter…and this turned out to be an unreasonable oversell.

I will remind you that this is in full recognition of the type of excessive enthusiasm that we mentor types often think we need to include in the letter. Also with what I happen to think is a reasonable sympathy for the exigencies of life that can cause people’s work to be somewhat below the stellar, even for extended intervals of time.

This particular trainee sucked.

And it wasn’t just me, either. We’re talking all around failure to perform in the context of multiple obligations of this particular training dealio. It happens, and this is not the main point.

The main point is the original letter writer who testified to the skills of this particular individual in a scientific/laboratory context. There is no way in hell the letter could have been an accurate reflection. No way this person performed well in the past…or even performed at average. No way.

So my opinion of this letter writer is now and forever somewhere less than dirt. For certain sure I would never trust any other recommendations that this person might make.

I learned a lesson, my friends, a very powerful one.

You need to keep your recommendations within bounds. Do NOT ever give a glowing recommendation for someone if you know that they are going to turn out to perform significantly below average.

Because if you get burned, that mud comes back on you.

Authorship Order Rules: The Departed are Demoted

This comes up not infrequently in laboratories. Suppose one person, a trainee or postdoc, leaves the lab with his or her manuscript not completed*. Sure, this dearly departed individual may have started the project and/or done the bulk of the work on it.

But still, it isn’t a manuscript.

And it therefore isn’t going to be a paper, ever, until someone else steps up and does the work. Finishes the draft at the very least. Polishes off the figures. Submits the damn thing. Fields the original criticisms. Marshals the response to review. Creates the revision.

If one other remaining/subsequent person in the laboratory does all this, the dearly departed loses the first author slot. Arguments about the scientific importance of the original idea or the key data pale at this point.

Because if it isn’t published it didn’t happen.

There is an important practical concern for mentors and you will want to think very closely about this. It opens the door for any subsequent trainee to leave unfinished (as in unsubmitted) projects behind and then later insist that they have the right to be first author when someone else finishes it up. The motivational impact on your trainees’ behavior is somewhere damn close to disastrous.

__
*Yes, there will be some wiggle here about “Oh, I submitted a complete draft to the PI and all it needed was a little editing” when it wasn’t even close to being submittable.

Obama digs NCATS

Despite Congressional skepticism for Collins’ plan to scrap NCRR in favor of a new translational Center:

Today, for example, my administration is announcing a new center that will help companies reduce the time and cost of developing lifesaving drugs. When scientists and researchers at the National Institutes of Health discover a new cure or breakthrough, we’re going to make it easier for startup companies to sell those products to the people who need them. We got more than 100 universities and companies to agree that they’ll work together to bring more inventions to market as fast as possible. And we’re also developing a strategy to create jobs in biotechnology, which has tremendous promise for health, clean energy and the environment.

I think that means this ship has sailed beyond the point of recalling, folks. Get used to it.