Dibs, PhysioProf.

September 30, 2011

Has anyone else heard this surgeon guy with the “professional coach” book he’s shilling?

“The coaching model [is] what you think of with athletes and singers, who have someone who coaches them all the way through their career, even if they’re one of the best in the world. But violinists and surgeons — at least in our theory of how we’re supposed to do it — we don’t. You go to medical school, you go to Juilliard, and you graduate. You get a degree, you get in your 10,000 hours of practice, and then some cream [is] supposed to rise to the top.

“But I was really struck by how different these models are and tried to understand it … I had a fascinating discussion with Itzhak Perlman, the great violinist, and I said, ‘Why don’t violinists have coaches, but singers do?’ And he said, ‘I don’t know, but I think it’s a mistake.’

“He said he had a coach his entire career. In his case, his wife …

Do we need coaches, Professoriat? At the lectern or at the bench, okay, Atul Gawande covered that. But what about the rest of it?

Reviewing manuscripts or grants. Writing them. Advising trainees. Collaborating.

Would we benefit from professional coaching?

First PhysioProf baited me with this ridiculous commentary in Nature from John P. A. Ioannidis. Then Genomic Repairman took a cast and then PiT threw a big box of chumsicle into the water.

What could I do, DearReader? What could I possible do but chime in?

Before I get into this, a minor caveat to my remarks. I do like the fact that Ioannidis proposed a number of different fixes and suggested a pilot-study type of approach. I’ve made similar proposals for trying new ideas on this very blog. However. Proposals for change should be based on a clear and honest recognition of the present reality.

Ioannidis:

Leading thinkers and experimenters worthy of unconditional support could be identified through peer assessment of their work and credentials. Appraisals of project-based proposals already take a scientist’s merit into account, but they typically give less weight to it than to the project plan. Peer assessment does not work well for early-career scientists, who have a short track record. But for those more established in their field, a career trajectory offers a wealth of information. By contrast, an isolated project is only a snapshot.

This is a not uncommon suggestion. Our old friend Noonan was proposing this just a little while back.

It is based in large part upon this false belief that the NIH system actually works as formally designed. It does not. Or at least, not so far as it is supposed to be a strictly project-based (i.e., proposal-based) system of grant funding. Ioannidis deploys “typically” in a context that makes it clear that he equates this with “nearly exclusively”. This is total nonsense.

Admittedly, I had an experience in the very first months of my independent career that emphasized the person-based nature of the NIH funding system. In spades, with emphasis and I will admit quite shockingly. I mean, I had grasped the substantial table-slanting toward the established investigator already as a late-postdoc. But this was a whole ‘nother bit of whammo.

So in my case I have been aware of this for some time. I didn’t understand how it really worked, insidiously and no doubt unconsciously for the most part, until my first study section meeting. That’s all it took.

I have related my anecdotes regarding this before. The first time, I think my jaw was literally hanging open in disbelief. It happened in my presence many times after and I have heard similar anecdotes from friends on other study sections. I have no reason to think that “our” end of the NIH world is so startlingly unique. It is encapsulated by the review of a grant, typically a competing continuation but sometimes a new proposal, from a very well respected and established scientist. The reviewer(s) get to the end of a rambling critique in which it is made emphatically clear that the proposal is jam-packed with stuff, confusing, devoid of logical design, consideration of alternatives, hypothesis testing and generally full of StockCritique Bait common to that particular section. In short, a crappy proposal that would be kicked unceremoniously to the curb, were it anyone less legendary. Then the reviewer finishes by saying “But I know Professor Grey Fox’s lab is going to knock our socks off with great stuff because she has such a fantastic track record of unbelievable contributions. Post-discussion score, 1”.

The perception that the NIH system is in part a Person- or Program-based funding system is reinforced by experiences with the Programmatic pickup behavior. When times started getting grim I had personal conversations with POs in which they stated nakedly that keeping “their established investigators” in grant funding was a high priority…they were deaf to my observations that new investigators who were not able to launch were also a high priority and a better long term investment.

When you are are considering a competing review, on study section, you get to see the Summary Statement and scores for the original proposal. Sometimes, that original Summary Statement makes it clear that the proposal sucked, was lucky to get a marginal score in review and Program picked up that dog turd anyway! Then, to add insult to injury, sometimes the current competing app is just as bad as the original one, the critiques you make are similar to many criticisms made by that earlier group of reviewers…and you later find out that Program picked up this dog turd too!!!

GAaaahhhh!

Unless it is ME, of course. Dude, I totally have NIH proposals under review these days and I am no longer a NoobProf. I mean, what am I saying here? Nevermind that stuff. If you are a reviewer out there looking over my proposals and notice any StockCritique Bait, feel free to ignore any deficiencies in my proposal writing. In favor of the fact that you know me and I’ve managed to publish a paper or two. And for the most part been “productive” on my prior awards. C’mon now. I need some of that People-based funding love! (Oh, and for any of my POs that are reading, I was KIDDING about that dog turd stuff- friends?)

Returning the NoobProfs who are now screaming in dismay back to earth, let me note that despite this fact, the Project-based part of the NIH system does work too. Good proposals from less known investigators get funded all the time. Just go to RePORTER and look up some Noobs in your subfield. Some of them get funded. From my study section experiences it became clear that in many of these cases the Noob got funded by proposing something excellent, not just because she happened to have postdoc’ed with ol’ Horace Grizzler or Grey Fox. Great proposals from young (and not so young) investigators who are not tied cosily into the system get funded.

So when you hear a guy like Ioannidis implying that the system needs novel introduction of “person-based” grant funding, realize that we already have a balanced system. It is not exclusively person-based, nor is it exclusively proposal-based. There is a mixture.

Naturally, people’s assessment of the current “balance” is greatly influenced by their perceptions (and misperceptions*) of their own status within the system and their predictions about how to make it easier for them to get the grant money that they deserve. I have a pretty short fuse for such nakedly self-serving myopia.

__
*How do they know they would be the one judged to have a track record that merits some sort of BSD-based funding?

A new post up at RockTalk has an interesting figure.

Figure 1. UCSF graduate student career preferences. Courtesy of C.N. Furman (sic) et al., CBE Life Sciences Education, 2011.

One would not assume that 40% of graduate students are going on to PI level appointments in academia.

Oh WAAAAAIIIITTTTT.

This is the fraction of ~3rd year graduate students that say they want an academic, PI type career.

That makes more sense. Unfortunately the paper is one long tilt at the straw man that everyone wants/expects doctoral students to all end up in Academia with PI level jobs. Nobody really believes that anymore…..do they? The paper exhorts local institutions and national funding bodies (Hi, NIH) to get on board with the notion of branching career outcomes as all being “successful”. Ok, good enough.

but guys. Really. The question here is whether those that end up wanting a PI job can secure one.

Right?

All else is just handwaving. These authors would have done a better service by tracking down UCSF grads from 5, 10, 15 and 20 years ago and see what jobs they were actually able to obtain.
__
Fuhrmann et al, Improving graduate education to support a branching career pipeline: recommendations based on a survey of doctoral students in the basic biomedical sciences. CBE Life Sci Educ. 2011 Fall;10(3):239-49. [PDF, PubMed]

The NIH policy change to allow multiple individuals to serve as the Principal Investigator (or Program Director for BigMech Awards) has been a whopping and unqualified success from my perspective. There are several situations that were addressed by this change, most importantly the extension of credit to a more-junior scientist. Since the PI reputation and track record was always such a big deal for the review of grants, it was often the case that the BiggestCheez was listed as the PI. Regardless of the amount of involvement of this individual. It was not rare to have very senior investigators allocating less than 10% effort as “PI” when another faculty member (perhaps even Associate Professor level) was allocating much more, say 15-30% effort, and clearly running the show for that particular project. For no official credit.

There was also a much harder to assess impact on collaborative science. In theory, there might be collaborative projects that did not get proposed precisely because neither partner was willing to do it without PI credit. Perhaps this resulted in the inefficiencies of two projects being funded when one would have sufficed? Perhaps it resulted in the collaboration never happening.

At any rate, I viewed the initial round of Multi-PI grants with interest. The first round or two featured a lot of discussion. Of the Leadership Plan. Of the need for a Multi-PI versus single PI application for that particular project. After that? Barely a peep. Nobody seemed to give it much thought at all. I’ve seen relatively recent summary statments where the fact that it is a MultiPI application is barely even discussed by the reviewers!

From my point of view this has become a non-issue for grant review and my advice, accordingly, is for people to go for it* if they think it fits their plans. I am curious as to whether my Readers have any experiences on either side of the review process to share?

The NIH, perhaps reacting to the reality that peer review panels don’t give a hoot about Multi-PI leadership plans and rationales, has decided to let local institutions alter the Single/Multi designation of a project without competitive review. I will remind you that they already permit changes in the research team, including the PI, at the whim of the applicant institution (to whom the grant is actually awarded, of course). So this is not a major new step. Just an…..alignment of policies.

The recent Notice (NOT-OD-11-118) describes how the NIH will permit additional flexibility in the use of the Multi-PI/PD option.

After several years of experience with the multiple-PD/PI model, NIH has determined that there are legitimate circumstances under which it would be in the best interest of an active project to change either from a multiple-PD/PI model to a single-PD/PI model, or from a single-PD/PI model to a multiple-PD/PI model, and that peer review of the new leadership team and Leadership Plan may not be essential in these cases… a request to change an active award from a single-PD/PI to a multiple-PD/PI model, or from a multiple-PD/PI model to a single-PD/PI model, must be made by the grantee organization and should be based on the scientific needs of the project. Justifications based on administrative convenience will not be considered. If the arrangements proposed by the grantee, including the qualifications of any proposed replacement or addition, are not acceptable to the NIH awarding IC the grant may be suspended or terminated.

and since you will be wondering about the newbs…

A New Investigator who is added as a PD/PI on a substantial NIH independent research award after initial peer review will not lose their New Investigator status.

This latter makes for some very interesting grant strategy indeed. A noob could, in theory, be writing grants in collaboration with a more-senior PI to be submitted with that person as solo PI. With the understanding that if it gets funded, they will wait a year or two and then slip JuniorMint (who actually wrote the thing) into the Multi-PI slot. Then the Noob could be busily submitting her own ESI/NI qualified grants, all the while enjoying de facto major grant funding for which she will eventually get at least partial PI credit.

Hmmm…..

__
*I don’t think this is a good idea for Noob Faculty before they’ve landed their first major award, however.

A query to the blog is a very typical reaction of those new (and not so new) to the NIH Grant game. As you will see in my answer. First, the question:

I’m now confused about this whole Program Announcement thing. The PO said that my application would be judged normally, just as part of whatever else the study section was reviewing, and that there was no special money set aside for the PA. If that’s the case, what’s the point of the PA in the first place? I had been under the impression it would be judged with other grants responding to the PA, but apparently that’s not true.

Help?

It is the RFA that generally routes applications into a dedicated, special emphasis panel type study section for review. For those Funding Opportunity Announcements (FOA) there is indeed a set aside pool of money and generally just a single receipt date. The PAS (Program Announcement with Set Aside Funds) also has dedicated funding, generally for the first round of submissions, then it converts to a regular old Program Announcement (PA) type FOA. Applications submitted for a PAR (Program Announcement with special receipt, referral and/or review considerations) is reviewed by a special panel, generally within an IC.

Regular old PAs are open for 3 years and generally use the standard receipt dates. In the ICs of my greatest experience they tend to be renewed and thus may represent essentially permanent PAs for much of your grant writing life. As per the reader query, the applications are reviewed in standard, CSR study sections with the appropriate domains of coverage and expertise. Alongside those applications that use the generic, mechanism based FOA. I would argue that you would only use the latter if you had to. Again, in the ICs of my greatest experience the PAs can be incredibly broad. Take “PA-10-268 Neuroscience Research on Drug Abuse (R01)” as an example. If your IC of interest has such broad topic PAs…you might as well use them.

Now as the reader question intimates, there is no overtly special benefit to your chances of getting funded. And there may be no benefit at all. Hard to tell. Because of course this sort of business only matters* when Program is considering the grey zone pickup funding. Is there a slant or a formula for how many approximately equivalently scored grants they will select under one of their PAs versus the generic parent FOA? I would suspect so, else why have such things? But I can’t say for sure. Maybe it is just make work for Program staff….to lay out their priorities. Or maybe it is a defensive excuse for those rare cases when they decide to stiff a grant that came in under the payline “Sorry PI Squirrel, it didn’t fit any of our Programmatic Interests…don’t you read the PAs?”.

The bottom line here for those new to the system is not to get all that excited when language in a PA seems directed at your research program. It isn’t *that* good of a bennie. But you might as well have some idea what is in the PAs and respond to them when you can. Because you just never know when it might help.
__
*assuming you have a modicum of sense and are not submitting stuff that is clearly not going to be of interest under the generic R01 parent FOA.

Muffy would never hurt a fly

September 27, 2011

[h/t: jekka]

Innocence

September 26, 2011

As you are probably aware there was a lot of hoopla from the lefty libby dirty hippies in the US and kibitzing OldEuro types on social media because the State of Georgia killed this guy. The reasons are pretty well captured in the accompanying article

Davis has repeatedly said he did not kill MacPhail, and seven out of nine witnesses who gave evidence at his trial in 1991 have recanted or changed their testimony.

No murder weapon was ever found, no DNA evidence or fingerprints tie him to the crime, and other witnesses have since said the murder was committed by another man — a state’s witness who testified against him.

This is not hard to grasp.

I have donated to the Innocence Project because I believe in this part of their mission statement.

The Innocence Project’s groundbreaking use of DNA technology to free innocent people has provided irrefutable proof that wrongful convictions are not isolated or rare events but instead arise from systemic defects. Now an independent nonprofit organization closely affiliated with Cardozo School of Law at Yeshiva University, the Innocence Project’s mission is nothing less than to free the staggering numbers of innocent people who remain incarcerated and to bring substantive reform to the system responsible for their unjust imprisonment.

I did not do so because I oppose capital punishment. As it happens, another unhappy soul was also executed recently, this time in the State of Texas.

Texas executed Lawrence Brewer, a white supremacist who was unquestionably guilty of the gruesome dragging death slaying of a black man in 1998.

I intentionally linked to the leftie-libby DFH argument that these are morally and ethically the same events because I disagree. here’s his crux:

The death of James Byrd Jr. — the black man who was tied to the back of a pickup truck in Jasper, Texas and dragged to his death — is shocking to recall, almost 15 years later. His murder is almost unimaginably cruel; it is impossible to read the details without being overcome with anger and revulsion. Yet this is what James Byrd’s sister had to say on the eve of Lawrence Brewer’s execution: “If I saw him face to face, I’d tell him I forgive him for what he did. Otherwise I’d be like him.”

I pay exactly as much attention to victims’ pleas for mercy as I do to their pleas for vengeance. The reason we have a rule of law in the first place is that justice and punishment have to come from a reasonably detached (blind lady justice?), societal point of view. Remember Dukakis and his famous flail on the question of what he would do if his wife were raped? I think Kerry managed to ass that one up too. The real answer Dems should espouse is my answer.

“I’d want to go to work on the homes here with a pair of pliers and a blowtorch. Are you fucking kidding? Anyone would. And given half a chance I damn well would. But there is no place for that sort of gutter, BronzeAge revenge-of-the-powerful jurisprudence in a just society. And THAT is why I support the rule of law.”

But in a democratic society we also meander towards approximations via what is, at root, barely managed democracy. The will of the people, so to speak. And the will of this person is that we, as a society and after due process, execute a guy like Lawrence Brewer. And this asshole too. People like this. maybe this gang of assholes.

But I also think our crime solving and crime convicting systems suck and are tremendously error prone. And have incredibly naked and thoroughly established racial and socio-economic biases.

So I donate to a project that wants to improve that. Even if they do, at root, have goals that are at odds with my support of capital punishment as a valid societal option.