Thought of the Day

March 28, 2014

One of the obvious desires and needs of the newly minted Assistant Professor is to rapidly establish his or her independent laboratory focus. To show the world, in both formal and informal ways, that all that brilliant work has indeed been driven forward by this new Principal Investigator.

As part of this it is necessary to take full credit for the work that has been done primarily by this young person’s laboratory. It can be acceptable in some situations to take a bit of extra credit by inference when the work has been of a collaborative nature, particularly when only that Assistant Professor is under review.

It is dangerous, however, to fail to modulate these claims of credit for collaborative work when all of the participants in the collaboration are under simultaneous review. On the tactical level, you do not want your reviewers thinking that two, three or more labs are taking credit for the exact same thing. On a strategic level, you ARE going to piss off your collaborators. And this is the sort of thing that induces collaborators to stop collaborating with you and just to do it themselves.

When you are the more-junior partner in this scenario, the odds predict that the more-senior person is going to have more relative ability (funds and personnel) to cut you off and continue by other means.

As a related issue, one of the skillsets you need to develop as a scientist is a decent Spidey-sense for collaborators. Some are going to be selfish and some are going to bend over backward to let you take credit, to help your career along and to promote you. These latter are ESSENTIAL to your success. The former must often be tolerated and you do well to protect yourself from them. However, if you cannot discern the two different types relatively rapidly and act accordingly, you run the risk of really pissing off people* who would otherwise be your champion.

Don’t do this.

__
*Remember that unless the person has “Emeritus” after their title, bending over backward to allow you to take credit is not necessarily immaterial to them. This is a reality. No matter how seemingly established a more-senior colleague is, they are worried about the future. There is always the next grant review. Doing a colleague a solid costs them something. The fact that they think this is the right thing to do, regardless, doesn’t mean that they do not do so with a conscious nod to the costs involved.

This is another guest post from @iGrrrl, a grant writing consultant.


A few comments I’ve seen around, on top of my experience working with applicants for K-flavored and other career development grants, make it clear that they think the required career development parts are just window dressing. I hear complaints that they have to write a mentoring plan, and then they never do anything that is on it.

Is it the mentor’s fault? The people who signed letters to be on the mentoring committee? No. (I’m going to switch voice now and talk at you K99/R00 or other K and F applicants/awardees.) And whose fault is it?

The fault is YOURS. No one cares about your career as much as you do, and even if it went in as fiction on paper, it is YOUR responsibility to make it reality. Otherwise you’ll never know if it would have made a difference to tap into the brains on your mentoring committee, to impress them with your initiative and willingness to learn. Making someone feel smart and important to you (while also getting good advice) is a good way to increase their sponsorship of you–inviting you present at meetings, to small subdisciplinary meetings, talking positively about you.

I think it’s easy for young people to underestimate the impact of the positive regard of more senior faculty, or for you young folks to know how that plays out in reality. No, they’re not gossiping about you; they have better things to do. But that ‘dream team’ remembers that they signed letters for you and then never heard from you again.

The FOA for the K99/R00 says that a different institution is “encouraged (but not required)“.

I will confess I thought this was somewhat stronger in tone but the original announcement was similar.

Applicants are encouraged to consider independent positions at departments and institutions different from where they conducted their mentored research. Should an awardee wish to activate the independent phase of the grant award at the same department or institution at which they conducted their mentored research, the individual must provide justification addressing the decision to remain at the same institution.

Still, I guess we can bench race what this would mean in our personal interpretation or recommendation. I would think this an exception for extreme circumstances… so more like 10%?

Well, a comment on a prior thread asserted that something around 25% of R00 awardees had remained at the same Institution in which they had been “training” under the K99 phase.

Naturally, I was interested in what my ICs of greatest interest have been up to. Of currently active R00 grants, one IC came in at 21% at the same institution as the K99 phase and another came in at either 29% or 35%. This latter one had a single award for which the K99 phase moved in the second year and then the R00 was continued in that new location.

Huh.

Huh.

Color me surprised.

I am no big fan of enforced academic nomadism. Were I the Boss of Science I may have omitted that little “encouraged but not required” clause, frankly. Maybe. But the clause is most assuredly in there and it gives an indication about intent. Nice to have exceptions for unusual circumstances, yes, but I would tend to interpret that statement as meaning rare.

At 25% of the awardees, clearly the NIH does not agree with me on this meaning.

Either that, or the reality of the (lack of) academic nomadism under the current system has essentially overcome the NIH’s intent.

Here we have a fancy grant award that gives postdocs some negotiating room in where they would like to be. As we’ve been discussing, these are presumably some top-quality candidates, going by the usual academic seals of approval. And a quarter are disregarding the expectation of nomadic dispersal at the transition to faculty level independence.

To me this underlines the opinion I have on nomadism.

There is something seriously wrong with the expectation that academic scientists travel all over creation for their jobs, in the current era of dual-career families and “training” phases that extend well into the third decade of life.

Quite some time ago I started scrutinizing CVs of visiting speakers, grant applicants, etc to see their trajectories. I didn’t formalize it but I came to the conclusion that a substantial (20-25% would probably have been my estimate prior to today’s exercise) number had violated the “expectation”. Yet it persists. Here we have fairly accomplished scientists (grant winners and invited speakers) violating the truthy truism of academic careers. Substantial numbers of them too. Yet the culture is to sneer at this as an intentional trajectory when it comes to advising junior scientists.

Shaking my head.

This sentiment comes up, now and again, when we discuss the K99/R00 NIH grant mechanism.

This inevitably rings as critical to my ear. As if there is something suspect or underhanded about the K99/R00. At the very least as if there is something bad about it. I disagree.

The K99/R00 mechanism is the only true transition mechanism in the NIH stable. First announced in 2006, the Pathway to Independence award was designed to be won by pre-tenure-track academics (the K99 phase) who could then transition to Research funding (R00 phase) once they won a tenure track job.

Prior attempts to assist new investigators in winning research grant funding failed to cross the transition from trainee to faculty. There were postdoctoral fellowships and there were faculty level awards but nothing that you could apply for as a postdoc and carry forward into a faculty position.

You can whinge all you want about the growing reluctance of Universities to cough up research-focused tenure track Assistant Professorships with substantial startup packages. You can shout as loudly as you like about the way things should be. But that won’t change reality at University of State.

Nor will it change the reality that many of us really work for the NIH.

It is in their interest to figure out how to get their future workforce into the shell “job” (aka, Assistant Professor at University of State) as rapidly as possible. If, that is, they are dismayed by the infamous analysis showing the gradual increase of age to first R01 and other dismal demographic statistics.

In my day, youngsters, there was only the Burroughs-Welcome Career award. They have cancelled this because of the K99/R00 but it used to be the only game in town. And it was quite a plum. But IIRC the University itself controlled who could apply as a postdoc. The numbers awarded were very small. It was a program that was out of reach for many types of science…basically it was all about GlamourMag science in the place I was at at the time I considered using this mechanism.

I saw the issuance of the K99/R00 program to be a hugely important step for the NIH. It was a recognition of reality and a way to get people, young people, hired into jobs now instead of in three more years’ time. IMO.

It was democratic in the sense that all the ICs could play, meaning topic domains were not excluded by a University committee or Dean or local political power playing. It was democratic in the sense that there were quite obviously going to be more R00 PIs than there ever had been Burroughs-Welcome PIs. It was democratic in the sense that peer reviewers were going to be deciding who got the primary awards.

My fields of interest have seen many K99/R00s awarded and the awardees have transitioned into faculty positions. Which are going quite well in my estimation.

Are there people who missed out on K99s getting overlooked in faculty job searches? Undoubtedly. Is this some sort of epic and systematic tragedy?

No.

There has always been an element of career success that is driven by something other than a pure and idealistic assessment of talent. It might be due to the accidents that brought you to a particular training lab. Due to the accident of being highly successful in your graduate student or postdoctoral project. Due to the accident of who else happens to be on the job market when you are. Due to the accident of which University Departments happen to be hiring in your field and in which sub-area of concentration.

None of this is fair.

The processes that lead to a K99 award are not fair either, but they have many upsides. Some of which I’ve mentioned above.

The factor of peer review by people who sit on study sections should not be lightly dismissed. Sure, they are subject to many sorts of biases but they are much less subject to the vagaries of what Departments happen to be hiring in which topic area. They are much less subject to group think because, on the whole, the areas of scientific expertise, career type, geographic region, etc are diverse by construction.

They add another layer, another chance for a supplicant to win over a group of people. That’s a good thing.

The successful K99/R00 Assistant Professor has to win over a search committee too, you know. These jobs are not automatic. And despite rumors on the Twitters, yes, Departments are indeed still hiring people without K99/R00 awards.

I really don’t see why people are so antagonistic toward the program.

__

ps: Changes in the eligibility criteria aren’t unfair either.

pps: The NIH had a sweet deal for Intramural postdocs in place for some time.

ppps: In disclosure, I personally think the NIH should shutter the F32 individual NRSA for postdocs and apply the money to the K99/R00 program. For real.

The Twitt @tehbride raised an interesting mentoring question:

 

As you are likely aware Dear Reader, due to the accident and intent of where I tend to sit on the scientific spectrum, the scooping type of competition is not a huge part of my professional life. That is, I have managed to get by to this point by not being terribly afraid of people knowing what I am working on or what I plan to work on. Part of this has to do with playing at a level of publication that is not obsessed with the very first person to demonstrate something. Part of it is selecting research questions that are not densely populated with dozens or scores of other laboratories trying to scratch the same flea. Part of it is my overweening and misplaced self-confidence that we did it better, dammit, so who cares who published first.

 

Part of it is pure wrongheadedness on my part, no doubt.

When it comes to grants, specifically, I was always around people who were reflexively generous with sharing their applications when I was a late-postdoc and an early-career faculty member. As time has gone on and more people are asking me for my proposals than I feel the need to ask, I have given mine out to anyone who requests them. (Usually with a little lecture about how my “successful” apps are no more informative than my triaged ones, of course.)

So take that into account.

On a purely tactical level, it is possible for the postdoc in this situation to simply refuse. We can extend this to PIs who are asked for their successful grant applications. You can just say no.

It seems to me to be unwise to do so, particularly when it comes to an application that has been successful. Even if you cannot stand the person who is asking. It just seem churlish when the cost to you is so low.

Is it going to give this person ScienceEnemy little boost ahead? Sure. But remember, the odds of funding are still very steep. So it isn’t like you are handing them an award. They still have to write a credible application. And get lucky. So why not*? It costs you essentially nothing to email over your application.

On a strategic level, this person could be your colleague in science for a long time. They could very well be in a position to review you and your work, particularly if they are in a related area of science. And even if they annoy you, it isn’t necessarily the case that they have so much as noticed. Lots of annoying people are kind of unaware… So why make an enemy?

And there is one more thing to consider. If you act within a professional capacity on personal whim and dislike, what does this say about your behavior as an objective peer reviewer? Shouldn’t you be able to set aside personal dislike to effectively review the scientific content of a paper or grant proposal? Yes, yes you should.

 

__

*Now, if you think the person is a data fraud or something…well that is entirely different.

When I first started noticing the opportunity to submit a “Graphical Abstract” for my papers I was initially perplexed as to why I would bother. Then I realized that the Graphical Abstract (at Elsevier titles anyway) could be a way to get the primary data figure out in front of the paywall. So I thought maybe we should do that.

Some joker has apparently concluded that he should use the Graphical Abstract space for being a sexist jerk.

via Dr. Isis, via this article. Elsevier has promised to pull the image so it may not last at the journal link.

Hur, hur, dudes, hur, de-hur, de-hur.

As detailed by Dr. Zen, Pier Giorgio Righetti is an author on at least four articles with highly sexualized Graphical Abstracts. Professor Righetti apparently responded to a query about the wisdom of one of these images with:

I wonder if you have been trained in the Vatican. As you claim to be a professor of Physiology, let me alert you that this image is physiology at its best!

This sounds remarkably like Dario Mastripieri who famously lamented the lack of attractive “super-model type” women at the Annual Meeting of the Society for Neuroscience on his Facebook page. This sexualization of women in a professional scientific and/or academic context has to stop. This is harassment of women in science. It lets all women in this job sector know that these dudes, senior figures with some influence mind you, see them as nothing other than potential sexual conquests. It is unfair, it is rude, it is detrimental to science and it is utterly unacceptable.

Professor Righetti  is on the Editorial Board of several journals, including the offending Journal of Proteomics where he is listed as the expert under the heading of “Proteomics of Body Fluids and Proteomic Technologies“. Eww.  And it gets better. @Drew_lab queried the Journal’s EIC Juan Calvete and received a dispiriting response.

At least it wasn’t a complete brush off such as Professor Righetti gave. But it isn’t a whole lot better.

I hope to settle the case as soon as possible to devote to the lab, which is what should take me up most of the day.

…this translates in my ear to “this is some absolute triviality and sure, sure, we’ll take down the images but really don’t you people have better things to worry about?

Not really, no. The EIC Calvete has himself identified why this is the case. All scientists would prefer to use their time and energy in ways that are devoted to lab business. Unfortunately, reality intervenes. And when male scientists are hitting on, slavering over, disrespecting, leering at, joking about and generally treating female scientists as property, this takes away from the energy the women (and indeed other men who have to witness this crap) have available to devote to science.

So what would really be great is if an EIC like Calvete identified this sort of inappropriate image (hint: it IS inappropriate, not “may be inappropriate”) in advance and prevented it from being published in the first place. It would be great if authors such as Righetti avoiding submitting these things. It would be great if Professors like Mastripieri kept their nasty little observations locked up tight inside their own heads.

 

Now go read Isis’ post. Reason #140 Why Sexist Bullshit in Academia is Not Okay

Craziness.

From Drug and Alcohol Dependence:

Drug and Alcohol Dependence will now be offering a new submission format, Registered Reports, which offers authors the opportunity to have their research protocol reviewed before data collection begins, with acceptance of the protocol providing acceptance in principle of the eventual results, irrespective of the nature of the results.

 

and more specifically….

Manuscripts which comprise the introduction, hypotheses, methods, analysis plan (including a sample size justification, for example based on a power calculation) and pilot data if applicable can be submitted via this format, and will first be considered by Drug and Alcohol Dependence‘s Editor in Chief or one of its Associate Editors. Those Registered Report manuscripts considered of appropriate interest and value will be sent for peer review, and then either rejected or accepted in principle. Following acceptance in principle, the study can begin and the authors are expected to adhere to the procedures described in their initial submission. When data collection and analysis is complete, the authors are to submit their finalised full manuscript for final peer review. As long as the procedures originally described have been followed, and the results interpreted sensibly, the manuscript will be published, irrespective of the nature of the findings.

 

Yeah…. let me get right on that.

 

 

Mad about school funding

March 20, 2014

The economic impact of the March Madness basketball tournament is immense. The much-quoted Challenger, Gray and Christmas analysis estimated a few billion dollars in lost productivity alone, due to workers slacking off to watch games and obsess over their brackets. Cities that host games might draw in the neighborhood of $4-6 million, according to the above linked article. Regardless of whether March Madness basketball games are a net gain for cities (or Universities) the main point is that money is expended. Huge sums.

bluebirdhappinessAt the personal level, how many individual citizens kick down some money to throw a watch party for their alma mater’s game? How many overpay for a hamburger, fries and a few pints of crappy beer at the local sports bar?
Well, I have a proposition for you, Dear Reader. Actually, the bluebird of Twitter happiness known as My T Chondria and Scientopia’s own Gerty-Z of Balanced Instability have a proposition for you.

The proposition is that you head on over the ESPN to fill out a bracket challenge.

The group is DarwinsBalls, the password is Darwin

Then, after you have done so (or even if you see this too late), head over to Donor’s Choose and donate $5 (or more, hey, just order one fewer beers or skip the fries, eh?) to a project supporting science education in under privileged schools.

Gerty-Z and MyTChondria suggested a few, they are now on challenge number 5

UPDATE v. 5: DAMN! It’s hard to keep up with you all!! Here is the next project, since Mrs. Brown’s class is now set up. Let’s bring home the Bacon (Francis Bacon) for Mr. Kovach’s class in Chicago!

We didn’t do the October bloggers drive for Donor’s Choose this year. So perhaps you, as I have been, keep hearing a nagging voice in the back of your head. For those of you who are new, Mr Kovach’s appeal will give you a flavor of the type.

I teach in an inner city public elementary school. Our students are always eager to learn; we are an extremely dedicated community that is proud to serve one of the highest populations of students currently living in homeless shelters out of hundreds of schools in our city. We strive to practice highly engaging, meaningful, challenging, cooperative, and most of all FUN learning experiences that will inevitably inspire all of our students to not only graduate high school, but to pursue college as a part of their long-term goals. Many of our students continue on to be the first members of their families to graduate with a college degree, and many are now pursing degrees in various areas of Science.

They sound very worthy. So what is the project?

This proposal seeks fetal pig specimens and dissecting kits so that our 7th grade students can continue to work in small cooperative groups in order to explore anatomy and physiology in the most realistic manner possible. It has been exciting for our 7th graders to show the younger students and their teachers of our school the process of conducting fetal pig autopsies, and to see little eyes peeping into our classroom door each day of the project.

Ah yes, the old fetal pig dissection model. C’mon, be honest…..isn’t this one of the 3-5 things you can actually remember from your biology education in primary or secondary school? Isn’t this something that would be tops on your list if you were designing Biology curricula? ….and aren’t you slightly gobsmacked to realize this is something that doesn’t fit into the Biology teacher’s budget anymore?

Well, it doesn’t for poorer (and not so poorer) schools.

So, I ask you. Find it in your hear to kick down a few bucks for childhood science education.

(and by all means, if this project or classroom doesn’t work for your, there are a ton more at Donor’s Choose. Search by academic domain, by poverty level, by geographical region. There will be something to tug your heartstrings.)

Thought of the day

March 20, 2014

If you don’t ever publish papers that are only of interest to yourself….that’s sad.

erickttr observed:

I notice that you go to RePORTER for information to help solve mysteries and gather data and strategic thinking respecting grant strategies and for a feel for national trends. Do you ever bring this up in conversation with POs? For example, “I noticed in RePORTER that only 5 R01s have been funded from this PAR, none from my IC of interest (which had a part in creating the FOA) …. what’s up with that?” Or is that too …. something … seems like something you’d learn in business school …. not grad school (where we learn to pipette and run gels).

The things that I talk about on this blog are things that I learned, sometimes the hard way, as a faculty level scientist who was expected to land major research grants to fund his laboratory. A few things I picked up as a postdoc, but my education really accelerated after my career world said “Okay, show us what you can do, junior.”.

Much of what I relate to you I learned in bits and pieces over a very long period of time. Just this very month, btw, I learned yet a new wrinkle on NIH behavior when it comes to grants. I am always learning new stuff.

Obviously, I think it is imperative for my continued career existence that I keep my head up about where the lab’s funding comes from. I blog because I think you Readers should do so as well. Most of this stuff isn’t rocket science, just information. Information that you will over time come to value, information that you will find to be incorrect for your situation and information that may never be of use.

It is my belief that the more academic folks who plan NIH extramurally funded careers know about the NIH system, the better for them. And even grad school isn’t too early to start to pick up the basics.

When it comes to Program Officers and, yes, Scientific Review Officers, my answer to erickttr is a simple “Heck yes!”. Even people who are part of the system don’t necessarily know everything about the system. Not even “their” piece of the system!

You may recall my various frustrations over the years with aspects of the NIH system that participants in the system seem to ignore. Rockey’s assertion about PhD job prospects. The amazing discovery that NI awards, prior to the invention of the ESI category, were going to highly established PIs who simply hadn’t been NIH funded yet. Program Officers who told people in soft-money jobs that “well, that’s not a very good job, you shouldn’t be taking those”. POs who tell investigators they just need to “write better grants”. SROs who were entirely unaware of the A2 traffic-pattern effect as it was developing (“What do you mean this study section rarely funds A0 applications?“)

The list goes on and on.

Areas of scientific study that are woefully underfunded by your favorite IC are no different. YOU, scientists, serve an educational purpose. You do this by virtue of the grant applications you submit. You do this by virtue of the reviews that you supply when asked to serve on study section. You do this in your annual Progress Reports.

And you do this by chatting up your friendly POs on the phone or at scientific meetings.

Part of your argument can be derived from RePORTER. Of course. Particularly when you want them to fund you to do X and there are hardly any grants funded on X at the moment. Maybe you can point to a study section which should be handling X but never seems to let any proposals out with a fundable score. Who knows, maybe you will eventually get a Program Announcement or Request for Applications funded.

Go RePORTing folks.

Addendum based on this (wisecrack?) comment from SidVic.

You are poorly served by these idiots, and it is shameful that the NIAAA portfolio doesn’t contain at least 4-5 projects addressing x and z. Hey you should really do your duty to humankind and pick up my grant…

Obviously you want to be polite. But the real point here is that you are playing the long game. When you front people with the deficits in their system, they are not going to immediately agree you are right and hand you a new grant award. Not the way the world works. You are trying to shape their own beliefs. This can take time. And you are also trying to give them (your advocates) the ammunition that they need to make their case with higher-ups. (When text that is suspiciously similar to your rantings shows up in the RFA, just quietly pat yourself on the back and consider it a job well done!)

As you are aware, Dear Reader, despite attempts by the NIH to focus the grant reviewer on the “Innovation” criterion, the available data show that the overall Impact score for a NIH Grant application correlates best with Significance and Approach.

Jeremy Berg first posted data from NIGMS showing that Innovation was a distant third behind Significance and Approach. See Berg’s blogposts for the correlations with NIGMS grants alone and a followup post on NIH-wide data broken out for each IC. The latter emphasized how Approach is much more of a driver than any other of the criterion scores.

This brings me to a query recently directed to the blog which wanted to know if the commentariat here had any brilliant ideas on how to effectively focus reviewer attention on the Innovation criterion.

There is a discussion to be had about novel approaches supporting innovative research. I can see that the Overall Impact score is correlated better with the Approach and not very well with the Innovation criterion score. This is the case even for funding mechanisms which are supposed to be targeting innovative research, including specific RFAs (i.e., not only the R21).

From one side, it is understandable because reviewers’ concerns over the high risk associated with innovative research and lack of solid preliminary data. But on the other side, risk is the very nature of innovative research and the application should not be criticized heavily for this supposed weakness. From my view, for innovative research, the overall score should be correlated well with Innovation score.

So, I am wondering whether the language for these existing review criteria should be revised, whether additional review criterion instructing reviewers to appropriately evaluate innovation should be added and how this might be accomplished. (N.b. heavily edited for anonymity and other reasons. Apologies to the original questioner for any inaccuracies this introduced -DM)

My take on NIH grant reviewer instruction is that the NIH should do a lot more of it, instead of issuing ill-considered platitudes and then wringing their hands about a lack of result. My experience suggests that reviewers are actually really good (on average) about trying to do a fair job of the task set in front of them. The variability and frustration that we see applicants express about significantly divergent reviews of their proposals reflects, I believe, differential reviewer interpretation about what the job is supposed to be. This is a direct reflection of the uncertainty of instruction, and the degree to which the instruction cannot possibly fit the task.

With respect to the first point, Significance is an excellent example. What is “Significant” to a given reviewer? Well, there is wide latitude.

Does the project address an important problem or a critical barrier to progress in the field? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field?

Well? What is the reviewer to do with this? Is the ultimate pizza combo of “all of the above” the best? Is the reviewer’s pet “important problem” far more important than any sort of attempt to look at the field as a whole? For that matter, why should the field as a whole trump the Small Town Grocer interest…after all, the very diversity of research interests is what protects us from group-think harms. Is technical capability sufficient? Is health advance sufficient? Does the one trump the other? How the hell does anyone know what will prove to be a “critical” barrier and what will be a false summit?

To come back to my correspondent’s question, I don’t particularly want the NIH to get more focused on this criterion. I think any and all of the above CAN represent a highly significant aspect of a grant proposal. Reviewers (and applicants) should be allowed to wrangle over this. Perhaps even more important for today’s topic, the Significance recommendations from NIH seem to me to capture almost everything that a peer scientist might be looking for as “Significance”. It captures the natural distribution of what the extramural scientists feel is important in a grant proposal.

You may have noticed over the years that for me, “Significance” is the most important criterion. In particular, I would like to see Approach de-emphasized because I think this is the most kabuki-theatre-like aspect of review. (The short version is that I think nitpicking well-experienced* investigators’ description of what they plan to do is useless in affecting the eventual conduct of the science.)

Where I might improve reviewer instruction on this area is trying to get them to be clear about which of these suggested aspects of Significance are being addressed. Then to encourage reviewers to state more clearly why/why not these sub-criteria should be viewed as strengths or lack thereof.

With respect to second point raised by the correspondent, the Innovation criterion is a clear problem. One NIH site says this about the judgment of Innovation:

Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?

The trouble is not a lack of reviewer instruction, however. The fact is that many of us extramural scientists simply do not buy into the idea that every valuable NIH Grant application has to be innovative. Nor do we think that mere Innovation (as reflected in the above questions) is the most important thing. This makes it a different problem when this is co-equal with criteria for which the very existence as a major criterion is not in debate.

I think a recognition of this disconnect would go a long way to addressing the NIH’s apparent goal of increasing innovation. The most effective thing that they could do, in my view, is to remove Innovation as one of the five general review criteria. This move could then be coupled to increased emphasis on FOA criteria and an issuance of Program Announcements and RFAs that were highly targeted to Innovation.

For an SEP convened in response to an RFA or PAR that emphasizes innovation….well, this should be relatively easy. The SRO simply needs to hammer relentlessly on the idea that the panel should prioritize Innovation as defined by…whatever. Use the existing verbiage quoted above, change it around a little….doesn’t really matter.

As I said above, I believe that reviewers are indeed capable of setting aside their own derived criteria** and using the criteria they are given. NIH just has to be willing to give very specific guidance. If the SRO / Chair of a study section make it clear that Innovation is to be prioritized over Approach then it is easy during discussion to hammer down an “Approach” fan. Sure, it will not be perfect. But it would help a lot. I predict.

I’ll leave you with the key question though. If you were to try to get reviewers to focus on Innovation, how would you accomplish this goal?

___
*Asst Professor and above. By the time someone lands a professorial job in biomedicine they know how to conduct a dang research project. Furthermore, most of the objections to Approach in grant review are the proper province of manuscript review.

**When it comes to training a reviewer how to behave on study section, the first point of attack is the way that s/he has perceived the treatment of their own grant applications in the past***. The second bit of training is the first round or two of study section service. Every section has a cultural tone. It can even be explicit during discussion such as “Well, yes it is Significant and Innovative but we would never give a good score to such a crappy Approach section”. A comment like that makes it pretty clear to a new-ish reviewer on the panel that everything takes a back seat to Approach. Another panel might be positively obsessed with Innovation and care very little for the point-by-point detailing of experimental hypotheses and interpretations of various predicted outcomes.

***It is my belief that this is a significant root cause of “All those Assistant Professors on study section don’t know how to review! They are too nitpicky! They do not respect my awesome track record! What do you mean they question my productivity because I list three grants on each paper?” complaining.

Beer companies were pulling their sponsorships of St Patrick’s Day Parades this year. Because of the long-running battle between Parade organizers and LBGT folks that wish to join the celebration.

Man, we’ve come a long way, haven’t we?

This post was live-blogged on Mar 15, 2009.


Step One: Make sure at least one of the Spawn is napping, visiting a friend or otherwise out of your hair.

Spd1-300.jpg

Why, whatever do you think we are celebrating today, Dear Reader?

Step Two: Make final check on materials and reagents. Run to store to get the remaining critical items. Sing loudly to your favorite ethnic folksongs to get in the mood.

stay tuned, Dear Reader, stay tuned…

Read the rest of this entry »

I am wondering whether anyone else is noticing any trends for prioritizing multiple-lab NIH grant proposals.

I recently got busted on, somewhat randomly given the proposal, for not including enough other faculty level investigators. At the time I shrugged it off as an annoying hobby-horse issue of one reviewer.

But Multi-PI proposals have been going over well for some time now, from appearances. So perhaps it is a trend and study sections will start punishing single-lab grants?

I am not sure what to make of this, should it become a trend.

The comment I received smelled to me like “why are you not bringing your junior faculty along for the ride?”…bu perhaps I am over interpreting.

Reminder. You are going to have advocates and detractor reviewing your grant proposals. Your goal is to give the advocate what she needs to promote your proposal.

No matter how much the advocate might love the essential ideas in your application, nothing good is going to happen if you violate every rule of basic grantsmithing.

At the least you should be able to put together a proposal that gets it mostly right. Credible. Serious. Without huge gaping holes or obvious piles of StockCritique bait lying around everywhere.

It should not be hard to give the advocating reviewer something they can work with.

This guest post is from @iGrrrl, a grant writing consultant. I think I first ran across her in the comments over at writedit’s place, you may have as well. She brings a slightly different, and highly valuable, perspective to the table.


iGrrlCartoonThere was an old New Yorker cartoon of two people at a party, and one tells the other, “I’m a fiction writer in the grant-proposal genre.” I hate putting fiction in grant applications, especially the type that will be due shortly in response to the Ginther report.

For those who have been worrying about their own grant applications, the Ginther report detailed the relationship of race and ethnicity to NIH grant funding at the R01 level, and NIH has created a few initiatives to try to change the pattern of lower success rates for African American applicants. In early April, the applications are due for the Building Infrastructure Leading to Diversity (BUILD) Initiative, which would fund large-scale projects within individual institutions, and the NIH National Research Mentoring Network (NRMN), which is designed to build a network of mentors. In other words, diversity interventions writ large, with millions of dollars behind them.

The NSF ADVANCE program started in a similar way, and some of the early ADVANCE projects included programs with a limited evidence base. The successes of components of these programs were variable, and the recent RFA included a social science research component to complement the required evaluation of ADVANCE-funded activity. It started with good intentions, and eventually became clear that more than good intentions were needed. Part of the NRMN announcement calls for pilot programs, but I would argue that in the first year, the network should do the social science work to consolidate the anecdotes African Americans tell about their mentoring experiences into hard data, so that the pilot programs can be based upon addressing the needs identified by African Americans who have been through, or training right now in, the current system.

At the recent AAAS meeting in Chicago there were sessions on building diversity in science. At one I learned that explicit bias has reduced in the last 30 years, but implicit bias hasn’t. We think we have made progress, and that our conscious intentions are enough. But they clearly are not. Expecting trainees to overcome biased behaviors (to which the actors are blind) places an undue burden on those who are discriminated against. There are studies showing that education about implicit bias helps to reduce such biased behavior, but education attempts can also be done badly and backfire. As pointed out in a recent piece in Science by Moss-Racusin, et al., there is an evidence base now for doing intervention well. If NIH is putting money into large-scale intervention, I hope the existing science will be part of the applications, and expected by the reviewers.

I’ve spent a lot of my professional life working on exactly the kind of large, infrastructure-based grant application represented by the BUILD and NRMN programs. It is easy for PIs to make assumptions that interventions that sound good on paper will actually have any impact. My concern is that what will be proposed by the applicants to BUILD and NRMN may miss the strong social science work that exists, and that still needs to be done. In fact, some of the best research on effective mentoring is the business literature, a place where few biomedical scientists would think to look.

Grant applications shouldn’t be pure fiction, but based on solid evidence. Every grant application represents a possibility, a reality, that could come to pass if the funds are awarded. In the mentoring literature, practices that improve the success for African Americans are often shown to improve the climate for everyone. There is an opportunity here for those in biomedicine to learn from other fields, to consider an evidence base that is outside their usual ken, and to improve the entire biomedical enterprise by improving the overall environment. I hope that those applying for BUILD and the NRMN include the social sciences, and even more importantly, include the voices and ideas of the very people these programs are meant to serve.

NIH has a long history of using dollars to encourage cultural change, with mixed results, because applicants can have varying levels of commitment to the NIH vision while being happy to take NIH dollars. The ADVANCE program at NSF had some hiccups as they worked out what worked to improve the climate for women in STEM. The leadership teams for BUILD and NRMN should include people with a deep knowledge of the research and scholarship on bias and on mentoring, and who can do the rigorous analysis of the current state of affairs for African Americans in biomedical science. I hope I’m wrong here in worrying that such people won’t be included, but I’ve seen fiction in grant applications a few too many times.