Overall Impact Scores Correlate Best with Approach Criterion Scores: The NIH-wide Analysis

Who else? Jeremy Berg strikes again.

OER’s Division of Information Services has now analyzed 32,608 applications (including research project grant, research center and SBIR/STTR applications) that were discussed and received overall impact scores during the October, January and May Council rounds in Fiscal Year 2010. Here are the results by institute and center:

Recreational Mephedrone Brief (09/30/10)

Mephedrone, or 4-methylmethcathinone, is a recreational drug that got very popular in the UK in recent years, no doubt due to it being legal to sell and possess up until April of this year. There is not a tremendous amount known about the pharmacology of this drug at present, however we can deduce quite a bit about where we should start looking from user experiences. I am currently intrigued by the fact that if you look at online user forums you can get Ecstasy fans describing mephedrone as being sortof like Ecstasy…only not as good, or not quite the same. In addition, a recent paper which surveyed a certain subset of users found that many of them report intranasal mephedrone to be as good or better than intranasal cocaine. You will recall, of course, that I have a great deal of blog interest in discussing MDMA-related fatalities.
A Case Report that recently appeared in the Lancet helps us to connect up some dots. Sammler and colleagues report on the case of a 15 year old girl who presented to their emergency department one afternoon with “altered mental status, vomiting and nausea”. She had been out drinking the night before and had also consumed a “white powdery substance”. The clinical workup contained a few interesting clues:

-the cerebrospinal fluid (CSF) opening pressure during lumbar puncture in the lateral decubitus position was raised at 350 mm of water.
-Blood tests showed profound hyponatraemia at 118 mmol/L.
-Serum osmolality was low at 256 mmol/kg, whereas urine osmolality was high at 742 mmol/kg.

Well, well, well. Hyponatraemia is frequently reported in cases of MDMA-related medical emergency and death. This is very likely related to an effect on vasopressin / antidiuretic hormone release that would cause the kidneys to retain water, perhaps in combination with induced polydipsia (urge to drink) or intentional (albeit misguided) prophylactic strategies. This is likely driven by the serotonin transporter inhibition properties of MDMA, this indirect agonist effect perhaps working through the serotonin 3, 2C, 4 and/or 7 receptor subtypes to induce vasopressin release.

I have no good stats but there is a distinct impression from reading MDMA case reports that young women may be particularly liable to hyponatremia following MDMA. This is something I need to take up at some point- is there evidence for increased sensitivity of adolescent women to fluid balance dysregulation?

Returning to the topic at hand, is this just a case of MDMA-induced hyponatraemia?

Fortunately, the doctors ran the tox panels:

We suspected drug intoxication and did gas chromatography-mass spectroscopy of the patient’s urine; this was unequivocally positive for mephedrone metabolites, but was negative for opioids, methadone, barbiturates, cocaine, cannabinoids, alcohol, benzodiazepines, and amphetamines including ecstasy. Analysis of the white powder was consistent with mephedrone.

Interesting. This suggests to me, as it did to the authors, that there is a MDMA-like component to this mephedrone stuff. It may be a dopamine transporter inhibitor and/or dopamine releaser like cocaine, amphetamine, methamphetamine but the hyponatraemia suggests an additional (significant) serotonergic component of the pharmacological response to mephedrone. This would be consistent with those users who report it as being at least somewhat like Ecstasy.

As I discussed before, one prior paper reported on the subjective effects of several cathinone analog compounds using the drug-discrimination assay. The cathinone structure if very similar to amphetamine and supports parallel modifications. The question becomes whether the same modifications of the cathinone and amphetamine core structures convey similar changes in the pharmacology.

In very brief overview of the drug-discrimination procedure, you train rats to tell you if the drug you have just given it is similar to a reference drug such as amphetamine or MDMA. The prior paper found that methylenedioxycathinone (MDC) and methylenedioxymethcathinone (MDMC) fully substituted for MDMA at reasonably similar doses. MDMC also fully substituted for amphetamine whereas MDC did not; in both cases the potency was much lower-higher doses had to be employed for comparable effect to the reference amphetamine.

This is complicated, because if anything MDA is closer in subjective and behavioral effect to amphetamine than is MDMA. And if there are any data on the 4-Methylmeth modification of amphetamine, I am unaware of them. Nevertheless it provides some clue that we are not totally out of line to suspect that the 4-Methylmeth modification to cathinone adds on a serotonergic agonist component, very likely mediated by blockade of the serotonin transporter (with perhaps some releasing effect)…just like one sees with the methylenedioxymeth modification of amphetamine in the case of MDMA.

Final note: The 15 year old girl in this Case Report made a full recovery. That’s a very good thing.
__
Sammler EM, Foley PL, Lauder GD, Wilson SJ, Goudie AR, & O’Riordan JI (2010). A harmless high? Lancet, 376 (9742) PMID: 20801405

Donors Choose Challenge 2010

It is my favorite time of year for blog-related group action activities.
Welcome back to the science blogosphere’s efforts to raise money to support classroom projects with Donor’s Choose.

As always, the key consideration is that every little bit counts. If prior years are any judge, the grad students in my readership are unbelievably generous, given their limited incomes. Just sayin, PIs, just sayin.
Also, I’ve selected a bunch of projects for your consideration that caught my eye for undoubtedly random and personal reasons. If you don’t see anything that catches your fancy, take a look through other bloggers’ lists or search for yourself at the Donor’s Choose site. No matter what you choose to support, kids win. And that means we all win.
__
Addendum:
Now, I must confess that I blew it this year. I was distracted working on some grant or other and blasted up my Donor’s Choose challenge page (Science up the Schools 2010!) ahead of the official launch date for the competition between blog collectives/interests. My total bad. but then The Gam launched….and our new MortalBlogEnemies launched…and what was I gonna do? Now PZ has thrown down with his 8 billion pound commentariat.
Sorry Janet. I’m not having a good few weeks here….

NIH Grant Basics: That post-Council pause

My feed for writedit’s comments keeps being populated with impatient applicants who have managed to land a decent-looking score on a NIH grant proposal. See this one.

The September council met on the 21st, and I have heard nothing. My R01 was scored at the 12th percentile and I am an ESI. I haven’t heard a thing yet, and my Commons status is “Council Review Completed.”

or this one:

The council meeting was over on September 1, and 3rd, mt erA commons sais `council review completed’. It still says the same. My PO is unreachable (has gone back to India until October 1st week). Does this mean I did not get the grant?

for the type.
This is understandable, given the importance of each grant award to the PI in question. But still, this is not mysterious stuff here folks. The timeline for getting a grant awarded is pretty clear.
Let’s take the current Fall Council rounds as an example. The applicants submitted their proposals back in Feb-Mar in most cases- with continuing submission, HIV-related grants and the odd RFA-related study sections, anywhere from January to mid April. These proposals were reviewed for the most part in Jun-Jul with the applicants receiving their scores about a week after the meeting and the summary statements several weeks later.
So they’ve been waiting a long time already. A borderline score makes things extra important when it comes to information about likely funding. Will the proposal sneak under the wire after all funds are accounted for in the IC? Will it really have zero chance, no matter the theoretical nonzero chance? Should, in point of fact, the applicant busy herself with revising the proposal for November or working on a new one for October? Or start planning preliminary studies for a Feb/Mar submission?
But folks, this part of the process is readily understandable.
Even if you have a 1%ile grant, NIH is not going to tell you it is funded until the Notice of Award is actually prepared. And that takes place in the week or two just before the first possible funding date. In this scenario, December 1. So no, you will not hear anything definitive until late November at the very earliest. Stop driving yourself crazy asking over at writedit’s place if the Commons status line will tell you anything. It won’t.
Now the bad news is that this particular round for funding has an extra-special annoying caveat. It is the first one of the new US federal fiscal year. That means that the ICs can’t commit to new awards until Congress passes the appropriations bill funding the NIH for the next year.
They never seem to get that passed on time and it often waits until Congress returns from vacation in late Jan/early Feb.
Sorry about that but you just have to chill.

Take your NIH PO's comments with a grain of salt

As my disclaimer, I am one of those that reminds new (and not so new) applicants to the NIH for research funding to talk to the Program Officer. Early and often, just like voting. In the pre-submission phase you want to identify the line POs down in the Division and Branch structure of one of the funding Institutes or Centers of NIH who might be interested in your work. After review, the PO who was actually assigned to your application can give you invaluable feedback about how the review of your application went down. So I recommend calling this person.

Take their comments, however, in context. They don’t know everything. Even if they are sitting in the room, paying attention to the discussion of grants, this does not always mean they truly understand what is going down.

One of the times I get really frustrated with POs is when they are so fixated on the objective truth of peer review. They often act as though they believe that the review process really does work nearly perfectly. Most often when it comes to newbies. Consequently if you are coming up short on your proposals, in their worldview they think you are “not writing well enough”. Or need to (somehow without funding) provide more/better preliminary data.

So this advice gets reflected back on the poor applicant who 1) drives herself crazy trying to “improve” her writing (which is just fine already) or 2) goes back to the lab to find that perfect figure which which will guarantee this grant gets funded (no such thing).

Disastrously, this prevents said newbie from doing what she really needs to be doing which is to submit multiple good-enough grants. In the face of budgets which allow the funding of only a subset (a third? quarter?) of the grants which are excellent and interesting and impactful and all that jazz, review becomes variable. Meaning the difference between making it into a fundable score and just missing a fundable score takes on the appearance of chance. The only way to beat such odds is to give yourself more chances at the game. This means writing and submitting multiple applications (on different topics, of course).

Don’t mistake me. There IS a learning process for grant writing and it remains good advice for the new (and not so new) investigator to seek feedback from peers prior to submitting a grant and when doing the post-mortem after receiving the summary statement. But this can’t be taken too far. At some point, you are just driving yourself crazy with conflicting (good) advice from people who are in different situations from yourself or each other. And anyway, you can apply the stylistic and structural advice you have received to new application just as well as to the revision of your existing application, right?

On being a “neuroscientist”.

If you are in a position that Journal of Neuroscience is to be sneered at for an insufficiently high IF, you are a GlamourMag scientist. Topic is irrelevant at that point.

On keeping abreast of the literature: A highly loaded poll question

Once I get a fair number of responses I’ll put my real thoughts in the comments…

NIAAA-funded researchers need to stop whining and sack the hell up!

Someone going by Addiction Scientist dropped this comment on the Sb blog:

Unfortunately, the questionable efforts by Dr. Volkow and her staff to “market” this merger at grantee meetings (“It’s a done deal!”), despite clear directions not to do so, has really created an adversarial relationship between the scientific staff of each Institute. Many of NIAAA’s best scientists are considering leaving NIH rather than work under Volkow. It’s hard to be a creative scientist in a hostile work environment, and people are really angry with Volkow and Leshner (her predecessor at NIDA). An additional insult to NIAAA is that this merger may signal the end of a carefully developed and nurtured alcohol research community that has contributed so much to the welfare of the American people. Alcohol researchers actually have developed useful pharmacotherapeutics, behavioral treatments, technologies, and clinical tools that have actually had impact on people. Aside from a “war on drugs” under ONDCP’s auspices, its hard to quantify NIDA’s contribution, beyond the reward pathway. Even the best endocannabinoid research is supported under NIAAA. So, who will this help and how? Does Volkow really need a larger empire?

Call the waaaahmbulance.

Read my lips, Volkow can not possibly be selected as the head of a new combined Addictions Institute. Not going to happen.

It is going to have to be a person who has feet firmly planted in both alcohol research and other-drugs research.

An additional insult to NIAAA

You need to get over this “insult” business. And I mean the collective you. It is not an insult. Honestly now, if you start with the assumption that Institutes are to be merged, which other ones rise to the top of the list? Seriously.

this merger may signal the end of a carefully developed and nurtured alcohol research community

Why that almost sounds like you think that alcohol research scientists suck and cannot compete on an even footing with those who are currently under the NIDA umbrella. Who’s insulting who now?

From my limited experience serving on a study section that received grants assigned to NIAAA and NIDA my opinion is that there is no competitive disadvantage/advantage for either types of applications that we reviewed. It also happens that I know quite a number of people who have held or do hold awards from both NIAAA and NIDA.

If we think about human populations, sure maybe there is more of a tendency to specialize but c’mon. The co-morbidity issue is huge in alcohol research- so get your grants from NIMH. Also, many human users of other drugs also co-abuse alcohol- perfect opportunity to steal a march on the other-drug-focused investigators.

C’mon NIAAA-funded extramural researchers, sack the heck up!

(if you were referring to intramural researchers in your comment about ‘best scientists’ leaving, well can’t help you there. not a fan of the intramural research program)

Now, getting us back on track with the politics, we still have not heard from powerful lobby interests and their pet CongressCritters. And by “powerful”, I mean not namby pamby patient advocacy groups and the like. I mean the “beverage” industry. Brewers, distillers and vintners, oh my.

My completely unfounded suspicion is that they will be very motivated to keep alcohol from being explicitly defined as a drug like any other, which is how this is going to look.

I love it when the data support my position

We occasionally lapse into discussions around here on various career and grant related topics. One that emerges now and again is what constitutes an acceptable Impact Factor of a journal. Yes, higher is better but out in the real world there are a lot of us publishing most of our work in journals with impact factors that are an order of magnitude lower than Glamour.

I make a big point out of field specific expectations- this is a well known, well discussed….. and generally ignored property of the IF. Nevertheless if we limit ourselves to NIH-funded biomedical research and excuse ourselves from discussing outlier scores…..”society level” journals are generally going to be below 10 IF, most frequently in the 2-6 range and with a gut-feeling point of deflection around IF 4 or 5. As in the number of journals really thins out past this point. Whether that reflects some objective or arbitrary standard of quality…that’s a debatable point. At any rate, there are those that sneer at journals of almost any IF below Glamour level (i.e., north of 20). Some, such as my coblogger PhysioProf, are known to make comments suggesting that the expected norm is something other than what I describe it to be.

Something else we talk about now and again has to do with the desired target lab size as expressed in Direct Costs value of extramural funding. This is the cash value you have to spend on supplies, personnel, equipment, etc in a given year. The R01 from the NIH is theoretically unlimited however the cap for writing a modular budget (instead of full itemization of expenses) is $250,000 / yr and you have to get special permission to even submit one for over $500,000 per year. Under the presumption that most newbies try to stay under the modular cap (and they should in most cases) my recommendation is that even a starting-out lab should be trying to land 2 R01s. I.e., I suggest that you need $500,000 in direct costs just to give yourself a fighting chance at scientific survival and (modest) success. There are those, likely those that are still fighting just to get one-R01 level funding, that argue that this is wild excess and the source of all that is wrong with success rates at the NIH grant getting game.

Well, wouldn’t you know, Director Berg of NIGMS has yet MORE data for us to geek over and it is relevant to these two discussion points. He has a new post up on his blog reviewing the scientific output of NIGMS grants first funded in 2006. Out of nearly 3,000 investigators

…the median annual total direct cost was $220,000, the median number of grant-linked publications was six and the median journal average impact factor was 5.5.

I was particularly struck by the second figure.

A plot of the median number of grant-linked publications from 2007 to mid-2010 (red circles) and median average impact factor for journals in which these papers were published (blue squares) for 2,938 investigators who held at least one NIGMS R01 or P01 grant in Fiscal Year 2006. The shared bars show the interquartile ranges for the number of grant-linked publications (longer red bars) and journal average impact factors (shorter blue bars). The medians are for bins, with the number of investigators in each bin shown below the bars.

Look at that interquartile range for the IF of the journals in which the papers were published. Up to the total laboratory R01/P01 funding level of $500K in direct costs per year, the 75th percentile is still only around an IF of 7.5-8. And the 25th percentile is below 5.0. So even if you do not account for subfield differences already there is plenty of evidence about what is a “normal” IF level for NIGMS awardees. And that norm reads one heck of a lot closer to my concept of normal than that of the GlamourAdvocates around these parts.

The second point is the obvious support for my position that around $500K / year in direct costs is a good target because the number of publications increases pretty steadily up to this point and then flattens out from ~$600K-$1M in direct costs.

There are a couple of interesting questions that naturally occur to one upon reviewing these data. First, the relationship between number of publications linked to a grant may be increasing because you can list more than one grant for a given paper. So if you have two or three grants, it is possible that you have a sort of convenient synergy. Sometimes one project is going better and leads to an extra publication- if you list both of your grants, this might be slightly overcounting the impact of the project which isn’t going so well and would not have produced that paper in isolation. Second, it could be that as you get into the zone well north of $500K, you start reaching a different type of laboratory. One that focuses more heavily on GlamourPubs is going to (inevitably?) trade IF for number of publications.

Women’s Health irritates the scientist-reader

Scicurious picked up an issue of Women’s Health magazine while working out at the gym. She grew gradually unamused.
Go Read. and Read. and Read.

Levi Leipheimer: Cyclist, Philanthropist….Idiot

Of course there is no particular reason to think that bike racing celebrity types should be any smarter than your average Hollywood actor or even one of those ruthless self-promoting celebrities who you can’t quite figure out why they are famous.

Levi Leipheimer is a US professional cyclist who has become, over the course of a long career, a top talent with a long list of accomplishments. Recently his accomplishments have been in association with Lance Armstrong who is an absolute pitbull when it comes to battling cancer. I’ve mentioned before that following his Twitter gives you a whole new appreciation for how much this guy works at the whole LiveStrong charity.

And it isn’t like Levi is just a passive participant. He puts the name of some kid with “pineoblastomas, a rare and aggressive brain cancer that afflicts less than 2% of all juvenile brain cancer patients” on his bike. Or remembers the name of a junior high school counselor who fell to colon cancer in another race.

He sponsors and promotes “Levi’s Gran Fondo“, a charity ride to raise money for various causes including “The Lance Armstrong Foundation received a $10,000 donation from the GranFondo for their ongoing funding of cancer research”.

Research.
So what in the hell is he doing
tweeting this?

Odesssa and I hanging w/ @richroll and @jaiseed at the 30th anniversary [famous ARA wackanut organization-DM] Gala. Great night http://yfrog.com/3upsxnj

I’ve said it before…we really need to get Lance Armstrong focused on including animal research as part of his message.

Grants of Obligation, Grants of Necessity, Grants of Desire

Grant applications, that is…..

I was recently thinking back on my primary motivating reason for writing some of the many grant applications I’ve written. I concluded that one possible way of categorizing is the triumvirate of Obligation, Necessity and Desire.

Grants of Obligation are motivated by the need to satisfy an obligation to others. Perhaps it is a group project to which you contribute. A departmental training grant, a research center with a lot of expensive toys that go “ping!”, a collaboration with the Dean of whatnot or perhaps support for a research conference.

Grants of Necessity are motivated by the need to fund your laboratory before something dire occurs. Startup funds are dwindling or perhaps you are down to the last year of your last R01. Maybe you are merely facing losing a key staff member or resource.

Grants of Desire are motivated intrinsically. Perhaps there is a research question that is just eating at you. Maybe it is what you’ve always wanted to do your entire scientific life and you finally see the chance. Or maybe you are in job situation where you would never even think of writing a grant unless it was something of deep personal interest.

Naturally there is overlap. You never have only one motivating reason for writing a grant applications and in some senses “Necessity” is always part of the picture. If I had private philanthropic funds being showered on my group….well, I don’t like writing grants that much, if you know what I mean.

One thing I was also pondering is if the motivating factor is associated with 1) better or worse grant application writing/preparing; 2) better or worse review outcome or 3) better or worse scientific outcome for those grants that get funded.

I have concluded the answer to all three questions is “No”. I have for sure written some real clunkers that arose out of my fondest heart’s desire, scientifically speaking. And while I’m hesitant to review my own writing, I would be loathe to claim that any of these three factors lead me to prepare my best or worst applications that I’ve ever submitted. I’ve had some pretty smoooooth Grants of Obligation…and some ones that looked like an obligation. Outcome? Well, if we’re going on the funded/not funded axis I certainly have received funding for grant proposals I’ve written primarily from all three of these motivators. And you know that means that I’ve come up short for proposals under all three categories as well.

How about you DearReader? Do you write under these motivations? Check all that apply.

A question for grad students who think they are underpaid and overworked

I had been meaning to make this a post but the moment simply arose.

In response to this comment:

Our grad students get 27K annually, a formal hourly rate that is just a little over what our dishwasher gets ($11/hour). And they always work more than 40 hours a week. So as a “real job” grad school totally sucks.

I said:

You know, I’ve had a post or poll idea nagging at me for a while- might as well ask it here. How many people whining about grad student salaries have ever worked a real, low paying job? One with hard work, long hours and a view of the 40-50 something workers telling you that there might not be any significant change on the horizon.

What was your worst job, how old were you and how long did you stick it out? (no, parenthood not included!)

and this:

I should point out for the nonscientists that “dishwasher” in this context refers to someone hired to wash laboratory glassware so the grad students and scientific technicians don’t have to do it. Not some poor chump sweating his/her way through a shift in a cramped kitchen loading the dishwasher for a restaurant.

also, US Federal minimum wage is $7.25 per hour. At 2080 hours per year (52 wks X 40 hrs) that works out to be $15K per year.

So how about it folks? What actual low-paying real jobs have you had? None of them worse than the conditions of being worked and paid like a graduate student?

Socio- versus Economic Privilege in Academic Careers

As might suspect, Dear Reader, I am one who is entirely unable to hold back from drenching trainees in career-relevant advice whenever there is the slightest opportunity. As you also know, I am not in a job category that require regular instruction of general populations of undergraduate students so most of my interactions with them come under the general heading of “lab experience”. One of the things that I tend to blurt out very early in my discussions with a student who wants to work in my group is “You know being a graduate student is a job that pays, right?”.
It turns out that many of them are unaware of this fact.

What is this My RePORTER for anyway?

The NIH’s search engine for funded grants, RePORTER, has a new beta feature called “My RePORTER”. Apparently you can save your searches so that you can repeat them later.
Naturally I had to sign up. But I’m thinking over my usual ways of using RePORTER and wondering what the point is. Okay, I guess I might search “Physio-wimple nucleus” now and again and perhaps “Bunny Hopper” (albeit very rarely) and maybe “Namnezus pinqaz” ever so often…
But I’m not sure I need something to store my searches.
You?
I guess my concern would be that this is set up primarily so that the usual Congress Critters can set up their triggers for all the research that they hate, you know, AIDS, sexuality, drug abuse…anything that sounds psychological and easily denigrated, in case they need something to say out on the stump to fire up their base.