Impact Score Versus Percentile Scatter Plot

Jeremy Berg–Director of the National Institute of General Medical Sciences–just posted this scatterplot of impact score versus percentile for hundreds of R01s assigned to NIGMS at his blog:

Overall Impact Score is the average final score from 1-9 given to the grant application by all the members of the study section, x10, and rounded (up?) to the nearest whole number. The percentile is the percent of applications reviewed by that same study section over the current round and the previous two that had better impact scores than that, and rounded up to the nearest whole number.

For those of us fascinated by inside NIH grants baseball, this is some serious fucken catnip, as there is a fuckton of interesting stuff in there. One of the most fascinating to me is the differences it reveals in scoring behavior for different study sections.

For example, looking at “milestone” impact scores of 20 and 30 reveals dramatic differences in “score inflation” in different study sections. One study section only scored 3% of its grants better than 20, while another study sections scored 20% of its grants better than 20. And one study section only scored 10% of its grants better than 30, while another scored 38% of its grants better than 30.

What would be truly fucken fascinating would be to redraw this scatterplot, with the dots representing funded grants drawn in green (for money!) and the dots representing unfunded grants in brown (for poop!).

The Scientopia Shop

If you have been reserving a place in your coffee cup cabinet or tee-shirt drawer since last Monday you need wait no longer. Point yourself to http://www.cafepress.com/scientopiablogs and you will find a few items with the Scientopia logo for your granfalloon pleasures.

“Now DM”, you are saying to yourself, “what about all this noncommercial, commie collective stuff”? Well, at the moment since Scientopia is not really set up to handle cashflow*, this shop is set up without any additional charges for the items beyond what cafepress requires. So we’re not making a dime.

“What about transparency? I thought you guys are all about the radical transparency?” Yes, well opinions vary on that one but FWIW my goals are threefold. First, to order up some schtuff for my owndangself. Selfish goals are tremendous motivators, you know. Second, to do some marketing of the Scientopia enterprise (whoops, “collective”) via exploiting you, DearReader. What? Too honest? It’s true. All true- bwaaahaaaahaaahaaa. Ahem.

Third reason is, of course, building community. Which is an experimental endeavor. I have some hypotheses about this (ok, ok, I learned everything I know from Kurt Vonnegut’s Cat’s Cradle) and I plan to test ’em. We may need stuff to hand out to our community now and again and this will facilitate that goal. We may also need a venue for commemorating special events, down the line, ditto on that.

You can use this thread for critique about the shop or email me with special requests from the cafepress catalog if there is something** I haven’t put up yet. I can also do some swapping of the front/back location on teeshirts if you have a pressing need.

__
*and the nature of any cash-handling future behavior of Scientopia is not yet established.
**not the thong though, please, not the thong.