Percentiles for NIH Grants are all rounded up

June 16, 2010

A reader who may or may not want to take credit Comrade PhysioProf has pointed me to an interesting factoid on the NIAID site. Page down to the section on “Comparison of NIH’s Old and New Peer Review Processes” and you will find:

[Old way] Percentiles range from 0.1 (best) to 99.5 (worst).
[New Way]Percentiles range from 1 to 99 in whole numbers. Rounding is always up, e.g., 12.1 percentile becomes 13.

So not only do we have scores clustering around the even-integer increments allowed reviewers, we have additional clustering of applications based on percentile rounding.
As far as a Program Officer is concerned, your 9.1 %ile application is the same as the next person’s 10.0%ile application.
Man, they really were serious about intentionally generating a lot of tied scores, weren’t they?

5 Responses to “Percentiles for NIH Grants are all rounded up”


  1. It was me! I found it! I KICK ASS!

    Like

  2. Cashmoney Says:

    It is at the point where we need some number jockey to make us up a new model of what disparity in discussion scores are really worth discussing. If the range is 2-3 for example, is there ever going to be an outcome where it matter what fraction of the panel goes for the 2 and what fraction goes for the 3?

    Like

  3. CWB Says:

    You may find this work interesting Cashmoney. It shows how capricious the cut-off can be sometimes. http://www.pnas.org/content/105/32/11076

    Like

  4. whimple Says:

    In a related vein, this article:
    http://iai.asm.org/cgi/content/full/77/3/929
    which cites the above article does an excellent job spelling out the systematic problems facing NIH-funded science today.

    Like

  5. andrew Says:

    @1 I don’t think this was really Comrade PP, it is almost polite and has no “fucking”.
    Someone tell PP that there is an imposter about please

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: