Percentiles for NIH Grants are all rounded up

June 16, 2010

A reader who may or may not want to take credit Comrade PhysioProf has pointed me to an interesting factoid on the NIAID site. Page down to the section on “Comparison of NIH’s Old and New Peer Review Processes” and you will find:

[Old way] Percentiles range from 0.1 (best) to 99.5 (worst).
[New Way]Percentiles range from 1 to 99 in whole numbers. Rounding is always up, e.g., 12.1 percentile becomes 13.

So not only do we have scores clustering around the even-integer increments allowed reviewers, we have additional clustering of applications based on percentile rounding.
As far as a Program Officer is concerned, your 9.1 %ile application is the same as the next person’s 10.0%ile application.
Man, they really were serious about intentionally generating a lot of tied scores, weren’t they?

No Responses Yet to “Percentiles for NIH Grants are all rounded up”

  1. It was me! I found it! I KICK ASS!


  2. Cashmoney Says:

    It is at the point where we need some number jockey to make us up a new model of what disparity in discussion scores are really worth discussing. If the range is 2-3 for example, is there ever going to be an outcome where it matter what fraction of the panel goes for the 2 and what fraction goes for the 3?


  3. CWB Says:

    You may find this work interesting Cashmoney. It shows how capricious the cut-off can be sometimes.


  4. whimple Says:

    In a related vein, this article:
    which cites the above article does an excellent job spelling out the systematic problems facing NIH-funded science today.


  5. andrew Says:

    @1 I don’t think this was really Comrade PP, it is almost polite and has no “fucking”.
    Someone tell PP that there is an imposter about please


  6. […] a bit of an aside, we also learned along the way that percentile ranks are always rounded […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: