Percentiles for NIH Grants are all rounded up
June 16, 2010
A reader who may or may not want to take credit Comrade PhysioProf has pointed me to an interesting factoid on the NIAID site. Page down to the section on “Comparison of NIH’s Old and New Peer Review Processes” and you will find:
[Old way] Percentiles range from 0.1 (best) to 99.5 (worst).
[New Way]Percentiles range from 1 to 99 in whole numbers. Rounding is always up, e.g., 12.1 percentile becomes 13.
So not only do we have scores clustering around the even-integer increments allowed reviewers, we have additional clustering of applications based on percentile rounding.
As far as a Program Officer is concerned, your 9.1 %ile application is the same as the next person’s 10.0%ile application.
Man, they really were serious about intentionally generating a lot of tied scores, weren’t they?
DrugMonkey 
June 16, 2010 at 5:03 pm
It was me! I found it! I KICK ASS!
LikeLike
June 16, 2010 at 5:03 pm
It is at the point where we need some number jockey to make us up a new model of what disparity in discussion scores are really worth discussing. If the range is 2-3 for example, is there ever going to be an outcome where it matter what fraction of the panel goes for the 2 and what fraction goes for the 3?
LikeLike
June 17, 2010 at 8:42 am
You may find this work interesting Cashmoney. It shows how capricious the cut-off can be sometimes. http://www.pnas.org/content/105/32/11076
LikeLike
June 17, 2010 at 10:27 am
In a related vein, this article:
http://iai.asm.org/cgi/content/full/77/3/929
which cites the above article does an excellent job spelling out the systematic problems facing NIH-funded science today.
LikeLike
June 17, 2010 at 1:18 pm
@1 I don’t think this was really Comrade PP, it is almost polite and has no “fucking”.
Someone tell PP that there is an imposter about please
LikeLike
June 25, 2013 at 10:14 am
[…] a bit of an aside, we also learned along the way that percentile ranks are always rounded […]
LikeLike