My Research Rocks! (and Yours Sucks)

August 17, 2007

Frequent commentor Physioprof has the call: this made me laugh my ass off:

S/he’s referring to a News of the Week in the 17 Aug issue of Science which reports on a survey of NSF applicants and reviewers. Quote from the article authored by J. Mervis.

Sent last fall to everyone who submitted a research proposal to NSF in the past 3 years (more than half were also reviewers), the survey also paints a picture of the typical applicant. He’s someone (three-fourths are men) who underestimates his chances of success but would have a go regardless of the odds. He needs the money primarily to keep his lab intact and is prepared to try and try again if his initial application is rejected. He’s reviewed up to a half-dozen proposals for NSF in the past 12 months, sometimes cutting corners, and thinks that few contain potentially transformative ideas. Yet he believes his own research, if funded, stands a good chance of shattering the existing paradigm in the field.

As Physioprof notes, this accompanying figure tells us all we need to know, say about characters like this.


2 Responses to “My Research Rocks! (and Yours Sucks)”

  1. PhysioProf Says:

    None of us would *ever* overestimate the importance of our own research, and underestimate that of others. Only those losers over there would do that.


    Seriously, what made me laugh about the survey isn’t that it reveals that there are a bunch of delusional arrogant scientists out there who deserve our scorn. Rather it reveals a totally predictable universal feature of human nature. It is exactly the same bias that accounts for the fact that the vast majority of car drivers consider their driving skills better than average.

    The effect of this bias on peer review must be accounted for in the design of effective peer review systems.

    The NIH study section system does a much better job of this than journal peer review, because NIH reviewers are known to one another and have to justify their reviews to each other. Journal reviewers are anonymous even to each other, and their biases and self-interested motives are thus subject to much less restraint.


  2. Piled Higher, Deeper Says:

    What fraction of research needs to be “transformative”? the huge majority says that 25% or less of applications are “transformative”. if true, and on the glass-full side, this suggests that the only proposals getting funded are “transformative”. At least in NIH funding this is patently false. The relative fraction of the budget spent on competing renewals tells us this.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: