Here We Go!!

September 18, 2008

I just received an e-mail from NIH concerning the implementation of new peer review practices in the aftermath of the Enhancing Peer Review navel-gazing effort. Noteworthy implementation plans are excerpted inside the crack.

Allow Flexibility through Virtual Reviews: Pilots will be conducted in 2009 on the feasibility of using high-bandwidth support for review meetings to provide reviewers greater flexibility and alternatives for in-person meetings.
Improve Scoring Transparency and Scale: Review criteria-based scoring on 1 to 7 scale commences in May 2009. Reviewers will provide feedback through scores and critiques for each criterion in a structured summary statement.
Provide Scores for Streamlined Applications: In 2009, streamlined applications will receive a preliminary score.
Shorten and Restructure Applications: Shorter (12-page research plan) R01 applications (with other activity codes scaled appropriately) will be restructured to align with review criteria for January 2010 receipt dates.
Fund the Best Science Earlier and Reduce Need for Resubmissions: To ensure that the largest number of high quality and meritorious applications receive funding earlier and to improve system efficiency, NIH is considering separate percentiling of new and resubmitted applications and permitting one amended application.
Review Like Applications Together: NIH is establishing an Early Stage Investigator (ESI) designation. In 2009, NIH will evaluate clustering ESI applications for review. The same approach will be considered for clinical research applications.

I have said it before, and I’ll say it again, shortening the R01 to 12 pages is going to differentially harm new investigators, who basically need to overwhelm a study section with preliminary data. It is thus absolutely essential that new investigator R01 applications be reviewed in their own clusters.

No Responses Yet to “Here We Go!!”

  1. Lorax Says:

    I’ll play DM, I think shortening the R01 to 12 pages will differentially harm mid-career investigators (those looking at their first renewal). There are mechanisms being pushed to help new investigators; senior investigators get the benefit of the doubt with prestige often overshadowing grant proposal deficiencies. Only my opinion of course, but I guess we will see what happens.


  2. juniorprof Says:

    Personally, I like all the recomendations and look forward to the changes. Perhaps the ESI designation will take care of the 12 page thing for junior investigators. I think its a step in the right direction…


  3. juniorprof Says:

    Just to expand a bit, I also like the virtual reviews. I think the discussion forum idea was bad; however, I envision this one much differently. With the money saved on travel costs the NIH could afford to buy every study section member a big-ass LCD screen for the wall of your office with a hookup so that you can see everyone on the screen, in their offices, having a big happy chat. Sure, its not as good as getting everyone in the same room, but think of the savings over the long-term. That must mean more than a few extra grants per year. If we’re not anticipating funding going up again anytime soon we better get creative on stretching the money!


  4. jonathan Says:

    As a postdoc – my experience in this area is near to zero; but I do have a question. My understanding of the way study sections are operated is that they place a high priority on the confidentiality of the applicants beyond the walls of the conference room. Conflicts of interest are identified – and people are asked to leave. Note taking is limited, or prohibited. etc etc.
    Now, I no doubt expect that all the reviewers sign this or that agreement prior to ‘volunteering’ – but how will the privacy of the grant applications be preserved if each study section is located …well.. nowhere. More to the point – if the giant LCD’s are installed in Professor X’s office; who’s to say that the conversation couldn’t be recorded; or that the whole lab is listening in Just off camera; or worse… that the grant applicant is sitting just off camera, watching and listening in on the review process and passing little notes to their buddy Prof. X?
    I’m likely off base – but it seems that there is something very important about meeting face to face in terms of benefiting the grant applicants. Sure, … its a pain in the arse to fly to bethesda X number of times a year; but catering to the study section reviewers instead of the grant applicants seems like the wrong focus to me. Public service is about, … service, not latte’s and comfy chairs and kick’back sessions with your NIH approved 52″ LCD filled with 15 talking heads. OK, I jest – but what am I missing here?
    just my 2


  5. DSKS Says:

    “With the money saved on travel costs the NIH could afford to buy every study section member a big-ass LCD screen for the wall of your office with a hookup so that you can see everyone on the screen, in their offices, having a big happy chat.”
    Time and resources would only be saved if there was a way to prevent all those fancy screens form being hooked up to XBox Live every Friday afternoon.


  6. DrugMonkey Says:

    Lorax: huh? I don’t think I’ve gotten into the issue of the yr 5-10 person looking to renew or acquire that second (or third) R01, have I? Orac does that beat, though. Not that I don’t agree with you in part, I just think they (ok, “we”) are at as great a risk and new and transitioning investigators.
    jp@#2: I agree with PP. Shorter apps are going to disadvantage the newer investigators because they have fewer papers published to make the arguments for them. Unlike PP, I also think that the detailed research plan is a place for the new investigator to shine with really good grantsmanship. This is unrelated to my views on whether this sort of thing should impress, the fact is that it is a very important component of review.
    I’m skeptical on the change from in-person review to internet-based systems but I’m listening. The asynchronous chat room idea seems very idiotic to me, perhaps because of my experiences with blogs, newsgroups, etc. Simultaneous discussions hold more promise in my view.
    One of the more critical problems with this approach is that I think it will push even more towards the direction of only the assigned reviewers having meaningful input. In your office, nobody knows you are laughing your ass off reading instead of listening to the boring discussion if you hit ‘mute’…


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: