More thoughts on the new bullet point NIH summary statement

The change to a very brief, bullet point format (sample of old narrative version) for the written critique of NIH grant applications will have many effects, bad and good. One I’ve been pondering recently has to do with the review of applications for which a previous version has been reviewed. This could be for an A1 revision of a grant application which did not receive a fundable score the first time it was submitted or for a competing continuation of a grant which has already been funded.
This change of format is going to make these reviews really, really difficult.

As I have previously noted, the review of such applications takes place on the basis of incomplete information because the reviewer does not have access to the prior grant application (funded or not).

The reviewer of the amended application is provided with the most recent summary statement and the revised application. Period. There is no provision of the actual prior application(s) or prior summary statements (for an A2 revision) and certainly not the prior score ranges, the identity of the prior reviewers or anything else.
The first and biggest problem is the failure to provide the prior application to the reviewer to facilitate assessment of “responsiveness” to the prior critique. This means that a new reviewer for the application is basically guessing on the basis of the summary statement what was contained in the prior application.

In the past, there was a good chance that the prior critique would provide enough overview of the application that you had a good idea what was in it. Now this will not be the case, there will just be evaluation limited to the 2-3 most important points.
It is going to be impossible to tell whether a competing continuation application represents new directions from the original, whether the project did anything related to what it said it would, whether it exceeded the prior goals, whether the PI heeded any of the critical comments after getting funded, etc. It will be impossible to tell if a revised application actually represents any sort of revision or not.
This may be good or it might be bad, there are certainly things about this aspect of review that I think have significant positive and negative effects on the process. The one certainty is that it sure as heck is going to be difficult for reviewers to meet the responsibility to evaluate certain issues with competing continuation and revised applications.