March 4, 2014
A communication to the blog raised an issue that is worth exploring in a little more depth. The questioner wanted to know if I knew why a NIH Program Announcement had disappeared.
The Program Announcement (PA) is the most general of the NIH Funding Opportunity Announcements (FOAs). It is described with these key features:
- Identifies areas of increased priority and/or emphasis on particular funding mechanisms for a specific area of science
- Usually accepted on standard receipt (postmarked) dates on an on-going basis
- Remains active for three years from date of release unless the announcement indicates a specific expiration date or the NIH Institute/Center (I/C) inactivates sooner
In my parlance, the PA means “Hey, we’re interested in seeing some applications on topic X“….and that’s about it. Admittedly, the study section reviewers are supposed to conduct review in accordance with the interests of the PA. Each application has to be submitted under one of the FOAs that are active. Sometimes, this can be as general as the omnibus R01 solicitation. That’s pretty general. It could apply to any R01 submitted to any of the NIH Institutes or Centers (ICs). The PAs can offer a greater degree of topic specificity, of course. I recommend you go to the NIH Guide page and browse around. You should bookmark the current-week page and sign up for email alerts if you haven’t already. (Yes, even grad students should do this.) Sometimes you will find a PA that seems to fit your work exceptionally well and, of course, you should use it. Just don’t expect it to be a whole lot of help.
This brings us to the specific query that was sent to the blog, i.e., why did the PA DA-14-106 go missing, only a week or so after being posted?
Sometimes a PA expires and is either not replaced or you have happened across it in between expiration and re-issue of the next 3-year version. Those are the more-common reasons. I’d never seen one be pulled immediately after posting, however. But the NOT-DA-14-006 tells the tale:
This Notice is to inform the community that NIDA’s “Synthetic Psychoactive Drugs and Strategic Approaches to Counteract Their Deleterious Effects” Funding Opportunity Announcements (FOAs) (PA-14-104, PA-14-105, PA-14-106) have been reposted as PARs, to allow a Special Emphasis Panel to provide peer review of the applications. To make this change, NIDA has withdrawn PA-14-104, PA-14-105, PA-14-106, and has reposted these announcements as PAR-14-106, PAR-14-105, and PAR-14-104.
This brings us to the key difference between the PA and a PAR (or a PAS):
- Special Types
- PAR: A PA with special receipt, referral and/or review considerations, as described in the PAR announcement
- PAS: A PA that includes specific set-aside funds as described in the PAS announcement
Applications submitted under a PA are going to be assigned to the usual Center for Scientific Review (CSR) panels and thrown in with all the other applications. This can mean that the special concerns of the PA do not really influence review. How so? Well, the NIDA has a generic-ish and long-running PA on the “Neuroscience Research on Drug Abuse“. This is really general. So general that several entire study sections of the CSR fit within it. Why bother reviewing in accordance with the PA when basically everything assigned to the section is, vaguely, in this sphere? And even on the more-specific ones (say, Sex-Differences in Drug Abuse or HIV/AIDS in Drug Abuse, that sort of thing) the general interest of the IC fades into the background. The panel is already more-or-less focused on those being important issues. So the Significance evaluation on the part of the reviewers barely budges in response to a PA. I bet many reviewers don’t even bother to check the PA at all.
The PAR means, however, that the IC convenes their own Special Emphasis Panel specifically for that particular funding opportunity. So the review panel can be tailored to the announcement’s goals much in the way that a panel is tailored for a Request for Applications ( RFA) FOA. The panel can have very specific expertise for both the PAR and for the applications that are received and, presumably, have reviewers with a more than average appreciation for the topic of the PAR. There is no existing empaneled population of reviewers to limit choices. There is no distraction from the need to get reviewers who can handle applications that are on topics different from the PAR in question. An SEP brings focus. The mere fact of a SEP also tends to keep the reviewer’s mind on the announcement’s goals. They don’t have to juggle the goals of PA vs PA vs PA as they would in a general CSR panel.
As you know, Dear Reader, I have blogged about both synthetic cannabinoid drugs and the “bath salts” here on this blog now and again. So I can speculate a little bit about what happened here. These classes of recreational drugs hit the attention of regulatory authorities and scientists in the US around about 2009, and certainly by 2010. There have been a modest but growing number of papers published. I have attended several conference symposia themed around these drugs. And yet if you do some judicious searching on RePORTER you will find precious few grants dedicated to these compounds. It it no great leap of faith to figure that various PIs have been submitting grants on these topics and are not getting fundable scores. There are, of course, many possible reasons for this and some may have influenced NIDA’s thinking on this PA/PAR.
It may be the case that NIDA felt that reviewers simply did not know that they wanted to see some applications funded and were consequently not prioritizing the Significance of such applications. Or it may be that NIDA felt that their good PIs who would write competitive grants were not interested in the topics. Either way, a PA would appear to be sufficient encouragement.
The replacement of a PA with a PAR, however, suggests that NIDA has concluded that the problem lies with study section reviewers and that a mere PA was not going to be sufficient* to focus minds.
As one general conclusion from this vignette, the PAR is substantially better than the PA when it comes to enhancing the chances for applications submitted to it. This holds in a case in which there is some doubt that the usual CSR study sections will find the goals to be Significant. The caveat is that when there is no such doubt, the PAR is worse because the applications on the topic will all be in direct competition with each other. The PAR essentially guarantees that some grants on the topic will be funded, but the PA potentially allows more of them to be funded.
It is only “essentially” because the PAR does not come with set-aside funds as does the RFA or the PAS. And I say “potentially” because this depends on their being many highly competitive applications which are distributed across several CSR sections for a PA.
*This is a direct validation of my position that the PA is a rather weak stimulus, btw.
As always when it comes to NIDA specifics, see Disclaimer.
February 19, 2014
…or maybe it is.
One of the things that I try to emphasize in NIH grant writing strategy is to ensure you always submit a credible application. It is not that difficult to do.
You have to include all the basic components, not commit more than a few typographical errors and write in complete sentences. Justify the importance of the work. Put in a few pretty pictures and plenty of headers to create white space. Differentiate an Aim from a hypothesis from an Experiment.
Beyond that you are often constrained by the particulars of your situation and a specific proposal. So you are going to have to leave some glaring holes, now and again. This is okay! Maybe you are a noob and have little in the way of specific Preliminary Data. Or have a project which is, very naturally, a bit of
a fishing expedition hypothesis generating, exploratory work. Perhaps the Innovation isn’t high or there is a long stretch to attach health relevance.
Very few grants I’ve read, including many that were funded, are even close to perfect. Even the highest scoring ones have aspects that could readily be criticized without anyone raising an eyebrow.
The thing is, you have to be able to look at your proposal dispassionately and see the holes. You should have a fair idea of where trouble may lie ahead and shore up the proposal as best you can.
No preliminary data? Then do a better job with the literature predictions and alternate considerations/pitfalls. Noob lab? Then write more methods and cite them more liberally. Low Innovation? Hammer down the Significance. Established investigator wanting to continue the same-old, same-old under new funding? Disguise that with an exciting hypothesis or newish-sounding Significance link. (Hint: testing the other person’s hypothesis with your approaches can go over great guns when you are in a major theoretical dogfight over years’ worth of papers.)
What you absolutely cannot do is to leave the reviewers with nothing. You cannot leave gaping holes all over the application. That, my friends, is what drops you* below the “credible” threshold.
Don’t do that. It really does not make you any friends on the study section panel.
*This is one case where the noob is clearly advantaged. Many reviewers make allowances for a new or young-ish laboratory. There is much less sympathy for someone who has been awarded several grants in the past when the current proposal looks like a slice of Swiss cheese.
December 13, 2013
Apparently Potnia is going to do a series over at Mistress of the Animals blog. This statement is one of those mnemonic gems you should paste on your monitor edge.
Aims should be general enough to require a project (1-2 papers per aim), but specific enough that they are a project.
Elevating a comment from Chris:
IMPORTANT UPDATE: My study section (originally scheduled for Oct 1-2, then cancelled until Feb) is now back on! I just heard from our chair that we will be holding an online meeting in the next few days (thanks for the advance notice). According to our chair, the “significant pressure” put on the NIH from the extramural community has led them to reconsider their decision to cancel everything and bring things back online. I have no authority on this, but would assume that this will happen across the board. So take heart, all may not be lost for this round…
I am likewise hearing rumor that the CSR is reconsidering what they are going to do.
Stay tuned folks, this ride ain’t over yet.
Updated: ps, your comments at Rock Talk can’t help but be viewed as part of the “significant pressure”. Go to it.
October 17, 2013
There’s at least one early indicator of what is going to happen with the study section rounds that were cancelled because of the government shutdown.
This has all sorts of implications, one of which was brought up by Professor Jentsch in a subsequent tweet. It is related to the NOTice just issued which says that all October deadlines will be pushed forward into the “November timeframe”.
Let’s say your submitted a new proposal in June or perhaps a revised or competing renewal proposal in July. And like a busy little beaver you’ve continued to work on the project. Perhaps you have some excellent new data that further supports your awesome ideas and the killer experiments that you’ve proposed.
There is only one thing to do. Pull the grant from consideration and resubmit it, with the new data, once the NIH picks some November deadlines.
Remember, o ye NIH grant seeking Readers, that your peers are supposed to be reviewing the grants you submitted in the June/July interval right about now. And thanks to the House of GOP shuttering the Federal government, the study sections are being cancelled.
You see, maybe a Continuing Resolution will be passed….tonight? or tomorrow morning? or Friday at 2?
And then the study section meetings for next week will be back on.
So the reviewers have to struggle along and finish up their jobs as best they can, not knowing if it will be for the meeting that is scheduled….or if it will be some sort of replacement meeting later in the month or year.
From what I am hearing, your friendly peers are stepping up to the damn plate and getting their grants reviewed.
Even without access to eRA Commons (where all the grants are stored, hardly anyone bothers to get a CD anymore). So the “read phase” that is supposed to take place the week before a study section meeting is going to be difficult. Hard to read the other reviewers’ comments on the grants you are assigned because you don’t know who they are! All that is supposed to be automatic on Commons you see. Well, from what I hear around the campfire, the sections are doing what they can, no doubt with heroic work from the Chairs and a little illegal subterranean rogue work from the government employee SROs.
I thank you all.
As we all know, much of the evaluation of scientists for various important career purposes involves the record of published work.
More is better.
We also know that, at any given point in time, one might have work that will eventually be published that is not, quiiiiiite, actually published. And one would like to gain credit for such work.
This is most important when you have relatively few papers of “X” quality and this next bit of work will satisfy the “X” demand.
This can mean first-author papers, papers from a given training stint (like a 3-5 yr postdoc) or the first paper(s) from a new Asst Professor’s lab. It may mean papers associated with a particular grant award or papers conducted in collaboration with a specific set of co-authors. It could mean the first paper(s) associated with a new research direction for the author.
Consequently, we wish to list items that are not-yet-papers in a way that implies they are inevitably going to be real papers. Published papers.
The problem is that of vaporware. Listing paper titles and authors with an indication that it is “in preparation” is the easiest thing in the world. I must have a half-dozen (10?) projects at various stages of completion that are in preparation for publication. Not all of these are going to be published papers and so it would be wrong for me to pretend that they were.
Hardliners, and the NIH biosketch rules, insist that published is published and all other manuscripts do not exist.
In this case, “published” is generally the threshold of receiving the decision letter from the journal Editor that the paper is accepted for publication. In this case the manuscript may be listed as “in press“. Yes, this is a holdover term from the old days. Some people, and institutions requiring you to submit a CV, insist that this is the minimum threshold.
But there are other situations in which there are no rules and you can get away with whatever you like.
I’d suggest two rules of thumb. Try to follow the community standards for whatever the purpose and avoid looking like a big steaming hosepipe of vapor.
“In preparation” is the slipperiest of terms and is to be generally avoided. I’d say if you are anything beyond the very newest of authors with very few publications then skip this term as much as possible.
I’d suggest that “in submission” and “under review” are fine and it looks really good if that is backed up with the journal’s ID number that it assigned to your submission.
Obviously, I suggest this for manuscripts that actually have been submitted somewhere and/or are out for review.
It is a really bad idea to lie. A bad idea to make up endless manuscripts in preparation, unless you have a draft of a manuscript, with figures, that you can show on demand.
Where it gets tricky is what you do after a manuscript comes back from the journal with a decision.
What if it has been rejected? Then it is right back to the in preparation category, right? But on the other hand, whatever perception of it being a real manuscript is conferred by “in submission” is still true. A manuscript good enough that you would submit it for consideration. Right? So personally I wouldn’t get to fussed if it is still described as in submission, particularly if you know you are going to send it right back out essentially as-is. If it’s been hammered so hard in review that you need to do a lot more work then perhaps you’d better stick it back in the in preparation stack.
What if it comes back from a journal with an invitation to revise and resubmit it? Well, I think it is totally kosher to describe it as under review, even if it is currently on your desk. This is part of the review process, right?
Next we come to a slightly less kosher thing which I see pretty frequently in the context of grant and fellowship review. Occasionally from postdoctoral applicants. It is when the manuscript is listed as “accepted, pending (minor) revision“.
Oh, I do not like this Sam I Am.
The paper is not accepted for publication until it is accepted. Period. I am not familiar with any journals which have accepted pending revision as a formal decision category and even if such exist that little word pending makes my eyebrow raise. I’d rather just see “Interim decision: minor revisions” but for some reason I never see this phrasing. Weird. It would be even better to just list it as under review.
Final note is that the acceptability of listing less-than-published stuff on your CV or biosketch or Progress Report varies with your career tenure, in my view. In a fellowship application where the poor postdoc has only one middle author pub from grad school and the two first author works are just being submitted…well I have some sympathy. A senior type with several pages of PubMed results? Hmmmm, what are you trying to pull here. As I said above, maybe if there is a clear reason to have to fluff the record. Maybe it is only the third paper from a 5 yr grant and you really need to know about this to review their continuation proposal. I can see that. I have sympathies. But a list of 8 manuscripts from disparate projects in the lab that are all in preparation? Boooo-gus.
July 9, 2013
Interesting exchange on the twitts today with someone who is intimating that the process of selecting peers to serve as grant reviewers on NIH study sections requires some transparency and fixing.
As my longer term Readers are aware, my main objection along these lines is that I think Assistant Professors should not be excluded and that the purge urged on by Toni Scarpa back some years ago was misguided. I will also venture that I think it is ridiculous that the peer review pool is limited to those Professorial rank people who have already won funding from the NIH (for the most part). If really pressed, I’ve been know to suggest that it is even unfair that the more senior postdoc types who have not yet won a faculty-level appointment cannot review grants.
Other than that, I am generally down with the official mandates to seek ethnic/racial, gender and geographic representation on panels. My personal experience has been that the SROs do a pretty good job at this. Also, because of these factors, I have found that the types of institutions represented spans the range pretty well..small mostly teaching profs, big Research Uni profs, research insitutes of various sizes, public Unis, private Unis, Med Schools and academic departments.
So it is with some confusion that I read someone asserting that there is a problem with who is selected.
My query of the day, therefore, is to ask you if you know of people who seek to serve on study section but cannot seem to land an invite. Alternately, do you know of categories of investigators that are routinely overlooked?
“The proposal is extremely well-written and clear.”
This is my query of the day to you, Dear Reader.
We’ve discussed the basics in the past but a quick overview.
1) Since the priority score and percentile rank of your grant application is all important (not exclusively so but HEAVILY so) it is critical that it be reviewed by the right panel of reviewers
2) You are allowed request in your cover letter that the CSR route your NIH grant application to a particular study section for review.
3) Standing study section descriptions are available at the CSR website as are the standing rosters and the rosters for the prior three rounds of review (i.e., including any ad hoc reviewers).
4) RePORTER allows you to search for grants by study section which gives you a pretty good idea of what they really, really like.
5) You can, therefore, use this information to slant your grant application towards the study section in which you hope it will be reviewed.
A couple of Twitts from @drIgg today raised the question of study section “fit”. Presumably this is related to an applicant concluding that despite all the above, he or she has not managed to get many of his or her applications reviewed by the properly “fitting” panel of reviewers.
This was related to the observation that despite ones’ request and despite hitting what seem to be the right keywords it is still possible that CSR will assign your grant to some other study section. It has happened to me a few times and it is very annoying. But does this mean these applications didn’t get the right fit?
I don’t know how one would tell.
As I’ve related on occasion, I’ve obtained the largest number of triages from a study section that has also handed me some fundable scores over the past *cough*cough*cough* years. This is usually by way of addressing people’s conclusion after the first 1, 2 or maybe 3 submissions that “this study section HATES me!!!“. In my case I really think this section is a good fit for a lot of my work, and therefore proposals, so the logic is inescapable. Send a given section a lot of apps and they are going to triage a lot of them. Even if the “fit” is top notch.
It is also the case that there can be a process of getting to know a study section. Of getting to know the subtleties of how they tend to feel about different aspects of the grant structure. Is it a section that is really swayed by Innovation and could give a fig about detailed Interpretations, Alternatives and Potential Pitfalls? Or is it an orthodox StockCritiqueSpewing type of section that prioritizes structure over the content? Do they like to see it chock full of ideas or do they wring their hands over feasibility? On the other side, I assert there is a certain sympathy vote that emerges after a section has reviewed a half dozen of your proposals and never found themselves able to give you a top score. Yeah, it happens. Deal. Less perniciously, I would say that you may actually convince the section of the importance of something that you are proposing through an arc of many proposal rounds*.
This leaves me rather confused as to how one would be able to draw strong conclusions about “fit” without a substantial number of summary statements in hand.
It also speaks to something that every applicant should keep in the back of his or her head. If you can never find what you think is a good fit with a section there are only a few options that I can think of.
1) You do this amazing cross-disciplinary shit that nobody really understands.
2) Your applications actually suck and nobody is going to review it well.
3) You are imagining some Rainbow Fairy Care-a-lot Study section that doesn’t actually exist.
What do you think are the signs of a good or bad “fit” with a study section, Dear Reader? I’m curious.
*I have seen situations where a proposal was explicitly mentioned to have been on the fourth or fifth round (this was in the A2 days) in a section.
May 10, 2013
The attitude “I’m happy to debate” should be replaced with “I’m happy to explain”.
and there it is.
April 30, 2013
Anyone out there ever reviewed SBIR or STTR grants for the NIH? Any thoughts on what seems to be most important, common pitfalls and the like? Any thoughts on how the review discussion tends to differ from standard R01 review?
I have essentially zero experience with these mechanisms and some reader was asking…..
January 31, 2013
Combative responses to prior review are an exceptionally stupid thing to write. Even if you are right on the merits.
Your grant has been sunk in one page you poor, poor fool.
January 31, 2013
There is nothing like a round of study section to make you wish you were the Boss of ALL the Science.
There is just soooo much incredible science being proposed. From noob to grey beard the PIs are coming up with really interesting and highly significant proposals. We’d learn a lot from all of them.
Obviously, it is the stuff that interests me that should fund. That stuff those other reviewers liked we can do without!
Sometimes I just want to blast the good ones with the NGA gun and be done.
Notice of Grant Award
January 19, 2013
Do not EVER spend so much time geeking away about the amazingly swell trees that you will be characterizing that you forget to convince the reviewer that the forest itself holds any interest. And I mean ANY interest…..Seriously dudes, I’m trying to help you out here but you are giving me absolutely nothing to work with. There is barely any point in me even reading your experimental manipulations….I can tell already there is no overall justification for doing them in the first place!