Ask the DM Blog Braintrust: Advice on First Study Section?
September 24, 2014
A query came it that is best answered by the commentariat before I start stamping around scaring the fish.
I’m a “newbie” heading to study section as an ESR quite soon…
I’d really, really appreciated it if you could do a post ona) your advice on what to expect and how to … not put my foot in my mouth
b) what in an ideal world you’d like newbies to achieve as SS members
Thoughts?
DrugMonkey 
September 24, 2014 at 1:31 pm
I remember my first study section meeting. My advice: Know what your assignments are and prepare both your written comments and talking points carefully. Then read the specific aims of the other applications and read more of the ones that you are interested in or for which you feel that you might have something to contribute. Parenthetically, I recall another newbie who did not know that grants were assigned to particular reviewers who tried to prepare thoroughly for all of them…not fun.
I would speak up if you feel you have something to say, but do not overdo it.
LikeLike
September 24, 2014 at 2:02 pm
what in an ideal world you’d like newbies to achieve as SS members
Just don’t be a numpty like the rest of ’em!!!
LikeLike
September 24, 2014 at 2:30 pm
Introduce yourself around when you get there. Tell a couple folks you know/know of on study section you’d appreciate their one on one feedback on if you are calibrated right. I always write the Chair and program officer seperately afterwards thanking them and asking them for their thoughts.
Bring your own fuckken food, cuz there won’t be any. Don’t throw your paper copy in the middle when done. Those days are over. Play Candy Crush with the volume off.
And sweet jaysus….don’t be funny. We just want to get the fuckke out. I’m talking to you PhysioProf.
LikeLike
September 24, 2014 at 2:57 pm
Dude, didn’t we write a whole fucken series on this back in the day?
LikeLike
September 24, 2014 at 3:01 pm
Your job isn’t to show how smart you are by finding as many problems with the grants as you can – wish someone had told me this before my first meeting.
Read the other reviewers’ critiques on your assigned proposals before you get there. Don’t be afraid to change your mind after reading them if they see something you didn’t, but don’t feel like you need to agree just to agree.
Bring the chair chocolate (but that one may not generalize).
LikeLike
September 24, 2014 at 3:34 pm
On a more practical note: bring a laptop/tablet. It wasn’t mentioned to me that scores would be entered online during the meeting (after each application is discussed). Guess things have advanced since the example “what happens at study section” videos were recorded.
LikeLike
September 24, 2014 at 5:11 pm
If you’re reviewing at CSR the cafeteria food isn’t that great but you probably won’t have time to go out to eat lunch. Therefore, don’t expect much though the view is nice. Bringing snacks is a good idea. Also, if the SRO “invites” the SS to dinner that evening, he/she won’t be paying. Expect network issues at CSR, which may interfere with entering your scores into eRA Commons.
LikeLike
September 24, 2014 at 5:53 pm
Go to the meeting 10 min in advance to set up and figure out how to access the internet. If you arrive right at 8am for the 8am meeting, you’ll be the only one trying to find a place to plug in your laptop cord, while they are starting the formal introductions.
Expect issues with the microphones (that you have to speak into because that’s how reviewers on the phone hear you and how program people in the next room hear you). We almost always have problems with some not working, or feedback, or volume is too low.
It’s not a formal affair, more like meeting clothes.
Also, I’ll second what SusieP said, don’t try to point out all the problems or potential problems. Instead try to figure out the critical issues to hash out. Remember that the chair is going to have to summarize the discussion the reviewers and group have, as soon as it is over. So give her something to work with.
LikeLike
September 24, 2014 at 7:11 pm
Read as many of the written critiques as you possibly can during the Read phase prior to the meeting. This is a tremendous jump start on understanding the culture, expectations, hot button issues, etc. It also usually helps increase your confidence about your own reviewing.
LikeLike
September 24, 2014 at 7:13 pm
It’s okay to go take a whiz. During Reviewer #2s turn is usually the place to bail on a review. Unless #1 starts repeating themselves.
LikeLike
September 24, 2014 at 7:22 pm
Don’t be afraid to speak up, chances are high that at least two other people are thinking the exact same thing.
Realize that nobody is the expert on everything. Even if they blather with great authoritah! (Don’t say a word, SP!)
Don’t read your written review in whole part- ever.
Go to the dinner unless there are super extenuating circumstances.
LikeLike
September 24, 2014 at 8:34 pm
Don’t go to the boat party. Its a trap.
LikeLike
September 25, 2014 at 4:24 am
So here’s something that I feel weird about. I’ve been invited to my first SS next month, and it’s the same SS that reviewed my A0 R01 at 12% last round. So my pending submission is really on the fence right now and resubmission planned for Nov.
Anyway, I feel honored and all that… but also self-conscious about serving on this SS without having my first R01. It’s not that I lack confidence as a reviewer, it’s just that I know people look up reviewers to see what they’ve done to deserve be in this position. It just doesn’t seem right to officially review other people’s R01s when NIH hasn’t seen fit to fund mine.
Plus, I have to admit that I’m sort of pissed to be in this position. I mean, I’m fighting tooth and nail to get my grant funded as an Asst Prof, then they ask me to serve. I realize it is an amazing learning experience but it is a HUGE time commitment at a very sensitive time (tenure clock, my own grant deadlines, etc.). And part of me wishes I could say “sure, I’ll do this, AS LONG AS…”. In the end, I felt obligated to accept the invitation. Definitely don’t want to piss off the SRO or PO at this stage. My only hope is that I will see the PO who is handling my grant, so I can beg some more.
So, is there anything that I can read into (relative to my grant) based on the invitation to ad hoc? Any advice on how to handle this situation?
(And please don’t tell me to just look on the bright side and ‘learn as much as you can’ from this tremendously valuable opportunity. I know how much of a privilege this is and I take it seriously. But my #1 concern right know is my imminently expiring tenure clock and literally keeping the lights on in the lab. I’ve run out of time to apply any valuable lessons from the SS at this point.)
LikeLike
September 25, 2014 at 5:38 am
You were invited to serve because your grant was fucken awesome. This is a great opportunity to impress the panel with your qualities as a scientist, which might have some bit of positive influence when they review your resubmission. BTW, what fucken IC isn’t funding 12%ile R01s of ESIs???
LikeLike
September 25, 2014 at 6:15 am
Thanks for the sentiment. Yes, I forgot that one of the direct benefits is of getting to know (and potentially leaving a good impression) on reviewers who will be looking at my A1 soon.
No ESI status for me! (MPI grant where the co-PI recently became ineligible). And my PO was not sympathetic in least on that front. Apparently, if he can’t claim credit officially for funding an ESI grant, then there’s no benefit/interest for them to do so.
Plus – Since it’s a grey area score on an A0, they are taking very “wait in line like everyone else” stance.
LikeLike
September 25, 2014 at 6:19 am
Why the fucken fucke did you submit a multi-PI grant with an ESI ineligible PI when you are ESI eligible? Someone should have told you not to do that.
LikeLike
September 25, 2014 at 7:11 am
@On the fence
Are you sure this same study section will review your A1? Will your serving on the panel not cause them to send your A1 to an ad-hoc group?
LikeLike
September 25, 2014 at 7:36 am
Will your serving on the panel not cause them to send your A1 to an ad-hoc group?
No, it will not. Serving as an ad hoc member of a study section does not create any conflict of interest, and the A1 will still be reviewed there.
LikeLike
September 25, 2014 at 8:11 am
I disagree enthusiastically with the claim that only those who have won R01 funding can review a grant properly. For the record.
LikeLike
September 25, 2014 at 8:19 am
My answer to b), btw, is be the change you want to see. It’s tricky b/c with just the ad hoc one time visit you don’t have much time to build credibility. But if something just doesn’t sit well with you, even if half the panel is nodding along, you should speak up.
LikeLike
September 25, 2014 at 8:48 am
Emphasizing what others have said here – the pre-meeting read phase is your friend. Use it to calibrate yourself against the other reviewers, especially those who reviewed the same proposals as yours.
I’m guessing you’ve been invited onto SS because of your perceived expertise in a given area. For that reason if a particular question comes up in review (even on a proposal that you’re not a 1/2/3 reviewer on) and it falls slap-in-the-lap of your area, then be prepared to chime in with an opinion. You may even be asked directly “what do you think about this?” Out of your 8-10 grants, probably 3 are directly in your area and the rest are peripheral, maybe a couple completely outside. BUT, there will always be grants in the big pile where you look over the list and think “hey that should have been assigned to me”. The assignment process is a mystery, but you should be reading all the other grants directly in your area, not just the assigned ones.
People read at different speeds, but it’s a good idea to start reading about a month out, and try to spend at least half a day on each proposal. So 10 proposals = a full work week of your time. Thus, block-off your calendar 2 days a week for the month before the meeting. Don’t do it too early, or you’ll have forgotten about everything by the time the meeting comes around.
Also people do this differently, but I like to read and scribble on all my assignments before writing a single word about them. If you write the full review for a proposal, then find similar mistakes (or a better way of doing something) in later one, then you have to go back and edit your reviews. It’s a better idea (my opinion, YMMV) to read everything and bench-mark the proposals against each other before diving into the writing.
Oh, and if no-one else said it, FFS get your reviews posted early. #1 way to piss off the SRO is being late.
LikeLike
September 25, 2014 at 9:30 am
@CPP
I knew it was putting myself behind the 8-ball to forego ESI status… but the truth is, out of 3 or so R01 projects where I am PI, this is the one that always had the best chance of funding. The percentile score confirmed this. I can’t exactly relegate the other PI on this one. I figured it was better to go in at multiPI than not at all.
LikeLike
September 25, 2014 at 9:59 am
I suspect we all remember our first review meeting. For good reason, your performance is being evaluated. The Chair and the SRO will chat afterwards and discuss new reviewers. So, be polite. Don’t get argumentative if someone disagrees. Take it as a learning experience. Frankly I learned a lot by reviewing proposals. You see what makes strong proposals and why other proposals are weak. Enjoy the experience.
LikeLike
September 25, 2014 at 11:21 am
I agree it is better to put your strongest foot forward than to worry about ESI eligibility.
LikeLike
September 25, 2014 at 12:43 pm
Other than what others have already said…
I always find it stressful to make sure I remember what I want to say when I’m summarizing, so I prepare a “my notes” version of the critique form with an extra paragraph (above the actual “Overall Impact” paragraph) with these four categories about the proposal described in bullet points: “Why”, “Who”, “What”, “How” (the Morgan Giddings steps). This part has my narrative about the proposal outlined so I can explain it to the panel, and I write it while I’m reading through the proposal (it also helps me understand it). I use that for the initial description of the proposal, and then give the strengths and weaknesses that I included in the “Overall Impact” section. I find it really helps me hit the important points and help others understand the proposal while keeping myself concise so I don’t ramble off track or forget something. I actually submit this (with the rest of the critique form sections filled in as I would in the final version) as my preliminary critique form, and I’ve never had an SRO complain or have anybody seem confused about it. But see below about revising the doc…
I also put the numbers for my scores next to the titles of the sections (i.e. Overall Impact, Significance…etc.) so I remember how enthusiastic (or not) I was feeling about things (and for the initial stating of preliminary scores). Otherwise I always forget and have to scramble to look them up on the IAR scoresheet when I’m called to give mine as review begins. I also take notes on any changes to my critique that come up during discussion so I don’t forget.
Key to all of these things is that I always have to do my revisions to my critiques right after the meeting ends. There’s usually time to just stay in the meeting room before heading out to catch my flight, or if not, I do it at the airport or on the plane. Otherwise any important changes I might have wanted to make to better communicate to the applicant go totally out of my brain. I also have to delete my “My Notes” section and the numbers next to the section titles so it doesn’t create extra work for the SRO.
LikeLike
September 25, 2014 at 1:31 pm
Nothing beats having been in the room for understanding the process. I wouldn’t commit to being a regular member before tenure, but it’s an experience that can’t be replaced, especially for things like figuring out how to interpret summary statements. If you haven’t had to write them, you wonder why reviewers can’t get their point across. Then it’s your turn and you start to understand why that’s not so easy.
I will repeat what was said above – do not read your critique verbatim. Don’t even try to cover every point or criterion. Concentrate only on the points that drove your score. If you are not the first reviewer, and the reviewers before you covered everything you have to say, the appropriate thing to do is to say “I agree with everything Drugmonkey and SusiePsycho have already said.” This really will not make you look weak or ill-informed. Too many people feel like they need to have a chance to talk since they spent so much time reviewing and writing the critiques. But if everything has been covered, there’s no need to rehash it.
LikeLike
September 25, 2014 at 2:54 pm
I agree with everything Drugmonkey and SusiePsycho have already said.
/things that never happened but should have
LikeLike
September 25, 2014 at 5:21 pm
Avoid writing the grant for the applicant by stating what he/she should have done. With the exception of some SBIRs, the applicants are pretty smart and know why they did it this way rather than that way. But they should explain why they did it this way.
It can be hard to judge budgets, but avoid cutting budgets because they seem too big (especially when monkeys are involved). The budget should be commensurate with the project. So if you think that the work can be done in 4 years, rather than 5, say so but be flexible because not everyone may agree. If you think that the work can be done by a post-doc and a tech instead of two of each, or if the PI and the subcontract PI are doing the same thing, say so. If it’s not clear what a key personnel’d role is, say so.
Don’t act like a newbie by offering to take everyone’s Starbucks order.
Don’t be a party boy by saying “Let’s wrap this up in time for happy hour.”
Don’t ask “Why didn’t you motherfuckers give me a better score last time?”
LikeLike
September 25, 2014 at 6:40 pm
Don’t quote lines from your own summary statements written by this panel to see who twitches.
LikeLike
September 26, 2014 at 1:48 am
http://drugmonkey.scientopia.org2008/06/11/study-section-act-i/
LikeLike
September 26, 2014 at 12:13 pm
Thanks for this, DM.
Speaking of twitching, I can so far say: watch self for a flare-up of imposter syndrome
LikeLike
September 26, 2014 at 12:16 pm
I would suggest that reading the critiques and seeing the ….variability….may help with understanding that they all are just folks, with knowledge and gaps thereof, just like anyone else. It’s going to be nervewracking but you’ll see…it’s pretty cool and I’ve always found study section members to be friendly and welcoming. especially to noobs.
LikeLike