On showing the data

September 5, 2013

If I could boil things down to my most fundamental criticism of the highly competitive chase for the “get” of a very high Impact Factor journal acceptance in science, it is the inefficiency.

GlamourDouchery of this type is an inefficient way to do science.

This is because of several factors related to the fundamental fact that if the science you conduct isn’t published it may as well never have happened.

Science is an incremental business, ever built upon the foundations and structures created by those who came before. And built in sometimes friendly, sometimes uneasy collaboration with peers. No science stands alone.

Science these days is also a very large enterprise with many, many thousands of people beavering away at various topics. It is nearly impossible to think of a research program or project that doesn’t benefit by the existence of peer labs doing somewhat-related work.

Consequently, it is a near truism that all science benefits from the quickest and comprehensive knowledge of what other folks are doing.

The “get” of an extremely high Impact Factor Journal article acceptance requires that the authors, editors and reviewers temporarily suspend disbelief and engage in the mass fantasy that this is not the case. The participants engage in the fantasy that this work under consideration is the first and best and highly original. That it builds so fundamentally different an edifice that the vast majority of the credit adheres to the authors and not to any part of the edifice of science upon which they are building.

This means that the prospective GlamourArticle authors are highly motivated to keep a enormous amount of their progress under wraps until they are ready to reveal this new fine stand-alone structure.

Otherwise, someone else might copy them. Leverage their clever advances. Build a competing tower right next door and overshadow any neighboring accomplishments. Which, of course, builds the city faster….but it sure doesn’t give the original team as much credit.

The average Glamour Article is also an ENORMOUS amount of work. Many, many person years go into creating one. Many people who would otherwise get a decent amount of credit for laying a straight and true foundation will now be entirely overlooked in the aura of the completed master work. They will never become architects themselves, of course. How could they? Even if they travel to Society Journal Burg, there is no record of them being the one to detail the windows or come up with a brilliant new way to mix the mortar. That was just scut work for throwaway labor, don’t you know.

But the real problem is that the collaborative process between builders is hindered. Slowed for years. The dissemination of tools and approaches has to wait until the entire tower is revealed.

Inefficiency. Slowness. These are the concerns.

Sure, it is also a problem that the builders of the average Glamour Article tower may not share all their work even after the shroud has been removed. It would be nice to let everyone know just where the granite was found, how it was quarried and something about the brand new amazing mortar that (who was that anonymous laborer again? shrug) created. But there isn’t really any pay for that and the original team has moved on. Good luck. So yes, it would be good to require them to show their work at the end.

Much, much more important, however, is that they show each part of the tower as it is being created. I mean, no, I don’t think people need to work with a hundred eyes tracking their every move. I don’t think every little mistake has to be revealed, nor do I think we necessarily need to know how each laborer holds her trowel. But it would be nice to show off the foundation when it is built. To reveal the clever staircase and the detailing around the windows once they are installed. Then each sub-team can get their day in the sun. Get the recognition they deserve.

[And if they are feeling a little oppressed, screw it, they can leave and take their credit with them. And their advances in knowledge can be spread to another town who will be happy to hire this credentialed foundation builder instead of some grumpy nobody who only claims to have built a foundation.]

The competition for Glamour Article building can’t really catch up directly, after all it takes a good bit of work to lay a foundation or create a new window design. They can copy techniques and leverage them, but there is less chance of an out and out scoop of the full project.

So if the real problem is inefficiency, Dear Reader, the solution is most assuredly the incremental reveal of progress made. We don’t need to watch the stirring and the endless recipes for mortar that have been attempted, we just need to know how the successful one was made. And to see the sections of the tower as they are completed.

Ironically enough, this is how it is done outside of GlamourCity. In Normalville, the builders do show their work. Not all of it in nauseating detail but incrementally. Sections are shown as they are completed. It is not necessary to wait for the roof to be laid to show the novel floorplan or for the paint to be on to show the craft that went into the floor joists.

This is a much more efficient way to advance.

It has to be, since resources are scarce and people in Society Burgh kind of give a shit if one of their neighbors is crushed under a block of limestone. And care if an improperly supported beam cracks and they have to get a new one.

This is unlike the approach of Glamour City where they just curse the laborer and draft three new ones to lift the block into place. And pull another beam out of their unending pile of lumber.

BikeMonkey Post
The sport of cross-country running is a fine one. I mean, who doesn’t get behind a brisk run in the woods for 5 km at a stretch? So much more interesting than going around a 400 m track or running on the city streets.

Cross-country features paths and forest roads, asphalt and grass sections. Up hill and down dale. Sometimes it gets a little narrow and the footing can be slippery or rocky…but for the most part it isn’t all that technical. We’re not talking parcour here.

As with most running, there’s no cheating, faking or room for much strategy. You run your race, you work your pace and you try to outrun the rest of the pack. Guys who were fastest on my cross-country team in highschool were the fastest on the road and the fastest on the track. There was maybe a tiny margin for guys who were slightly lighter to put the relative hurt on in the climbs, for some guys to downhill slightly faster or for the longer and shorter limbed to have slight advantages on the curvy or the straight. But the margins were slim.

Fast is fast in running.

Mountain biking is a different story although the differences from running aren’t as extreme as in road cycling. MTB racing tends to be a bit more of a solo effort with little advantage to be gained from sitting in a giant pack of riders. And as with cross-country running, the terrain varies from asphalt to grass to trails. Wide fire-roads and single-track paths, smooth, rutted or rockey. Sustained miles of mountain ascending, short power-climbs and flat terrain. What is up must come down and you have to be good at getting a little sideways now and again.

I always was about 10 pounds over the top end of the ideal range for running and biking competition from highschool through college. Ascending climbs of any duration was never my strong suit. The MTB racing phase came along later in life and I’d put on a good 8 more pounds by that point. So the sustained climb of anywhere from 10 to 20 minutes duration that occurred in the typical MTB course was a big hurdle.

I was not usually in contention to win MTB races. Top 10 finishes tended to be my goal. Sometimes you just have to be realistic about your individual abilities within the competition you have chosen to join.

One race day we faced a course that started and finished with a paved park road, maybe a mile long. On the out-leg, the course jumped off the road to a track-and-a-half with a creek crossing right before the main climb. Which was steep. Being a MTB race, the pack rolled out kind of slowly on the paved road. No sense in pulling the peleton along with you in a MTB race, right?

About 300 m from the start of the trail section, I just nailed it. Like a sprint finish, I mean. I’m sure everyone else thought I was insane…but there wasn’t anyone near me until I was well into the climb. Per usual, the climbers eventual caught and passed me and I kind of lost track of how many folks were in front of me. With multiple start groups on the same course, it can be hard to tell when you are just buried trying to go fast. The downhill section was pretty technical and I am sure I passed some folks here and there, some with flat tires. But still no idea what the race looked like until I crossed the line first.

It turned out the hole shot was everything in that particular race. The most important thing was getting a big enough lead at the start that my deficits at climbing were minimized. Anyone smart enough to get on my wheel at the beginning couldn’t get by and took a big face full of creekwater and a bad line. Climbers who reacted too late, or were caught up in the pack, let me have a few extra precious minutes on the climb. Then, in the rest of the course the matter was decided by the other parts of the MTB racer’s skill set. Descending, handling, power climbs and the final hammer to the finish on paved road. Some of those things were to my advantage.

Minimize the impact of your deficits so that your strengths can carry your through. Sometimes this requires advance planning* to pull off. It almost always benefits from full commitment to the initial move.

Somewhere, between cross country running and MTB racing, there is a lesson for science careers.
*yeah, I’d planned the holeshot when I previewed the course earlier.

Thought of the Day

August 29, 2013

What “best predicts” the success of a junior scientist is handing her a laboratory and R01 level funding.

The notion that past publication record predicts anything independently from these two factors is arrant nonsense.

Academic comfort levels

July 23, 2013

I have a question for you, Dear Reader.

During what fraction of the time you spent at each major career stage in academics – undergrad, grad school, postdoc, faculty level (TT or no, plz specify) – did you feel comfortable?

Not did you feel it was easy exactly, but did you feel as though you had it handled? As though there was little doubt you were doing a good job of what you were expected to do.

For me, undergrad all 4 yrs, grad school 3 yrs, postdoc maybe a scattered 2 yrs total time.

At the faculty level maybe my first three years and again for maybe 6 mo last year.

For faculty, make special note of the tenure decision- were you feeling comfortable in the few years leading up to that?

You no doubt remember the advice of Scott Kern and that of Mu-ming Poo with respect to what it takes to survive and thrive in an academic career.

It boils down to “work harder, no I don’t care how much you are already working, you need to work more. and baby’s are dying of cancer and something, something growth cones. so there”

One Radhika Nagpal has written a counter argument from her current position as Full Professor at Harvard.

Go read

The-Awesomest-7-Year-Postdoc or: How I Learned to Stop Worrying and Love the Tenure-track-faculty-life

So with some humor to balance my fear, here’s goes my confession:

Seven things I did during my first seven years at Harvard. Or, how I loved being a tenure-track faculty member, by deliberately trying not to be one.

  • I decided that this is a 7-year postdoc.
  • I stopped taking advice.
  • I created a “feelgood” email folder.
  • I work fixed hours and in fixed amounts.
  • I try to be the best “whole” person I can.
  • I found real friends.
  • I have fun “now”.

right now. I can wait.

Here’s what I think about

Work Life balance

The paradox of the poles of the work-life balance discussion in Nature News is this.

Nobody who succeeds at work and then claims balance really knows if they just got very, very lucky in their career.

Nobody who works around the clock and drives their lab to similar performance knows that this was *required*.

The hidden side is that both balanced and St Kern/Poo’d types also fail in their careers.

UPDATE 09/06/11: Plus also, StKern/Poo’d types can also succeed in their careers really, really well….and still fail to cure cancer.

And about Protecting Your Time

Yes, I for damn sure wish for more hours in the day. Yes. Of course. And at each and every major stage there were things being neglected so that I could pursue some other thing. Either in the proximal, days to weeks, or in the long-haul, years to decades(!), perspective. But I have never been an obsessive and any fair read would fail to find any major imbalance.

How did I do it?

I think the most useful and general approach is that you have to be willing to fail.

Let me say it again: YOU HAVE TO BE WILLING TO FAIL!!!!!

And one of my accidental mentors taught me this:

There were several areas in which I picked up either positive (“gee, that seems useful”) or negative (“not gonna go there”) PI patterns from this person. One of the former was this guy’s role as father and scientist. Whenever one had to find this PI, if he wasn’t around because of father duties his whole lab knew about it. “Oh, he’s at Opening Day.” or “He had a sick kid today, he’ll be back later”. or “He’s taking his kid to [SportingActivityX]”. This guy has a perfectly viable career with nice pubs, great NIH grant support, always seems to have at least 4-5 postdocs and a similar number of techs, serves study sections, organizes symposia, etc. In short, he’s well respected and does not appear to have paid any obvious sort of career price to date. This had a great impact on YHN as I was transitioning both as PI and father.
The power of this example for me was basically “Screw it, if he doesn’t worry about being known at work as a guy who takes his role as father seriously then I’m not going to worry about it either”.

The interesting thing, which is emerging on the Twitters, is that Professor Nagpal’s advice is really no different from a host of women who write “You can’t have it all” screeds and lament the fact “You have to be twice as good as a man to succeed”. The ones that describe not being their enough for their kids or the ones describing being their too much and failing. The men too, although their career advice is thinner. See Kern and Poo links at the top–they describe how “it has to be” because this is how it was for them.

And you know what? It is ALL true. All of it. Because these are personal anecdotes tied to the career history and success of the person giving the advice or reciting their life-story in academics (or other professional life).

My advice, scattered throughout this blog, is no different.

And just as the winners of global social politics write the histories of what happened and for what reason, likewise those who have succeeded in academic science tend to write the prescriptions for our careers.

I don’t think Professor Nagpal has written some amazing revelation here. It is not hugely different than arguments that I make myself on this very blog.

It is true that she has been very, very successful by appearance.

But there are also, I would wager, tons of people who made the choices she made and were denied tenure. Many who washed out of academic professordom entirely. And that is the point about this advice which runs from Nagpal’s 50 hrs max to the Kern/Poo maxim of “always more”.

It is no guarantee.

“The proposal is extremely well-written and clear.”

Note for PIs

July 2, 2013

If your lab requires a “weekly support group” meeting, there is no scenario wherein you are doing it right.

I think I’ve done a post on this before but it arose again on the Twitts today.

As a lab head, I give all the trainees access to our funded grant proposals..and often the applications I am working on. I would certainly give them to someone in my lab upon request if I had forgotten to email something to them (or not bothered in the case of our current firehose of applications).

I am at a considerable loss to imagine why any lab head would have a problem doing this.

Does anyone have any new insight on why a PI would not make the funded grant proposals available? Doesn’t everyone in the lab need to have at least some understanding of what is supposed to be accomplished?

Now, benign neglect, I can sort of understand. Not all the PIs out there understand how important it is to get the trainees thinking about the grant cycle as early as possible. Opinions vary on that. Some would rather trainees not be “distracted”. I get that…but I think it outmoded.

But outright refusal to hand the grant over if asked? That is odd….almost to the point of suspecting shenanigans.

Unfunded Overhead

June 6, 2013

It struck me today

thanks to the referenced comment from Jim Woodgett that we’ve never really had a discussion of unfunded overhead situations, despite several discussions of overhead rates in the ongoing effort to determine TheRealProblemTM with NIH budgets these days. It is worth bringing up, particularly for anyone who might be job seeking or negotiating in the near future. As we continue, you’ll see what you need to ask about, and what you need to get in writing along with your job offer.

As a brief introduction the overhead (or Indirect Costs; IDC) associated with a research grant award is the amount that disappears into the University, research institution (or what have you) instead of going into the PI’s account to spend.

When it comes to federal awards from the NIH (and some other agencies beloved of my Readership) the IDC rate varies across the Universities, research institutes and varied other applicant institutions. For discussion’s sake, I’ll throw out that the general rate for larger public Universities is about 56%. Smaller (private) Universities and not-for-profit research institutes tend to have higher ones with overhead rates of over 80% not uncommon. Rumors abound of 100% overhead rates but I’ve not directly seen one of those myself. To my recollection. This research crossroads site used to have a handy database of the federally-negotiated overhead rates but it has been down for some time now and I suspect it is defunct. I don’t know where they were scraping their data from but presumably these overhead rates are public info.

There are numerous non-federal sources of funding that a given PI might see as appropriate to pursue for her laboratory. Contracts with biotech or Big Pharma companies. Larger or smaller disease focused foundations (American Heart, Michael J Fox). Less-focused foundations (like Bill and Melinda Gates Foundation). Local philanthropic donors. State foundations or funds (like those diverted from tobacco or alcohol taxes). In many, if not most, cases these funding streams do not wish to pay your University the federally-negotiated overhead rate.

The differential can be large. Such as a foundation that will pay 10% maximum…and your federal rate sits at 70%. Perhaps a donor doesn’t want to pay any overhead at all and expects the full donation to go into the research lab’s coffers.

The ways that Universities and research institutions deal with this issue varies considerably. Across institutions, of course. But also within an institution depending on the money source, the amount of funding involved, the identity of the PI, etc.

The best case scenario for PIs is the institution that doesn’t care. Money is money and….they’ll take it. I’ve heard rumor of such things but it is fantasy as far as I am concerned.

What is more common is that the University has a way to cover the “unfunded overhead” situation to make it appear that the full federally negotiated rate is being applied to each and every grant of consequence*. Sometimes this is accomplished through the mumbo-jumbo of money being fungible and the University simply using their endowment proceeds or some other source of funds not easily connected to a grant to “cover” the overhead. This is good, if you can get it. That is, if your University has a default, no-questions-asked way to do this for a given source of grant support. That’s a supportive place to be.

Considerably less-good is the situation where the PI is supposed to “cover” this for herself. Now sometimes it is the case that the Chair of the Department covers it through a slush fund and, obviously, this would be a more limited pool of money. Consequently, the Chair has to balance who gets the slush. This leaves a lot of room for shenanigans having to do with departmental politics. A lot of room for problems based on how many faculty are trying to tap this pot of slush money in a given year. This is why you, as a prospective new hire, need to ask how these situations are covered and get as much in writing as you can.

There are two remaining horrible options which I hesitate to rank.

Some Universities will pull the overhead out of the new-hire’s startup funds. That’s a dicey game for a new faculty member to play. It might be worth it, it might not. Why would it be worth it? Well, that startup is a fixed, nonrenewable pool of money that is supposed to get you launched, right? This means, in essence, to help you secure a grant. Having grant funding awarded to your lab is a good thing and catapults you into the “funded investigator” category. Depending on the size of it, your use of startup to secure that award, instead of continuing the uncertain game of generating more preliminary data, may be advisable. You just have to look at the leverage that contributing startup to the unfunded overhead will give you.

Some places (and here I find the very high overhead, small not-for-profit research institutes to raise their heads) simply refuse to let faculty (even new hires) apply for anything that doesn’t come with full overhead.

Yes, this seems an unbelievably stupid policy and a way to cripple the prospects of your newly-hired faculty, but there you have it.

For anybody on the job market that is reading this, the conclusions are clear. If the unfunded overhead policies of your prospective institutions are not handed to you when you visit, ask. Determining what grants you will and will not be allowed to apply for in your first few years (or across your career) should not be left up to the (entirely logical) assumption that any grant available is attractive to your University.

ETA: A comment from Jim Woodgett

In essence, NIH subsidizes those agencies and philanthropists that don’t allow or who restrict overhead.

reminded me I forgot to address why the Universities are doing this. My assumption is that if the federal negotiators thought this statement sufficiently true, they would lower the IDC rate for that University. As I said, my assumption. I’ve never been able to get an institutional official to verify this directly though.

Additional reading: Cost principles, Proflike Substance on what overhead pays for.
*there can be blanket exceptions for trainee fellowships or exclusions based on an upper limit on the “award”.

“Oh, we know nobody is actually going to buy anything. We’re just here to make connections with postdocs for down-the-road.”

I generally like Stephen Curry’s position on the Journal Impact Factor. For example, in today’s confessional posting, he says:

mostly because of the corrosive effect they have on science and scientists.

In this we agree. He also posted “Sick of Impact Factors” and this bit focused on UK scholarly assessment. I enjoy his description of the arguments for why the Journal Impact Factor is leading to incorrect inferences and why it has a detrimental impact on the furthering of scientific knowledge.

But he pulled an academic nose sniffer / theological wackaloon move that I cannot support.

I was asked by a well-known university in North America to help assess the promotion application of one of their junior faculty. This was someone whose work I knew — and thought well of — so I was happy to agree. However, when the paperwork arrived I was disappointed to read the following statement the description of their evaluation procedures:

“Some faculty prefer to publish less frequently and publish in higher impact journals. For this reason, the Adjudicating Committee will consider the quality of the journals in which the Candidate has published and give greater weight to papers published in first rate journals.”

He then, admirably, tried to get them to waver on their JIF criterion….but to no avail

The reply was curt — they respected my decision for declining. And that was it.
I feel bad that I was unable to participate. I certainly wouldn’t want my actions to harm the career opportunities of another but could no longer bring myself to play the game. Others may feel differently.

So by refusing to play, he has removed himself as a guaranteed advocate for change. By drawing a hard, nose-sniffing line in the sand that he refuses to play if the game doesn’t change.

I prefer a more practical approach to all of this. I think I’ve alluded to this in the past.

I certainly agree to review manuscripts for journals where they are overtly concerned with “impact and importance” and the maintenance of their Journal Impact Factor. Certainly. And no, I do not ignore their obvious goals. I try to give the editor in question some indication of where I see the impact and importance and whether it deserves acceptance at their high falutin’ journal.

But I use my standards. I do not just roll over for what I see as the more corrosive aspects of Glamour Chasing. I rarely demand more experiments, I do not throw up ridiculous chaff about “mechanism” and other completely subjective bullshit and I do not demand optogenetics as the threshold for being interesting.

Stephen Curry could have very well done the same for this tenure review. He could have emphasized his own judgement of the impact and importance of the science and left the JIF bean counting to other reviewers. He could have struck a blow in support of the full and comprehensive review of the actual meat of this poor young faculty members’ contributions. Instead, he simply left the field, after sending up an impotent protest flag.

I think that is sacrificing actual progress on ones goals for the fine feeling of chest thumping purity. And that is a mistake.

Frequently commenter miko offered this up for consideration:

Being only ever first (or last) might make you look independent but also might make you look uncollaborative or, more likely, that no one likes you.

Fascinating remark. Have you ever heard anyone say this for real? Not in terms of mere publication numbers, I’ve heard that one more than once (as in “Can’t you just get on some more papers” from people concerned about a thin publication record from an earlier-career scientist). But in terms of your ability to collaborate productively. Play well with others, so to speak.

I can’t really remember hearing this. Mostly people are only mentioning collaborative, middle-author contributions to denigrate the person’s level of independence and/or to try to subtract credit from their publication count.

Greybearded TDWFs FTW

March 21, 2013

this is making the rounds…

From the PBS News Hour

Read the rest of this entry »

from a self described newProf at doc becca’s digs.

Last week, the first NIH proposal I wrote with PI status was rejected… I knew things were bad, but it still hurts…Problem is, I don’t know how to allocate my time between generating more preliminary data/pubs and applying for more grants. How many grants does the typical NIH- and/or NSF-funded (or wannabe-funded) TT prof write per year before getting funded?

It is not about what anyone else or the “typical” person has done.

It is about doing whatever you possibly can do until that Notice of Grant Award arrives.

My stock advice right now is that you need to have at least one proposal going in to the NIH for each standard receipt date. If you aren’t hitting it at least that hard, before you have a major award, you aren’t trying. If you think you can’t get out one per round…. you don’t really understand your job yet. Your job is to propose studies until someone decides to give your lab some support.

My other stock advice is take a look at the payline and assume those odds apply to you. Yes, special snoflake, you.

If the payline is 10%, then you need to expect that you will have to submit at least 10 apps to have a fighting chance. Apply the noob-discount and you are probably better off hitting twice that number. It is no guarantee and sure, the PI just down the hall struck it lucky with her first Asst Prof submission to the NIH. But these are the kinds of numbers you need to start with.

Once you get rolling, one new grant and one revised grant per round should be doable. They are a month apart and a revision should be way easier. After the first few, you can start taking advantage of cutting and pasting a lot of the grant text together to get a start on the next one.

Stop whining about preliminary data. Base it on feasibility and work from there. Most figures support at least a half dozen distinct grant applications. Maybe more.

I never know for sure how hard my colleagues are working when it comes to grant submissions. I know what I do…and it is a lot. I know what a subset of my other colleagues do and let me tell you, success is better correlated with effort (grants out the door) than it is with career rank. That has an effect, sure, but I know relatively older investigators who struggle to maintain stable funding and ones who enjoy multi-grant stability. They are distinguished to some extent by how many apps they get out the door. Same thing for junior colleagues. They are trying to launch their programs and all. I get this. They have to do a lot of setup, training and even spend time at the bench. But they also tend to have a very wait-and-see approach to grants. Put one in. Wait for the result. Sigh. “Well maybe I’ll resubmit it next round”. Don’t do this, my noob friends. Turn that app around for the next possible date for submission.

You’ll have another app to write for the following round, silly.

In email chatting with PP, as is our wont, I had the following query.

Do you think these “do it to Julia” muppethuggers really think they have the best objective solution? Or do they really know they are just looking out for número uno?

What do you think, Dear Reader?