Thought of the Day

May 4, 2013

Listed-third author gets to refer to it as a second-author-paper when the first two are co-equal first authors, right?

One duffymeg at Dynamic Ecology blog has written a post in which it is wondered:

How do you decide which manuscripts to work on first? Has that changed over time? How much data do you have sitting around waiting to be published? Do you think that amount is likely to decrease at any point? How big a problem do you think the file drawer effect is?

This was set within the background of having conducted too many studies and not finding enough time to write them all up. I certainly concur that by the time one has been rolling as a laboratory for many years, the unpublished data does have a tendency to stack up, despite our best intentions. This is not ideal but it is reality. I get it. My prior comments about not letting data go unpublished was addressing that situation where someone (usually a trainee) wanted to write up and submit the work but someone else (usually the PI) was blocking it.

To the extent that I can analyze my de facto priority, I guess the first priority is my interest of the moment. If I have a few thoughts or new references to integrate with a project that is in my head…sure I might open up the file and work on it for a few hours. (Sometimes I have been pleasantly surprised to find a manuscript is a lot closer to submitting than I had remembered.) This is far from ideal and can hardly be described as a priority. It is my reality though. And I cling to it because dangit…shouldn’t this be the primary motivation?

Second, I prioritize things by the grant cycle. This is a constant. If there is a chance of submitting a manuscript now, and it will have some influence on the grant game, this is a motivator for me. It may be because I am trying to get it accepted before the next grant deadline. Maybe before the 30 day lead time before grant review when updating news of an accepted manuscript is permitted. Perhaps because I am anticipating the Progress Report section for a competing continuation. Perhaps I just need to lay down published evidence that we can do Technique Y.

Third, I prioritize the trainees. For various reasons I take a firm interest in making sure that trainees in the laboratory get on publications as an author. Middle author is fine but I want to chart a clear course to the minimum of this. The next step is prioritizing first author papers…this is most important for the postdocs, of course, and not strictly necessary for the rotation students. It’s a continuum. In times past I may have had more affection for the notion of trainees coming in and working on their “own project” from more or less scratch until they got to the point of a substantial first-author effort. That’s fine and all but I’ve come to the conclusion I need to do better than this. Luckily, this dovetails with the point raised by duffymeg, i.e., that we tend to have data stacking up that we haven’t written up yet. If I have something like this, I’ll encourage trainees to pick it up and massage it into a paper.

Finally, I will cop to being motivated by short term rewards. The closer a manuscript gets to the submittable stage, the more I am engaged. As I’ve mentioned before, this tendency is a potential explanation for a particular trainee complaint. A comment from Arne illustrates the point.

on one side I more and more hear fellow Postdocs complaining of having difficulties writing papers (and tellingly the number of writing skill courses etc offered to Postdocs is steadily increasing at any University I look at) and on the other hand, I hear PIs complaining about the slowliness or incapabability of their students or Postdocs in writing papers. But then, often PIs don’t let their students and Postdocs write papers because they think they should be in the lab making data (data that might not get published as your post and the comments show) and because they are so slow in writing.

It drives me mad when trainees are supposed to be working on a manuscript and nothing occurs for weeks and weeks. Sure, I do this too. (And perhaps my trainees are bitching about how I’m never furthering manuscripts I said I’d take a look at.) But from my perspective grad students and postdocs are on a much shorter time clock and they are the ones who most need to move their CV along. Each manuscript (especially first author) should loom large for them. So yes, perceptions of lack of progress on writing (whether due to incompetence*, laziness or whatever) are a complaint of PIs. And as I’ve said before it interacts with his or her motivation to work on your draft. I don’t mind if it looks like a lot of work needs to be done but I HATE it when nothing seems to change following our interactions and my editorial advice. I expect the trainees to progress in their writing. I expect them to learn both from my advice and from the evidence of their own experiences with peer review. I expect the manuscript to gradually edge towards greater completion.

One of the insights that I gained from my own first few papers is that I was really hesitant to give the lab head anything short of what I considered to be a very complete manuscript. I did so and I think it went over well on that front. But it definitely slowed my process down. Now that I have no concerns about my ability to string together a coherent manuscript in the end, I am a firm advocate of throwing half-baked Introduction and Discussion sections around in the group. I beg my trainees to do this and to work incrementally forward from notes, drafts, half-baked sentences and paragraphs. I have only limited success getting them to do it, I suspect because of the same problem that I had. I didn’t want to look stupid and this kept me from bouncing drafts off my PI as a trainee.

Now that I think the goal is just to get the damn data in press, I am less concerned about the blah-de-blah in the Intro and Discussion sections.

But as I often remind myself, when it is their first few papers, the trainees want their words in press. The way they wrote them.

__
*this stuff is not Shakespeare, I reject this out of hand

LPU Redoux

April 12, 2013

Another round of trying to get someone blustering about literature “clutter” and “signal to noise ratio” to really explain what he means.

Utter failure to gain clarity.

Again.

Update 1:

It isn’t as though I insist that each and every published paper everywhere and anywhere is going to be of substantial value. Sure, there may be a few studies, now and then, that really don’t ever contribute to furthering understanding. For anyone, ever. The odds favor this and do not favor absolutes. Nevertheless, it is quite obvious that the “clutter”, “signal to noise”, “complete story” and “LPU=bad” dingdongs feel that it is a substantial amount of the literature that we are talking about. Right? Because if you are bothering to mention something under 1% of what you happen across in this context then you are a very special princess-flower indeed.

Second, I wonder about the day to day experiences of people that bring them to this. What are they doing and how are they reacting? When I am engaging with the literature on a given topic of interest, I do a lot of filtering even with the assistance of PubMed. I think, possibly I am wrong here, that this is an essential ESSENTIAL part of my job as a scientist. You read the studies and you see how it fits together in your own understanding of the natural world (or unnatural one if that’s your gig). Some studies will be tour-de-force bravura evidence for major parts of your thinking. Some will provide one figure’s worth of help. Some will merely sow confusion…but proper confusion to help you avoid assuming some thing is more likely to be so than it is. In finding these, you are probably discarding many papers on reading the title, on reading the Abstract, on the first quick scan of the figures.

So what? That’s the job. That’s the thing you are supposed to be doing. It is not the fault of those stupid authors who dared to publish something of interest to themselves that your precious time had to be wasted determining it was of no interest to you. Nor is it any sign of a problem of the overall enterprise.

UPDATE 2:
Thoughts on the Least Publishable Unit

LPU

Authors fail to illuminate the LPU issue

Better Living Through Least Publishable Units

Yet, publishing LPU’s clearly hasn’t harmed some prominent people. You wouldn’t be able to get a job today if you had a CV full of LPU’s and shingled papers, and you most likely wouldn’t get promoted either. But perhaps there is some point at which the shear number of papers starts to impress people. I don’t completely understand this phenomenon.

Avalanche of Useless Science

Our problem is an “Avalanche of Low Quality Research”? Really?

Too Many Papers

We had some incidental findings that we didn’t think worthy of a separate publication. A few years later, another group replicated and published our (unpublished) “incidental” results. Their paper has been cited 12 times in the year and a half since publication in a field-specific journal with an impact factor of 6. It is incredibly difficult to predict in advance what other scientists will find useful. Since data is so expensive in time and money to generate, I would much, much rather there be too many publications than too few (especially given modern search engines and electronic databases).

Idle thought

March 6, 2013

Relevant to Sci’s recent ranting about the paper chase in science…

Sorry reviewers, I am not burning a year and $250K to satisfy your curiosity about something stupid for a journal of this IF.

SevenTierCakeOccasionally during the review of careers or grant applications you will see dismissive comments on the journals in which someone has published their work. This is not news to you. Terms like “low-impact journals” are wonderfully imprecise and yet deliciously mean. Yes, it reflects the fact that the reviewer himself couldn’t be bothered to actually review the science IN those paper, nor to acquaint himself with the notorious skew of real world impact that exists within and across journals.

More hilarious to me is the use of the word “tier”. As in “The work from the prior interval of support was mostly published in second tier journals…“.

It is almost always second tier that is used.

But this is never correct in my experience.

If we’re talking Impact Factor (and these people are, believe it) then there is a “first” tier of journals populated by Cell, Nature and Science.

In the Neurosciences, the next tier is a place (IF in the teens) in which Nature Neuroscience and Neuron dominate. No question. THIS is the “second tier”.

A jump down to the IF 12 or so of PNAS most definitely represents a different “tier” if you are going to talk about meaningful differences/similarities in IF.

Then we step down to the circa IF 7-8 range populated by J Neuroscience, Neuropsychopharmacology and Biological Psychiatry. Demonstrably fourth tier.

So for the most part when people are talking about “second tier journals” they are probably down at the FIFTH tier- 4-6 IF in my estimation.

I also argue that the run of the mill society level journals extend below this fifth tier to a “the rest of the pack” zone in which there is a meaningful perception difference from the fifth tier. So…. Six tiers.

Then we have the paper-bagger dump journals. Demonstrably a seventh tier. (And seven is such a nice number isn’t it?)

So there you have it. If you* are going to use “tier” to sneer at the journals in which someone publishes, for goodness sake do it right, will ya?

___
*Of course it is people** who publish frequently in the third and fourth tier and only rarely in second tier, that use “second tier journal” to refer to what is in the fifth or sixth tier of IFs. Always.

**For those rare few that publish extensively in the first tier, hey, you feel free to describe all the rest as “second tier”. Go nuts.

@mbeisen is on fire on the Twitts:

@ianholmes @eperlste @dgmacarthur @caseybergman and i’m not going to stop calling things as they are to avoid hurting people’s feelings

Why? Open Access to scientific research, naturally. What else? There were a couple of early assertions that struck me as funny including

@eperlste @ianholmes @dgmacarthur @caseybergman i think the “i should have to right to choose where to publish” argument is bullshit

and

@eperlste @ianholmes @dgmacarthur @caseybergman funding agencies can set rules for where you can publish if you take their money

This was by way of answering a Twitt from @ianholmes that set him off, I surmise:

@eperlste @dgmacarthur how I decide where to pub is kinda irrelevant. The point is, every scientist MUST have the freedom to decide for self

This whole thing is getting ridiculous. I don’t have the unfettered freedom to decide where to publish my stuff and it most certainly is an outcome of the funding agency, in my case the NIH.

Here are the truths that we hold to be self-evident at present time. The more respected the journal in which we publish our work, the better the funding agency “likes” it. This encompasses the whole process from initial peer review of the grant applications, to selection for funding (sometimes via exception pay) to the ongoing review of program officers. It extends not just from the present award, but to any future awards I might be seeking to land.

Where I publish matters to them. They make it emphatically clear in ever-so-many-ways that the more prestigious the journal (which generally means higher IF, but not exclusively this), the better my chances of being continuously funded.

So I agree with @mbeisen about the “I have the right to choose where I publish is bullshit” part, but it is for a very different reason than seems to be motivating his attitude. The NIH already influences where I “choose” to publish my work. As we’ve just seen in a prior discussion, PLoS ONE is not very high on the prestige ladder with peer reviewers…and therefore not very high with the NIH.

So quite obviously, my funder is telling me not to publish in that particular OA venue. They’d much prefer something of a lower IF that is better respected in the field, say, the journals that have longer track records, happen to sit on the top of the ISI “substance abuse” category or are associated with the more important academic societies. Or perhaps even the slightly more competitive rank of journals associated with academic societies of broader “brain” interest.

Even before we get to the Glamour level….the NIH funding system cares where I publish.

Therefore I am not entirely “free” to choose where I want to publish and it is not some sort of moral failing that I haven’t jumped on the exclusive OA bandwagon.

@ianholmes @eperlste @dgmacarthur @caseybergman bullshit – there’s no debate – there’s people being selfish and people doing the right thing

uh-huh. I’m “selfish” because I want to keep my lab funded in this current skin-of-the-teeth funding environment? Sure. The old one-percenter-of-science monster rears it’s increasingly ugly head on this one.

@ianholmes @eperlste @dgmacarthur @caseybergman and we have every right to shame people for failing to live up to ideals of field

What an ass. Sure, you have the right to shame people if you want. And we have the right to point out that you are being an asshole from your stance of incredible science privilege as a science one-percenter. Lecturing anyone who is not tenured, doesn’t enjoy HHMI funding, isn’t comfortably ensconced in a hard money position, isn’t in a highly prestigious University or Institute, may not even have achieved her first professorial appointment yet about “selfishness” is being a colossal dickweed.

Well, you know how I feel about dickweedes.

I do like @mbeisen and I do think he is on the side of angels here*. I agree that all of us need to be challenged and I find his comments to be this, not an unbearable insult. Would it hurt to dip one toe in the PLoS ONE waters? Maybe we can try that out without it hurting us too badly. Can we preach his gospel? Sure, no problem. Can we ourselves speak of PLoS ONE papers on the CVs and Biosketches of the applications we are reviewing without being unjustifiably dismissive of how many notes Amadeus has included? No problem.

So let us try to get past his rhetoric, position of privilege and stop with the tone trolling. Let’s just use his frothing about OA to examine our own situations and see where we can help the cause without it putting our labs out of business.

__
*ETA: meaning Open Access, not his attacks on Twitter

Dump Journals

January 17, 2013

To be absolutely clear, I use the term “dump journal” without malice. Some do, I know, but I do not. I use it to refer to journals of last resort. The ones where you and your subfield are perfectly willing to publish stuff and, more importantly, perfectly willing to cite other papers. Sure, it isn’t viewed as awesome, but it is….respectable. The Editor and sub-editors, probably the editorial board, are known people. Established figures who publish most of their own papers in much, much higher IF journals. It is considered a place where the peer review is solid, conducted by appropriate experts who, btw, review extensively for journals higher up the food chain.

What interests me today, Dear Reader, are the perceptions and beliefs of those people who are involved in the dump journal. Authors who submit work there, the Editor and any sub-editors….and the reviewers. Do we all commonly view the venue in question as a “dump journal”? Or are there those that are surprised and a bit offended that anyone else would consider their solid, society level journals as such a thing?

PatheticImpFactorAre there those who recognize that others view the journal as a dump journal but wish to work to change this reputation? By being harsher during the review process than is warranted given the history of the journal? That approach is a game of chicken though…if you think a dump journal is getting too uppity for its current IF then you are going to just move on to some other journal for your data-dumping purposes, are you not? If a publisher or journal staff wanted to make a serious move up the relative rankings, they’d better have a plan and a steely nerve if you ask me.

This brings me around to my fascination with PLoS ONE and subjective notions of its quality and importance. What IS this journal? Is it a dumping grounds for stuff you had rejected elsewhere on “importance” and “impact” grounds and you just want the damn data out there already? That would qualify as a dump journal in my view. Or do you view it as a potential primary venue…because it enjoys an IF in the 4s and that’s well into run-of-the-mill decent for your subfield?

Furthermore, how does this color your interaction with the journal? I know we have a few folks around here who function as Academic Editors. Are you one of those that thinks PLoS ONE should be ever upping its “quality” in an attempt to improve the reputation? Do you fear it becoming a “dump journal”? Or do you embrace that status?

Are you involved with another journal that some might consider a dump journal for your field? Do you think of it this way yourself? Or do see it as a solid journal and it is that other journal, 0.245 IF points down, which is the real dump journal?

Damaged Goods

January 14, 2013

Have you ever had a manuscript severely damaged by the process of peer review?

by way of example, I can recall one time where the Editor demanded I chop off two experiments..and I did so*.

Otherwise, I’m generally of the opinion that peer review has a positive impact on the manuscript.

___
*Those figures have yet to see the light of day and may never get published. A shame, but then, we got the paper published and the main point was one of the other figures.

For some reason the response on Twittah to the JSTOR downloader guy killing himself has been a round of open access bragging. People are all proud of themselves for posting all of their accepted manuscripts in their websites, thereby achieving personal open access.

But here is my question…. How many of you are barraged by requests for reprints? That’s the way open access on the personal level has always worked. I use it myself to request things I can’t get to by the journal’s site. The response is always prompt from the communicating author.

Seems to me that the only reason to post the manuscripts is when you are fielding an inordinate amount of reprint requests and simply cannot keep up. Say…more than one per week?

So are you? Are you getting this many requests?

First, let’s all enjoy the bliss of, count ’em, EIGHT authors who….

1D.S., A.B., M. Maroteaux, T.J., C.P.M., R.S., J.-A.G., and G.S. contributed equally to this work.

To make it extra hilarious please note that the first four are listed first authors and the last four are…listed last authors.

This is ridiculous. Going by the affiliations of the first four and the last four (and knowing a little something about the careers status of several of the last four) it looks very much like typical trainee-PI pairings in a multi-group collaboration. Consequently it would make considerably more sense to identify the four trainees and the four PIs as contributing equally compared with each other…but not across the trainee/PI divide.

But really, the discussion of the day is raised by a troll communication to the blog.

As you know there are style guides for journals as to how previous studies are to be cited and how they are to be referred to in the text. One typical style guide might suggest that you use “As shown by Gun et al (2009), the PhysioWhimple nucleus is critical in…“. You might also resort to the more conversational “Gun and colleagues demonstrated…“.

Very good, right?

Now what about when the paper in question indicates co-equal contribution, eh? Then you should say “Genedog, Tideliar and colleagues showed….“. Right? You should absolutely insist on including the name of the co-equal authors, should you not?

Especially if you are one of those who insists that this designation is meaningful…

h/t: a certain troll

So one of the Twitts was recently describing a grant funding agency that required listing the Impact Factor of each journal in which the applicant had published.

No word on whether or not it was the IF for the year in which the paper was published, which seems most fair to me.

It also emerged that the applicant was supposed to list the Journal Impact Factor (JIF) for subdisciplines, presumably the “median impact factor” supplied by ISI. I was curious about the relative impact of listing a different ISI journal category as your primary subdiscipline of science. A sample of ones related to the drug abuse sciences would be:

Neurosciences 2.75
Substance Abuse 2.36
Toxicology 2.34
Behavioral Sciences 2.56
Pharmacology/Pharmacy 2.15
Psychology 2.12
Psychiatry 2.21

Fascinating. What about…
Oncology 2.53
Surgery 1.37
Microbiology 2.40
Neuroimaging 1.69
Veterinary Sciences 0.81
Plant Sciences 1.37

aha, finally a sub-1.0. So I went hunting for some usual suspects mentioned, or suspected, as low-cite rate disciplines..
Geology 0.93
Geosciences, multidisc 1.33
Forestry 0.87
Statistics and Probability 0.86
Zoology 1.06
Forestry 0.87
Meteorology 1.67

This a far from complete list of the ISI subdisciplines (and please recognize that many journals can be cross-listed), just a non-random walk conducted by YHN. But it suggests that range is really restricted, particularly when it comes to closely related fields, like the ones that would fall under the umbrella of substance abuse.

I say the range is restricted because as we know, when it comes to journals in the ~2-4 IF range within neuroscience (as an example), there is really very little difference in subjective quality. (Yes, this is a discussion conditioned on the JIF, deal.)

It requires, I assert, at least the JIF ~6+ range to distinguish a manuscript acceptance from the general herd below about 4.

My point here is that I am uncertain that the agency which requires listing disciplinary medians JIFs is really gaining an improved picture of the applicant. Uncertain if cross-disciplinary comparisons can be made effectively. You still need additional knowledge to understand if the person’s CV is filled with Journals that are viewed as significantly better than average within the subfield. About all you can tell is that they are above or below the median.

A journal which bests the Neurosciences median by a point (3.75) really isn’t all that impressive. You have to add something on the order of 3-4 IF points to make a dent. But maybe in Forestry if you get to only a 1.25 this is a smoking upgrade in the perceived awesomeness of the journal? How would one know without further information?

PSA on Journal selection

November 27, 2012

Academic trainees should not be publishing in journals that do not yet have Impact Factors. Likewise they should not be publishing in journals that are not indexed by the major search database (like PubMed) used in their field.

Co-Communicating Author

November 15, 2012

What is “communicating author” attribution for in your field? How is it interpreted?

(and do you note it on a CV or job application in any way?)

A question arrived about publication expectations for trainees at the blog mailbox recently.

I was wondering if you would consider a blog post and perhaps encouraging discussion on a related topic, on how do you evaluate your student/postdoc performance and how common is the 1 paper/yr “rule”?

At the outset I was skeptical that much use would come of trying to answer this because the real answer is “It depends very much on subfield and ultimate career aspirations, therefore broad sweeping pronouncements are of little value.“. And this is true. But what the heck? I’ll give you my thoughts from my point of view, no doubt some others will go shitnutz about how it is clearly different and maybe we can hash out the space of useful answers.

Some detailed stuff that I thought about, but often are not discussed thoroughly include:
– I always assumed that when people talk about 1 paper/yr it refers to 1 first-author paper but not in a top-tier journal (usually “best in the sub-field” journal, e.g. Org. Lett., J. Med. Chem., etc.)

Yeah. I think one paper per year is a pretty good general starting point. Emphasis on general. For trainees, I think this average will be lower, ditto if you only count first-author papers. But it is a pretty good target expectation for the central tendency. One first author per year in a “top tier” journal is a ridiculously absurd expectation for postdocs. Even one per year in a “top tier” journal as senior author is only possible for the very top laboratories and is therefore not the expectation for everyone. If you can do it, good on you, but it ain’t typical. So if you are in a place where you think this is the standard for postdocs? please. I’m familiar with a lab that has probably one of the highest CNS counts ever and the postdocs do not hit one CNS pub per year as first author. They have not done so over the ~15 years I’ve been watching the lab’s production. So anyone who does this out there in the whole postdoc population is the rare exception.

– How do you factor in non-1st author papers? Ignoring the effects of journal IF, would one 1st-author paper = two 2nd-author paper?

There is no direct relationship, I would argue. Non-substitutable quantities. No amount of non-first author papers makes up for not having any first-author papers. They are just that important in the minds of many people, including me. Conversely, the existence of some 2nd-Xth author papers is better than not having any, because more is better when it comes to publications on the CV. I suppose at some point there would be a balance point in which too many Nth author papers starts to subtract from the credit generated by the first-author list. It would be related to the thought of “why doesn’t this trainee have more firsts if she is this experimentally productive?”.

– Do people even consider anything greater than 2nd-authorship (i.e. having 3rd authorship is basically useless or not counted)? If so, does the level of the prestige of the journal change this perception (i.e. having 3rd authorship in PNAS is equivalent to a 1st-author in some 2nd-tier journal like Biochemistry)?

In my view, no, the Nth author on an article in a higher IF journal doesn’t trump first-author in a lesser journal. See above, the Nth authorships count but I would say they are independent of the first-author credits. So within the sphere of Nth authorships, sure, the higher IF is better.

– How do you factor in the prestige or IF of the journal? Does publishing in Science/Nature/Cell count as having 2-3 1st-author papers in 2nd-tier journals?

Indubitably the CNS first-author counts more than several first-authors in lesser journals. One might even suggest that CNS first-author as a postdoc trumps infinity non-CNS first-authors. For some situations. There are those that assert that the presence/absence of very specific journals on the CV is the difference between round-filing an application for an Assistant Professor position and placing it on the long-list for consideration. I credit these assertions but would also point out that there are many perfectly acceptable jobs that would not have this absurd criterion.

– Do people take a time-average (i.e. as long as you get 5 papers in 5 years it’s fine), or is having a regular output more important (i.e. would prefer to have 1 paper every year as opposed to 2 papers in 1st year and 3 papers in 5th year but nothing in between)?

I would say that it is only once one becomes a PI that it is ever reasonable to look at consistency of output. This particular example would not even be noticed, I would say. And even then it sort of depends on the type of work you do. I know of multiple types of work in my areas of interest (particularly human studies) that have years of data collection followed by a flurry of papers.

When I have recommended shooting for consistent output, being concerned with whether a manuscript submitted to Journal X at this point in the year will have a pub date from this year, etc it has to do mostly with motivation. Most of the time the pace of submission for a postdoc is not going to be easily controlled. The experiments have their own timeline. Things come up. New things need to be done to wrap up the paper. Then there are the many sources of delay in the review process. There is no reason to obsess about 2 in first year / 3 in fifth over meeting a strict rate of 1 per year for 5 years.

The clock is ever ticking, however and since one cannot go back and fill in missing publication-years, one is best keeping one’s eye on the prize. If you haven’t had a paper in a two year span, well maybe it is better to dump out a quick one, give up on hitting the highest possible IF, etc. You have to make this judgement thinkingly, of course. And no, there are no formulaic answers such as my correspondent seems to be seeking.

Balance. That is my best suggestion.

This one is mostly for the PIs in the audience but I’m sure trainees will have experiences to share as well.

What fraction of the people who have spent time in your laboratory have ended up with authorships on published papers?

(Including students and techs)