Finishing projects
December 30, 2016
If you are paid by the taxpayers, or generous private philanthropists, of your country to do science, you owe them a product. An attempt to generate knowledge. This is one of the things that orients much of my professional behavior, as I think I make clear on this blog.
If you haven’t published your scientific work, it doesn’t exist. This is perhaps an excessive way to put it but I do think you should try to publish the work you accomplish with other people’s money.
Much of my irritation with the publication game, prestige chasing, delusions of complete stories, priority / scooping fears and competition for scarce funding resources can be traced back to these two orienting principles of mine.
My irritation with such things does not, however, keep them from influencing my career. It does not save me from being pressured not to give the funders their due.
It is not unusual for my lab, and I suspect many labs, to have thrown a fair amount of effort and resources into a set of investigations and to realize a lot more will be required to publish. “Required”, I should say because the threshold for publication is highly variable.
Do I throw the additional resources into an effort to save what is half or three-quarters of a paper? To make the project to date publishable? I mean, we already know the answer and it is less than earth shaking. It was a good thing to look into, of course. Years ago a study section of my peers told us so to the tune of a very low single digit percentile on a grant application. But now I know the answer and it probably doesn’t support a lot of follow-up work.
Our interests in the lab have moved along on several different directions. We have new funding and, always, always, future funding to pursue. Returning to the past is just a drag on the future, right?
I sometimes feel that nobody other than me is so stupid as to remember that I owe something. I was funded by other people’s money to follow a set of scientific inquiries into possible health implications of several things. I feel as though I should figure out how to publish the main thing(s) we learned. Even if that requires some additional studies be run to make something that I feel is already answered into something “publishable”.
DrugMonkey 
December 30, 2016 at 11:25 am
This is what review articles are for… I always try to stuff a few unpublished findings or other snippets that never made it to full paperhood, into reviews or other non-paper outputs (published abstracts for big meetings, published posters, archived online webinars, &c.)
Having said that, we do have a large folder on the lab cloud drive labeled “orphan data”, containing many of the types of projects of which you speak… rotation students that never panned out, summer undergrad projects &c. Occasionally I go back and comb through it, and very very occasionally might find something that can make an extra figure for an in-progress paper. I guess after I retire there’ll be plenty of material to “mine” for post-career output (just like they keep on finding and publishing new stuff by Vonnegut long after his demise).
LikeLike
December 30, 2016 at 12:36 pm
I know in my head that you are right about review articles. I just never manage to let go of the thought that a stand-alone pub should be attainable.
LikeLike
December 30, 2016 at 12:38 pm
Ola- re: orphan data. To what extent can you convince trainees to troll that collection for a starting point for an “easy” side project to finish for a quick pub?
LikeLike
December 30, 2016 at 12:40 pm
We need a a journal called “Ahah”, that’ll have entries indexed to any paper with more than a dozen primary, non-self citations. Any data relevant to that paper can be compiled continuously with just a figure, legend and methods. No pre publication peer review, buyer beware, but at least it is out there.
Speaking of which, has anyone tried Matters yet? The one shot result publishing format from CSH?
LikeLike
December 30, 2016 at 12:41 pm
I try to publish as much of the experimental work we do as possible for any given grant project. Sometimes this is high impact; mostly it’s just the major journal in the field; and occasionally only PLoS (though I did get called out on its impact factor in my last grant review).
But if we did the work, and know the answer, I agree that we owe it to our funders to put the data out there.
Ola, I also have an orphan data file- it is interesting to see how long it takes for things to move out of there, but I would say 30-50% eventually turn into papers.
LikeLike
December 30, 2016 at 1:00 pm
In my limited experience I absolutely agree that we owe funders a published product. If the whole enterprise goes pear-shaped, then we ought to document our experience to help other researchers avoid the same mistakes—not the most exciting of publications, but adding to the scientific record. Hopefully we do our science carefully enough that something is learned for the next time around regardless of the “result”.
KNOWING we owe someone a paper and DOING it are different things. I’m sitting on a few datasets that “should” be written up and that will be…”as soon as I have time”. (gulp)
LikeLike
December 30, 2016 at 1:14 pm
Jmz4: do you mean science matters? https://www.sciencematters.io
If that is the one, I don’t think it’s form CSH.
I am gonna argue that if something is not indexed in Pubmed or is not even picked up by google scholar it virtually does not exist. Would not send even orphan data to an invisible outlet.
LikeLike
December 30, 2016 at 1:41 pm
Maybe slightly off topic but what if you’re a postdoc and you have a job offer but that mean project stops if job accepted? PI’s job to see it finished? Or should postdoc owe it to funder (PI) as well?
LikeLike
December 30, 2016 at 5:58 pm
We try to pub everything but worry that study sections of future look down on funded but eventually incremental work from past
LikeLike
December 30, 2016 at 6:10 pm
I’ve sent some orphan data to other labs in my field, or talked about it one-on-one at conferences with other PIs. This has led to 1) a coauthored manuscript, and 2) a fun discussion that ended up giving me a new direction to pursue. These approaches do require you to have colleagues that aren’t jerks, and not be against coauthoring manuscripts.
My lab is dealing with this right now, as PI is retiring as soon as I finish my postdoc, and PI doesn’t want all the old half-completed stories to simply vanish.
LikeLike
December 31, 2016 at 12:59 am
@drugmonkey: I find that unfinished projects can linger in the mind and distract from the next big idea, and the best way to prevent them from doing that is to publish: http://serialmentor.com/blog/2013/11/3/no-one-reads-your-paper-either
The absolute worst strategy (which however I see in a lot of graduate students) is to hang on to a weak project and try to polish it into something great, rather than just publishing the weak findings and moving on. But even if you’re not actively working on a project anymore, just having the project in the back of your mind can distract you from more useful work.
Finally, if the project is 75% finished, why not just write it up as is and send to a low-tier journal rather than wasting effort doing the last 25%. In any case, the reviewers will likely ask for something, and that something will be different from what you think the last 25% are, so better submit at 75% and use the last 25% of effort to respond to the reviewer comments.
@Pinko Pinko: In my experience, as long as you also publish high-quality projects with some regularity, nobody holds the lower impact work against you. It’s more likely that people will consider you productive and thorough.
LikeLike
December 31, 2016 at 10:24 am
Now if you just remember that owing something to the taxpayers means you need to not only publish, but publish in open access journals that the taxpayers can read, we are getting somewhere. Your loyalty should not be to Springer or Elsevier and their ilk. You owe those parasites *nothing*.
LikeLike
December 31, 2016 at 10:35 am
Publishing OA means you publish less because it costs so much. As you well know. Using a publishing outfit’s journals lets you publish more work and it is available in a year. Good bargain.
LikeLike
December 31, 2016 at 12:03 pm
Good bargain for the commercial publishers, anyway, as it props up their obsolete business model for a few years longer. And no, it doesn’t “cost so much”. I’ve paid more than a typical OA fee to publish in closed access publications (color figures ain’t cheap). Not that publishing is ever more than 1% or 2% of a grant anyway.
LikeLike
December 31, 2016 at 2:55 pm
JB- that was your choice. I have paid pub fees on five percent or fewer of my papers in my career and 30% of those were for open access.
LikeLike
December 31, 2016 at 3:00 pm
Oh and 1-2% of grant is the wrong question to answer. How much of the discretionary account after staff costs (which are majority)? How far does the OA fee go towards new experiments? In my case $1300 (PONE fee, lower end) to $3,500 (not uncommon) is easily a publishable figure or three. This is not a trivial amount as you are trying to suggest with your 1-2% comment.
LikeLike
December 31, 2016 at 4:54 pm
Serialmentor, I agree. Since time is limited, putting time into publishing these smaller papers can be distracting from the better ones. I still do it, sometimes, but I wouldn’t claim that it is the most efficient use of my time. Some reasons to do it are: a trainee needs it, it’s very cool and I just will love to see it out, easy picking for a publication, a way to test that odd journal we haven’t submitted to (don’t want to try it with a “main result”), preventing someone from spending time doing what we already did, or encourage someone considering it to look into this in more detail, etc.
As for “owing it to the funders”, no. We owe them to be as productive as possible. Put out the best and most science we can. If small papers distract from better things, I don’t see how they get best value for the dollar. If anything, I think I owe more to the animals we “sacrificed” for the study, or the people who donated their organs. Those are the ones who deserve that we get it out there. Maybe also the trainees who put their time.
Publishing takes a lot of effort.
LikeLike
January 2, 2017 at 2:14 pm
@Pippso
You’re right, I don’t know why it got stuck in my head that it was associated with CSH.
“I am gonna argue that if something is not indexed in Pubmed or is not even picked up by google scholar it virtually does not exist. Would not send even orphan data to an invisible outlet.”
-I think at least some of them are picked up by Scholar (just did a quick test and I could find all the titles I copied from the website to Scholar search). You can also add them on your ResearchGate account, which means they show up in 2/3 of the ways I find papers.
@serialmentor
“The absolute worst strategy (which however I see in a lot of graduate students) is to hang on to a weak project and try to polish it into something great, rather than just publishing the weak findings and moving on.”
-Yeah, this is often the result of bad mentorship.
LikeLike
January 2, 2017 at 2:35 pm
JL- I absolutely agree with you that artificial career implications that keep us from publishing as much of our data from animals as possible is a violation of the spirit of our responsible use of animals obligation. Particularly when our failure to publish something means one or more labs go on to replicate it, using more animals. If that *also* turns out “unpublishable” for [reasons] the cycle continues.
LikeLike
January 7, 2017 at 12:30 pm
check the RESULT tab in RePorter , it’s empty more often than not
LikeLike
January 8, 2017 at 10:24 am
I don’t find this to be the case at all, especially if you only start counting after two years, which is reasonable. Now, sometimes the listed papers have very little obvious connection to the description of the project. But from my reading I think PIs tend to realize they have to connect something published to that award.
LikeLike
December 23, 2017 at 9:49 pm
Taxpayers get their moneys worth. Publications only is a crap strategy of measuring professional worth. Sometimes taxpayer money goes into training someone on an unpublishable result. Sometimes tax payer money goes to outreach. Sorry, NIH. You’re paying for this too. We aren’t the one dimensional creatures they buy w grant $$
LikeLike