Well respected addiction neuroscientist Nick Gilpin posted a twitter poll asking people about their favorite part of doing science.

Only 4% of the respondents voted for “Reading the literature”. Now look, it’s just a dumb little twitter poll and it was a forced choice about a person’s favorite. Maybe reading is a super close second place for everyone who responded with something else as their first choice, I don’t know.

However. Experience in this field, reviewing manuscripts submitted to journals, reading manuscripts published in journals, fielding comments from reviewers about our manuscripts and trying to help trainees learn to write a scientific manuscript suggests to me that it is more than this.

I think a lot of scientists really don’t like to read deeply into the literature. At best, perhaps they weren’t ever trained to read deeply.

As a mentor, I tip toe around this issue a lot more than I should. I think, I guess, that it would be sort of insulting to ask a postdoc if they even know how to read deeply into a literature for the purposes of writing up scientific results. So I take the hint approach. I take the personal example approach. Even my direct instruction is a little indirect.

The personal example approach has a clear failure point. The hint may suffer from that somewhat as well. The direct statements of instruction that I do manage to give “Hey, we need to look into this set of issues so we know what to write” is only slightly better. The failure point I am talking about is that part of reading deeply into the literature is a triage process.

The triage process is one of elimination. Of looking at a paper that might, from the title or eve the Abstract, be relevant. In the vast majority of cases you are going to quickly figure out that it is not something that contributes to the knowledge under discussion in this particular paper. To me this is under the heading of reading the literature. It doesn’t mean you read every paper word for word, at all. Maybe this is part of the problem for some people? That they think it really means read every frigging search result from start to finish? I can see how that would be daunting.

But it IS work. It takes many hours, at times, of searching through PubMed, Google Scholar or Web of Science using several variant key word searches. Of then scanning papers as needed to see if they have relevant information. Sometimes you can triage based on the abstract. Sure. But a lot of the time you have to download the paper and take look through it. Sometimes, you are just checking for a detail that didn’t make it into the abstract and finding that, nope, the paper isn’t relevant. But sometimes it is and you have to read it. But then, maybe it would be a good idea to look at the citations and follow the threads to additional papers. Maybe you should use Web of Science to find out what subsequent papers cited that particular paper. All of this work to come up with three or two or even one sentence with three surviving citations.

The person who is following my writing by example doesn’t necessarily see this. They may think I just pulled three cites at random and kept on going. They may think that somehow it is my vast experience that has all of this literature in my head all at the same time. Despite me saying more or less the above as reminders. When I say things like “hey, everybody works differently, but when I do a PubMed search on a topic, I like to start with the oldest citations and work forward from there”, I mean this as a hint that they should actually do PubMed searches on topic terms.

(I admit in early graduate school I was really intimidated by the perception that the Professors did have encyclopedic knowledge of everything, all of the time. I don’t think that any more. I mean, yes, my colleagues over the years clearly vary and some are incredibly good at knowing the literature. But some are just highly specific in their knowledge and if you get too far outside they flounder. Some know lots of key papers, sure, but if you REALLY are digging in deeply, you figure out they don’t know everything. Not the way a grad student on dissertation defense day should.)

So I’m not talking about reading the literature in a “keep up with the latest TOCs for all the relevant journals” kind of way. I’m talking about focused reading when you want to answer a question for yourself and to put it into some sort of scholarly setting, like a Discussion.

I have no recollection of being taught to read a literature. I just sort of DID that as a graduate student. I was in the kind of lab where you were expected to really develop your own ideas almost from whole cloth, rather than fitting into an existing program of work. Some of my fellow grad students were in labs more like mine, some were in labs where they fit into existing programs. So it wasn’t the grad program itself, just an accident of the training lab. Still, it wasn’t as though I chatted much with my fellow students about something like this so I have no idea how they were trained to read the literature. I have no recollection of how much time anyone spent in the library (oh yes, children, this was before ready access to PDFs from your desktop) compared with me.

So about this poll that Professor Gilpin put up. How about you, Dear Reader? How do you feel about reading the literature? Were you taught how to do it? Especially as it pertains to writing papers that reference said literature (as opposed to reading to guide your experiments in the first place, another topic for another day). Have you tried to explicitly train your mentees to do so or do they just pick it up? Is it always the case that reviewers of your manuscripts hand wavily suggest you have overlooked some key literature and they are right? or is it the case that you know all about what they mean and have triaged it as not relevant? Or you know that what they think “surely must exist” really doesn’t?

Despite evidence to the contrary on this blog, some people who don’t like to write have occasionally said things in the vein of “oh, but you are such a good writer”. Sometimes this is by way of trying to get me to do some writing for them in the non-professional setting. Sometimes this is a sort of suggestion that somehow it is easier for me to write than it is for them to write, in the professional setting.

I don’t know. I certainly used to be a better writer and my dubious blogging hobby has certainly contributed to making my written product worse. Maybe I’m just getting that Agatha Christie thing early (her word variety constricted towards her final books, people suggest that was evidence of dementia).

But for decades now, I view my primary job as a writing job. When it comes right down to the essentials, an academic scientist is supposed to publish papers. This requires that someone write papers. I view this as the job of the PI, as much as anyone else. I even view it as the primary responsibility of the PI over everyone else, because the PI is where the buck stops. My personnel justification blurb in every one of my grants says so. That I’ll take responsibility for publishing the results. Postdocs are described as assisting me with that task. (Come to think of it, I can’t remember exactly how most people handle this in grants that I’ve reviewed.)

Opinions and practices vary on this. Some would assert that no PI should ever be writing a primary draft of a research paper and only rarely a review. Editing only, in the service of training other academic persons in the laboratory to, well, write. Some would kvetch about the relative ratio of writing effort of the PI versus other people in the laboratory. Certainly, when my spouse would prefer I was doing something other than writing, I get an earful about how in lab X, Y and Z the PI never writes and the awesome postdocs basically just hand over submit ready drafts and why isn’t my lab like that. But I digress.

I also have similar views on grant writing, namely that in order to publish papers one must have data from which to draw upon and that requires funds. To generate the data, therefore, someone has to write grant proposals. This is, in my view, a necessary job. And once again, the buck stops with the PI. Once again, practices vary in terms of who is doing the writing. Once again, strategies for writing those grants vary. A lot. But what doesn’t vary is that someone has to do a fair bit of writing.

I like writing papers. The process itself isn’t always smooth and it isn’t always super enjoyable. But all things equal, I feel LIKE I AM DOING MY JOB when I am sitting at my keyboard, working to move a manuscript closer to publication. Original drafting, hard core text writing, editing, drawing figures and doing analysis iteratively as you realize your writing has brought you to that necessity…I enjoy this. And I don’t need a lot of interruption (sorry, “social interaction”) when I am doing so.

In the past year or so, my work/life etc has evolved to where I spend 1-2 evenings a week in my office up to about 11 or 12 after dinner just writing. I dodge out for dinner so that my postdocs have no reason to stick around and then I come back in when the coast is clear.

I’m finding life in the time of Corona to simply push those intervals of quiet writing time earlier in the day. I have a houseful of chronologically shifted teens, which is awesome. They often don’t emerge from their rooms until noon…or later. Only my youngest needs much of my input on breakfast and even that is more a vague feeling of lingering responsibility than actual need. Sorry, not trying to rub it in for those of you with younger children. Just acknowledging that this is not a bad time in parenthood for me.

So I get to write. It’s the most productive thing I have to do these days. Push manuscripts closer and closer to being published.

It’s my job. We have datasets. We have things that should and will be papers eventually.

So on a daily and tactical level, things are not too bad for me.

I was trained to respond to peer review of my submitted manuscripts as straight up as possible. By this I mean I was trained (and have further evolved in training postdocs) to take every comment as legitimate and meaningful while trying to avoid the natural tendency to view it as the work of an illegitimate hater. This does not mean one accepts every demand for a change or alters one’s interpretation in preference for that of a reviewer. It just means you take it seriously.

If the comment seems stupid (the answer is RIGHT THERE), you use this to see where you could restate the point again, reword your sentences or otherwise help out. If the interpretation is counter to yours, see where you can acknowledge the caveat. If the methods are unclear to the reviewer, modify your description to assist.

I may not always reach some sort of rebuttal Zen state of oneness with the reviewers. That I can admit. But this approach guides my response to manuscript review. It is unclear that it guides everyone’s behavior and there are some folks that like to do a lot of rebuttal and relatively less responding. Maybe this works, maybe it doesn’t but I want to address one particular type of response to review that pops up now and again.

It is the provision of an extensive / awesome response to some peer review point that may have been phrased as a question, without incorporating it into the revised manuscript. I’ve even seen this suboptimal approach extend to one or more paragraphs of (cited!) response language.

Hey, great! You answered my question. But here’s the thing. Other people are going to have the same question* when they read your paper. It was not an idle question for my own personal knowledge. I made a peer review comment or asked a peer review question because I thought this information should be in the eventual published paper.

So put that answer in there somewhere!

___
*As I have probably said repeatedly on this blog, it is best to try to treat each of the three reviewers of your paper (or grant) as 33.3% of all possible readers or reviewers. Instead of mentally dismissing them as that weird outlier crackpot**.

**this is a conclusion for which you have minimal direct evidence.

Citing Preprints

May 23, 2018

In my career I have cited many non-peer-reviewed sources within my academic papers. Off the top of my head this has included:

  1. Government reports
  2. NGO reports
  3. Longitudinal studies
  4. Newspaper items
  5. Magazine articles
  6. Television programs
  7. Personal communications

I am aware of at least one journal that suggests that “personal communications” should be formatted in the reference list just like any other reference, instead of the usual parenthetical comment.

It is much, much less common now but it was not that long ago that I would run into a citation of a meeting abstract with some frequency.

The entire point of citation in a scientific paper is to guide the reader to an item from which they can draw their own conclusions and satisfy their own curiosity. One expects, without having to spell it out each and every time, that a citation of a show on ABC has a certain quality to it that is readily interpreted by the reader. Interpreted as different from a primary research report or a news item in the Washington Post.

Many fellow scientists also make a big deal out of their ability to suss out the quality of primary research reports merely by the place in which it was published. Maybe even by the lab that published it.

And yet.

Despite all of this, I have seen more than one reviewer objection to citing a preprint item that has been published in bioRxiv.

As if it is somehow misleading the reader.

How can all these above mentioned things be true, be such an expectation of reader engagement that we barely even mention it but whooooOOOAAAA!

All of a sudden the citation of a preprint is somehow unbelievably confusing to the reader and shouldn’t be allowed.

I really love* the illogical minds of scientists at times.

I am a consumer of the creative arts and, really, have always been in awe of creative artists. Looking back chronologically over my lifetime, my greatest consumption and appreciation has been fiction writing, music and cartooning (particularly the political variety). I’m not a big fan of flat art (sculpture speaks to me much more) but I am definitely amazed by what some people can paint, draw and the like. I do like moving picture arts but I don’t think I have any particular sense of awe for them as a craft and certainly not for the participants as creative artists*. I get that others can see this, however.

Anyway, the creative artists are amazing to me.

A couple of days ago it occurred to me that understanding the process of creative arts might help cross what I find to be a somewhat frustrating bridge in training other people to write scientific manuscripts.

Sidebar: I am pretty sure we’ve discussed related topics before on the blog, but I can’t remember when so I’m probably going to repeat myself.

When I first started to write scientific manuscripts I quite reasonably suffered the misunderstanding that you sort of did the experiments you planned and then wrote them (all of them) up in chronological order and badda boom, published it somewhere. That is because, I assume, many scientific manuscripts read as if that is how they were created. And there are probably some aspects of “Research Design 101” instruction that convinces young scientists that this is the way things work.

Then, when it is your own work, there are two additional factors that press down and shape your writing process. First, a sense of both pride and entitlement for your effort which tells your brain that surely every damn thing you worked on needs to fuel a publication. Second, a sense that writing is hard and you want to know in advance exactly what to write so that no effort is wasted.

“Wasted”.

And this is where the creative arts come in.

Now, I’ve never lived cheek by jowl with a creative artist and I am only superficially familiar with what they do. But I am pretty convinced it is an iterative, inefficient process. Flat art folks seem to sketch. A lot. They work on an eye. An expression. A composition. A leg. Apple. Pair of shoes. Novelists and short story authors work on themes. characters. plot elements. They write and tear their hair out. Some of this is developing skill, sure, but much of this for a reasonably mature creative person is just working the job. They create copious amounts of material that is only leading up to the final product.

And the final product, I surmise, is built from the practice elements. A plot or character for a story. A curve of a mouth for a portrait. Melody. Chord progressions. A painted sunbeam. The artist starts stitching together a complete work out of elements.

I think you need to get into this mode as a scientist who is writing up manuscripts.

We stitch together a work out of elements as well. Now in our case, the elements are not made up. They are data. That we’ve collected. And we spend a heck of a lot of time on the quality of those elements. But eventually, we need to tell a story from those parts.

N.b. This is not storyboarding. Storyboarding is setting out the story you want to tell and then later going out and creating the elements (aka, figures) that you need to tell this particular story. That way lies fraud.

The creative process is looking at the elements of truth that you have available to you, from your labors to create good data, and then trying to see how they fit together into a story.

The transition that one has to make as a scientist is the ability to work with the elements, put in serious labor trying to fit them together, and then being willing to scrap the effort and start over. I think that if you don’t get in there and do the work writing, writing, writing and analyzing and considering what the data are telling you, you make less progress.

Because the alternative is paralyzing. The alternative is that you keep putting off the creative process until something tells you how to write “efficiently”. Maybe it is that you are waiting for just the right experimental result to clarify a murky situation. Maybe you are waiting for your PI or collaborator or fellow trainee to tell you what to do, what to write, how to structure the paper.

I suppose it may look like this to a relatively inexperienced writer of manuscripts? That its a bit daunting and that if only the PI would say the right words that somehow it would be magically easy to “efficiently” write up the paper in the right way that she expects?

When I hear generic muttering from trainees about frustration with insufficient feedback from a mentor I sometimes wonder if this is the problem. An over expectation of specific direction on what to write, how to write and what the story is.

The PI, of course, wants the trainee to take their own shot at telling the story. Whereupon they will promptly red pen the hell out of all that “work” and tell the trainee to rewrite most of it and take a totally different tack. Oh, and run these two more experiments. And then the trainee wonders “why didn’t my PI tell me what she wanted in the first place instead of wasting my time??? GAh, I have the worst possible mentor!

I realized within the past year or so that I have the same problem that I have criticized on the blog for years now. I tell new professors that they need to get away from the bench as quickly as possible and that this is not their job anymore. I tell them they have to find a way to get productivity out of their staff and that doing experiments is not their job anymore. I never had this problem as a transitioning scientist…I was fine getting away from the bench**.

But my equivalent is data analysis. And I’m not talking high falutin’ stuff that only I can do, either. I want to see the data! Study by study. As it rolls in, even. I want to examine it, roll it around in it. Create graphs and run some stats. Think about what it means and how it fits into my developing understanding of a research direction in our laboratory. I can’t wait to think about how this new figure might fit into one of our ongoing creative works…i.e., a manuscript.

I cannot give it up.

I create a lot of sketches, half plotted stories and cartoon panels. Elements. Themes. Drafts.

Many of these will never go into any published manuscript. If lucky some of these building blocks will make their way into a slide presentation or a into a grant as preliminary data. I never feel as though the effort is wasted, however. Making these bits and pieces is, to me, what allows me to get from here to there. From blank page to published manuscript.

Ideally, as I am supposedly training people to become independent scientists, I would like to train them to do this in the way that I do. And to get there, I have to get them across the hurdle of the creative artist. I have to get them to see that just rolling up your sleeves and doing the work is a necessary part of the process. You cannot be told a route, or receive a Revelation, that makes the process of creating a scientific manuscript efficient. You have to work on the elements. Make the sketches. Flesh out the plotlines.

And then be willing to scrap a bunch of “work” because it is not helping you create the final piece.

__
*I have a friend that is behind the camera on teevee shows. Big name teevee shows that you’ve heard of and watch. I see his work and I’m not really Seeing. His. Work. But this guy casually takes a few vacation pictures and I’m amazed at his eye, composition, etc. He doesn’t seem to even consider himself a still camera artist, acts like he considers himself barely a hobbyist at that! So clearly I’m missing something about moving picture photography.

**I’m not actually a bench scientist, the ~equivalent.

The currency of science news

September 23, 2015

Ok, I take the point that journalism should not only talk about science upon the publication of a paper. 

Absolutely.

Science news can be much more fluid and the semi-public knowledge of a finding precedes formal publication.

But if there is a paper then it should be cited. Not merely linked obscurely, but properly cited
Scientists have been complaining about the failure of journalists to cite papers associated with their science news stories for ages. Ed knows this as well as anyone in science journalism. So I am confused as to what he is about here.

The life of the academic scientist includes responding to criticism of their ideas, experimental techniques and results, interpretations and theoretical orientations*.

This comes up pointedly and formally in the submission of manuscripts for potential publication and in the submission of grant applications for potential funding.

There is an original submission, a return of detailed critical comments and an opportunity to respond to those critiques with revisions to the manuscript / grant application and/or argumentative rebuttal.

As I have said repeatedly in this forum, one of my most formative scientific mentors told me that you should take each and every comment seriously. Consider what is being said, why it is being said and try to respond accordingly. This mentor told me that I would usually find that by considering even the most idiotic seeming comments seriously, the manuscript (or grant application) is improved.

I have found this to be a universal truth of my professional work.

My understanding of what I was told by my mentor, versus what I have filled in additionally in my similar comments to my own trainees is now very fuzzy. I cannot remember exactly how extensively this mentor stamped down what is now my current understanding. For example, it is helpful to me to consider that Reviewer #3 represents about 33% of peers instead of thinking of this person as the rare outlier. I think that one may be my own formulation. Regardless of the relative contributions of my mentor versus my lived experience, it is all REALLY valuable advice that I have internalized.

The paper and grant review process is not there, by any means, to prove to you beyond a shadow of a doubt** that the reviewer’s position is correct and you are wrong. A reviewer that provides citations for a criticism is not by any means the majority of my experience…although you will see this occasionally. Even there, you could always engage cited statements from an antagonistic default setting. This is unwise.

The upshot of this critique-not-proof system means that as a professional, you have to be able to argue against yourself in proxy for the reviewer. This is why I say you need to consider each comment thoughtfully and try to imagine where it is coming from and what the person is really saying to you. Assume that they are acting in good faith instead of reflexively jumping behind paranoid suspicions that they are just out to get you for nefarious purposes.

This helps you to critically evaluate your own product.

Ultimately, you are the one that knows your product best, so you are the one in position to most thoroughly locate the flaws. In a lot of ways, nobody else can do that for you.

Professionalism demands that you do so.

__
*Not an exhaustive list.

**colloquially, they are leading you to water, not forcing you to drink.

via Twitter and retractionwatch, some hilarity that ended up in the published version of a paper*.

Although association preferences documented in our study theoretically could be a consequence of either mating or shoaling preferences in the different female groups investigated (should we cite the crappy Gabor paper here?), shoaling preferences are unlikely drivers of the documented patterns both because of evidence from previous research and inconsistencies with a priori predictions.

Careful what sorts of editorial manuscript comments you let slip through, people.

__
*apparently the authors are trying to correct the record so it may not last in the official version at the journal.

One precious tic of academic writing I implore you to avoid is “…in the literature”.

Novel contribution to the literature…

Unknown in the literature…

Fill a gap in the literature…

You know what I mean.

The impression you create is that this is some silly self-referential game with only internal measures of importance.

Whether this is how you see science or not…..avoid creating this impression.

Talk about knowledge or understanding instead.

Attention FLOWbies!

September 3, 2014

A solution for you!

http://twitter.com/dtollerton/status/507180601397760000

Writing Process

August 31, 2014

I start a paper draft very early in the process.

Sometimes it is started before even a single bit of data has been collected.

It starts, often, with some literature that I am reading that starts to gel an idea. So I’ll jot down the author/date and some words related to my thinking at the moment. Could be a full manuscript ready sentence, sometimes just a few words.

At this point I don’t even insert “introduction” and “discussion” headings because I’m not sure where it is going. As time goes on there will be a tipping point where I take an hour to put in the structure.

Title page, headings, maybe some cut and paste methods that we’ll be modifying later.

I didn’t use to do this, but I have gotten better about writing up figures as they roll off the assembly line. Even before I know the end analysis, etc. So maybe I waste a little time if I have to redo analysis with more groups or something and reconfigure the graph.

I’ve found that it helps me later to know what we have and what we don’t have.

So now I might actually start a draft around a key figure that I really like. Stare at that graph in the Word file and the ideas start coming.

The key for me is to trigger early on just getting some words down on the paper in approximation of what I am thinking. At the moment.

Thoughts often change. I write many times more words in the drafts than will ever appear anywhere in print.

This helps me to think. To see.

On making progress

August 11, 2014

90% of the progress on my manuscripts and grants takes place during 20% of the time I am ostensibly working on them.

Anyone who thinks this is a good idea for the biomedical sciences has to have served as an Associate Editor for at least 50 submitted manuscripts or there is no reason to listen to their opinion.

When you are reviewing papers for a journal, it is in your best interest to stake out papers most like your own as “acceptable for publication”.

If it is a higher IF than you usually reach, you should argue for a manuscript that is somewhat below that journal’s standard.

If it is a journal in which you have published, it is in your interest to crap on any manuscript that is lesser than your typical offerings.

It is sometimes easy to forget that the trainees do not have a boatload of their writing published yet.

Even if their way of phrasing it kind of sucks ass……there’s that.