Reading the literature and writing

Well respected addiction neuroscientist Nick Gilpin posted a twitter poll asking people about their favorite part of doing science.

My favorite part of science is:

— Gilp (@The_Gilp) April 17, 2020

Only 4% of the respondents voted for “Reading the literature”. Now look, it’s just a dumb little twitter poll and it was a forced choice about a person’s favorite. Maybe reading is a super close second place for everyone who responded with something else as their first choice, I don’t know.

However. Experience in this field, reviewing manuscripts submitted to journals, reading manuscripts published in journals, fielding comments from reviewers about our manuscripts and trying to help trainees learn to write a scientific manuscript suggests to me that it is more than this.

I think a lot of scientists really don’t like to read deeply into the literature. At best, perhaps they weren’t ever trained to read deeply.

As a mentor, I tip toe around this issue a lot more than I should. I think, I guess, that it would be sort of insulting to ask a postdoc if they even know how to read deeply into a literature for the purposes of writing up scientific results. So I take the hint approach. I take the personal example approach. Even my direct instruction is a little indirect.

The personal example approach has a clear failure point. The hint may suffer from that somewhat as well. The direct statements of instruction that I do manage to give “Hey, we need to look into this set of issues so we know what to write” is only slightly better. The failure point I am talking about is that part of reading deeply into the literature is a triage process.

The triage process is one of elimination. Of looking at a paper that might, from the title or eve the Abstract, be relevant. In the vast majority of cases you are going to quickly figure out that it is not something that contributes to the knowledge under discussion in this particular paper. To me this is under the heading of reading the literature. It doesn’t mean you read every paper word for word, at all. Maybe this is part of the problem for some people? That they think it really means read every frigging search result from start to finish? I can see how that would be daunting.

But it IS work. It takes many hours, at times, of searching through PubMed, Google Scholar or Web of Science using several variant key word searches. Of then scanning papers as needed to see if they have relevant information. Sometimes you can triage based on the abstract. Sure. But a lot of the time you have to download the paper and take look through it. Sometimes, you are just checking for a detail that didn’t make it into the abstract and finding that, nope, the paper isn’t relevant. But sometimes it is and you have to read it. But then, maybe it would be a good idea to look at the citations and follow the threads to additional papers. Maybe you should use Web of Science to find out what subsequent papers cited that particular paper. All of this work to come up with three or two or even one sentence with three surviving citations.

The person who is following my writing by example doesn’t necessarily see this. They may think I just pulled three cites at random and kept on going. They may think that somehow it is my vast experience that has all of this literature in my head all at the same time. Despite me saying more or less the above as reminders. When I say things like “hey, everybody works differently, but when I do a PubMed search on a topic, I like to start with the oldest citations and work forward from there”, I mean this as a hint that they should actually do PubMed searches on topic terms.

(I admit in early graduate school I was really intimidated by the perception that the Professors did have encyclopedic knowledge of everything, all of the time. I don’t think that any more. I mean, yes, my colleagues over the years clearly vary and some are incredibly good at knowing the literature. But some are just highly specific in their knowledge and if you get too far outside they flounder. Some know lots of key papers, sure, but if you REALLY are digging in deeply, you figure out they don’t know everything. Not the way a grad student on dissertation defense day should.)

So I’m not talking about reading the literature in a “keep up with the latest TOCs for all the relevant journals” kind of way. I’m talking about focused reading when you want to answer a question for yourself and to put it into some sort of scholarly setting, like a Discussion.

I have no recollection of being taught to read a literature. I just sort of DID that as a graduate student. I was in the kind of lab where you were expected to really develop your own ideas almost from whole cloth, rather than fitting into an existing program of work. Some of my fellow grad students were in labs more like mine, some were in labs where they fit into existing programs. So it wasn’t the grad program itself, just an accident of the training lab. Still, it wasn’t as though I chatted much with my fellow students about something like this so I have no idea how they were trained to read the literature. I have no recollection of how much time anyone spent in the library (oh yes, children, this was before ready access to PDFs from your desktop) compared with me.

So about this poll that Professor Gilpin put up. How about you, Dear Reader? How do you feel about reading the literature? Were you taught how to do it? Especially as it pertains to writing papers that reference said literature (as opposed to reading to guide your experiments in the first place, another topic for another day). Have you tried to explicitly train your mentees to do so or do they just pick it up? Is it always the case that reviewers of your manuscripts hand wavily suggest you have overlooked some key literature and they are right? or is it the case that you know all about what they mean and have triaged it as not relevant? Or you know that what they think “surely must exist” really doesn’t?