"Science 2.0 Open Access Lab Notebooks" Is Completely Absurd
April 23, 2008
Bora linked to a recent article at the Scientific American Web site concerning “Web 2.0” and its relationship to the conduct of science. The basic concept is that “THE WEB TOTALLY CHANGES EVERYTHING!11!!!!!1!ELEVENTY!11!!!!!1!” and scientists are and/or should be going to completely alter their way of communicating their findings to one another. One aspect of this is the notion of “Open Access Lab Notebooks”, pursuant to which scientists will keep their lab notebooks on publicly accessible Web sites where other scientists can view them and update the information they contain on a daily basis. In other words, scientists will essentially be continuously live-blogging their experimental activities in the lab. This is a totally fucking stupid idea.
Bill Hooker was quoted in the article very cogently explaining his perception of the appeal of this idea:
“To me, opening up my lab notebook means giving people a window into what I’m doing every day,” Hooker says. “That’s an immense leap forward in clarity. In a paper, I can see what you’ve done. But I don’t know how many things you tried that didn’t work. It’s those little details that become clear with an open [online] notebook but are obscured by every other communication mechanism we have. It makes science more efficient.”
This is completely nuts. Reading other people’s lab notebooks will decidely not provide a “leap forward in clarity”. You can barely understand your own lab notebook entries weeks or months after they occur. Lab notebooks contain a huge amount of totally irrelevant obscure information, the vast majority of which relate to failed experiments.
Half of the pages of my post-doctoral lab notebooks contain stuff like “God fucking damnit!!! Ran desired fragment off end of gel while drinking in bar!!!! Fuckit!!” Who has the time or inclination to wade through all of that irrelevant boring shit, just to find some nugget of useful information?
The whole point of publishing papers is that scientists wade through vast amounts of data, extract those that are informative, and generate figures that accurately and concisely distill out the useful features. Why would a working scientist want to expend the huge amount of effort necessary to wade through, extract, and distill this same information on her own?
My prediction is that “Open Lab Notebooks” is an idea that is going to go absolutely nowhere, at least in the biomedical sciences, and for a very good reason.
DrugMonkey 
April 23, 2008 at 10:28 am
You may be right, but this post makes me want to read more of your lab notebooks….
LikeLike
April 23, 2008 at 10:41 am
Umm.
A blog should be started where scientists can upload some ridiculous lab notebook entries. Kind of like http://www.skineart.com, but with sprinkles of science, chemical stains, and frustration.
LikeLike
April 23, 2008 at 10:45 am
“Lab notebooks contain a huge amount of totally irrelevant obscure information…”
What, like, ya know, TEH INTERNETS ™?!
Come on, PP, there are worse things out there taking up bandwith than lab notebooks.
*heroically resists temptation to be snarky and say something about this blog, which is really rather immensely more enjoyable to read than most lab notebooks*
“My prediction is that “Open Lab Notebooks” is an idea that is going to go absolutely nowhere, at least in the biomedical sciences…” I agree with you 100% on that- but I think it is not going to go anywhere for a rather poor reason- people are terribly afraid of getting scooped.
Also, some of them are even embaressed to publically admit how many gels they waste because they were drinking in bars. Which is a pity, as it can make a trainees life seem much less bleak to know how many things go wrong before something goes right.
I bet Edison’s notes from the famous “I’ll try a million things to get the right filament” experiments are now publically available. I’m not gonna go wade through that mess, but I can see why someone might think it would be cool to do so.
Also, I second Dave’s comment- now we want to see a sample PP lab notebook entry! Ideally with lots of ranting.
LikeLike
April 23, 2008 at 10:53 am
I think the online notebook is a waste of time, but I do wish that we could ask our seminar speakers to give two talks when they visit–one the standard “Here’s Our Fabbo Science” talk, the other a chalk talk entitled “Shit We Tried That Didn’t Work, Sometimes Repeatedly.”
I’d love to hear PIs summarize some projects or questions in their lab that, despite best efforts, ended up squandering a month or year because no one could get the experiment to work, for technical or unknown reasons.
This would achieve some of the goals of open notebooks (increased understanding of the path of science, and of dead ends) without the wading-through-people’s-chickenscratch part.
I know I’m dreaming, though.
LikeLike
April 23, 2008 at 11:04 am
You’re probably right – if a human were to attempt to read through a lab notebook from start to finish it would likely be a huge waste of time and effort. In my mind, the reasons for advocating open notebook science are:
1) We can develop search and indexing algorithms to extract the important information from the fluff, and deposit them in a database. As long as the database is searchable and well structured, this would be a very valuable asset – both to the scientists themselves (when writing papers for instance) as well as to the community.
2) In the event that someone is looking for specific information on a certain protocol, they will have easy access to the raw information from someone who has done it before. For instance, my notebooks have weeks of entries in which I am testing different variables in a procedure, and finally one works. These failures are what advocates of Open Notebooks are interested in archiving for other scientists.
LikeLike
April 23, 2008 at 11:14 am
I’m not sure the open notebook idea sits well with the concept of peer review either. There’s a reason why journals don’t random collections of half-completed experimental ramblings until they are collated into some sort of cohesive, readable document that other scientists have to approve. Isn’t there a danger to opening up the former type of document in the public domain? The people who are advocating the idea seem to be assuming that only scientists would read them – what’s to stop a journalist from doing so and reporting rubbish based on unpublished non-reviewed data?
LikeLike
April 23, 2008 at 11:34 am
I like Dr. J & Mrs. H’s suggestion on seminars. There’s one other version of such talks I’d enjoy. First the Fabbo Science Talk, where all the experiments are presented in logical order, and the authors look like savants who efficiently zeroed in on the truth.
Then the ‘reality’ talk, where they admit they were trying to do something else entirely, and had the complete wrong idea about things, only realized what was happening near the end, then hastily went back to do the initial ‘foundational’ experiments after the fact, so it would make for a coherent story.
And yeah, I agree the open notebook idea’s a bust. It would be too much work, wouldn’t usually make any sense to anyone, and anyone doing anything really interesting would be too worried about being scooped. In certain corporate science environments (biotech, pharma), having e-notebooks on a company intranet make senses, but that’s different.
LikeLike
April 23, 2008 at 11:45 am
Maybe my lab notebooks were different from PhysioProf’s. Years later, I could look back at mine and have an excellent idea what I was doing on a given day and how the preceding days’ of trying to get the experiments to work had motivated what I was trying this time around.
Part of what’s interesting about doing science is that it’s not always obvious at first what parameters are important. A lab notebook can thus be almost like a treasure hunt for the researcher trying to nail down what’s happening in a system s/he’s working with.
A lot of what’s in the notebooks may seem utterly irrelevant to most of the people who might read it, but there sure were times that it would have been helpful to me to see the notebooks of the people whose experiment I was using as a jumping off point for my own work. The “Materials and Methods” sections of the relevant journal articles were short on some crucial details.
I was expecting this was going to be a post worried about the conflict between open notebooks and protecting one’s intellectual property. To my mind, Jean-Claude Bradley makes the best case for open notebooks in areas of research where there’s no expectation of finding anything patentable and where effective collaborations are really important to answering the scientific questions in your sights.
LikeLike
April 23, 2008 at 1:00 pm
Phew. I thought I was the only one who filled his lab-books with “WTF?!?!?!” and “screw this, I’m done” type comments.
LikeLike
April 23, 2008 at 1:17 pm
I don’t see this idea going anywhere in paleontology. Our “lab notebooks” consist of field notes and museum notes. These theoretically could be uploaded, but I see a number of things standing in the way. My field notes often contain GPS coordinates for fossil sites that should *not* be publicly available – it invites site vandalization (and these data are available in museum accession records, for qualified investigators). Full notes are archived in a public institution, also. My museum notes often contain working ideas or reference to undescribed species – this too would be potentially disastrous to be on-line, due to the problems associated with taxonomic priority. Sure, there aren’t patent issues in paleo, but taxonomic and intellectual priority are pretty darned important!
LikeLike
April 23, 2008 at 2:29 pm
Probably the person advocating open lab notebooks didn’t think the idea through far enough. If you did want to share your WTF moments (or more pleasant ones), you could establish a Twitter network and twit what was going on to your friends/colleagues. This way you could share as much or little as you want and still share support and info.
LikeLike
April 23, 2008 at 3:19 pm
What about the simple act of having more transparent research agendas ? What is the amount of money that is spent on overlapping projects and races to be the first ego to produce X. Try explaining that to the funders. Exposing ongoing results also attracts collaborations/criticism/helpful suggestions.
Regarding the “noise”, you would subscribe to the amount of detail on any given topic that you wish for. It is not that far off from what already happens today if you think about the different journals and their degree of specialization. It is just adding another degree of detail.
LikeLike
April 23, 2008 at 4:04 pm
Why would a working scientist want to expend the huge amount of effort necessary to wade through, extract, and distill this same information on her own?
Two words to solve that problem: teh so-called “search enginez”???!!!!!11!!!!!
And you never know how information can be of value. Personally I’d love to know how many fragments I can run off gels without ruining my career so I can maximize drinking productivity.
LikeLike
April 23, 2008 at 6:56 pm
The information in bioscience research lab notebooks is of a nature and structure that making it “Googleable” is pretty much a total pipedream.
LikeLike
April 23, 2008 at 7:11 pm
I can see where Dr. Free-Ride comes from, but the days of science for the sake of science are gone. One could lose too much (career, reputation, money, etc., for rvealing the details of how one’s lab is operated or how one’s brain function (or dysfunction). As long as the published data stand the scientific scrutiny by other scientists, that record is sufficient to be archived.
LikeLike
April 23, 2008 at 9:37 pm
Oh man, the Borg™ wouldn’t load comments, this has been just killing me, I’ve been waiting all day to say this…
PP, you are utterly full of rancid batshit!
They wouldn’t — any more than they’d read every issue of every journal cover-to-cover in order to find the papers relevant to what they’re doing. It’s called a search engine, tucheslacker.
What’s not searchable about standard terms like “Western blot”, “antibody”, etc etc?
You can barely understand your own lab notebook entries weeks or months after they occur.
As Janet pointed out, speak for yourself. Not only could I read my notebooks from years ago and figure out what I was doing and how I was doing it, but so could any other scientist.
Your notes might not be as anal-retentive as mine, but they’re nowhere near as bad as you make out either, or you’d have never published anything! Projects can run over months or years, hundreds of experiments: no one can keep all that information in their head, but you need to have it available when you’re writing the ms at the end.
Irrelevant to you, right now — but what about later, or someone else? And as for failed experiments, if I’ve already fucked something up seven times and finally figured out all the tricks to make it work, why would you NOT want access to that information? You sure as shit won’t find it in any paper.
On top of that, what if the experiment “failed” in a different sense: all the controls worked fine, but the null hypothesis stands up? You know yourself it can be next to impossible to publish that kind of “negative result” — but it should be part of the knowledge base.
LikeLike
April 23, 2008 at 10:02 pm
Hey, dude! I was wondering where the fuck you were!
What about all the gel pictures and diagrams and all that other shit that people paste into lab notebooks? How’s that gonna be searchable? This shit just ain’t gonna work with the way bioscience researchers take notes, and no working scientist is going to be willing to spend any extra time or effort keeping notes in a way that will make it possible.
Computer-based lab notebooking systems have been around for a number of years. While they may be used to some extent in for-profit environments where a COO can simply mandate their use, they are not used in academia, and for good reason. So how is all the scrawling and drawings and other happy horseshit in people’s notebooks going to end up in searchable form.
I am willing to bet $100 that five years from now, Web-based Open-Access Lab Notebooking is at most a far-out fringe activity in the biosciences, done only by people with a ideological attraction to the idea.
LikeLike
April 23, 2008 at 10:06 pm
I somehow feel passionate frustrations in the author’s comment about Open Lab Notebooks, and I do not seem to comprehend where it comes from – it is not a big deal!!
Open Lab Notebooks can be really useful locally within a circle of collaborators. Since everything is typed up, scientists tend to think twice about what to write online, unlike hand-written ‘wtf’ containing raw contents.
The problem is when Open Lab Notebooks go completely public to people around the world – most importantly for intellectual property reasons… Who would want to put everything online for free? Authors of Open Lab Notebooks will have to make a choice whether or what content one is going to put online for public use.
So, Open Lab Notebooks should not be ‘raw’ like what we see in our written notebooks but rather, contents should be selected, filtered and polished. If this requirement is met, Open Lab Notebooks will be powerful tools for scientists. For those ‘purists’ of what science notebooks should be about “wtf??, raw, etc..”, online notebooks may not be appealing, but this kind of concern is insignificant compared to possible pluses that open notebooks can provide the scientists with…
LikeLike
April 23, 2008 at 10:18 pm
Ahhhh, now that I’ve got that off my chest, I have a more general point to make. “Open Access Lab Notebooks” is your own term, which appears to conflate Open Access (as in publishing) with Open Notebook Science, which is the extreme end of a spectrum of increasing transparency and cooperation. Extreme, as in it isn’t for everyone, and no one is arguing that it won’t work for every lab or every project. But you can’t argue with success (well, you can, but you’d have to be a douchebag): Jean-Claude Bradley’s lab has gone from scratch to lead compounds in their search for antimalarials; they’ve done it with undergraduate hands, in under two years if I have the timeline right, and they’ve done it entirely in the open. The reason they’ve been able to achieve this is that their open approach has enabled collaborators to find them long before any publication, and it’s their network of collaborations, from structure predictions to bioactivity testing, that has powered their success.
That example notwithstanding, it may well be that completely open notebooks will remain a small niche. Still and all, I contend that less competition and more transparency in science would be good things. Why not have one Open project in your lab — put it out there, and see whether you don’t find yourself with a nice new network of collaborators too?
And finally, do you really think that the internets haven’t OMG TOTALLY CHANGED EVERYTHING???!!! I met my wife online, and she’s someone I would almost certainly never have met any other way; I have friends all over the world of whom the same is true. I haven’t bothered with a phone book or street directory in years. I buy my books and plane tickets and pizza online. Want the lyrics to some song that’s stuck in your head? Want to read some Shakespeare? Want to know who won the Nobel for Physics in 1963? It’s on the internets. Hell, you and I can only have this conversation because of the internets. Think back to when you first got online (for me, that’s about 1993): would you have predicted any of this? Amazon, eBay, Google, blogs, YouTube, PubMed Central?
Now, compare all of that to what we’ve done in science. We’ve taken dead-tree papers, turned them into pdfs and made them (partly) searchable. Even that small step into web-powered possibility has made a huge difference, as you know yourself if you’re old enough to remember Index Medicus! It’s easy to be dismissive, and difficult to predict the future — but if you think that the web isn’t going to change the way science is done, you’re betting against a pretty fair track record.
LikeLike
April 23, 2008 at 11:11 pm
I somehow feel passionate frustrations in the author’s comment
Nah, that’s just PP. We’ve tried getting him to switch to half-caf but he claims he can taste the difference.
PP, check your spamfilter, my last comment (posted before I saw your #17) had a couple links in it.
More later, including “I’ll take that bet”, but right now I gotta set a blot up and go home and eat dinner.
LikeLike
April 24, 2008 at 12:35 am
What about all the gel pictures and diagrams and all that other shit that people paste into lab notebooks?
Usually they print the digital version in order to paste the hard copy into their notebooks! If their notebooks were digital, they could actually save work. Even if we’re talking polaroid gel photo, it’s a minute’s work to scan that sucker.
Computer-based lab notebooking systems have been around for a number of years. While they may be used to some extent in for-profit environments where a COO can simply mandate their use, they are not used in academia, and for good reason.
Sure, but that’s a good example of the sort of “betting against track record” I’m talking about. Remember the Walkman? It’s a half-ounce, fifty zillion gigabyte iPod now. Ever see an early mobile phone? They were the size of briefcases.
Electronic lab notebooks are not widely used in academia (I know a couple of people who use ’em) because they aren’t very good — yet. I’m all about the idea, and I’m still using a paper notebook, but with the advent of the ultraportable (e.g. Asus EEE) I’m hoping to try to switch fairly soon. The only thing I can think of that might be more cumbersome is hand-drawn diagrams, but (for me; ymmv) most of those will be easy enough to do with clip art in Corel Draw or something like that.
I am willing to bet $100 that five years from now, Web-based Open-Access Lab Notebooking is at most a far-out fringe activity in the biosciences
OK then. On April 23 2013, I will owe you or your charity of choice $100 if I cannot find two dozen papers (in regular, peer-reviewed journals) that are traceable wholly or in large part (say, more than half the figures) to Open notebooks. Deal?
LikeLike
April 24, 2008 at 3:32 am
Holy inflationary bangs! I remember when a single “ONE” in your string of 3-5 exclamation points was sufficient to indicate your ironic disdain for any hint of genuine optimism or excitement.
We need to tie this shit to a gold standard.
LikeLike
April 24, 2008 at 6:25 am
That’s a nice sentiment, but even in very small fields with no practical applications there are assholes who refuse to share their knowledge but don’t mind getting yours for free.
I think sharing lab notebooks and trying to make them readable is a good idea within the research group, though (start small before putting everything on web2.0). It would really help people who take over the work of someone who’s not at the lab anymore. My trilingual lab notebooks probably are a lost cause there though…
LikeLike
April 24, 2008 at 6:33 am
Who has time to “select”, “filter”, and “polish” 95% of the shit that goes in a lab notebook that is not good data and that never makes it into a peer-reviewed publication? And if you do all this shit, it ain’t a lab notebook anymore. It may be some kind of database, but it ain’t a fucking lab notebook.
For example, in my post-doc lab notebooks, a huge amount of the information concerns me dicking around with DNA, trying to make complicated transgene constructs. Once I successfully obtained the constructs, all the complicated weeks or months of dicking around became instantly irrelevant and uninteresting to anyone. No one cares about this shit, and throwing it out there on the Internet is a waste of time and effort for everyone.
Two dozen papers in the entire peer-reviewed literature could still be totally “fringe”. Here’s my counter-offer: You find six papers published in Cell in the calendar year 2013 each with more than half of the figures traceable to publicly accessible Web-based notebooks, and I’ll give you $100. And this has to be Open Notebooks, not just that some of the processed data were made available through a database prior to publication, or manuscripts made available on pre-print servers.
Cell publishes roughly 300 research articles per year, so this would only be 2%, implying a very generous definition on my part of the line between “fringe” and “mainstream”.
The Internet has changed a lot of shit. That doesn’t mean that uploading the lab notebooks of bioscience researchers onto publicly accessible Web servers isn’t a stupid idea.
I want my colleagues to filter information before they show it to me! And there are plenty of venues for bioscientists to share information well before publication as peer-reviewed articles, but in filtered processed comprehensible form.
LikeLike
April 24, 2008 at 6:33 am
Couldn’t resist the opportunity to weigh in here. Serious question though. I really do find this argument a bit disturbing. It seems to go ‘My lab book is rubbish, therefore everyone else’s is, therefore there’s no point making them available’. If I can’t read my student’s lab books and find things in them then they don’t go in the lab until its up to scratch. This is part of the training of a good scientist surely? Are you really saying that the primary data you base your publications on can’t be found? Or worse, that you think it isn’t important to be able to find it?
Quick note on the technical issues, particularly in terms of gels etc. We don’t print them, we upload them, and there is work going on on ways to make the searchable. Early days yet but it will come. We’ve got some tools to annotate the digital gels (yes you can circle a band and write WTF! on them if you really want, even better, I can choose whether to bother looking at your comments 🙂 and making that searchable is technically feasible with existing tools. Glueware will be a nightmare though. Its got a way to go though but what I would say is that we are trying to develop the tools that will solve what I think are the underlying issues (legible and comprehensible lab books that are searchable) you are getting at.
LikeLike
April 24, 2008 at 7:03 am
In most biosciences fields, the overwhelmingly vast majority of “data” isn’t even in lab notebooks. It’s in digital data files stored on servers: images, physiology traces, spreadsheets, etc. The lab notebooks document the process of performing experiments, but the data doesn’t go into the notebooks.
I’m not saying you couldn’t figure out some system for ongoing processing of useful data and uploading to Web servers in a comprehensible and useful searchable form. But 95% of all the information in a typical biosciences lab notebook is not going to be useful to anyone. And if people do go forward with systems like this, it would be totally misleading to call it “Open Notebook”.
Call it “Open Data” or something, but if it’s useful, it ain’t gonna be primarily the contents of real lab notebooks. No one gives a flying fuck about looking at gels of restriction digests of subcloning experiments.
LikeLike
April 24, 2008 at 7:30 am
I agree with the PP that digital lab notebooks will be as frequent as whole-cell recording experiments streamed live over the internet. A few people might do it for kicks. Methinks it will peter out quickly once people realize that they can’t do any damn experiments because they spend all their time blogging their experiments.
I hope that e-lab notebooks don’t catch on, because they lose a lot of character when ‘people are watching’. I remember doing in vivo recordings, sometimes for ~20 hours, in grad school. If the neurons were alive, I was recording. Things were pretty automated, so I would pass the time looking at the oscilloscope during stimulation trials while drinking Busch light. After about 10 beers one night, I finally realized what the neurons were doing and got a C/N/S paper out of it. I remember looking at my notes from that fateful night, scribbled with the drunken uncoordination that is only found on pub toilet walls: “THEY’RE FUCKING OSCILLATING!!!!” I wrote this about 10 times as I went back and looked at the sweeps from previous experiments that I had pasted into my notebook. Then, I perfused the animal, drank 2 more beers, had a smoke in our fume hood, and passed out. I was awoken by our TurboTech at 7 am.
I still have the notebook, complete with beer stains and coffee stains (and probably some drool). That’s not something I’d be willing to live-blog, or even transfer onto the web, though. It could only be appreciated as a scratch-and-sniff YouTube video; or in my blurred memories of grad school.
LikeLike
April 24, 2008 at 10:48 am
Ha! What a a great little anecdote. Hope you don’t mind I ganked it and posted over at Bayblab.
LikeLike
April 24, 2008 at 11:19 am
Quit yer wrigglin’. Here’s the definition of Open Notebook Science:
Let’s not get too hung up on the “notebook” part; it’s about information, the form doesn’t matter. I don’t care if you scribble it in a book, save it to a hard drive or tattoo it on your ass.
As you yourself point out, the overwhelmingly vast majority of experimental output is now digital (and I’d add that whatever isn’t, can easily be made so). Those files don’t mean shit without explanatory notes, though. Suppose I show you a Western blot — unless you know how the samples were prepared, which lane is which, and so on, it won’t be any use. For me, if an electronic notebook can be made workable, it would actually mean less work to be able to simply annotate the digital files directly. Both the files and the notes are what Open Notebook Science aims to capture.
In the current system, Jo(e) Postdoc does fifty or a hundred experiments and writes up a manuscript which contains five figures and a brief, often vastly inadequate description of the materials and methods. If I’m reading that manuscript and I want to use a novel method from it, I have to email J for the details or fuck around for three weeks working them out for myself.
It’s also entirely possible that J made some observations that didn’t fit the story for the manuscript, and didn’t make any sense to him/her — but would be just the final piece of whatever puzzle I’m working on. Those observations are lost in the current system — they stay entombed in J’s notes. Granted, in order for me to find those observations even if J keeps an Open Notebook, there has to be a searchable connection between my puzzle and the observations — say, a protein we both work on but neither of us realizes is active in both our systems. I’m reaching a bit here, but see also “betting against track record”.
Further, I’m taking J’s word for it that those five figures are representative of all of the data. Not to sound too much like S Rivlin here, but frankly there are a lot of lying motherfuckers in science these days and I’d rather be able to see the raw data, thankyouverymuch. There’s no space limit online; there’s no reason not to make ALL your data available. It makes no difference whether that’s in a manuscript supplement or an Open notebook (apart from the real-time aspect of the latter, which is a considerable plus). Most of the time, most readers won’t need the additional information — but the ones looking the hardest, the researchers doing related work, should have that information. You take a pretty hard line on how a scientist should respond to criticism of their work, and I agree with you. I’m saying that a scientist should arm his or her critics as well as possible. The data either stand up or they don’t; there should be no need to pick the best blot or cut that odd timepoint out or whatever — but we all know that happens, and more than it should.
Here, we agree. That’s why I find electronic notebooks so appealing — they allow for making “dark data” available without requiring extra time spent select/filter/polishing.
LikeLike
April 24, 2008 at 11:29 am
The problems with ELNs discussed here are true for the majority of ELNs or their substitutes out there. However, you guys are not computer scientists, and probably that is why you don’t know that most of these problems are in fact resolvable with the modern technology. E.g., you can break note documents in different parts and exclude certain ones from being shared. A step in that direction have been taken by iPad ELN – a free soft that is at the same time simple/flexible AND allows you to produce notes that are structured and clear for others, and share stuff with people in a controlled way. PARTIALLY open science is already a reality, the only question is how far this partiality will stretch.
LikeLike
April 24, 2008 at 11:35 am
“Who has time to “select”, “filter”, and “polish” 95% of the shit that goes in a lab notebook that is not good data and that never makes it into a peer-reviewed publication? And if you do all this shit, it ain’t a lab notebook anymore. It may be some kind of database, but it ain’t a fucking lab notebook.”
1. Everyone doesn’t or has time at the same time – not a relevant point.
2. Sounds like a lot of people in biomedical sciences seem to be enjoying a vicious cycle of doing research for 6 month ~ 2 years writing things on the notebook, struggle to publish in a peer-reviewed journal, get a NIH grant and go back to the first step all for no obvious reason (maybe they want power and fame by only thinking about publishing in peer-reviewed journals). Like a hamster on the wheel you know.. At least the internet offers an opportunity to connect with others you would never know otherwise (in and out of science), and will give someone a sense of direction of how one can do something at least relevant to a lot of others in the society. Open access lab notebooks with user feedback system may offer such opportunity.
“But 95% of all the information in a typical biosciences lab notebook is not going to be useful to anyone. ”
If typical = writing shits in lab notebook, of course, but remember that people in not-so-typical biosciences lab can be more professional and don’t write too many emotional shits like ‘wtf’. And they would write about how one would go about solving a problem, rather than being emotional about it. This scientific process reflected in notebook is what is going to be relevant and useful to a lot of budding and even struggling scientists.
Bottomline: if one wants a lab book to be personal for whatever reason, go ahead.. if one wants to make a lab book more relevant to general audience who may give a shit, put it online. There is no right or wrong answer. Stop rhetorical bullshit, and do something about it; whether to ‘ban’ open lab notebooks or to improve them in a useful way.
LikeLike
April 24, 2008 at 12:34 pm
Further, I’m taking J’s word for it that those five figures are representative of all of the data. Not to sound too much like S Rivlin here, but frankly there are a lot of lying motherfuckers in science these days and I’d rather be able to see the raw data, thankyouverymuch.
bill, I think that you seriously underestimate the degree to which this aspect would be used in a malicious way against normal scientists who are not “lying”. You seem to take a “well, what are you hiding?” position on this. Overlooking the amount of time that will be tied up if someone like a woo-meister, academic opponent, animal rights activist or Congressional points scorer comes after someone. Maybe you don’t make decisions in your scientific behavior that would lead to a toehold for a motivated idiot to question you endlessly, but I suspect every scientist does. Imagine every assumption or long-established “practice” in your techniques being questioned. Can you really prove beyond any doubt, with overweening references why you do things?
PP claimed he was at the bar and just fucked up….is there any documentation for this? How do we know he isn’t just hiding an adverse outcome? Huh? That antibody is “bad” you say? Where are the pubs from multiple labs proving this? The company asserts it works. Are you just lying?
From my end of the world. Did you mass spec every single drug you got from Sigma to make sure it was the right thing? take blood to verify the drug actually got in the rat? no? you say you clean the open field, but how do you know the rat can’t smell the lingering cues? all the rats are fucked up today and you just “know” the care staff turned on the lights when they weren’t supposed to, eh? can you proooooove it?
I can go on and on.
If you want a concrete example, wander on over to MAPS (www.maps.org) and take a look at how they went after Ricaurte. the point is not whether he was a lying motherfucker or not since we still do not know. the point is to see how a motivated opponent can go after you, even for stuff that is commonly accepted field-practice. if he wasn’t such a BSD with so many friends in NIDA his career would’ve been over. and that is without “open” access to his lab’s books….
LikeLike
April 24, 2008 at 2:43 pm
Can you really prove beyond any doubt, with overweening references why you do things?
Do I really have to? And when I can’t, isn’t it better to have those assumptions and elisions out in the open, in case I am doing something wrong? (Not morally wrong, but making-a-mistake, not-producing-reliable-information wrong.)
And if some ulterior-motive-packin’ shitweasel does come after me, what more can I do than open my notebook and say, have at it? The data are what they are, and there comes a point where you just have to cut certain conversations off; you can’t play whack-a-loon with every wackaloon that wanders along. If your data are public, then interested parties can make their own decisions.
I get that motivated miscreants (ID-ers, Congresscritters looking to provide value for those drug company bribes, etc) can mount endless attacks. I don’t get how transparency is more ammunition for them than it is defense for me. They’re gonna do what they do no matter what I do. Will I not come off looking a whole lot better if my data are public when I say “OK jackass, I’m done with you” after a couple rounds make their bad faith clear?
OK, that’s my usual rejoinder; now let me try something new. Thinking about running my own lab (HA! HA! but bear with me), I want to be as Open as I can, but the model is very new for biosciences and there are plenty of bad actors and free riders out there. So what to do? Apart from making as much use of Nature Precedings and similar, what I am thinking about is a semi-open notebook/data collection. What I mean is something that Google can index, but when you click on the link you have to tell me who you are before I let you read the whole thing. That way I can block access to known shitweasels, and prove who read what when if someone ever tries to scoop me.
Whaddaya think?
(@Physioprof: I think I’m getting screwed on the deal with the Cell restriction, but I just used up all my between-expts time on DM so I’ll have to get back to you on our bet.)
LikeLike
April 24, 2008 at 3:29 pm
It is an empirical question whether real working bioscientists are going to buy into this fancy vision. My prediction is they will not in any significant numbers.
It was an empirical question whether bioscientists would participate in on-line wiki-like discussion of peer-reviewed literature. It is turning out that working bioscientists have little or no interest in doing so. There is no reason to think this will change any time soon, regardless of fancy technological advances.
This all has to do with the nature of scientific information, scientific discourse, and allocation of professional credit. All the fancy technology in the world ain’t gonna mean jack fucking shit in the absence of fundamental changes in these structural features of the bioscientific enterprise. And there is no particular prospect that I see for such changes to occur any time in the foreseeable future.
These empirical issues are independent of whether such changes are a good idea.
LikeLike
April 24, 2008 at 3:50 pm
“[…]all the rats are fucked up today and you just “know” the care staff turned on the lights when they weren’t supposed to, eh? can you proooooove it?”
Yes. Go to here and then click on the ‘data’ image (ignore the error message and press ok). That red line is the light levels in one of my labs today. The others are temperature and other sensors. Of course 99.99% of the information is useless. Its the 0.001% that you care about that’s important. Capture it all and it may come in useful one day.
Of course it’s not perfect, and you may have availability issues (link seemed really slow when I just tried it but I got through eventually) but that’s why we’re working on it.
LikeLike
April 24, 2008 at 5:39 pm
While that is pretty cool Cameron, it seems like a lot of effort and expense to go to when you can just scribble a note in the book a couple of times a year!
I echo PhysioProf’s points about all the extra headache and overhead for no apparent productive gain.
bill, my point is that being open the way you want lowers the bar for annoying harassment. sure, if someone wants to go after you, they can FOIA you to death. sure. but it is a high barrier to overcome. rightly so. lower the barriers and broaden the amount of data you make available and the lower-threshold nutjobs will arrive at your door. no, I don’t agree that the obscure benefits of potential fact-checking and good-collaboration overcomes what I see as the overwhelming probability of nuisance harrassment.
…and I haven’t even gotten to the scoopage issue yet.
LikeLike
April 24, 2008 at 6:28 pm
what I see as the overwhelming probability of nuisance harrassment
This is something I don’t get. Maybe I haven’t been looking in the right places, but I don’t see an epidemic of nuisance harrassment. Splain me who is going to go after Jean-Claude Bradley, f’rinstance, and why they would do that?
I also don’t understand why I would have to respond to every nutjob would-be harrasser. A simple “fuck off, nutjob” goes a long way.
Nor have you explained what I’d be making available to the nutjobs that would be more useful to them than the standard “what’s he hiding, why won’t he let us see his notes?” — notes which, as you point out, they can FOIA anyway. If they’re low-rent nutjobs who wouldn’t bother with FOIA, then presumably they aren’t making any sense so why can’t I just tell them to fuck off?
I guess I don’t understand where the nutjob hordes that are apparently massing on your horizon get all their power.
LikeLike
April 24, 2008 at 6:30 pm
@DM: have you written about this before? Do you have more examples to hand? I went to the site you linked, but there’s no quick way to get the story there.
I gotta think I’m missing something, because your concern seems so overblown to me.
LikeLike
April 24, 2008 at 6:58 pm
With respect to MAPS/Ricaurte, start here
http://www.maps.org/research/mdma/#ricaurte , yeah, you have to click a few things but the way they went after this guy should be apparent. Again, not debating the relative merits. Just the fervor with which someone might choose to go after a scientist.
I gotta think I’m missing something, because your concern seems so overblown to me.
I think you are entirely unimaginative about the motivations of those who oppose certain selective types of science. Just on Sb alone you can find hints of the animal rights activist, the vaccine / autism nutjob, the homeopathic remedy and Complementary/Altie Medicine types. The Cannabis and Ecstasy avengers come out on occasion when I post something but for the most part you may not have seen the real nutjobs, they exist nevertheless. There was a recent “why are we doing sex research” post that links in my mind to prior Congressional efforts to defund research on an individual grant basis just because a single Congress critter thinks it inappropriate. Can you imagine if a Toomey set his staffers onto the lab books first and picked out some ambush arguments? I haven’t followed it recently but in the early days the anti-HIV research was a palpable thing- ’cause it was a “gay” disease and they “deserved it”, you know. frankly we still see some of this with drug abuse research. in a related event, Limbaugh, for example, decided to go after one of the drug-vaccine guys as a lark (covered here)-can you imagine if his legions could then go and sift through that lab’s databooks for grist? latched onto some crap toehold and cycled it back to Rush? Who needs that sort of headache?
There is another factor here which is that in my career to date since grad school, I’ve paid attention when labs generating papers that I like / follow disappear temporarily or permanently. Often times it is because of various stripes of nutjob attacks. When subtle, non news-making stuff has occurred on campuses that I’ve been on, the impact of an “investigation” into potential wrongdoing has been clear to me.
Again, I’m not opining on right or wrong on any case whether it be disgruntled trainee accusations of stealing, accusations of science fraud or animal protocol violations. I’m saying that justifying oneself against an attack can be absolutely devastating. Even when one is clearly in the right. I’ve seen it put people totally out of science. Crippled some labs output for years. The impact is quite high.
Against this the “gain” seems very uncertain and of minor magnitude.
LikeLike
April 25, 2008 at 8:18 am
While that is pretty cool Cameron, it seems like a lot of effort and expense to go to when you can just scribble a note in the book a couple of times a year!
Less effort and expense than the storage costs for a rat in the UK over a year (but that doesn’t include the development costs admittedly). Come on though. You say its not worth putting the data out cos it’s all rubbish, we say some of it is useful, then you say its too hard, I say we’re trying to make it easier, and then you say its all rubbish. If you think light exposure of your rats is an important variable, why on earth aren’t you measuring it?
Our technical aim is to make it easier to stream all the data out of a lab. And I mean all the data that we practically can get at. That’s every instrument, a bunch of sensors, various other bits of useful info like who is where, and the general notetaking stuff that stiches it all together. In fact without that, the rest is pretty worthless. We are a reasonable way down this road and if you are worried about your rats then I would say some of this might be useful to you. Given that funding agencies are moving towards compulsion in the proper storing of data and making it freely available you may need some of these tools in the future to make it easier for you to comply. That to me makes our technical research in this area worthwhile, regardless of whether you choose to use it to be fully open or not.
Our working social theory is that by making the data available we do better science ourselves and enable others to do better science. As PP says it’s an empirically testable hypothesis. We are now testing it. Evidence so far is patchy, but positive, but this may change as the numbers of people involved rise and as the spread of disciplines change. Frankly the type of environment you describe is so nasty I wouldn’t want to go anywhere near it.
LikeLike
April 25, 2008 at 8:58 am
What evidence do you have? (I assume it’s streaming on the Web somewhere!)
LikeLike
April 25, 2008 at 9:42 am
“Shit We Tried That Didn’t Work, Sometimes Repeatedly.” I LOVE it!! The best talk I ever went to was Paul Nurse. He started, ‘Everyone comes and talks about their beautiful results and correct hypotheses. I’m going to tell you about all the things that went wrong.’
Lab notebooks: my favorite entry in mine is: “[date here]: PI wears Chairman Mao hat to lab meeting.”
LikeLike
April 25, 2008 at 10:15 am
What evidence do you have? (I assume it’s streaming on the Web somewhere!)
Naturally, but I feel I should make you search for it 🙂
The best example is Jean-Claude Bradley and collaborator’s recent identification of novel anti-malarial lead compounds. This project has been done completely openly but what is perhaps more important is that the collaborators who did the screening came across the synthetic work through the online lab book of JC’s group. This is the only case I am aware of of a project being open right through from conception to results stage. You can follow the links back from http://tinyurl.com/5bdjo8
More generally I’ve made some random requests for help, some more successful than others, which have helped us take a bit of work forward: http://tinyurl.com/5cf6wq and http://tinyurl.com/559f59
All little things as I say. Small communities and patchy but I would contrast that against no negative experiences of the type you suggest whatsoever so far across JC’s and my experience, and others such as the OpenWetWare group. At least none that have involved unethical or nasty behaviour (beyond people actively telling us we are idiots for making our data available of course :).
As for evidence that making data openness compulsory can have big benefits; I submit a) GenBank and b) PDB (which incidentally is requiring deposition of the raw experimental data these days). This is of course after publication but it makes the case for data availability. Before Genbank people questioned the value of making sequence data available because no-one knew what to do with such a mass of data. Now it is a crucial part of bioscience. Our case is that the uses (and re-uses) of other data won’t be obvious until clever people get their hands on them.
LikeLike
April 25, 2008 at 10:20 am
I should add that those little bits of evidence (that are currently stuck in the moderation queue – damn links) were done with essentially zero funding and the spare time of a couple of students.
LikeLike
April 25, 2008 at 3:41 pm
I think there may be a disconnect in the discussion because of unstated assumptions. Every researcher participating in some form of Open Science has their own motivations for doing so. When we project onto others our own motivations, their behavior can seem irrational.
Not everyone is interested in patenting every single outcome of every single project. Not only is it expensive but also a tremendous time sink. If you’re working for a company you probably don’t have the luxury of ignoring intellectual property but in academia many can and do every time they publish a paper. And that’s a very good thing – otherwise we would not have unpatentable scientific fields like cosmology.
Not everyone is interested in appearing to be perfect to the detriment of sharing the messy truth of scientific research.
Not everyone derives satisfaction mainly from defeating an opponent in a competitive game. For some just creating something new and useful is enough. Of course people want credit for what they did but it doesn’t have to be a zero sum game.
Not everyone writes comments on blog posts to convince others of the error of their ways 🙂 Some do it to increase the probability of finding like-minded individuals.
Cameron, Bill and others have made some strong points to explain their interest in Open Notebook Science. But I don’t think anyone here is proposing to impose this on anyone.
People will keep doing what they are motivated to do and information systems will be built and science will progress. Some of us have a hypothesis that science can move more quickly with ONS and we’ll see how it plays out. I’m not sure the number of papers in Cell is the best way to quantify scientific progress. There are some very good reasons in the current system why articles in Cell are useful for researchers and they have little to do with objective scientific progress.
LikeLike
April 28, 2008 at 12:54 pm
Finally figured out what’s wrong with this, from my perspective: I wouldn’t publish in Cell, even if they’d have me — they’re not OA! I suspect that the vast majority of Open Notebook types are going to feel the same way.
But you know what? I can’t think of a significantly better way to frame the conditions, so, fuckit, I’ll just put my money where my mouth is. You have yourself a bet, my man.
LikeLike
April 28, 2008 at 1:08 pm
Schweet!!!! Hopefully 100 smackeroos will still be enough to buy a beer and a shot of fucking Jameson in 2013!
LikeLike
June 5, 2009 at 8:43 pm
A lot of what’s in the notebooks may seem utterly irrelevant to most of the people who might read it, but there sure were times that it would have been helpful to me to see the notebooks of the people whose experiment I was using as a jumping off point for my own work. The “Materials and Methods” sections of the relevant journal articles were short on some crucial details.
LikeLike