Why GlamourMag Science is a Disaster for Scientific Progress, Part Eleventy
April 14, 2009
Browsing over DamnGoodTechnician’s recent posts for the one I was going to excoriate gently discuss, I ran across this gem:
Part of my project has been to recapitulate the results from a fairly recent Nature paper. I’m not sure how many of you have attempted this feat, but I believe deciphering the Rosetta Stone may have been simpler. What concentration of these ingredients did you use? WHICH of these ingredients did you use? How long? How many media changes? Transfection? Infection? Gack. The kicker is that the protocol induces a switch in cell fate, and the timecourse for that change is more or less two weeks, so any conditions I set up today as a “Let’s see if this set of conditions proves you guys weren’t lying” experiment won’t be ready to go until nearly May.
I’ve been banging my head on this protocol for about two months now
Word.
As we all know, personnel are the most expensive part of research. Wasted staff (techs, grads, postdocs and PIs) time is wasted money and wasted opportunity. This stupid failure to actually publish the real methodology (yes, including how many times the prep failed before you lucked onto the one that worked and really don’t know why!) wastes my taxpayer dollars. I oppose all the primary forces which encourage shenanigans with Materials and Methods sections including lame excuses about print journal real estate.
Competitiveness is the main one, however. This idea that you are better off if nobody can replicate your work has to stop. We have to break these contingencies that value selfish science above collaborative science.
I have a sort of dual perspective on this because I work in areas that are much more open and much less suspicious and competitive than what I deduce is the average for certain other biomedical subfields. It certainly helps that there are relatively fewer labs working on topics in the behavioral pharmacology of substance abuse compared with a gajillion bench jockeys working on some freaking esoteric transcription factor or whatnot.
But this gives me an appreciation for the benefits as well. I feel a twinge of selfish concern only very, very, very, very rarely when it comes to sharing methods and technical info with all comers. I try to describe my Methods as completely as possible and never fear making it too easy for the competition. I take questions* from anyone that wants to do something we’ve published and try to help trouble shoot as best I can.
Guess what? This a very nice way to work. More of us should try it.
__
*and for some reason a lot of times the answer is “um, do what we said we did, not the shortcut modification you pulled out ’cause it seemed easier. there’s a reason we wrote that!”
DrugMonkey 
April 14, 2009 at 12:49 pm
FWIW, methods – or lack thereof – in papers are a real pet peeve of mine. qaz had suggested that we contact the journal to see what we can do about getting actual information about how to do the experiment. In fact we did one better – we invited the PI to come give a seminar here at MassivePharma, and participate in some round-table discussions (I mentioned this in a veiled “pencil” analogy). When asked about this protocol, the PI hemmed & hawed and suggested we should open up a “consultation agreement” with their lab instead. Holding a protocol hostage? Now that’s pretty damn lame.
Also – putative excoriation?? WTF did I do?
LikeLike
April 14, 2009 at 1:28 pm
From a completely different corner of the world [1], I can only say:
Vast understatement.
[1] Commercial semiconductors. Some of the most rewarding times in my career have been collaborations with direct (as in cutthroat) competitors.
LikeLike
April 14, 2009 at 1:42 pm
Call me crazy but … one would think that with the adoption of supplementary information (I know M&M isn’t supplementary) inclusion in most major journals that “page restrictions” wouldn’t really matter any longer. Any claim by the authors that the M&M would be too long should just be told “Include it as Supplementary Material”.
When I review papers I always spend a large amount of time on reviewing the M&M, and I’m always asking for clarification and elongations. If I don’t know exactly what they did, how can I come to the same conclusions they did in the Results and Discussion? Perhaps something I’m thinking (but can’t say for sure because I don’t know 100% what they did) might have been answered satisfactorily if I had a complete M&M.
*shrug*
So, at any rate … I concur. More complete methodology would be preferable. I try to push this agenda by demanding it in the papers I review. If others feel the same way, perhaps they should consider doing the same thing.
LikeLike
April 14, 2009 at 1:54 pm
When asked about this protocol, the PI hemmed & hawed and suggested we should open up a “consultation agreement” with their lab instead. Holding a protocol hostage? Now that’s pretty damn lame.
It is scientific misconduct in my view. As qaz noted at your blog, there is explicit language in some journal policies (I believe Nature is one) that make it clear that this is an obligation. to describe your Methods clearly and provide reasonable access to necessary reagents. Helping to clarify and troubleshoot a protocol may not be explicit but it sure as heck is implied.
Omitting methods is one level of bad but resisting when asked directly is even worse because it cannot be excused as an oversight. It has to be interpreted as intentionally violating the spirit of communicating replicable science.
I hate that we have come to this.
LikeLike
April 14, 2009 at 2:02 pm
The extent to which this is a problem is highly field specific. In my current field/subfield, people are extremely free and open about sharing reagents and information.
In the field I started my post-doc in, it was the opposite. We would routinely receive empty pBluescript instead of the cDNAs we had requested, or different point-mutants than we had requested. This happened way more frequently than could be attributed to honest mistake.
LikeLike
April 14, 2009 at 2:07 pm
I guess some people don’t believe that pharmaceutical companies count as places where your work ought to be replicated. The shittiest part is that academic science is funded on the principle that someday their work can “improve human health by increasing scientific knowledge related to disease and health” (from NIH.gov). When a DRUG COMPANY is asking about your stuff – you know, the kind of company that can translate that science into medicines – that seems like the best place to be super-explicit about how you did your experiments.
But maybe that’s just me. And in the end, I did manage to recapitulate what they describe – after a couple months of trial & error & error & error & error.
LikeLike
April 14, 2009 at 2:25 pm
When I entered my current lab as a graduate student, I heard some horror stories about attempting to get certain transgenic mouse strains that had been described in papers only to find out that the strains had not been saved in any way, often due to simple loss of interest and a seeming inability to competently set up a colony or hand it off to someone else.
LikeLike
April 14, 2009 at 2:27 pm
I guess some people don’t believe that pharmaceutical companies count as places where your work ought to be replicated.
Some of my best friends are in pharmaceutical companies….ha!
This is a slight complication, admittedly. I find people on the research end of pharma to be wonderfully naive about how academics should interact with them. You all seem to think that you should be treated just like any other not-for-profit scientist. I do not think it should be the same- but the point hinges on the effort involved, not the obligation to be forthcoming.
In short, I think that if you work for a for-profit company you should have to compensate the academic as a consultant if they are to spend real time on your issues. Correspondingly, if the company is offering a consult gig, the scientist should do what they can to be accommodating.
Discl: I give lots of advice to for-profit company scientists gratis but I’ve also operated on a paid consulting basis when the amount of effort involved was substantial.
LikeLike
April 14, 2009 at 2:39 pm
I suppose if the pharma researcher is asking you for unpublished work, or for long slogging hours of consulting on a project with no compensation, then sure – hold out the empty purse & wiggle it around. On the other hand, if you’re refusing to clarify published work without $$ changing hands? Under those circumstances you might have to ask if you’d do the same thing to a colleague at another university. I don’t believe that’s my naiveté talking, but feel free to correct me if it is.
LikeLike
April 14, 2009 at 2:46 pm
It’s being naive about human nature. Academics feel under compensated, no matter the rank. They think of BigPharma as being hugely rich moneybags looking to profit off their hard work. (which in the larger sense is indubitably true. the BigPharma whining about their R&D costs always, always, always neglects to mention how everything is built on a rock solid foundation of public-funded work. )
If I was managing a unit in a for-profit company, I would argue for throwing down $1K/day consulting fees anytime a scientist was being brought in and asked for advice. The good will purchased, and the future help without additional compensation would make it a very good deal. How much time and effort, translated into $$ did you burn working on this? How much would have been saved if you just threw the scientist a one-day consulting contract?
LikeLike
April 14, 2009 at 2:59 pm
One of our careery-day things got me talking with an IP lawyer (I think he was actually working down at NIH). We had a great conversation; he really saw himself as fighting for the scientists at his institution.
Academics tend to think of materials and methods (like most things) as public domain. Yet methods/procedures are potentially patentable. It would actually not be unheard of to help a guy at BigPharma out, only to have them scaleup and patent your process (with maybe a few tweaks). It wouldn’t even be totally unheard of for BigPharma to then turn around and sue the academic researchers for infringement*.
It’s vaguely possible this academic who was so unforthcoming has simply been warned very effectively by his tech transfer office.
*note- obviously, in the case described, if the legal system works properly, this sort of suit would never, ever actually be successful; and indeed I’ve never heard of one being successful. It doesn’t have to be successful to be a PITA though.
LikeLike
April 14, 2009 at 3:03 pm
My department is set up to replicate exciting work from across the Ivory Tower, and translate it into a compound screen. Would this be more efficient with input from those academic labs? Hells yes. Would MassivePharma lose its IP advantage, since even with a CDA in place the information would undoubtedly leak? Hells yes as well. I’m not sure how much good will we’d get by insisting on a CDA in the first place..
I’m not trying to suggest that it’s all tears and whimpers here at MP: just that the *point* of a methods section is so that other scientists, wherever they may be working, can replicate your results. It’s not supposed to function as a gate so that you can grant entry to your little niche experiment to only those you want in.
[boy, that sounds much nastier than I mean it..]
LikeLike
April 14, 2009 at 3:13 pm
right. so the entire model of your unit is to take the ideas and intellectual property created by academic scientists under public funding for free, so that you can create drugs or therapies which you then sell back at exorbitant markups to the public. sounds like a nice little scam.
LikeLike
April 14, 2009 at 3:25 pm
No, it sounds like ‘economic development’. Don’t forget that this, plus training the technical labor force, is a major argument for academic science and public science funding in general. People don’t necessarily care about discoveries, unless they perceive those discoveries as useful. In American, ‘useful’ generally means someone can make money off them.
Cynicism aside, the translation of basic research into a real honest-to-god drug takes a lot of work and enormous $$ investment/risk on the part of big pharma. It doesn’t matter if you find a cure for cancer if no one is willing to manufacture or distribute it.
LikeLike
April 14, 2009 at 3:27 pm
Obviously, the answer is for everyone to publish their step-by-step methods in a reputable methods journal (preferably one that has no page charges and pays authors a royalty based on the use of their articles). Everybody wins, you get another publication to add to your CV, some $ in your pocket and other researchers get detailed protocols to help their work move forward. Then again, I’m kind of biased here.
LikeLike
April 14, 2009 at 3:36 pm
I know of a couple academic labs that are embracing high through put screening with some success. I don’t know if they will really compete with the big pharma folks.
LikeLike
April 14, 2009 at 3:38 pm
Oh, it ain’t free – quite often it’s for $$$$, like when we license a mouse model (as we’re doing for 90,000 euros/year for a line for part of my project) or specific cell line (as we did for $2500 for the use of a cell line for three months before the MTA insisted we destroy it) or use of a process (as we did for the $50,000/year we paid for the protocol for purifying a protein complex).
Moreover, it doesn’t matter that it’s a pharma company trying to repeat what you published – if your work can’t be replicated based on how you said you did it, then what is your “discovery” really worth?
LikeLike
April 14, 2009 at 3:50 pm
if your work can’t be replicated based on how you said you did it, then what is your “discovery” really worth?
on this issue I suspect we agree. OTOH, “how you said you did it” is always, always going to be incomplete. Methods descriptions assume an awful lot of generic expertise and sometimes there are lab lore issues that simply are not included. These are often fixed with a simple exchange of email or a phone call. or sometimes there are legitimate unknowns that take much effort to track down. these should be relatively easily distinguished from intentional deception or obfuscation intended to keep anyone from easily duplicating your work.
I’m sort of surprised that nobody’s launched the “it took us a lot of time and effort so why should anyone else just jump to the end on our coattails” gambit yet….
LikeLike
April 14, 2009 at 4:05 pm
not 6 months ago i networked my way to an invite to the originating lab of supercool technique x to learn it. they were totally awesome, more than happy to share. happy when i ran into them recently and i told them how great the assay is working. i’ve exchanged protocols with heavy hitters in my field for comparison and discussion purposes. maybe it’s just my field.
i guess if it’s pretty variable across fields, then maybe i take mine for granted.
LikeLike
April 14, 2009 at 4:19 pm
Self-preservation? The Nomex is out to the cleaners?
LikeLike
April 14, 2009 at 4:23 pm
My lab’s most recent paper (in press for a ~13 IF journal) had, when we originally submitted it, lots of methodological detail & discussion that we thought very important (in part because it showed how previous/widely-used methods are massive roads to crap).
Reviewer #1 had fits over the fact that we were trashing a cherished approach that’s garnered a lot of high profile attention & use lately. The editor ultimately told us to ignore this person.
But Reviewer #3 still told us that the methodological details along these lines were ‘a distraction’ and suggested they be removed or reduced to ‘a single sentence’. What could we do? We cut the stuff, as directed.
There are still details in the methods section and supplementary data, and ultimately we show how the old bad ways can be circumvented, but the really useful methodological stuff has all been omitted and/or whitewashed.
LikeLike
April 14, 2009 at 4:25 pm
Isn’t the point of publication to submit a piece of information to the community at large for verification? If a piece of information can’t be verified, then it has no worth, because people don’t know anything more than before you published the information. It’s actually worse than before, because others think they know something that (unknown to them) can’t be verified and thus may or may not be true.
If someone wants to establish a technique as a trade secret, they know where to go. Journals are not that place – if you publish, you are expecting recognition for your work in return for telling people what you did and how to do it. Telling people you have a neat result but not telling people how you did it is unuseful and dishonest, and the people who do so should be treated accordingly. If I want fiction or fantasy I can get it a lot cheaper than Nature or Science (OK, maybe not a lot), let alone JACS, Angewandte, or Org. Lett.
LikeLike
April 14, 2009 at 4:33 pm
How would a trainee go about learning more about IP law and protection of IP? Becca, you said that you found an IP lawyer at a career-day event. Would it be advisable for a program to bring in an IP lawyer to give a seminar to the grad students about these topics?
The business world and academia are so different, it’s a culture shock to interact between the two. I worry that as an academic I won’t know “how things work” enough to protect myself or play it smart.
LikeLike
April 14, 2009 at 4:35 pm
Yes and No. It depends on what you mean by “point”. If by “point” you mean “value to the scientific community”, then yes. If by “point”, you mean “value to the authors”, then this is just the beginning of the analysis, not the end.
LikeLike
April 14, 2009 at 4:56 pm
Yeah, the first one was what I meant. Publications are a quid pro quo – the correct publication (the first point) is the quid, and its consequences/rewards to the authors are the quo.
If you don’t specify what you did, there’s no quid – no contribution. No quid, then no quo. The quo might be the point for an author, and if he can get it without having to give something up, he will. It would seem to be in journals’ interest to increase the cost of nonfull compliance – otherwise, they end up being more like “Penthouse Letters” for scientists without the misogynism or sex than actual journals. (“I was in my lab one day and this dude in a lab coat and sunglasses and a gleaming lab coat said that he didn’t have money to pay for for his solvents but he would show me how to make transparent aluminum if I gave him the solvents for free.” …)
LikeLike
April 14, 2009 at 4:56 pm
As this post suggests, we need to be very careful when these two aspects come into conflict.
The problem for me is the obsessive focus on GlamorMagz, and the implicit suggestion that the only science that matters is published therein.
LikeLike
April 14, 2009 at 4:58 pm
Sara (#22): Call your Technology Transfer Office (maybe called ‘Office of Technology Management’, or ‘Intellectual Property Management’, or something like that). They almost certainly have someone who would love to come talk about what they do and what they can do for you.
Small non major research universities might not have an office like that. I am not sure what you do then.
LikeLike
April 14, 2009 at 5:08 pm
It would seem to be in journals’ interest to increase the cost of nonfull compliance
And since it appears to be in Science’s and Nature’s respective interests to encourage such practices (both with the page real estate dodge and with encouraging paranoid, competitive science in various ways)why would you think this?
LikeLike
April 14, 2009 at 5:09 pm
DM (aka “troll”): I’m sort of surprised that nobody’s launched the “it took us a lot of time and effort so why should anyone else just jump to the end on our coattails” gambit yet….
Here’s the background:
If I’m developing a new method and it takes me 3 years to get it worked out, what does that do for my “productivity” by this metric? If you insist on “productivity” (which you do), and you are competing against me for grant dollars (which you are), and you don’t take time out to help the field by developing new methods yourself (which you aren’t), how can I just give this method away to you free of charge and stay in business?
LikeLike
April 14, 2009 at 5:16 pm
“specific cell line (as we did for $2500 for the use of a cell line for three months before the MTA insisted we destroy it)”
“Warning! This cell-line will self-destruct in three months. *tickticktickticktick*”
Sorry. Sounds awful to deal with. But slightly amusing in the abstract. *cue Inspector Gaget song*
“Would it be advisable for a program to bring in an IP lawyer to give a seminar to the grad students about these topics?”
It sounds like a great idea to me! But then, I like talking to lawyers (as long as I don’t need their services, and I’m not being charged for it!). If you’re ignorant of the perils of IP and interfacing with the corporate world as an academic, at least you’re not alone.
I get the distinct feeling most academics are just annoyed by MTAs, and don’t really get why all these legal hoops are necessary (in fairness, it is very, very, very easy to get that feeling when you are impatiently waiting on a cell line that is available down the hall but you can’t use until the paperwork goes through, and that you don’t even want unless it makes IL-12… *grumble grumble*).
LikeLike
April 14, 2009 at 5:19 pm
One word: pipeline. If your lab is working on one thing at a time, you are completely fucking up. By the time your competition is doing shit with your just-published approach, you are doing shit with a different new unpublished approach.
LikeLike
April 14, 2009 at 5:29 pm
Because if they have no readers, their journals don’t have much value, and it will be harder to charge authors lots to publish there. If readers can’t trust the science published in your journal, they won’t read it (or won’t pay lots to read it, anyway, nor will their libraries). The journals get the big money because of their reputations, which powers the desire of people to publish there and the desire of people to read what’s published there. If enough irreproducible research is published in them, their reputations suffer, and eventually all that paper real estate assumes the value of an abandoned strip mall. If authors have to pay a lot to publish somewhere that doesn’t enhance their reputations much or in whom their research won’t be seen, and readers no longer care what’s in your journal, then authors will publish elsewhere, oand readers will read something else. There is also the possibility that if publication in a journal ceases to have a measure of reliability, people may decide not to bother and may make their research public in other ways, a shift which would not be in journals’ long-term interests, either.
LikeLike
April 14, 2009 at 5:35 pm
Sounds good, but you & I know that this isn’t really practical. Most everyone else knows it’s not practical either, which is why many leading labs…
1) Make their super duper reagent.
2) Do the awesome experiments they designed the reagent for.
3) Hit the talk circuit pumping the results.
4) Write a zillion grant applications using the preliminary data.
5) But DON’T publish, because they still need to…
6) Do all the other logical extension experiments that they thought up more recently (or which were suggested by talk audiences or grant reviewers)
7) Maybe write more grant applications, and
8) Milk this baby for all it’s worth until…
9) People start to grumble about why the reagent isn’t available yet, at which time…
10) PI with the reagent grumbles about what a hassle it’s been to get their paper published, and
11) Agrees to send the ‘unpublished reagent’ to powerful buddies…
12) While the paper is actually in press or newly in print, and therefore…
13) PI gets:
a) admiration for the great reagent,
b) sympathy for the publication hassles, and
c) a reputation for giving out ‘unpublished’ reagents.
Super WIN!
If you try to stuff your pipeline and keep it full, you are going to lose. There will always be someone able to hype your stuff better, milk it better, and (thanks to stealing your credit) beat you to the next punch thanks to all the resources they get for it. Do we have to name names?
LikeLike
April 14, 2009 at 5:36 pm
Hap (#31): What fantasy world do you suppose all this will happen in?
LikeLike
April 14, 2009 at 5:59 pm
Please explain the fantasy part – I’m trying to figure what part of “no readers, no writers, no journal” I’m missing. Unless readers are buying journals for the ads, and authors just want to see their names in print, and money isn’t an object, but that would seem to be slightly more fantastic than the scenario I proposed.
How do you think journals got where they are? Magic?
LikeLike
April 14, 2009 at 6:18 pm
Maybe that last question shouldn’t be as dismissive as I though it was. How did journals get to have the status they do? Why do you read (or publish in) specific ones? If disclosing reproducible research isn’t important for journals, what is? (I’m assuming that since there are many ads in most cases, ad revenue isn’t driving publication choices or financial well-being.)
LikeLike
April 14, 2009 at 6:26 pm
Obviously late to the party here…. Just curious, DamnGoodTechnician–did you guys ever try contacting the authors for clarification on their methods?
I’ll agree that GlamourMags’ M&M sections leave much to be desired, but between the supplemental materials that TomJoe mentions and contacting authors, I’ve usually been able to get a complete protocol. And I’m in a field where PIs will actually answer questions from the audience at national meetings with, “Yes, we did Super Cool experiment and got amazing results, but I can’t tell you what they are.”
BTW, if contacting the authors doesn’t work, contacting their competitors sometimes does.
LikeLike
April 14, 2009 at 6:56 pm
For people wanting to know a little history of how Nature grew into a hydra of Nature X journals, all with a bent for publishing what the editor thinks is hot, you might want to check out the obituary of a 22-year Nature editor.
LikeLike
April 14, 2009 at 7:13 pm
Same way every other magazine does. It advertises, hypes, and hopefully gets a reputation for being a ‘must read’ that is self-perpetuating. See DJ&MH’s post (#37). Also consider: http://en.wikipedia.org/wiki/Cell_Press
LikeLike
April 14, 2009 at 10:05 pm
It would actually not be unheard of to help a guy at BigPharma out, only to have them scaleup and patent your process (with maybe a few tweaks).
I doubt that this is a common occurrence if it happens at all. When something is disclosed (whether it be a poster, lecture, a casual conversation with someone outside your group, or a manuscript) prior to a patent submission it will not be recognized as patentable in many countries (I believe in the US you have a year to file after making such a public declaration, but in Europe you’re SOL). No degree of “scaling up” would make a difference, and unless those tweaks are significant (and can be proven as such) I doubt the patent would hold.
Outside of this, the beauty of the patent process is that, even if Pharma does manage to patent the idea, you can dispute it. Manuscripts should serve as excellent documents to stake your claim.
LikeLike
April 14, 2009 at 10:41 pm
IANAL, but the people in Legal have drilled this into me over many years:
* In Europe, public diclosure prior to filing is an absolute bar to patentability. However, an NDA renders a disclosure non-public.
* In the USA, public disclosure by the inventor starts a one-year clock ticking; at the end of that year the invention is barred if an application hasn’t been filed.
* Both in the USA and in Europe, disclosure by someone other than the applicant prior to filing of an application is a bar to patentability.
In other words, if you invent something today, and next month I publish (or ship as a product) the same or substantially similar invention the day before you file for a patent, you’re SOL.
Then again, IANAL.
LikeLike
April 14, 2009 at 11:44 pm
Wow things change on this blog fast in the course of a day. Turns out things are not always as they seem.
Turns out DGT is not an academic non-profit institution, but rather a “researcher” for BigPharma. (I guess I should have read the blurb to the side describing who DGT was.) Given this new information, my view on the scientist “holding his protocol hostage” changes pretty dramatically. [I am on the side of openness to other non-profit academics, but I can definitely understand the scientists’ attitude towards BigPharma trying to grab his work.] DGT, did you offer this scientist a CDA? Did you offer him/her compensation for his/her contribution to your work?
Let me put forward two important points.
1. How much of your BigPharma work is patented and not shared without compensation? Why would you expect an academic scientist to behave differently?
2. How much of your work that your BigPharma company makes billions on was originally done by some scientist working on taxpayer dollars at a public university? How have you compensated that scientist, the taxpayer, and the public for their work that you are now making billions on?
I have a colleague here at BigStateResearchUniversity who gives her transgenic animals free to any academic researcher who asks, but demands exorbitant payments from BigPharma. She funds a very large lab that way. And I say more power to her.
Becca #11 and TomJoe #39 – I can’t give concrete examples, but given some information I know of patent fights in the electronics industry, this would not be unheard of. [I know of suits brought not because they were justified but because party X knew that they had more money to pay lawyers than party Y. It was explained to me (by a consultant involved) as being like poker. If you can bluff the pot large enough that the other person can’t play, you win. Even if they have the winning hand.]
Finally, I should point out that there are really two issues here and we should not get them confused. One issue is whether methods are sufficient in GlamourMagz. To that issue, I would note that we do not know how helpful this scientist is to other academics trying to replicate his/her work. The other issue is whether an academic scientist is duty-bound to clarify protocols to BigPharma for free, and to help BigPharma replicate it, presumably to make a big profit from it, with all the evidence suggesting that the scientist, taxpayer, and university will not see a penny of that big profit. Those are different issues and should not be confounded.
LikeLike
April 15, 2009 at 1:04 am
Qaz: The other issue is whether an academic scientist is duty-bound to clarify protocols to BigPharma for free
Completely untrue. This research was probably conducted predominantly with taxpayer dollars. As far as I’m concerned, that means that any taxpayer deserves that the materials and methods be fully written out. Doesn’t matter in the slightest if that taxpayer happens to work at BigPharma, LittlePharma, or the corner deli. You publish it, you owe people an explanation of how you did it.
LikeLike
April 15, 2009 at 2:05 am
In my field, computer science, I’ve seen a number of really interesting papers that I *really* wished had included source of some kind. The ones I’m thinking of were graphics algorithms.
One of them was an algorithm for combining an over-exposed and an under-exposed image to make one correctly exposed – basic high dynamic range stuff. The really nifty part was their technique for dealing with motion. The paper had great sample images of trees blowing in the wind.
I would argue that source code of some form should be published as a matter of course for algorithms like this. Source of course makes it often quite easy to reproduce results, as well as compare different algorithms with the same data set. It also helps give you a feel for how well chosen the data sets used in the paper’s examples.
LikeLike
April 15, 2009 at 7:45 am
Gopi #43 – PsychReview requires all papers reporting simulations include code capable of regenerating the figures in the paper.
LikeLike
April 15, 2009 at 8:50 am
Fascinating scare quotes there.
LikeLike
April 15, 2009 at 9:30 am
Magazines are different though in that they can use ad revenue to subsidize their publishing so that they can get enough readers to make it both useful for others to read and useful to advertise in (because lots of people will see it). Journals don’t have much ad revenue to allow them to subsidize copies for readers to get them to read, and without the prospect of being read, no one is going to publish there. Magazine subscriptions are easier to sustain for readers and libraries because they’re cheap, while the same is not true of journals, so people are likely to not buy a journal if they don’t read it. Journals can hire important editors who can induce/convince/wheedle/threaten their friends to publish, which can drive readership early on, but probably won’t sustain the journal.
In chemistry, one of the major journals has had issues with articles that (in one case) had insufficiently description of what had been prepared (hexacyclinol) and was likely to have not been prepared (no SI other than a spectrum of product and penultimate intermediate, with a bizarrely shifted solvent peak and no satellites in the product spectrum), while neither the authors nor reviewers for another article appear to have searched the literature even cursorily for precedent (12annulenium ions, later found to have been Zincke reaction products). People in the blogosphere seem to have thought these instances to have diminished the journal’s reputation, but I don’t actually know whether there has been any effect on either readership or authors.
LikeLike
April 15, 2009 at 9:42 am
I know of a case where a lab recruited a PhD student for the main purpose of finding nits to pick in a rival lab’s glamour mag result…
LikeLike
April 15, 2009 at 10:14 am
“Researcher”? Wow. WTF is that about, qaz? I had no idea that academic labs had a monopoly on doing science. What the hell am I doing here, then? [hangs up pipettes, walks out door]
To answer qaz’s questions – would I expect patented work done here to be freely given away? Fuck no – and it doesn’t work like that in the opposite direction either. As I mentioned in #16, when we are using something that comes directly from an academic lab, it’s licensed for a hefty fee. I have no problem with that, and neither do the powers-that-be at my company. No one would never suggest that academic researchers should send us reagents or constructs or mice out of the goodness of their hearts. If the lab in question had decided to patent their method of differentiating these cells, then fine: tell us that you’re patenting it, and we’ll license the technology. But if it’s not being patented, then why are you being a huge dick about clarifying published methods? There would be no place for a CDA with that PI, because their work was published. You can’t keep confidential the method that you used to do your published experiments. This gets at the distinction of whether methods are sufficiently complete to repeat experiments.
The other question, about how the taxpayers aren’t seeing anything come from their $$ that big pharma is “stealing”: can you show me an academic lab that’s screening huge compound libraries to try to find new therapeutics? Can you show me an academic lab that’s running dozens of massive clinical trials? No. You can’t. This absurdly expensive exercise is the way that we find, validate and re-validate therapeutics to, you know, help treat & hopefully cure diseases. That’s the ultimate goal of what biomedical research funded by the NIH is, isn’t it? To “improve human health by increasing scientific knowledge related to disease and health”, like I said in #6? So what does an academic lab do, exactly, to “improve human health”? You “increase the knowledge” and I’ll make the drugs, mm-kay?
[/rant] I take umbrage at this attitude that anyone working outside academia has no worth in the scientific discourse. I think the distribution of shitty scientists-average scientists-fantastic scientists is pretty equivalent between an academic lab and an industry lab.
LikeLike
April 15, 2009 at 11:35 am
“Researcher”? Wow. WTF is that about, qaz? I had no idea that academic labs had a monopoly on doing science. What the hell am I doing here, then? [hangs up pipettes, walks out door]”
I sympathize DTG. The idea that “BigPharma” means “Bad” is a travesty, and ignores the many petty (and perhaps significant) interests that drive work in academia as well. I’ve always hated to click off on the box that says that “I have no significant financial interest” in submitting manuscripts, ’cause I don’t see how my need to continue in my position, get grants, place students can be differentiated clearly from someone who owns 10% stock in a company that makes a drug they’re publishing about.
The motivations in “BigPharma”, “Small Biotech”, “Big Research Institute”, “Big University”, “Big Med School” are not identical, and the different groups can have different alignment with the public good of science and medicine. But it’s simply wrong to assume that one’s status or employment automatically equates with the relationship to the public good.
Our task is to try to align the motivations of employees in any of these venues with what we think contributes best to the public good, and whether they’re shareholder owned, or a public institution, or a private non-profit is relevant, but not the discriminatory factor.
LikeLike
April 15, 2009 at 1:21 pm
DGT- first, the way the game is played these days, it doesn’t matter if the researcher wants to patent it. It doesn’t matter if the researcher wants to give it to you. What matters is whether the University has deemed your company’s needs an acceptable use of their intellectual property.
You say that you wouldn’t expect reagents (property) for free. Nor would you expect patented protocols (intellectual property) for free. Yet you expect unpatented protocols for free. Do you think that something becomes property by filing for a patent?
It’s hard to wrap your brain around, and it doesn’t always end up making sense, but in general the precedent is for IP to start counting as IP when you create it… not when you patent it (thus the importance of lab notebook dates).
Patenting it can be used to stop others from using it, but notpatenting it doesn’t (in theory) make it not yours. You sometimes avoid a lot of drama by patenting though.
However, there is yet another angle- whether wanting to know the details of what they did qualifies as “fair use”. I’m not a lawyer, and I don’t know what you want to do with the protocol, and how closely a profit of your company could be linked with the academic researcher’s efforts, so I really can’t speak to that.
(Also, you are definitely a researcher in my book. WTFOMGBBQ, qaz?)
LikeLike
April 15, 2009 at 2:10 pm
I take umbrage at this attitude that anyone working outside academia has no worth in the scientific discourse.
Me too. Some of my best friends really are in the private sector doing research. Better quality in some cases because of the way resources are allocated. Sometimes for a complete loss because the company decides to move in a different direction and totally closes up shop on a given project or even a whole research installation (Remember AnnArbor!).
LikeLike
April 15, 2009 at 2:12 pm
As a matter of fact, yes. The point of the patent system is to trade a period of commercial exclusivity (the patent) for disclosure. Those not willing to make that trade are welcome to treat the invention as a trade secret, taking the risk that someone else will come up with it independently.
Note that “publication” sort of blows the whole “trade secret” thing. IMHO an honest author would unwilling to share the Super Secret Methods should at least have the honesty to state that fact: “We used a trade-secret method to produce the published results. Gaze upon our mad skillz and despair!”
“Yours” and “not yours” are at best sentiments until you get to the point where you can keep others from using things that are “yours.” Which is what a patent is. However, there are some things that are outside of the patent system, including (to name one) natural laws. It might be possible to patent a laser, but not to patent the physics that make it possible.
Also, please note that patenting is effectively a form of publication: you have to specify how you do it in sufficient detail that “a person having ordinary skill in the art” could use the patent to reproduce the invention.
Alternately, there’s trade secret. See above.
As previously mentioned, IANAL. However, ISTR that practicing a patent for scientific, educational, research, etc. purposes is not actionable. More or less, only commercial practice of the patent is covered. A research lab using a patented technique might not be enforceable.
As always, consult a competent patent attorney for real advice; the above is just uninformed opinion by someone whose dealings with patent law have been entirely from the position of an inventor (but there are a few around in my name, for what it’s worth.)
LikeLike
April 15, 2009 at 2:17 pm
Yet you expect unpatented protocols for free.
So what’s the point of a methods section in a paper, then? Shouldn’t we all just put “For methods and reagents described in this manuscript, please contact the Office of Technology Development at University X”.
LikeLike
April 15, 2009 at 2:44 pm
That private enterprise benefits from publicly-funded research is a central element underlying the strategy by which publicly funded research helps the public. (At least, unless the system switches to a GSE-led monopoly, which would be an absolute disaster imho.)
The allocation of public funds to basic research aims in part to reduce the risk and cost (marginally at least), and thus the buy-in, that would otherwise drive away private capital investment and strangle production period, let alone innovation.
LikeLike
April 15, 2009 at 3:40 pm
This is gobsmackingly wrong. Publicly disclosing an invention without claimning it in a patent means it is in the public domain, and may be freely practiced by anyone who wants to.
This may have been arguable until a few years ago, but it is untrue now. Use of a patented invention in a university research lab can definitely constitute patent infringement.
LikeLike
April 15, 2009 at 3:47 pm
I do wonder if this is part of the quid pro quo for university labs being in the commercial technology development business.
Put another way, time was when science developed on the public dime in universities was public domain, and the law was reluctant to “advance science and the useful arts” by smacking them down for doing pro bono publico science. Then we got the wonders of Congressionally-encouraged Technology Transfer Offices and university labs became just another bunch of “will research for hire” businesses — or so a plaintiff could argue.
Well, TANSTAAFL.
LikeLike
April 15, 2009 at 3:50 pm
This was, indeed, part of the rationale for the judicial determination that universities are not immune from patent infringment claims based on research activity.
LikeLike
April 15, 2009 at 4:25 pm
DGT- I suspect simply because the norms for scientific publishing were already well-established before universities could own intellectual property (which is a comparably recent development).
CPP-
1)Technically, I was saying “unpatented” doesn’t imply “no property existing” (though as D.C. Sessions notes, this is more a theoretical point than anything practical unless one actually goes on to patent).
2) In general, publishing will prevent something from being patented. But…
(this is from wiki, so take with a grain of salt) “There is an exception to this rule, however: in U.S. (not European) law, an inventor may file a patent claim up to one year after publishing a description (but not, of course, if someone else published or used it first).”
(also, the paper itself is not in the public domain due to Nature’s copyright, but that is neither here nor there).
LikeLike
April 15, 2009 at 4:41 pm
That’s still the rule as far as my instructions from Legal go.
LikeLike
April 15, 2009 at 4:59 pm
The one-year grace period between publication and filing is correct, and is unique to US patent law.
LikeLike
April 15, 2009 at 5:37 pm
If I understand this correctly, it is not just publishing but presenting the work in any way. For example, if one presented work at a conference, the one year clock starts then. Is that correct, PP?
LikeLike
April 15, 2009 at 6:56 pm
Any public disclosure counts. This includes sampling a product practicing the invention. It even includes private presentations to someone not under duty of confidentiality. The latter could, for instance, include job candidates, students, or alumni taking a tour prior to writing a honking huge check.
Again: neither CPP nor I are lawyers. Even if we were, we’re not your lawyers. Accept no substitutes.
LikeLike
April 15, 2009 at 10:12 pm
DGT – Sorry about the “researcher” quotes. That was uncalled for. I have many friends doing research out of academia. I will not presume to determine whether you are a real researcher or not. I over-reacted when the narrative changed.
However, there are very different cultures in academia and industry (whether it be BigPharma or BigElectronics or BigSoftware). In academia, information is shared and when it is not shared, one becomes angry at the person hoarding the information. In industry, information is hoarded and should not be expected for free, either from other industry or from an academic researcher.
All of my experiences with intellectual property have been negative. All of my experiences with shareware have been positive. Maybe that’s just me, but the numbers on both sides have been large (n>10) and that’s statistically significant.
LikeLike
April 15, 2009 at 11:21 pm
The one-year grace period between publication and filing is correct, and is unique to US patent law.
Which of course is not going to be enforced in other parts of the world. So, while you might make a profit here (and enjoy legal protections), someone can use your efforts there at no charge and with no threat of legal repercussions.
LikeLike
April 16, 2009 at 6:37 am
Where do you get the cockamamie idea that it is a “comparably recent development” that universities can own intellectual property? Becca,you are again blithering about shit you know nothing about.
Why are you doing this so frequently? For example, I have observed you on other blogs giving people career advice about things–like faculty negotiating with chairs–that you can’t possibly have the slightest infinitesimal clue. Why don’t you stick to what you know, whatever that might be?
LikeLike
April 16, 2009 at 8:28 am
Yow, Comrade. I understand the herding instinct, but remember she’s a guest and should be treated with respect. If you drive away all contributors, it’ll only be you and DM here chattering about the good old days when people read your blog. If she’s wrong about something, point that out. But if she’s right, she’s right, and it doesn’t matter whether she got the information first hand, second hand, or in a psychic vision.
In any case, with regard to #66, she’s probably referring to the Bayh-Dole act. While she maybe should have said so specifically (I couldn’t remember how to spell it either and am only able to give the info now thanks to some desperate Google searching with a bunch of fuzzy semi-recollected variants of the name), I think it’s relevant to the discussion, and 1980 (the year that act passed congress) is reasonably recent.
In any case, I see nothing fundamentally wrong with her opinion, which she clearly stated as such. So settle down. In the part of Italy where I am from, we have a saying: “I merda è merda, ma se è lucido allora buon montato in una collana”, which translates roughly as “If a turd is shiny, you can still put it in a necklace.”
LikeLike
April 16, 2009 at 9:59 am
CPP- I would have been more explicit but it took me a while to Google up the name of the act (of course, I finally find it and then I get around to reading Luigi’s comment… I really tend to do things the hard way far too often).
Anyway-it was total hyperbole to say Universities “couldn’t own intellectual property”- I should have been more clear. But federally funded research didn’t have the same profit-potential for the Universities before the Bayh-Dole act; it really changed the game dramatically, at least acording to what the IP folks have told me.
I honestly believe that if the mindset that spawned the Bayh-Doyle act had been around prior to the establishment of the free-flow of scientific information (and a publishing fetishizing culture) Universities might well insist all methods sections simply refer to their tech transfer office.
Anyway, it’s not like grad students don’t take ethics classes (heck, my understanding is that NIH explicitly requested it!). So why rag on me for discussing the stuff that came up in that course?
LikeLike
April 16, 2009 at 2:35 pm
http://cafe.comebackalive.com/viewtopic.php?f=8&t=41020
LikeLike
April 17, 2009 at 10:04 am
And indeed, I’ve read Nature papers with 95 pages of supp. inf..
But then, of course, those were papers on dinosaur phylogenetics, not on something anyone could make money off of.
LikeLike
April 17, 2009 at 10:37 am
When you file for a patent, a patent can only be granted if you have disclosed your invention with sufficient detail for someone “skilled in the art” to make it. If you don’t put sufficient detail in the patent application, it is rejected.
Scientific research articles should have no less a standard. If the description of an experiment or procedure is insufficient for someone skilled in the art to replicate it, then those doing peer review should reject it.
Making a method or process known by publishing it is a complete and absolute bar to anyone patenting it. Even if a patent does issue on something published before the patent was applied for, the patent is invalid and cannot be enforced. No competent legal team will try and enforce an invalid patent because it will be declared invalid. An invalid patent that has not been declared invalid by a court still has some value. It has the same value that a suit of armor or a sword of papier mache has. They have only bluff value, a very slight value so long as they are never tested against a real sword or a real suit of armor.
The US patent system requires patent applications to include all prior art known to the inventor. To not disclose something that would invalidate the patent is to commit fraud on the patent office. That is criminal fraud for which you can go to jail. If a patent lawyer participates in that fraud, they can be disbarred and go to jail too.
LikeLike
April 17, 2009 at 11:03 am
WRONG! “Fraud on the patent office” is a very different legal concept than traditional criminal fraud. There may possibly be *extraordinarily* rare instances when fraud on the patent office could also constitute criminal fraud, but it would have to be adjudicated completely independently of any patent infringement litigation.
LikeLike
April 18, 2009 at 6:02 pm
As much as I enjoy Science and Nature, neither has a requirement for completeness; that is to say they routinely publish research that cannot be duplicated from the articles presented. Frankly, I’m pretty skeptical of Science and Nature claims until I see a published article in a journal that has the space and specialized audience for a complete article. I was personally involved in the commercialization of a Nature “invention” that turned out to be bullshit. There is no doubt in my mind that Nature’s poor editorial policy regarding completeness played a key role in misleading a lot of people in a high-profile field. There was a major article on the SNAFU–in Science!
The idea that universities routinely get their inventions stolen by the private sector is laughable. Just the opposite–universities have become pretty sophisticated at monetizing their IP resulting from research funded by the taxpayer. Remember the $540 million dropped on Emory 4 years ago? And what about the $500M+ judgment for the UC web browser applet patent(s)?
I think Bayh-Dole has been a major success. Universities have a revenue stream that reflects their significant contributions to research. Researchers with commercially relevant research–and interest in making money–can get an extra income stream to augment their salary.
Successful companies are by their very nature greedy and selfish. One of the world’s largest consumer product companies was shorting me on royalties for a technology that saved them, by their own calculation, $2Billion. What was surprising is they were cheating on royalties of only around $100k! One of their own employees was really upset because he believed their internal BS about honesty. It’s just part of the DNA.
On free consulting, I guess I have to agree with DrugMonkey. The editors and reviewers have approved an article. Chances are company researchers are just being lazy and greedy with a typical request for free information. A genuine omission could be addressed in a public forum (letter to the editor or paper/talk)–“We have been unable to duplicate RESEARCHERS findings due to the omission of a significant step…” That should really put them on the spot, and I think it can be very justified. Scientists that publish works that omit key details for duplication SHOULD be called out. Of course, if others were able to duplicate it you would look like an absolute fool (or just another greedy commercial exploiter!)
Great Discussion
LikeLike