One for the Journal-n-Impact Factor Geeks
January 22, 2008
DM, I’m looking in your direction. Anyway, Chris Rowan of Highly Allochthonous has a bit on the new Nature Geophysics journal. For the usual bioscience audience around these parts, I think you will see some familiar themes emerge and one comment that goes a bit off the path:
my own initial response to almost any article in Nature or Science is to treat it the same as I would an infomercial: claims presumed false until I see supporting communications or have a chance at my own interpretation of the data presented
Of course, one wonders what it is to “have a chance at [ones’ own] interpretation of the data” if not, err, reading the paper?
DrugMonkey 
January 22, 2008 at 1:57 pm
Of course, one wonders what it is to “have a chance at [ones’ own] interpretation of the data” if not, err, reading the paper?
Seeing a raw, complete dataset? Getting to press the researcher at a talk about the hypotheses that you’re pretty sure must have been tested but not corrected for?
I thought Clarkson’s comment was a bit petty (although I agree that the Nature specialty journals are much higher quality than the flagship) as it’s not like there’s a shortage of garbage in less flashy journals. But I don’t get your nitpick.
LikeLike
January 23, 2008 at 4:34 am
“my own initial response to almost any article in Nature or Science is to treat it the same as I would an infomercial: claims presumed false until I see supporting communications or have a chance at my own interpretation of the data presented”
This is ludicrous.
LikeLike
January 23, 2008 at 6:41 am
I agree… totally ludicrous, but a ludicrous position that I keep hearing over and over again… what is the cause of all this pessimism? or is it jealousy?
LikeLike
January 23, 2008 at 7:09 am
I’m not as extreme as the original comment, but I do get the general sentiment. There are two things working against accuracy in Nature and Science.
First: The short format makes it very hard to evaluate the quality of the research. The supplementary sections don’t always fill in all the blanks since they are not written to flow with the rest of the paper. Some Science/Nature articles are shortly followed by a second article on the same or similar data in a less prestigious journal that goes into the method and concept more clearly. Beyond getting a bonus paper on one’s CV, there’s a reason these papers were accepted.
Second: A goal of Nature/Science is to publish high impact findings, which are often unexpected and surprising. If a finding is unexpected, it’s more likely to be wrong. Only additional research can prove of the data is an outlier or a real new direction.
In the digital age, the publishers can figure out more ways to improve the first point and the second point isn’t something they necessarily want to change.
LikeLike
January 23, 2008 at 11:27 am
Although, it should be noted that Nature has recently started reserving the right to require independent laboratory confirmation of what they’d consider to be really extraordinary findings prior to publication. They did this with the recent “turn human somatic cells into human stem cells” thing for example.
LikeLike
January 23, 2008 at 12:01 pm
junior prof:
1) jealousy
2) a bit of truth
i was listening to a radio bit this morning, they were doing something on the film industry, academy awards, etc. just talking about the best-actor nominees alone they got into a conversation that maps almost directly onto this one. Marquee names on the shortlist for what the film critics thought were pedestrian performances but they get on the list of nominees because, well, they are marquee performers and have a lot of friends “in the biz”. Much discussion of quid-pro-quo for Clooney’s pro-writers stance. “voting is a closed shop,we’re all buddies here” type of stuff. the other guys (day lewis? lee jones? was it?), something about clearly superior performances (and these guys aren’t one-offs, obviously)….
LikeLike
January 23, 2008 at 1:30 pm
I don’t disagree at all on your perspective of a bit of truth, big names almost certainly have a leg up on getting in; however, this doesn’t mean the work shouldn’t be believed (“presumed false”). I have never been able to understand this type of pessimism… perhaps I’m naive.
My impression is that a big problem with Nature and Science papers (but not Cell) is that authors tend to be able to get away with too much wild speculation that goes beyond the normal discussion that you would get in a longer manuscript. Then again, perhaps this is just my perception based on the compressed writing form these journals require that often leaves out the caveats that would normally be raised.
I also fail to understand why, in this electronic age, Nature and Science can’t make their manuscripts more accessible to the average reader (and the scientist that will follow up on the work). As these papers become more and more dense they become more and more difficult to follow. Eventually its diminishing returns, you just don’t have the time or energy to go through these papers in detail… I’m not at that point yet but I can see how it could get there relatively quickly… is that doing a service to cream of the crop work?
LikeLike
January 23, 2008 at 1:38 pm
junior prof:
all smartassness from BM and my usual busting on the GlamourMags aside, I am not suggesting that “all” stuff in these should be “presumed false”. that was WAY over the top, IMO.
regarding “wild speculation” well, that’s the way to get in. it is not that the authors are “allowed to get away with it” it is that if they do not go WAY off on “we cured Alzheimer’s Disease! AND cancer! all with the same transcription factor!” type of crap they are not going to even go out for review…
LikeLike
January 23, 2008 at 1:46 pm
I wasn’t referring to you, sorry, the original quotation above said “presumed false” hence the quotes.
LikeLike
January 23, 2008 at 2:33 pm
Next up — does someone want to take a shot at the plagiarism” study in Nature today?
My impression: yeah, a lot of people probably copy chunks of methods text, introductions (“Cancer is a leading cause of death…”) and discussions (“…and may lead to a cure for cancer.”) Who cares?
LikeLike