Honestly Nature, you at least understand that this is misleading…don't you?

October 30, 2008

This would be funny if it weren’t going to be used, extensively, to defend policies which continue to block the transition and advance of younger scientists. A News bit in Nature overviews a paper [pdf] which reports on the number of publications produced by younger and older scientists. From the abstract:

Those who worry about the aging of scientists usually believe that the younger they are the more creative and productive they will be. Using a large population of 13,680 university professors in Quebec, we show that, while scientific productivity rises sharply between 28 and 40, it increases at a slower pace between 41 and 50 and stabilizes afterward until retirement for the most active researchers. The average scientific impact per paper decreases linearly until 50-55 years old, but the average number of papers in highly cited journals and among highly cited papers rises continuously until retirement. Our results clearly show for the first time the natural history of the scientific productivity of scientists over their entire career and bring to light the fact that researchers over 55 still contribute significantly to the scientific community by producing high impact papers.


A look at the paper will give you the confirming evidence. Paper numbers are pretty stable from 50 to 70 although there is a numerical decline. Interestingly the impact measures (top 10% of journals and some citation metrics) decline from 40-50 or so and increase from that point until 70. There would seem to be some obvious selection issues here…
At any rate, the Nature bit interprets these data as if they support the idea the oldsters are just as brilliant, innovative and “productive” in the scientific sense. To my read anyway.

If the mind is the first to go, it doesn’t seem to show up in the bibliography. A new study suggests that older scientists publish more than their younger counterparts.
A research team from the University of Quebec in Montreal, Canada, looked at the publication record of nearly 14,000 professors and found that those in their 50s and 60s published almost twice as many papers each year as those in their early 30s.

The comments over at Nature are all over this of course. No duh! The mid thirties to the mid fifties is the interval over which scientists transition from postdoc to independent scientist to a well-funded (hopefully), multi-doctoral-scientist group. The PI in her fifties is “productive” in a way that the postdoc in her thirties is not- i.e., she gets authorship on each and every paper coming from her lab group. This reality undercuts any possible application of these data to any real world question of whether “the mind is the first to go” or related. And oh, yes, there will be the related questions.

The survey, say its authors, is the strongest evidence yet that older scientists are at least as productive as their juniors, if not more productive. “We should not underestimate the older researcher,” says team leader Yves Gingras, a historian of science.

Right, except this argument will be used in an apples-to-oranges comparison. The question for the NIH at present is whether pushing careers back by encouraging endless postdoctoral training and un-encouraging rapid transition to independent research positions is a good idea. The question is not whether BigCheez PIs at 55 get on as many papers at this age as they did at 35. It is to ask if you get more productivity out individuals who transition to independence and start growing their group at 32 as you do for those individuals who transition at 42! These data do not really address that but I guarantee you they will be applied to this question.

Gingras became interested in the topic after reading that the average age for first grants from the US National Institutes of Health in Bethesda, Maryland, rose from 34 in 1970 to nearly 42 in 2004.

[Anthony van Raan, who studies science and technology policy at Leiden University in the Netherlands] says, the research makes a compelling case against mandatory retirement, which is commonplace throughout Europe. “It’s better to keep [researchers] and let them work,” he says. “They can be very productive.”

See what I mean?
Nature does an absolute disservice when they write just about the least defensible interpretation of the reasons for the data.
I just don’t understand how the general science journals can maintain their “news” and “opinion” sides in a way that looks more like mainstream media than it does the scientific content* of their journals. Why present the sexiest or most controversial take on something? Why not try to get at the most plausible or most logical interpretation of the data?
__
*whether there is in fact a disconnect between the way the general science mags report data and the news I will leave as an exercise for the reader.
Disclosure: I have been seriously annoyed at least once in the past by being interviewed for an opinion/news/fluff type of piece. It is my contention that the absolute least representative quote of an extensive conversation ended up being published. It was clear to me from the beginning of the conversation that the journalist was pursuing a particular angle and I tried very hard to disabuse the journalist of the apparent bias or take on the story. The quotation was the thing I said that came the closest to supporting this pre-existing position and least close to my actual position. I sort-of understand when this happens in the general media. Not one bit when it happens in a science journal that just happens to have a newsy/opinion-y side.
This is by way of disclosing why the sort of piece that is the subject of this post really ticks me off.

No Responses Yet to “Honestly Nature, you at least understand that this is misleading…don't you?”

  1. cashmoney Says:

    When Joe the Plumber takes over running the business, he gets credit for plunging ten times as many toilets, right?
    So he’s ten times as “productive” and deserves ten times the money.
    /eyeroll

    Like

  2. ScienceWoman Says:

    Dammit DM, I was going to blog on this and now you’ve saved me the effort. I felt like this piece was written by someone not in their right mind OR someone who really had no idea how scientific careers advanced. Either way, terribly unimpressive for Nature.

    Like


  3. Oh good grief … things like this make my head hurt.

    Like

  4. Becca Says:

    I vote for mandatory retirement from science for anyone over the age of 40!

    Like

  5. BilNerd Says:

    Do they account for the fact that at 55 many scientists are now in a position where they are in charge of younger researchers who are doing cutting edge work, and getting put down as authors even though they had little to nothing to do with the paper?

    Like


  6. Do they account for the fact that at 55 many scientists are now in a position where they are in charge of younger researchers who are doing cutting edge work, and getting put down as authors even though they had little to nothing to do with the paper?

    Jeezus motherfucking christ!!! How many times do we have to debunk this motherfucking nonsense!!

    Like

  7. juniorprof Says:

    Apparently at least one more time.

    Like

  8. Robin Says:

    It is apples and organges if postdocs are compared to PI’s without some appropriate scalar. To determine the productivity of a research lab, one needs to divide by the number of lab members (and perhaps it would also be interesting to try normalizing by research dollars spent). There are many older PI’s with large labs. You can hardly compare then annual output of a 25 member lab to a lab with 4-5 people. Some credit should be given for efficiency.

    Like

  9. TreeFish Says:

    I think a good metric would be to normalize the contribution of each lab member of a lab or number of authors (LMNA), for both young PIs and the PI-asauruses, and multiply this by number of pubs.
    For example, if the young PI has three pubs in 2009, each of which has 3 authors on it, that PI’s LMNA score would be [pubs/average number of authors per pub] = 3/3 = 1, and the normalized score would be 3. If PI-asaurus has 6 pubs, each of which has 10 authors on it, the LMNA score would be [6/10] = 0.6, and the normalized score (nLMNA) would be 3.6.
    Though there is still an advantage conferred to PI-asaurus, the contribution to each study by each lab member is very comparable.
    Then, you can control for impact by constraining your analysis to PIs who have an average citation # per pub, and see if the nLMNA score increases, decreases, or stays the same throughout a scientist’s career. By using roughly similarly statured scientists (objectively determined by citation # per pub), this approach would address whether productivity is truly increased, or whether it is more an emergent property of having a bigger lab.

    Like

  10. Becca Says:

    TreeFish- if I could, I would give you Halloween candy simply for the use of the term “PI-asauruses”.
    CPP- my frist snarky comment was going to be about how the relationship of “grad student” (or “post-doc”) to “PI” can be thought of as the relationship of a Renaissance artist to her patron. Then I realized that the patrons were usually monarchs or The Church. However, even the King of France or the Pope did not insist on signing their names on the paintings. So the artist:patron model does not fit the data.
    Ergo, “grad student/post-doc”:”PI” :: “pilgrim”:”God”.
    Anyway, the Emperor has no clothes! The reason it’s so difficult to pinpoint this is that scientific “productivity” is a collective figment of our imaginations. Nine times out of ten, anything we can identify as productivity is based more on what people think of you than what you actually do.

    Like

  11. Eric j. Johnson Says:

    This is a great metric, since it’s not like some papers are 100-10,000x more important than others. They are all equal.

    Like

  12. Eric j. Johnson Says:

    Comrade cashmoney, not every call you get in the trades is as trivial as a clogged can. Some are a lot more puzzling, hence the greater productivity of someone with lot of experience, and probably higher native intelligence, like joe the plumber. Also, certain kinds of work can be pretty serious in plumbing, due to the risk of permitting or contributing to a future house flood. Joe knows which of his peeps are suited to each situation, which is worth quite a lot. Not that i am against any and all redistro of wealth – but we ought to be clear on why we are doing it and what the downsides will be.

    Like

  13. neurolover Says:

    “the average number of papers in highly cited journals and among highly cited papers rises continuously until retirement”
    This is the sentence that struck me. Highly cited papers is good, but “papers in highly cited journals” doesn’t mean a lot if you think that PI-saurus’s get special entre’ into “highly cited journals.
    And, ComradePP, you know you can’t debunk the irrelevant PI myth with anecdotes. Just because a PI with a large lab can be an important contributor to a project does not mean that they all are.

    Like

  14. DrugMonkey Says:

    Just because a PI with a large lab can be an important contributor to a project does not mean that they all are.
    I’m not usually quite so fervent as the good PhysioProf on this issue but do you actually think about this?
    What world do you people live in where all the scientific resources necessary for a high profile paper are magically made available to lowly postdocs? Seriously. Where do you get this conceit?

    Like

  15. Bill Says:

    “What world do you people live in where all the scientific resources necessary for a high profile paper are magically made available to lowly postdocs?”
    Ah Ha! The true DM ventures forth from his throne room, er, office, to proclaim over the masses. You’re only adding insult to injury by calling postdocs “lowly”. But thanks for the warm words! I’ll remember to genuflect the next time my PI floats by.

    Like

  16. DrugMonkey Says:

    Do feel free to address the substance instead of perseverating in your postdoctoral pique, Bill.
    Do you grasp the essence of PP’s usual argument? Do you have an answer? Why should the PI be excluded from the credit for papers that were generating using resources that were provided through her efforts?
    Start with the fact that the PI provides a lab to work in and move forward to the specifics from there..

    Like


  17. Ah Ha! The true DM ventures forth from his throne room, er, office, to proclaim over the masses. You’re only adding insult to injury by calling postdocs “lowly”. But thanks for the warm words! I’ll remember to genuflect the next time my PI floats by.
    DM and PP, I would just let this shit go because, seriously, it’s making the head of the domestic and laboratory goddess hurt. Here’s how I figure it — if these folks really think they can do their science independent of the intellectual environment and financial support provided by a PI, they know where the door is. Let them have at it on their own.

    Like


  18. Why should the PI be excluded from the credit for papers that were generating using resources that were provided through her efforts?

    And just to be crystal clear–as the lovely, talented, and wise Dr. Isis pointed out–“resources” does not just mean money and space. It also means an intellectual environment, scientific guidance, salesmanship by the PI of the trainee’s work at meetings and conferences, etc, etc, etc.

    Like

  19. whimple Says:

    Actually, I thought this was the crux of the argument:
    ask if you get more productivity out individuals who transition to independence and start growing their group at 32 vs… (emphasis mine)
    When the paradigm is a career of exponential growth of human capital, as it was for the last 100 years, right up until the end of the NIH doubling in 2003, gaining in productivity the longer you remain in the system is the natural order of things. Now that funding has changed (at least for now) from exponential growth to steady-state, this natural order has broken down, so comparisons between the more senior PI that had a naturally easy time growing their group, and the junior PI of today for whom this growth is extraordinarily difficult aren’t really meaningful.

    Like


  20. Now that funding has changed (at least for now) from exponential growth to steady-state, this natural order has broken down, so comparisons between the more senior PI that had a naturally easy time growing their group, and the junior PI of today for whom this growth is extraordinarily difficult aren’t really meaningful.

    This is a very good point, and apropos of my colleague who is going to have an exceedingly difficult time growing her lab beyond about ten people when she can’t snag a third R01 because scurrilous asshole study sections members refer to her “high level of funding” in their reviews, thereby shitcanning her outstanding application.

    Like

  21. msphd Says:

    Thanks for posting on this. I think it’s perfect timing, to wit:
    Assumption A. Everything is controlled by the most senior scientists. With some advice from marketing.
    Assumption B. A lot of those senior scientists are having to justify why they can’t afford to retire now. The real reason is the economy, but instead they want to portray themselves as irreplaceable. See the article on delayed retirements in the Chronicle this week.
    Assumption C. According to the article in the Chronicle, at some places, 1 senior scientist that doesn’t retire = 2 junior faculty who won’t be hired.
    Conclusion: Senior scientists must make every effort to justify continuing to hang on to their jobs while exploiting junior scientists (in the form of super-cheap postdocs) to artificially raise their own “impact”. Why not? It’s much easier than promoting the most creative young bunch to be the new competition for R01s.
    I particularly like the way the article first tries to distinguish between “scientific” impact and what I would call “citation” impact, which have not always been the same thing.
    But in the next sentence, they conflate the two again and use this as a way to say “Look, the senior scientists aren’t doing anything scientifically useful, but at least they’re publishing their useless crap in high-citation journals!”
    At least, that’s how I read it.
    Yeah, I’m real happy with where science is going. Remind me again why I thought I wanted to be a professor?

    Like

  22. whimple Says:

    my colleague who is going to have an exceedingly difficult time growing her lab beyond about ten people when she can’t snag a third R01 because scurrilous asshole study sections members refer to her “high level of funding” in their reviews, thereby shitcanning her outstanding application
    Doubtless what the study section members meant to say was, “the investigator hasn’t lived up to her promise, and while she can write a nice grant, she doesn’t subsequently deliver on the science.”
    I guess she’ll just have to muddle through with the 10 people and 2 R01s she already has for a while and prove she can do something with what she’s already got before anyone will have enough confidence in her as a scientist to allocate further resources to her.

    Like


  23. While that is a scenario that occurs, this was the opposite. She has already been extremely productive publication-wise with both of her existing R01s, and the full comment was:

    Dr. So-and-So is an outstanding young investigator who has been very successful in establishing herself as an independent investigator. An important indication of her success is her high level of funding.

    It was clearly a “this fucker doesn’t need more money” ding, not a “this fucker hasn’t proven she knows how to turn money into publications” ding.

    Like

  24. Lab Lemming Says:

    I would like to point out that I know several researchers who have a productivity boom (real, not seniority-based) in their late 50’s/ early 60’s age range, simply because their kids grow up and they can go back to crazy hours for the first time in 20 years.

    Like

  25. kiwi Says:

    Christ yes, get rid of the kids and everything gets much simpler. Is there any difference in best productivity ages for women and men?

    Like

  26. neurolover Says:

    I think all NIH funded scientists should have to start putting G.W. Bush as the last author on their grants, since he’s in charge of the providing the resources for all of them.
    (BTW, I am not a post-doc, and I fully understand the role that PIs play in fostering projects, even when they visit the lab only once a month or quarter or even more occasionally. But, to argue that every PI must have provided that role, because their name is on a paper suggests a circularity of argument that is unworthy of anyone who wants to call themselves a scientist).

    Like

  27. Lu Says:

    “exceedingly difficult time growing her lab beyond about ten people”
    Comrade PhysioProf,
    That is crazy… Ten people at once…. Why would anyone have such a big group?
    I can safely assume that at least a half of those people are graduate students. Where would they go after graduation?

    Like

  28. Guy Muhl Says:

    Being healthy to travel around the globe. Use a beach home near the water.

    Like


Leave a comment