Have the data changed your mind?

January 3, 2008

The Edge annual question for 2008 asks:

WHAT HAVE YOU CHANGED YOUR MIND ABOUT? WHY?

Science is based on evidence. What happens when the data change? How have scientific findings or arguments changed your mind?”

out of their

163 contributors; 111,000 words

I’ve pulled a couple of interesting ones.

A DM fav author, David Brin almost boots it but ends up with:

I certainly expected that, by now, online tools for conversation, work, collaboration and discourse would have become far more useful, sophisticated and effective than they currently are. I know I’m pretty well alone here, but all the glossy avatars and video and social network sites conceal a trivialization of interaction, … everybody is still banging rocks together, while bragging about the colors. Meanwhile, half of the tricks that human beings normally use, in real world conversation, have never even been tried online.

Richard Dawkins may be an obvious read but hits on a DM theme of what’s wrong with grant review:

When a politician changes his mind, he is a ‘flip-flopper.’ Politicians will do almost anything to disown the virtue — as some of us might see it — of flexibility. … How very different is the world of science. Scientists actually gain kudos through changing their minds. If a scientist cannot come up with an example where he has changed his mind during his career, he is hidebound, rigid, inflexible, dogmatic!

While I can’t believe I’m giving props to Irene Pepperberg and she’s obviously speaking from personal bitterness, dangit, she’s right! Sometimes we need fishing expeditions.

First, and probably most importantly, I’ve learned that one often needs simply to sit and observe and learn about one’s subject before even attempting to devise a testable hypothesis…

Second, I’ve learned that truly interesting questions really often can’t be reduced to a simple testable hypothesis, at least not without being somewhat absurd…

Third, I’ve learned that the scientific community’s emphasis on hypothesis-based research leads too many scientists to devise experiments to prove, rather than test, their hypotheses.

Terry Sejnowski disappoints with what dear-FSM-I-hope is a straw pre-amble (WTF is in the water at Salk anyway?) but ends with a caution against scientific orthodoxy.

The way that neuroscientists perform experiments is biased by their theoretical views. If cortical neurons use rate coding you only need to record, and report, their average firing rates. But to find out if spike timing is important new experiments need to be designed and new types of analysis need to be performed on the data. Neuroscientists have begun to pursue these new experiments and we should know before too long where they will lead us.

The stadium rocker of Science Blogging PZ Myers notes:

In the context of pursuing science, however, there is a substantive context in which we do not change our minds: we have a commitment to following the evidence wherever it leads. … every good research program has as its goal the execution of observations and experiments that will challenge our assumptions — and about that all-important foundation of the scientific enterprise I have never changed my mind, nor can I, without abandoning science altogether.

I dunno who Anton Zeilinger is, but…well, YES! Take that you basic-science purists.

When journalists asked me about 20 years ago what the use of my research is, I proudly told them that it has no use whatsoever…When journalists ask me today what the use of my research is I proudly tell them of my conviction that we well have a full quantum information technology in the future, though its specific features are still very much to be developed. So, never say that your research is “useless”.

I like this Roger Highfield chap.

This idea that science is an objective fact-driven pursuit is laudable, seductive and – alas – a mirage.

Science is a never-ending dialogue between theorists and experimenters. But people are central to that dialogue. And people ignore facts.

…the motto of the world’s oldest academy of science, the Royal Society. Nullius in Verba was once taken as ‘on the word of no one’ to highlight the extraordinary power that empirical evidence bestowed upon science.

See any you particularly liked DearReader?

6 Responses to “Have the data changed your mind?”

  1. bill Says:

    Ping. I noticed some of the same ones you did; I thought there was something of a theme apparent.

    Like

  2. Schlupp Says:

    Anton Zeilinger is a physicist. Nice collection!

    Like

  3. drugmonkey Says:

    (crossposted)

    bill, I saw the same trend you’ve noted, although you picked up a bunch I hadn’t noticed. We’ve seen some of the same criticisms about the hypothesis-driven orthodoxy arising in the context of the NIH’s breastbeating over peer review over 2007.

    I think the key, as always, is going to be to really hash out what we mean in detail instead of creating this amorphous skepticism of the conduct of science. This latter is a possible interpretation of all those comments.

    We need to concentrate on ?? such as, what is the “good” type of fishing expedition and what is the “bad” that gave rise to the original orthodoxy?

    I’ll note that some of this is being resolved/driven by the gene array technologies. because the experiments are such obvious fishing expeditions in the good and bad senses. so for quite some time now in talks and grants people have been distinguishing between the “hypothesis generating” (read: don’t ding me on ‘fishing expedition’ grounds, please) and “hypothesis testing” aspects of the research program.

    Interestingly there are no objective reasons why this very excellent distinction can’t or shouldn’t be made in just about any type of research program. Pepperberg’s comments lay this out pretty well, I think.

    Like

  4. CC Says:

    I’d noticed the same trend, usually as a windy preface to “Why, even I was wrong about something once, several decades ago!” and had been surprised that everyone else was so rapturous over that feature.

    I’d put it down to selection bias. The scientists they tapped are mostly public figures, with a taste for self-promotion; a number of them barely qualify as active or recently-active researchers. They have their speech about the glories of The Scientific Method on auto-repeat.

    If you dragged Professor Whatever out of the lab and asked him the same question, he’d have plenty of examples about how he and everyone else believed [something no layman or any scientist in any other field cares about] and it turned out to be wrong. But who would want to read that?

    My favorite example of The Thing Everyone Knew That Turned Out To Be Completely Wrong: using sense RNA as a negative control for antisense studies.

    Like

  5. CC Says:

    Incidentally, what I (scientifically) changed my mind about in 2007: while moderate obesity may reduce quality of life, its effects on disease and mortality overall are clearly nowhere near as negative as they’ve been portrayed, and may not be negative at all. Same for dietary fat.

    And I don’t see SCIENCE! rushing to correct itself in light of the data, and certainly don’t see anyone apologizing for all the fear and guilt they caused by presenting their preconceptions and class prejudices as settled fact.

    Like

  6. neurolover Says:

    What I noticed was the frequency of scientists whose “re-evaluation” depended on work done in their own laboratory (Ledoux & Sejnowski both cite their own work). Ledoux talks of being “dragged” into doing an experiment by a member of his lab, one that he was “sure” would fail. Sejnowski’s experiment, of course, wasn’t at all counter to what he thought, and in fact he specifically went searching for an experiment that would support his ideas. Sejnowski’s paragraph doesn’t properly belong as something he has changed his mind about, more like something he still wants to change everyone else’s mind about, and is becoming more confident about.

    These problems dovetail with Pepperberg’s comment that folks “devise experiments to prove, rather than test, their hypotheses.” That’s certainly what Sejnowski did.

    Like


Leave a comment