“Legalize eet mon” the ongoing series: In which the “scientists” are blamed, yet again, for the failings of journalism.

The pot legalization folks are at least as organized as the MDMA advocates in attacking any and all science findings which might counter their firmly held beliefs. In fact one might assume that holding one’s fundraiser at the Playboy Mansion trumps some aging ex-GratefulDead-spouse [no permanent link so this may disappear]. Agent (provocateur) PhysioProf put me on the latest scent of drug-abuse science denial. In this case from one Bruce Mirkin, communications director of the Marijuana Policy Project has a screed on the “Latest Anti-Pot Quack Science” up on Alternet. He’s ticked about:

Recent weeks have seen a rash of new studies of marijuana hitting the mass media, generating scary headlines like “Smoking Pot Rots Your Gums,” “Cannabis Bigger Cancer Risk Than Cigarettes” and “Pot Withdrawal Similar to Quitting Cigarettes. Most of this coverage can be boiled down to a fairly simple equation:
Flawed science + uncritical reporting = misinformation.

“Flawed” science? Flawed science? We’ll see about that after the jump.

Let us consider “Cannabis Smoking and Periodontal Disease Among Young Adults” by W. Murray Thomson, PhD; Richie Poulton, PhD; Jonathan M. Broadbent, BDS; Terrie E. Moffitt, PhD; Avshalom Caspi, PhD; James D. Beck, PhD; David Welch, PhD; Robert J. Hancox, MD, published: JAMA. 2008;299(5):525-531. Mirken had this to say about the article from which he chose the post-colonic part of his title:

The researchers were careful to say they hadn’t proved cause and effect, but simply what scientists called an “association.” But that didn’t stop one U.S. reporter from writing that marijuana “could … destroy gum tissue” and an Australian headline writer from declaring that marijuana “makes teeth fall out.”
Reading the actual study — something one suspects most reporters never did — raises questions the media never asked.

Hmm. Okay, the researchers were “careful” but the reporters are the one giving Mirkin fits. How is this the fault of the scientists, again? Perhaps Mirkin’s critique of the science will provide a clue.

Why is there no indication that participants were questioned about use of alcohol or other illicit drugs, both of which are known risk factors for dental and gum problems? Why were they not asked about brushing and flossing habits?
Given the relatively small effect — the statistical margin of error meant that the increased risk could be as low as 16 percent — confounding by alcohol/drug use or poor dental hygiene could easily explain the whole difference. In other words, there is a very good chance this study found nothing real at all.

He’s also quoted in this Reuters article:

“I think they’ve raised an interesting question, but I don’t think they’re close to giving a meaningful answer,” Marijuana Policy Project spokesman Bruce Mirken said.
Mirken called the study’s definition of heavy marijuana use arbitrary and said additional factors like alcohol or other drug use may help account for the findings.

Looking over the methodology, this is a pretty good study:

The Dunedin Multidisciplinary Health and Development Study is a longitudinal study of a cohort of children born at the Queen Mary Hospital, Dunedin, New Zealand, between April 1, 1972, and March 31, 1973.11 That institution was Dunedin’s only obstetric hospital. The sample that formed the basis for the longitudinal study was 1037 children, all assessed within a month of their third birthdays, and represents 91.0% of consecutive births. The participants’ families represented the full range of socioeconomic status.
-snip-
At age 32 years, 1015 of the 1037 original participants (97.9%) were still alive, and periodontal examination data were available for 915. Cannabis smoking history information from at least 2 assessments between ages 18 and 32 years was available for 903 (98.7%) of those individuals, and all subsequent analyses were limited to those 903. … Individuals with low SES were underrepresented among the participants included in the analyses.

A nicely complete sample for an entire city and this is really, really low dropout rate for a study of this type. Good so far. Now what’s all this about the “arbitrary” cannabis exposure grouping?

At ages 18, 21, 26, and 32 years, participants were asked how many times in the previous year they had used cannabis. The exposure measure used in the current study was the mean of the usage over those 4 ages (for which Cronbach {alpha} was .79); participants were then assigned to 1 of 3 distributionally based cannabis exposure groups: the no exposure group, comprising those who reported 0 occasions of cannabis use; the some exposure group (a mean of 1-40 occasions of cannabis use during the previous year); and the high exposure group, who were those with a mean of 41 or more occasions of cannabis use during the previous year (the highest 20%).

Sidebar: How much cannabis use is this, really? PhysioProf submits the following evidence…

Getting back to the Mirkin complaint, is “arbitrary” a valid critique of the study? Not really. Unless there is clear evidence of post-hoc cherry picking of the data. Otherwise, it is generally acceptable to group a variable sample somewhat arbitrarily because it is next to useless to try to pigeonhole real humans’ behavior into neat spontaneous groups. You simply are not going to find natural groupings of people who smoke exactly the same number of joints on exactly the same schedule! So we are left with units such as deciles, quartiles, and halves of the sample to use as our grouping factors. Yes it can be arbitrary but this is really the only unbiased solution.
Moving along, the primary dependent measure on this study was combined attachment loss (CAL). You know, this is the “one, one, two, one, three, two, four…” chant the dental hygienist produces whilst sticking that painful little probe down your gumline. Four and five mm CAL depths are NotGood according to my hygienist (and the paper). They found that:

When regression analysis was used to control for the confounding factors of tobacco smoking exposure, sex, SES, irregular dental service use, and the amount of plaque present (Table 5), the relative risk of having 1 or more sites with 4 mm or greater CAL for those who were in the highest cannabis exposure group was 1.61 (95% confidence interval, 1.16-2.24) (in comparison with those who had never smoked cannabis). When the more stringent case definition (1 or more sites with 5 mm or greater CAL) was used, it was 3.13 (95% confidence interval, 1.53-6.38). The regression analyses were repeated with only those who had never smoked tobacco (and thus without tobacco exposure in the models).

Nice straightforward stuff here. Regression is the only way to deal with real human samples and gives us a way to attempt to account for the fact that this is not a controlled-subject study. People do different things and you cannot go around compiling samples of people that fall into nice neat categories with respect to potentially important factors which may explain part of your effect. So a statistical procedure, which is well accepted, has been adopted. One may have some complaints about the interpretation of such analyses and question the strength of the support for a given conclusions. But it is not evidence of “flawed science”. And wait, there is more!

As a validity check, all regression analyses were repeated using the top quartile and then the highest 10% for cannabis use (with all other variables remaining unchanged), and cannabis exposure remained a highly significant predictor in each model (data available on request).

In other words, if you didn’t like their “arbitrary” top two deciles, they provided the top decile and the top quartile as alternative “arbitrary” groupings and the data still support a similar conclusion. This consistency of outcome suggests that the described result is not an artifact of the way one chooses to group the “high” and “low” exposure groups. There are a few more analyses presented to address what might be some other complaints; bottom line, the central conclusion is still supported no matter how you slice the data.
Finally, the authors lead the second paragraph of the Discussion with:

Before considering the findings, it is appropriate to examine the study’s limitations.

I.e., they give you some caveats and limitations before they even get started with the strengths of the study and what they see as the importance of the data! How can this Mirkin guy complain about the quality of the science? I just don’t get it.
Now, back to the carping about possible alternative hypotheses and factors that the authors did not address. Scientists recognize implicitly and explicitly that any single study cannot possibly address all potential hypotheses. All studies are necessarily limited. This is, in some senses, the whole point of including a Methods section. It tells the reader how to fill in the blank in how we read a result: “we found blah, blah blah, under ______ conditions“. The fact that a scientist has presented a study under conditions which do not please a particular reader is not evidence in and of itself of “flawed” science! To advance this critique is essentially to deny that the scientific enterprise is capable of telling us anything at all.
We can’t put too fine a point on it. Mirken is guilty of exactly the same critique he levels at the media. Of overhyping to make his point. He’s furthermore guilty of unwarranted bashing of the scientists when his actual complaint is against the media sensationalism of the science. I take a particularly dim view of this latter.