Professional Differences
June 10, 2015
In real science, i.e., that that includes variability around a central tendency, we deal with uncertainty.
We believe, however, that there IS a central tendency, an approximate truth, a phenomenon or effect. But we understand that any single viewpoint, datum or even whole study may only reflect some part of a larger distribution. That part may or may not always give an accurate viewpoint on the central tendency.
So we have professional standards in place that attempt to honestly reflect this variable reality.
Most simply, we present the central tendency of effects (e.g., mean, median or mode) and some indication of variability around that central tendency (standard error, interquartile range, etc).
Even when we present a single observation (such as a pretty picture of a kidney or brain slice all hilighted up with immunohistochemical tags) we assert that the image is representative. This statement means that this individual image has been judged to be close to the central tendency of the images that were used to generate the distributional estimates that contribute to the numerical central tendency and variability graphs / tables presented.
Now look, I understand that it is a bit of a joke. There are abundant cracks and redefinitions that point out that the “most representative image” really means “the image that best makes our desired point”.
There is a critically important point here. Our profession does not validate least representative image as an acceptable standard. Our professional standards say that it really should be representative if we ever present N=1 observations as data.
The alleged profession of journalism does not concern itself with truth and representativeness at all.
Their professional ethical standards, to the extent they exist, focus on whether the N=1 actually occurred AT ALL. In addition it focuses on whether that datum was collected fairly by their rules- i.e., was the quote on the record. Accuracy, again for the alleged profession, focuses only on episodic truth. Did this interviewee literally string these words together in this order at some point in time during the interview? If so, then the quote is accurate. And can be used in a published work to support the notion that this is what that interviewee saw, experienced or believes.
It is entirely irrelevant to the profession of journalism if that accident of strung-together words communicates the best possible representation of the truth of what that person saw, experienced or believes. Truth, in this sense, is not the primary professional ethical concern of journalism.
If the journalist pulls a quote out of an hour of conversation that best fits their pre-existing agenda with respect to the story they are planning to tell, it literally does not matter if every other sentence spoken by that person tells a different tale. It’s totally okay because that interviewee literally said those words in that order on the record (and it is on tape!).
If a scientist processes twenty brains in the experiement, grabs the one outlier that tells the story they want to tell, trashes the 19 that say the opposite and calls it a representative image (even if by inference if not directly)….this is fraud and data fakery. Not okay. Clearly outside the professional bounds.
That, my friends, is the difference.
And this is why you should only agree to talk to journalists* that will send you a nearly final draft of their piece to ensure that you have been represented accurately.
If every single one of us scientists insisted on this, it would go a long way to snapping the alleged profession into line. And greatly improve the accurate communication of scientific findings and understandings to nonspecialist** audiences.
__
Representative image from here.
*They exist! I have interacted with more than one of these myself.
**Reminder, we ourselves are nonspecialist consumers of much of the science-media. We have two interests here.
On Responding to Prior Critique
June 10, 2015
The life of the academic scientist includes responding to criticism of their ideas, experimental techniques and results, interpretations and theoretical orientations*.
This comes up pointedly and formally in the submission of manuscripts for potential publication and in the submission of grant applications for potential funding.
There is an original submission, a return of detailed critical comments and an opportunity to respond to those critiques with revisions to the manuscript / grant application and/or argumentative rebuttal.
As I have said repeatedly in this forum, one of my most formative scientific mentors told me that you should take each and every comment seriously. Consider what is being said, why it is being said and try to respond accordingly. This mentor told me that I would usually find that by considering even the most idiotic seeming comments seriously, the manuscript (or grant application) is improved.
I have found this to be a universal truth of my professional work.
My understanding of what I was told by my mentor, versus what I have filled in additionally in my similar comments to my own trainees is now very fuzzy. I cannot remember exactly how extensively this mentor stamped down what is now my current understanding. For example, it is helpful to me to consider that Reviewer #3 represents about 33% of peers instead of thinking of this person as the rare outlier. I think that one may be my own formulation. Regardless of the relative contributions of my mentor versus my lived experience, it is all REALLY valuable advice that I have internalized.
The paper and grant review process is not there, by any means, to prove to you beyond a shadow of a doubt** that the reviewer’s position is correct and you are wrong. A reviewer that provides citations for a criticism is not by any means the majority of my experience…although you will see this occasionally. Even there, you could always engage cited statements from an antagonistic default setting. This is unwise.
The upshot of this critique-not-proof system means that as a professional, you have to be able to argue against yourself in proxy for the reviewer. This is why I say you need to consider each comment thoughtfully and try to imagine where it is coming from and what the person is really saying to you. Assume that they are acting in good faith instead of reflexively jumping behind paranoid suspicions that they are just out to get you for nefarious purposes.
This helps you to critically evaluate your own product.
Ultimately, you are the one that knows your product best, so you are the one in position to most thoroughly locate the flaws. In a lot of ways, nobody else can do that for you.
Professionalism demands that you do so.
__
*Not an exhaustive list.
**colloquially, they are leading you to water, not forcing you to drink.
DrugMonkey 