Why Scientists Hate Jury Duty: A Ray of Hope from the NAS

February 19, 2009

I hate jury duty. No, not because it adds yet more stuff into a fairly busy schedule as a parent-scientist. I love the theory of jury duty and the opportunity for public service and engagement of citizens in our public life. I do. What I hate with a burning vengeance is the system whereby truth seeking and the establishment of fact are a distant eleventh to the nonfactual, emotional manipulation of the jury to see the matter of guilt in a particular direction.
A panel convened by the National Research Council of the National Academies of Science has concluded that the forensic sciences need a serious overhaul in the US (full report).
What took us so long?

Listening to National Public Radio yesterday I was barraged throughout the day with this story (audio).
It resonated with the clarity of the purest crystal goblet. You see, my jury experiences have completely convinced me that the system is utterly broken and has no resemblance to anything rational or logical. (Yes, yes, my experiences are a tiny, microscopic minority of all the jury experiences in the US but this is blogging. )
Here’s the first thing that offends my sensibilities as a scientist person with at least a half a brain. There are two reference instructions about evidence that are particularly important. By the preponderance of the evidence and beyond a reasonable doubt. Scientists will recognize the former. It is our stock and trade. We draw tentative conclusions about the state of nature based on the currently available evidence. When evidence is in conflict, we attempt a thinking-person’s synthesis of the evidence to estimate the most likely truth of the matter.
Scientists do not recognize “beyond a reasonable doubt”.
This standard is foreign to us. So when faced with an instruction from the judge, we are being asked to be even more strict than we are in daily professional life! Even if we accept the “reasonable doubt” standard as being hyperbolic and meant for a lay audience, it should at the very least be a standard that is as strong as our scientific standard, right? Otherwise what does it mean?
My problem with juries and trials is that once the instructions have been laid down, the prosecution and defense try as hard as they bloody can to convince the jury that their evidence is in fact stronger than it actually is by playing on emotional strategies to trick the jury members. The supposed adherence to fact (and for that matter rules of evidence and exclusion, etc) is a total sham.
This brings me back to the NAS/NRC finding. At the risk of insulting your intelligence, the establishment of factual matters in a criminal case looks nothing like the TV version. Evidence is rarely convincing on the face, can be technically complicated to conduct or to explain and may rest on firmer or weaker methodological foundations. The NAS/NRC report emphasizes the latter.

A congressionally mandated report from the National Research Council finds serious deficiencies in the nation’s forensic science system and calls for major reforms and new research. Rigorous and mandatory certification programs for forensic scientists are currently lacking, the report says, as are strong standards and protocols for analyzing and reporting on evidence. And there is a dearth of peer-reviewed, published studies establishing the scientific bases and reliability of many forensic methods. Moreover, many forensic science labs are underfunded, understaffed, and have no effective oversight.

You know. Fingerprint matching (when was Pudd’n Head Wilson written?). Hair and fiber identification. Eyewitness testimony quality. Observer bias. DNA matches. Etc. The teevee would convince us this stuff is duck soup. Turn key. Infallible. Except when the antagonist of the drama intentionally puts a finger on the scale.
My experiences suggest this is total nonsense. I’ve seen situations in which the prosecution spent an hour bragging on the qualifications of an expert witness, how many training classes attended, etc so that said expert could get up and state that “in their professional opinion” the evidence pointed to a certain conclusion. Without putting up the data and showing the jury how the conclusions were reached. While briefly mentioning the caveat that no, there were not really any scientifically established standards for the field. So in the jury room let me tell you the scientist is looking around in jawdropping disbelief to find people accepting the expert statement but not even wanting to examine the weak-ass excuse for “evidence” that was entered but not actually presented by the expert!!!!!
Did I mention that I hate the jury process?
Okay, well perhaps I am too hard on issues of credentialed “expertise”. What did the report have to say?

Many professionals in the forensic science community and the medical examiner system have worked for years to achieve excellence in their fields, aiming to follow high ethical norms, develop sound professional standards, and ensure accurate results in their practice. But there are great disparities among existing forensic science operations in federal, state, and local law enforcement agencies. The disparities appear in funding, access to analytical instruments, and availability of skilled and well-trained personnel; and in certification, accreditation, and oversight. This has left the forensic science system fragmented and the quality of practice uneven. Except in a few states, forensic laboratories are not required to meet high standards for quality assurance, nor are practitioners required to be certified. These shortcomings pose a threat to the quality and credibility of forensic science practice and its service to the justice system, concluded the committee

Yeah, I love being right, big w00tang for me! Except….this forensic science business is kinda important.
More from the report that is totally congruent with my limited experiences:

Nuclear DNA analysis has been subjected to more scrutiny than any other forensic discipline, with extensive experimentation and validation performed prior to its use in investigations. This is not the case with most other forensic science methods, which have evolved piecemeal in response to law enforcement needs, and which have never been strongly supported by federal research or closely scrutinized by the scientific community.

but…but…this stuff has been in use for decades in many cases. Convicted, what, hundreds of thousands of alleged criminals. Are you saying nobody has bothered to figure out if it is complete bullshit?

As a result, there has been little rigorous research to investigate how accurately and reliably many forensic science disciplines can do what they purport to be able to do. In terms of a scientific basis, the disciplines based on biological or chemical analysis, such as toxicology and fiber analysis, generally hold an edge over fields based on subjective interpretation by experts, such as fingerprint and toolmark analysis. And there are variations within the latter group; for example, there is more available research and protocols for fingerprint analysis than for bitemarks.

Are you depressed yet?

The report points out the critical need to standardize and clarify the terms used by forensic science experts who testify in court about the results of investigations. The words commonly used — such as “match,” “consistent with,” and “cannot be excluded as the source of” — are not well-defined or used consistently, despite the great impact they have on how juries and judges perceive evidence.
In addition, any testimony stemming from forensic science laboratory reports must clearly describe the limits of the analysis; currently, failure to acknowledge uncertainty in findings is common. The simple reality is that interpretation of forensic evidence is not infallible — quite the contrary, said the committee.

Right. Another way to put this is “The system flat out lies about the quality of the evidence to sway the jury to make a decision on the basis of something other than the best interpretation of the evidence”.
Can we fix things by nudging the existing structures to do better? Hell no.

The existing forensic science enterprise lacks the necessary governance structure to move beyond its weaknesses, the report says. The recommended new National Institute of Forensic Science could take on its tasks in a manner that is as objective and free of bias as possible — one with the authority and resources to implement a fresh agenda designed to address the problems found by the committee. The institute should have a full-time administrator and an advisory board with expertise in research and education, the forensic science disciplines, physical and life sciences, and measurements and standards, among other fields.
The committee carefully considered whether such a governing body could be established within an existing agency, and determined that it could not. There is little doubt that some existing federal entities are too wedded to the current forensic science community, which is deficient in too many respects. And existing agencies have failed to pursue a strong research agenda to confirm the evidentiary reliability of methodologies used in a number of forensic science disciplines.

You know we ponder the role of science in society a lot around the science blogosphere. We talk about communicating the scientific thought process to nonscientists and frequently brag on how our approaches can make much of public life better.
Isn’t the finding of fact in criminal cases in which an individual is facing incarceration an issue which reaches the highest standard of importance?
The system is clearly doing it wrong.
Science can help.

No Responses Yet to “Why Scientists Hate Jury Duty: A Ray of Hope from the NAS”

  1. bobh Says:

    I haven’t read the NAS report so this may be covered there in which case excuse the intrusion. A more fundamental problem with forensic science exists. A good scientist will work hard to make sure he is not overlooking some piece of evidence or some problem in his experimental methodology to ensure his conclusions are not tainted by his expectations and own biases. Bad science (e.g. think cold fusion) results when the experimenter filters his results to find what he is looking for. The police and prosecutors are wanting, and expecting, to find evidence that the suspect is guilty. In all instances the crime labs are part of the same system and it should not be surprising if, within that culture, the forensic technician sees that their job is to find the evidence that the police and prosecutors are looking for. I know that this problem is recognized in some areas but it is a hard culture to break.


  2. drdrA Says:

    So- it’s not that you hate jury duty- it’s that you hate the subjective nature of a process that could be improved using science…to be less subjective?
    Sadly- the system calls Ph.D. students who don’t go on to academic careers ‘failures’… but it seems like there is a great opportunity for them …. in forensics!


  3. frog Says:

    bobh: I believe that your criticism was addressed. One suggestion (the one most strongly rejected by law enforcement and prosecution — surprise!) is to make forensic investigate independent of law enforcement and prosecution; their jobs should be solely to analyze evidence impartially, not to enforce or prosecute.
    You’d think it’d be obvious; the analysis of the evidence should be as advantageous to the defense as the prosecution if done properly. But we live in a reality of one-size-fits-all, where the antagonistic process is believed by lawyers to be a universal process.


  4. antipodean Says:

    You have a PhD and actually managed to get on a jury? From what I’ve heard you automatically get booted by both sides in most cases.


  5. DrugMonkey Says:

    You have a PhD and actually managed to get on a jury?
    Yes. I have several friends similarly equipped with the trappings of science who have also been selected for service.
    I feel certain there is at least one prosecutor in my locale who will never make such a mistake again…but who knows?


  6. Rose Colored Glasses Says:

    “Beyond a reasonable doubt” is a gimmick. Jurists and lawyers will never allow a criterion to be defined — the numerical probability that would identify this degree of certainty. This elastic quality allows them to assert that something is virtually certain, even if the odds are even-money or odds-off.
    If you want to get angrier yet, take a fresh hard-nosed look at voir dire, looking for how anyone would expect the process to serve its stated goal. Nothing like this exists outside of legal practice. In the real world we know we cannot improve on the fairness of a random draw by cherry-picking among the random selections. Voir dire is a game of chance with three players — prosecution, defense, and the bench — all working toward a shared goal, a jury stripped of people who have no faith in the police, lawyers, or judges. Failure to weed out that ilk guarantees no chance of a conviction.


  7. Tony P Says:

    I’m just an info sci guy but I know for a fact I’ll never be called for jury duty in my state.
    Reason being, I worked for the prosecutors for a couple of years. I know most of them. That’s an automatic toss from jury duty.


  8. Anonymous Says:

    ‘I feel certain there is at least one prosecutor in my locale who will never make such a mistake again…’
    That’s hilarious. I’ve been on one recently as well. Hung jury. When I wrote that on the questionnaire for the second jury duty… I ended up juror number 100 (like the very last one),… with not a snowballs’ chance in hell of being seated.


  9. Curt Fischer Says:

    My one experience as a juror in a civil case was enlightening and positive. The woman who brought the suit did not have a compelling case. She had her collision reconstruction expert witness and the defendant had his. The defendant’s explanation of the crash made physical sense, and the plaintiff’s just showed a lot of pictures of the crash and kept saying how fast it was and how much damage there was.
    We exonerated completely (by a 11-2 vote) the driver she’d pulled out in front of after 1 hr of deliberation. The two who wanted to find fault with the driver accepted that it would be a very low % of the fault (my state is a fault-sharing state), so that even then in her $X suit, if we had decided the accident was 5% his fault, she’d only get 0.05 * X * 3.
    (The three I found out later was due to some state law that automatically tripled all damages without telling the jurors; the state lying to its jurors did piss me off).
    Those two were in the minority anyway, so the plaintiff lost, and after the verdict was read the judge came in to thank us and to answer *general* questions we had about the criminal justice system, her career and background, etc. she spent 30 minutes chatting with us. All this, from a life-appointed judge.
    It was an eye-opening experience for me, I had assumed juries of your “average” non-sciencey types would be emotional and inattentive. Not the case at all. Maybe “regular” people aren’t as dumb as we assume they were…maybe the problem is just egoism on our end.


  10. Comrade PhysioProf got chosen to serve on a jury that included an attorney, an MD, and a medical student.
    We ended up convicting a dude of forgery of insurance documents after what I considered to be a very analytical dispassionate discussion of the evidence. When we had reached unanimity, one of the jurors–a retired tailor–made a very passionate little speech in which he remarked that he had never experienced such a high level of analytical discussion before in his entire life, and that he was proud to have served on the jury with us.
    This case had no contested forensic evidence, so is not really germane to the point of your post.


  11. drdrA Says:

    ‘I feel certain there is at least one prosecutor in my locale who will never make such a mistake again…’
    That’s hilarious. I’ve been on one recently as well. Hung jury. When I wrote that on the questionnaire for the second jury duty… I ended up juror number 100 (like the very last one),… with not a snowballs’ chance in hell of being seated.


  12. antipodean Says:

    My understanding is that all common law countries prohibit lawyers serving on juries. I was not aware that this was permissible in the USA. Thanks for the story. Sounds like a case study in how juries should function.
    It’s good to know that PhDs are sometimes allowed to serve on juries other than in patent cases.


  13. antipodean Says:

    … and to drag the discussion back on track…
    I take it TV lied to us again. There are not a lot of Gil Grissoms floating around running forensic ‘science’ labs.


  14. Tina Says:

    Comrade PhysioProf got chosen to serve on a jury that included an attorney, an MD, and a medical student.
    Really? An attorney? Wow. I didn’t even know that was possible.
    The most bothersome part is the expert who never presented the data. Ding ding ding, stop the train! How is that possible? Why is it in some cases this society is so damn trusting? I mean, even I thought the fingerprint analysis was it, and now I am reading that it is not.


  15. New Asst. Prof. Says:

    Yep, this is particularly close-to-home for me since I’m waiting for that little notice from our circuit court that I’m in the jury pool again. Last time was pre-Ph.D. (4 months before my defense, how’s that for timing), and not only was I selected for 2 juries that round, I was made foreperson on 1 of them. I truly hope there are more fellow jurors capable of “dispassionate analytical discussion” this time than there were last…forensic evidence or not!


  16. aphid Says:

    What’s always bothered me about my jury experiences is that you cannot ask questions. As a scientist, I want to know why some things are not presented as evidence. How many other places are you asked to make important judgements but not allowed to ask questions?


  17. Dunc Says:

    The thing that gets me is the whole theatrical nature of the adversarial system. Do you guys have “dock identification” over there? (“Do you see your assailant in court today?” – with the equivalent of a giant neon sign hanging over the accused.)
    Last time I was on a jury, I literally had to bite my tongue to avoid laughing out loud at some of the rhetorical tricks used. Somehow I don’t think it would go down too well in a murder trial…


  18. My jury experience was shortlived. We were told that police in Texas can cuff you and take you to jail, sans justification, if they pull you over for anything other than speeding. I asked if it was not the job of the police to enforce the law, not make it up as they go. I was done.
    Pity too. As as actuary, I pray to the FSM for the day I get some rhetoritician masquerading as an attorny to ask me if something is “possible”.


  19. Art Says:

    One fact may hinder, if not actually derail, any improvement in the standards for forensic sciences used in court cases: Most defendants are using court appointed and state funded representation. Often these lawyers are so overloaded with cases, often taking on ten or twenty at a time, that they see the file only 24 hours before the court date and may only see their client a few minutes before they are marched into court.
    By law, upheld by the supreme court, this fulfills the requirement for “adequate representation”. Fact being that defense counsel being asleep, drunk , and unaware of the core questions of the case is Not deemed to be evidence of inadequate representation.
    There is a clear reason for this. “Adequate representation” is, in practical terms, is defined as the level of representation that he taxpayers within a municipality are wiling to pay for. If the taxpayers of the county fund the office of the defender at $200 a case this is then deemed to be adequate representation. If they fund the defense at $20, or $2, then that level of defense is deemed to be the legal standard for “adequate representation”.
    At such low funding for the defense there is little need for the prosecution, which is always better funded, to raise the level, or even competency, of their game. Fact being that prosecuting attorneys earn their chops by locking people up. The majority of jurors automatically assume, despite being told otherwise by the judge and it being in opposition to a cornerstone of our legal system, that anyone brought to trial is guilty. If not of this particular offense then something else.
    Police testimony is typically given more weight than it deserves. Made worse because police are trained, and conditioned to use words and descriptions that specifically fulfill the conditions that justify their actions and will more likely impress a jury. Even when the phrasing implies things that are subjective or completely made up.
    In the end prosecuting attorneys fight the case on emotional ground because it works and it is cheap. It even relieves them of having to know the specifics of the case because raising alarm over the ‘decline of society’ and emotional pleas of ‘it could happen to you’ and ‘we need to send a message’ and ‘make an example’ are not dependent on the specifics of the case and are generic to all defendants. The defense usually has little or no access to physical evidence that hasn’t been specifically collected by the prosecution. Potentially exculpatory evidence of tests are usually ignored. The defense can’t afford their own experts or testing to point out the holes in the prosecutions evidence. The defense has no physical evidence game and the prosecution has no motivation to raise theirs.
    So both sides, one by choice and the other by necessity, wage war with emotional arguments. That isn’t going to change until the PD is funded and the playing field is leveled.


  20. Speaking of the role of bogus forensic testimony in jury trials, I just finished reading “Criminal Injustice”, about the Tankleff murder case. It’s an absolutely gripping read.


  21. MPW Says:

    aphid: “What’s always bothered me about my jury experiences is that you cannot ask questions.”
    A friend of mine was on a jury and got reprimanded for trying to take notes, which also amazed me. Is this standard or universal practice? (I’ve never been chosen for jury duty.)
    It was a fairly simple civil suit, if I recall correctly. But can you imagine being entrusted with the decision on whether or not to send somebody to jail, or even the electric chair, without being able to take notes on the evidence you’ve heard and seen? Would we even think of letting cops and prosecutors investigate cases while having to keep everything in their heads?


  22. Curt Fischer Says:

    MPW – In my state, note taking was allowed, and in fact jurors were provided with note pads by the court. This was to ensure that the court could destroy the notes immediately after the trial.
    I agree that jurors should be permitted to ask questions, perhaps if submitted in written form to the judge. That way the judge could remove questions like “Why does the defendant refuse to admit he is guilty?”


  23. Stephanie Z Says:

    I was reminded of this while writing about the person who really dug into the reliability of eyewitness testimony. It made me a little sad that the psychology, for which research has been done, is generally considered less sciency than the forensics.


  24. casey Says:

    I’d like to suggest that you separate out the criticism of the typical forensic scientist from the criticism of the techniques of the lawyers.
    I think the lawyers on both sides are really slimy, and present only whatever supports their positions. More often than not, though, it was the defense side of things that brought in the ridiculous courtroom drama. Sometimes that’s all they had going for them, because the facts were clear cut. Of course there were cases that are mixtures and there’s a whole ball of wax with regarding to interpreting mixtures. There are differing opinions with regards to mixtures, but when I was there, we did aim for the conservative approach.
    At least when I worked in forensics (5 years as a forensic DNA analyst), we did everything we could to build objective, scientifically supported case files. Though I worked in a state-run laboratory, I wasn’t working for the prosecution or the defense. I was there to present my data.
    I was always equipped to discuss the factual results at every case (bringing overheads in order to do so), but the lawyers usually decided not to present it.
    Our state-run laboratory was certified, our equipment was validated before being put into use, we used validated and standardized laboratory practices, and our analysts were trained by PhDs. I’m not saying ALL forensic laboratories are like this, but our laboratory had the funding and trained staff to keep things going well. At least we did, but I’m not sure about now, I’ve been gone for 10 years.
    There is a very real problem, though. There are too many crimes to keep up with. Of all the cases I had in laboratory that actually had potential suspects, there were thousands more sitting unsolved because they were older cases. It’s one of the reasons I decided to leave. There was just too many and I was beginning to have panic attacks because of how overwhelming it was. The advent of PCR and capillary electrophoresis was a great thing because you could process more cases, but it was a bad thing because you could process more cases. You still need to analysts interpret results and write reports.
    In my opinion, it’s just not that we need more scientists (though we do), it’s that we also need to figure out what the hell is wrong with people that commit the violent crimes in the first place. Something is not working here. But that’s a huge topic in itself.
    Just my $.02
    Oh, and I’m never going to have to go on a jury trial for a criminal case, but I’m not excused from being a juror on a civil case.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: