by Chuck Sharman

In the American courtroom, the word “forensic” carries a veneer of infallibility. Jurors hear it and envision the sleek laboratories of television crime dramas with sterile rooms humming with sophisticated technology, where evidence is processed by advanced algorithms that produce irrefutable results. This misguided belief persists despite a troubling reality: many forensic techniques routinely used to convict defendants rest on foundations no sturdier than intuition, and some have been thoroughly debunked by the very scientific community whose authority they claim.

The consequences of this disconnect are measured not in abstractions but in human lives. More than 3,750 Americans have now been exonerated after wrongful convictions, many imprisoned by forensic evidence whose probative value was grossly overstated. Collectively, these innocent people lost over 37,000 years to incarceration. To place that number in perspective: 37,000 years ago, Neanderthals were vanishing from their last refuges in Ice Age Europe, while woolly mammoths still roamed across three continents.

This article examines the persistent gap between forensic mythology and forensic reality. It traces how techniques from fingerprint analysis to bitemark comparison have evaded meaningful scientific scrutiny, how cognitive bias corrupts even well-intentioned practitioners, and how systemic failures – scientific illiteracy among jurors, overburdened defense attorneys, and judges ill-equipped to evaluate technical claims – allow unreliable evidence to determine guilt and innocence. Importantly, it explores what can be done: the legal tools, educational interventions, and institutional reforms that might, at last, align courtroom practice with scientific truth.

A Two-day Course and a 33-Year Sentence

Jessica Logan, 31, is currently incarcerated in Illinois for the 2019 murder of her infant son, Jayden. Except that it’s not at all clear there was a murder. Logan found the infant tangled and suffocated in his crib sheets and immediately dialed 911. But within a minute of listening to a tape of the call, Decatur Police Dept. (“DPD”) Det. Eric Matthews wrote in his notes that he was sure Logan had killed her child, ProPublica reported. How did he know?

Five months earlier, Matthews sat through a two-day course at Richland Community College taught by retired Ohio cop Tracy Harpster called 911 homicide: Is the caller the killer? Originally developed as a thesis for his graduate work, Harpster’s program purports to provide training in analyzing 911 calls for signs that the caller is guilty of the crime. His work has been thoroughly debunked by “seven federal government agencies, universities and advocacy groups,” ProPublica found. The FBI has also issued warnings about its use. Yet the Illinois Law Enforcement Training and Standards Board (“LETSB”) approved Harpster’s course for credit toward certification as a lead homicide investigator. Matthews was there on the public dime.

As his notes showed, he began applying what he learned to build a case against Logan. Meanwhile the Macon County coroner hired pathologist Dr. Scott Denton to perform an autopsy on Jayden. He and Matthews agreed that the detective would ask Logan to reconstruct events from that fateful night. Credulously, she agreed, using a doll and letting Matthews film her. After the detective passed along the footage, Denton finalized his report: The child’s death, he said, was caused by “smothering and compression of the neck.”

If Jayden was killed, of course, then someone killed him – and no one else was home at the time except Logan’s older child, still a toddler. Logan also let cops search her phone, where they found evidence of a Google search conducted before Jayden died asking, “How do you suffocate.” Except it’s not clear that the Google search happened when the cops said it did. Family members recalled that they and Logan wondered together in the hours after the death how it was possible to suffocate. They said that she made a phone search then, though they couldn’t say what she typed. But it’s also possible the search never happened at all. ProPublica reviewed the Google search file and was unable to verify what DPD claimed it extracted from the phone.

When asked how the LETSB decided to approve Harpster’s course for professional accreditation, Chief Legal Counsel John Keigher told ProPublica that he was unsure, but he guessed that a previous training manager “did his own research and Googled stuff.” ProPublica also submitted Denton’s autopsy to three other pathologists, and all disagreed with his conclusion that Jayden died from “smothering.” Fatal asphyxiation, they noted, is frequently accidental or the result of illness. None of this exonerating evidence was presented at Logan’s trial, however, because her attorney apparently never looked for it. The attorney also bumbled Matthews’ cross-examination, giving the cop an opening to share his “analysis” of the 911 call with jurors.

Crucially, ProPublica found, the contents of Harpster’s course have been carefully shielded from discovery by cops who take it. This secrecy enables what could be called parallel construction, a practice where law enforcement relies on a covert or inadmissible method to form a conclusion, then assembles a separate, ostensibly independent evidentiary record for court. In Logan’s case, officers appeared to use the debunked 911 analysis to decide she was the killer, then sought corroborating evidence, like the Google search and the pathologist’s determination, to support that conclusion.

This investigatory sleight of hand is fraught with peril for the accused. By insulating the original analysis from formal disclosure, police effectively shield it from judicial scrutiny. Because the 911 analysis was never cited as the investigation’s foundation, the defense was denied any opportunity to challenge its validity or to argue that subsequent evidence may have been gathered and interpreted through the distorting lens of that initial, unfounded determination. In fact, Matthews’ 911 call analysis had not been shared in writing with prosecutors, much less the defense attorney.

This secrecy laid a trap that the defense counsel bumbled right into. During cross-examination, Logan’s attorney mishandled his questioning of Matthews, and in doing so, inadvertently presented an opportunity. Matthews seized upon it by presenting his 911 analysis to the jury as though it were an established investigative technique. He testified that from his training, he had identified “mostly all guilty indicators” in the call Logan made. Blindsided by testimony that had never appeared in any filing or disclosure, her attorney failed to challenge it.

The effect was decisive. Combined with Denton’s cause-of-death determination and the disputed Google search, Matthews’ testimony gave jurors what they needed – an uncontested pronouncement of guilt wrapped in the veneer of scientific certainty. They convicted Logan of first-degree murder. In 2021, she was sentenced to 33 years in prison. Three years later, the state Supreme Court affirmed the conviction on direct appeal.

Zombie “Science”

It should be obvious that no one sitting in a classroom for two days comes out prepared to say with certainty that the way a distraught young mother talks to a 911 dispatcher about her dying child proves that she is guilty of killing him. The FBI has said as much – twice. But 911 call analysis is just one of several forensic methods which, like zombies, continue to stalk the American criminal justice system long after they’ve been declared dead.

Other discredited techniques purport to scientifically analyze footprints, shoe prints, and bitemarks. Because the methods are studied in a lab by a technician with some training in natural sciences, law enforcement authorities continue to insist that the analysis is science. But it’s not science. It’s what the Hoover Institution calls “scientism,” an exaggerated trust in the methods of natural science to explain areas of human behavior where they do not apply. Its spread is likely influenced by rampant scientific illiteracy in the U.S., where the National Center for Education Statistics (“NCES”) reported in 2019 that just 22% of high school graduates were judged proficient in science under the National Assessment of Educational Progress. Science Performance, NCES Annual Reports (May 2022); and The Dangerous Rise Of Scientism, Hoover Institution (June 2016).

In the realm of criminal justice, this means that far too many potential jurors imagine a forensics lab looks like what they see on TV dramas: a spacious room with big windows and flattering lighting, outfitted with advanced computer technology. Where crime-scene evidence like fingerprints and hairs is quickly rendered into component patterns and chemicals, which a computer matches to samples stored in massive databases. Where voice recordings turn into wave patterns that are analyzed the same way. Where grainy photos of blurred faces are brought into crisp resolution and magnified many times. Under this onslaught of bytes and algorithms, a suspect is quickly identified with certainty. Pathetic alibis of the guilty unravel just as fast.

But the reality is starkly different. In the small, windowless rooms where most technicians work, those grainy photos with blurred faces typically remain grainy and blurred because computer programs cannot accurately augment the existing pixels in the image. To do so, they would have to manufacture additional pixels that were never captured by the camera. While some new artificial intelligence programs attempt this, they result in “hallucinations” that compromise the evidence, generating details that don’t exist, much like the AI “slop” images flooding the Internet that give themselves away with extra fingers or limbs.

In the same way, crime-scene fingerprints, hairs, and voice recordings are analyzed for patterns and chemicals. But there is limited computer assistance in making matches, relying instead on the technician’s training and experience – and leaving the methods subject to shockingly high error rates. Yet when a suspect hears that forensic science has identified him as the perpetrator of a crime, he may imagine the sort of iron-clad results depicted on TV. His attorney and even the prosecutor may labor under the same misconception. All too often, that’s the point at which “forensics” exits the story, having helped prosecutors negotiate a plea. In the U.S., well over 95% of state and federal criminal convictions are resolved this way. Trials are the rare exception. [CLN, Dec. 2025, p.1.]

But even for those brave suspects and their legal counsel not blinded by the shiny claims of forensic science, the path to a trial verdict is mined with other dangers, beginning with the limitations of forensic scientists and their techniques. When called to testify, forensic technicians all too often oversell their product and discount its flaws. The hapless defense counsel, usually overloaded with cases, may be too distracted to conduct an effective cross-examination. To counter with testimony from another expert witness, he must also clear technical and procedural hurdles, after which jurors are still likely to discount what the defense expert says.

Littering this grim landscape are now some 3,750 cases of men and women who were later exonerated after being wrongfully convicted of crimes, often due to forensic evidence whose value was overstated. Many times, the defendants themselves were the ones bamboozled, but it happens to defense attorneys, too. Prosecutors aren’t obligated to make full disclosure of evidence before arraignment, which is when plea bargains frequently occur. Collectively, these exonerees have been robbed of over 37,000 years of their lives while they were wrongfully imprisoned. It’s a number that’s difficult to fathom, so consider this: 37,000 years ago, the only records of individual human lives were red ochre handprints pressed onto cave walls, the earliest surviving marks of people who refused to vanish without a trace.

Limitations of Modern Forensic Techniques: Fingerprint Analysis

As a field of study, fingerprint analysis was barely 30 years old when a U.S. prosecutor first successfully used it in 1911 to identify a criminal suspect and obtain a conviction. Illinois parolee Thomas Jennings was still wearing a blood-stained coat and carrying a pistol when he was apprehended for the fatal shooting of railroad clerk Clarence Hiller at his Chicago home. Crucially, his fingerprints were also matched to those that Hiller’s assailant had left behind on the home’s freshly painted stair railing, a chunk of which was sawn off and brought to court as evidence for Jennings’ trial.

His attorney, W.G. Anderson, attempted to demonstrate for jurors how unreliable this new “science” was by challenging a forensic technician to lift the attorney’s own prints from a piece of paper that he had touched. The technician then proceeded to produce a remarkably clear set of Anderson’s prints. The abashed lawyer never recovered from the impression that this left on jurors, who voted to find Jennings guilty of capital murder. As the Decatur Herald reported at the time, it was “the first conviction on finger-printing evidence in the history of this country.”

On direct appeal, the state Supreme Court upheld the conviction, boldly declaring that “[t]his method of identification is in such general and common use that the courts cannot refuse to take judicial cognizance of it.” People v. Jennings, 252 Ill. 534 (1911). With that, the Chicago Tribune reported, the state’s highest court had decided that fingerprint analysis was “sufficient basis for a verdict of death by hanging.”

As Smithsonian Magazine observed over a century later, the Jennings case also cemented the role of the expert witness at criminal trials. Even today, “courts have not significantly curtailed the use of fingerprint evidence, nor the reliance on examiners to interpret this evidence for the jury” – though the validity of an identification has been more often challenged as prosecutions increasingly rely on partial prints, unlike the clear set that Jennings left behind.

The foundation of fingerprint analysis rests on an assumption that has never been empirically proven: that every person’s prints are unique and unchanging over a lifetime. This may well be true, but proving it would require examining the fingerprints of every human being who has ever lived. Instead, the assumption has simply been accepted – first by courts, then by the public – as though repetition were the same as verification.

Perhaps the most famous modern failure of fingerprint analysis resulted in the arrest of Brandon Mayfield for bombing the Madrid train system in 2004. Spanish investigators lifted prints from a plastic bag they found, which was filled with detonator caps. The FBI helped them match those to a set of Mayfield’s prints, stored in what was then called its Integrated Automated Fingerprint Identification System (“AFIS”). The problem was that Mayfield, an Oregon lawyer, had never been to Spain. He didn’t even hold a valid passport. But he was arrested anyway, only to be released when Spanish authorities matched the same print to Ouhnane Daoud, an Algerian national who was among those convicted in Spain’s 2007 trial of the Madrid bombers.

The false match of Daoud’s prints to Mayfield was initially made by a computer program trained to look for similarities in the direction and characteristics of fingerprint “ridges.” Three FBI specialists then examined the two sets of prints and confirmed the bogus match, based on 15 “points” at which the prints purportedly appeared similar. How could they get it so wrong? There are several possible reasons. First, there is no standard set of points to compare, and no agreed-upon number of comparisons that must be made; both of those decisions are up to the judgment of the individual examiner. Add in real-world problems like partial, smudged, or otherwise low-quality prints, and the work of a fingerprint examiner becomes even more subjective, with results that are extraordinarily difficult to replicate.

Yet the ability to repeat an experiment and reliably replicate its results is a foundational scientific principle. Which made it all the more remarkable when a landmark study found that four of five analysts changed their assessments after being falsely told that the prints they had been given were those mismatched to Mayfield from the Madrid bombing. A rigorous scientific process should not be so dramatically affected by this sort of cognitive bias in its practitioners. Dror, I.E. & Charlton, D., “Why Experts Make Errors,” Journal of Forensic Identification (2006).

The FBI conducted its own review and found that supervisors had made the initial match between the prints from Spain and Mayfield, which apparently left their subordinate examiners feeling pressured to confirm the bosses’ work. In yet another example of cognitive bias, a controlled study of fingerprint analysts found that they were more or less likely to find a match based on whether a high match probability had been assigned by the AFIS software.

But the rot goes deeper than bad supervisors. In 2016, a landmark report by the President’s Council of Advisors on Science and Technology (“PCAST”) reviewed the few rigorous studies that actually tested examiners’ accuracy. Far from being infallible, the report found false-positive error rates as high as 1 in 18. The following year, a report by the American Association for the Advancement of Science concluded that because there is no way to know how many people might share similar ridge patterns, courtroom testimony claiming to identify a suspect “to the exclusion of all others” is scientifically “indefensible.”

None of this has kept fingerprint analysis from being admitted into evidence in criminal trials. Under standards first laid out in Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579 (1993), and later codified in Federal Rule of Evidence 702, judges serve as gatekeepers for scientific testimony, weighing whether the methodology (1) can be tested; (2) has been subjected to peer review; (3) has a known or potential error rate; (4) has standards and controls; and (5) is generally accepted in the relevant scientific community. Evidence satisfying these factors may be admitted, though critics argue the framework is far too permissive when applied to forensic techniques with documented flaws.

As the U.S. Court of Appeals for the Third Circuit explained in United States v. Mitchell, 365 F.3d 215 (3d Cir. 2004), the underlying theory in fingerprint analysis, that prints are unique and unchanging, met the first factor because it is scientifically testable. And although there is conflicting opinion over the third factor, whether a determinable error rate can be calculated for the standard Analysis, Comparison, Evaluation, and Verification (“ACE-V”) process used by fingerprint examiners, the Mitchell Court said that verification by enough analysts could provide something like scientific peer review to satisfy the second factor. The next factor was also met because standards and controls for fingerprint analysts are maintained by the Scientific Working Group on Friction Ridge Analysis, Study, and Technology. Moreover, clearing the last factor, courts have been accepting fingerprint evidence for a century, believing – much like the Illinois Supreme Court that heard Jennings’ appeal – that most of the “scientific community” was on board. Therefore, the fingerprint evidence in Mitchell and thousands of other cases was deemed admissible.

That leaves it to a battle of experts to convince jurors that their analysis was the one most faithful to the ACE-V protocol, making the identification they have made or disproven the most accurate result. But according to a November 2023 report by the National Institute of Justice (“NIJ”), 46% of 87 fingerprint analyses that were studied revealed at least one error, with 18% providing a false identification of a criminal suspect. The report stressed that most errors involved fraud by uncertified examiners who “clearly violated basic standards.” But this caveat offers cold comfort. It reveals a system in which unqualified analysts can operate undetected and in which even a fraction of that 18% error rate, applied to certified examiners, would be intolerable in a domain where liberty and life literally hang in the balance. The Impact of False or Misleading Forensic Evidence on Wrongful Convictions, NIJ (Nov. 2023).

These statistics are not abstractions. Consider the case of Charles Coleman, Jr., who was accused of a string of nine 1986 rapes and robberies in New Haven, Connecticut, based on fingerprint analysis. When the cases were consolidated, Coleman entered an Alford plea, but he moved to withdraw it just before sentencing. The motion was denied, and he was sentenced to 35 years in prison. After losing on direct appeal, he filed a federal habeas petition and was allowed to withdraw his pleas. He then went to trial again in three of the nine cases. Based on the fingerprint evidence, he was again convicted, this time earning sentences that totaled 170 years.

In 2021, Coleman successfully petitioned to submit the evidence from those decades-old sexual assaults for DNA testing. The results excluded him as a contributor to two of the samples; the third sample wasn’t tested because it had been lost by then. Importantly, the fingerprint evidence was also re-examined. Experts again reported a match in all three cases, with the same degree of certainty that they had expressed at Coleman’s original trials. Fortunately, a state court was persuaded by the exculpatory DNA test results to reverse the two convictions that they challenged. Since Coleman had already completed his sentence for the third conviction, he was released in 2024, after four decades in prison for crimes he did not commit.

Multi-Source and Trace DNA

There is a type of evidence that allows experts to make a criminal identification with a very high degree of certainty: Single-source DNA from a sample large enough and in good enough condition to pull the entire sequence of chromosomal pairs. If there is no match to a suspect’s DNA, he or she may be excluded as a possible donor. If there is a match, it indicates that the sample DNA came from the suspect. There is some theoretical uncertainty because assigning a unique DNA profile to everyone cannot be conclusively proven without subjecting the entire human population to testing, but the statistical probability of a coincidental match is infinitesimal.

New York’s Supreme Court for Bronx County was among the first in the U.S. to rule on the admissibility of DNA evidence at a criminal trial in 1989 – and the first to critically scrutinize it. Prosecutors argued that the DNA profile of 20-year-old Vilma Ponce matched that found in blood smeared across the face of a watch worn by Joseph Castro, the man accused of her 1987 murder. The standard then in place, before Daubert, came from Frye v. United States, 293 F. 1013 (D.C. Cir. 1923). Following a 12-week hearing, the court determined that the Frye test was met because “DNA forensic identification techniques and experiments are generally accepted in the scientific community and can produce reliable results.”

The sort of pretrial hearing that the court conducted was needed, the court said, “to determine if the testing laboratory substantially performed the scientifically … accepted tests and techniques, yielding sufficiently reliable results to be admissible as a question of fact for the jury.” Based on what it heard, the court found that “the DNA identification evidence of exclusion is deemed admissible as a question of fact for the jury.” Accordingly, prosecutors could inform jurors that the blood on Castro’s watch was not his. But “the DNA identification evidence … of inclusion is deemed inadmissible, as a matter of law,” the court said, because of testing lab failures that the hearing uncovered. Therefore, jurors could not be told that the blood on the watch was likely a genetic match to Castro’s blood. People v. Castro, 144 Misc. 2d 956 (N.Y. Sup. Ct. 1989).

Castro then pleaded guilty without going to trial. But his case established the judicial precedent for the admissibility of DNA evidence, including recommendations for discovery requirements, such as providing the defense with copies of all lab tests and reports, plus an explanation of the statistical probability calculations used to qualify a match and any errors observed in the chain of custody.

The same year that Castro was decided, DNA evidence was first used to exonerate a wrongfully convicted prisoner. Gary Dotson was freed from an Illinois prison where he had served 12 years for a 1977 rape that never occurred. The accuser, Cathleen Crowell, admitted making up the tale after having sex with her boyfriend and fearing she might have gotten pregnant. When she made that sensational confession in 1985, it touched off a four-year-long media circus, during which Dotson was paroled and re-incarcerated several times before DNA testing finally excluded him as a contributor of the DNA found in Crowell’s semen-stained underpants.

In a less-publicized case, also from 1989, the Minnesota Supreme Court took the recommendations for discovery from Castro a step further. “Ideally, a defendant should be provided with the actual DNA sample(s) in order to reproduce the tests,” the court said. But “[a]s a practical matter,” it admitted, “this may not be possible because forensic samples are often so small that the entire sample is used in testing. Consequently, access to the … data, methodology, and actual results is crucial so a defendant has at least an opportunity for independent expert review.” State v. Schwartz, 447 N.W.2d 422 (Minn. 1989).

Neither of these cases noted that among its limitations, DNA analysis cannot tell how a particular genetic profile came to be found on a particular piece of evidence. It can’t tell when it happened. But the analysis will tell whether a particular suspect is excluded from the portion of the population that might have donated the genetic material. It will even indicate that the suspect may have been the donor. That doesn’t definitively prove the suspect’s guilt, however, as the notorious case of the “Phantom of Heilbronn” shows.

In that incident, police in several European countries found the same female DNA on evidence collected from dozens of crime scenes all over the continent, dating back to 1993. By 2009 – two years after the same DNA was found at the scene of a German cop’s murder in the town of Heilbronn – the reward for the unknown woman’s capture had spiked to €300,000 (approximately $400,000 at the time). It was only then that officials realized they had duped themselves; the recurring DNA most likely came not from a suspect but a worker in a Polish factory, which manufactured the cotton swabs used to collect samples for testing. In 2016, the publication of ISO standard 18385 finally established a global protocol requiring “forensic grade” sterile swabs to prevent such contamination.

But factory contamination is not the only way DNA lies. The hyper-sensitivity of modern equipment has introduced a new peril: Secondary Transfer. We now know that DNA can be transferred from person A to person B via a handshake, and then left by person B on a weapon that person A never touched. This phenomenon nearly imprisoned Lukis Anderson for a California murder in 2012; his DNA was found on the victim’s fingernails, but he was in a hospital coma at the time. The vector was the paramedics, who treated Anderson and then the victim.

While single-source DNA analysis has been used in thousands of criminal cases, its most profound impact has come in exposing wrongful convictions like Dotson’s, leading so far to hundreds of exonerations. In 38 of those, the prisoner was freed from death row, according to the National Registry of Exonerations (“NRE”).

As the Schwartz court noted, crime scenes often yield only small amounts of genetic material, sometimes no more than a trace sample, with just a fragment of DNA. Samples may also contain a mixture of DNA from multiple sources. Sifting through this genetic material to match or exclude a suspect is an incredibly complicated process, one that is susceptible to a multitude of errors. If a trace amount of DNA shows a particular pattern in a particular chromosome, for example, that doesn’t “match” a suspect to the crime simply because he shows the same pattern on the same chromosome. The determination is statistical, not visual. Analysts compare the genetic profile from a crime scene to population databases and calculate the probability that a random person would share those markers, a figure known as the Random Match Probability (“RMP”). A very low RMP, such as one in a quadrillion, makes a coincidental match so improbable as to be dismissed in practice, though not in theory.

This mathematical component, combined with advanced techniques to magnify tiny DNA samples, has given rise to Probabilistic Genotyping (“PG”), as CLN reported. [CLN, Aug. 2025, p.1.] A relatively new branch of DNA analysis, it looks at trace and mixed DNA samples through the lens of mathematical probabilities to declare the likelihood that a suspect can be excluded as the source. Needless to say, this makes the work of the average juror exponentially harder, leaving him or her more reliant than ever on the testimony of experts – and not just experts in genetic science but also those trained in the advanced mathematics necessary to make probability calculations.

Further complicating the process for defendants, vendors of PG software typically refuse to share its source code, fearing competitors will use it unfairly. In Schwartz, the Minnesota Supreme Court acknowledged that Cellmark Diagnostics Corp., the private firm contracted to provide DNA analysis in the case, “ha[d] not yet published data regarding its methodology” in its “DNA fingerprinting” protocol. Yet the court denied a defense request “for more specific information regarding its methodology and population data base.” In fact, Cellmark had objected to giving up what it did, but the court said that even when “trade secrets may be at stake,” there are “[p]rotective measures [which] could be pursued … before denial of discovery is appropriate.” The same remains true of PG methodology. But it’s just as true that courts remain reluctant to challenge these trade secret claims.

Even more challenging for the defense is the sheer complexity of the science and math that PG involves. While the average juror may understand that each person carries in his or her cells a DNA sequence that is believed to be unique, all that can be determined from a DNA sample, even one that is complete and in good condition, is that a suspect can or can’t be excluded as the donor. When dealing with trace DNA samples, this already tricky process becomes even more challenging. Any DNA from other sources must be segregated and quantified in a way that explains the risk of a false match. So must the amount of amplification (known as PCR) to which the sample must be subjected, replicating the tiny number of cells billions of times in a lab, in order to test it.

Every bit of this, moreover, is ultimately expressed in mathematical formulas that attempt to quantify genetic test results in terms of RMP. Jurors never look at a magnified image of DNA inside a cell nucleus because it would be meaningless; a “match” to the image of DNA inside another cell nucleus cannot be made visually. Instead, each must be rendered into a graph showing each gene at each place it resides in a chromosome.

As these “alleles” pile up on the graph and pass predetermined threshold quantities, they reach “peaks” that may be counted as true expressions of the presence of a certain type of gene. However, the analyst must beware of “stutter peaks” produced by the electropherogram making the graph; those are not really peaks at all and must be disregarded. Ultimately, relying on an ever-expanding database of analyses conducted in the same way, the odds that genetic information is of value in excluding or identifying a suspect are expressed mathematically as the RMP.

The Castro Court took weeks just to weigh the DNA evidence presented by prosecutors, listening as experts explained how testing works, how allelic graphs are made, and whether or not the lab used in the case had followed generally accepted practices to arrive at an RMP. And that was with a DNA sample from a single source in a far larger quantity than PG uses. Even then, the court determined that only the DNA-exclusion analysis passed muster, not the DNA-inclusion analysis, so jurors never heard that Vilma Ponce was a possible donor of the blood found on Castro’s watch.

Over the intervening decades, DNA analysis has come to be considered the best forensic evidence by most criminal justice experts. No one has yet been exonerated from a conviction that was obtained primarily with DNA evidence. But the process is expensive, a factor limiting its use by prosecutors. It has helped overturn a number of wrongful convictions, though, when evidence from an older case was submitted to DNA testing that excluded the prisoner who was convicted of the crime as a possible donor. Perhaps righting an injustice justifies the expense more easily than avoiding injustice in the first place.

By 2020, as University of California at Berkeley Law School Prof. Andrea Roth noted in the chapter she contributed to Silent Witness: Forensic DNA Evidence in Criminal Investigations and Humanitarian Disasters, DNA analysis had accomplished what other discredited forensic techniques have failed to do, establishing an agreed-upon set of testing parameters in a specific group of locations on a DNA strand where allelic distribution is mapped for comparison and identification. The FBI requires the same 20 points to be mapped in every genetic profile uploaded to its Combined DNA Index System (“CODIS”). U.S. courts have all come to accept analyses that are mathematically expressed in RMPs.

What remains to be seen is how well the record of DNA testing holds up with its PG variant, where the risk of another “Phantom of Heilbronn” multiplies with each additional contributor to a multi-source DNA sample, and the risk of overstating a match probability rises in concert with the amount of amplification necessary to test a trace sample. Good luck to the average juror, who has just a 22% chance of being proficient in science, as he or she attempts to follow an expert’s explanation of this.

For all its limitations, DNA analysis remains the closest thing forensic science has to a gold standard – a method grounded in genuine biological principles, subject to statistical quantification, and capable of true exoneration. The same cannot be said for the techniques that follow. Where DNA evidence struggles with complexity and opacity, the forensic methods examined next suffer from a more fundamental defect: they were never science at all.

Junk Science: Analyzing Hair and Bitemarks

In many of the exonerations catalogued by the NRE, a wrongful conviction was obtained based on other types of evidence whose usefulness in making a criminal identification is now considered very low. Though each was at one point considered just as certain as DNA matching, all have been relegated to the category of “junk” science. That doesn’t prevent police from using these techniques to identify a suspect, the way Jessica Logan became a suspect in her son’s death by an “analysis” of her 911 call. But it does mean that the techniques are easier to challenge (1) by an effective defense attorney, provided that (2) prosecutors take the evidence to court.

Hair Analysis

Human bodies are covered in hair, which is constantly shed onto clothing and other surfaces. So finding incriminating hairs at a crime scene is not new. In the U.S., prosecutorial use of hair analysis at trial dates back at least to 1855, when John Browning and his son Gaston Browning were tried in Mississippi for the murder of John Neal, the overseer of a nearby plantation whose brick-weighted body was fished out of the Yazoo River with a broken neck. A rope found on Browning’s property was tied into a noose. In the noose were hairs “which upon comparison corresponded exactly in color and length with Neal’s hair,” as the state Supreme Court later recalled. Browning v. State, 33 Miss. 47 (1856).

From that 19th-century Mississippi courtroom to late-20th-century FBI labs, there was no significant change in the way forensic hair analysis was conducted: A hair found at a crime scene was observed for its characteristics, in later cases with the aid of modern microscopes; a similar observation was then made of a suspect’s hair; and finally, the two were compared. But there were no standard points of analysis, no agreed parameters for determining similarities, and no database showing the prevalence of hair characteristics in the general population. So there was no way to assign a probability to any match that was made. The method was never scientific because it couldn’t be replicated in controlled settings to produce predictable results.

Yet of all the DNA exonerations catalogued by the NRE, nearly one-quarter have been wrongful convictions obtained with hair analysis. Subjecting the evidence to DNA testing, it was later revealed that a false match had been made. Three exonerations involving microscopic hair comparison analysis (“MHCA”), all performed in FBI labs, eventually cost the District of Columbia almost $40 million in payouts to the innocent men convicted and imprisoned based on the flawed results.

The first exoneree, Donald Eugene Gates, was convicted in 1982 of raping and murdering Catherine Schilling the previous year, after FBI analyst Michael Malone testified that Gates’ hair was “microscopically indistinguishable” from those left behind on the victim’s body. A U.S. Department of Justice (“DOJ”) Inspector General later flagged problems with Malone’s work during a 1997 review of the FBI crime lab. But the report was never made public, and only prosecutors were notified. Gates’ attorneys didn’t find out until 2002, and it then took them another seven years to get DNA testing of the evidence, which excluded Gates as the source of the hairs found on Schilling’s corpse. Then 59, Gates was released from prison in 2009. He sued the City and its police detectives for his wrongful conviction, and a jury ruled for him six years later. The suit was settled in 2015 for $16.65 million.

The following year, fellow D.C. resident Santae Tribble took a $13.2 million payment under the city’s Unjust Imprisonment Act (“UIA”) for the 28 years he spent wrongfully locked up. Tribble was convicted of the 1978 murder of cabdriver John McCormick, 63, after another FBI analyst testified that Tribble’s hair matched those found in a ski mask discarded by the killer. He was paroled in 2003. Then, after DNA testing of hair evidence exonerated Gates in 2009, Tribble sought testing of the hairs that convicted him. DNA analysis concluded that the hairs came from someone else – in fact, from three other people, plus a dog. Four years after securing the payout for this error, Tribble, 59, died in 2020 of complications from HIV contracted while imprisoned.

The third exonerated D.C. prisoner, Kirk Odom, was an intellectually disabled 18-year-old with an IQ of 73 when he was convicted of a 1981 rape, again after testimony from an FBI analyst that his hair was “indistinguishable” from hair found on the victim’s nightgown. He was paroled in 2003. Following the exonerations of Gates and Tribble, evidence from Odom’s case was submitted in 2012 for DNA testing, which excluded him as a possible source. His claim under the City’s UIA was settled for $9.654 million in 2013; the same year, he accepted another $1.2 million to settle his claims against the U.S. government for the FBI’s misconduct.

After that damning 1997 DOJ Inspector General report on Malone’s work, the FBI limited use of hair analysis to cases in which the hair sample was also subjected to DNA testing. Instead of providing a basis for criminal identification, hair has come to be treated as any other type of crime scene evidence that must be genetically tested to see if a suspect could be eliminated as the donor. It is important to note that these three D.C. men – and dozens more also exonerated from convictions obtained with hair analysis – were imprisoned based not just on the faulty results of the testing but also on the testimony of FBI analysts, who declared with all that agency’s authority that MHCA had proven the hairs from the suspects were indistinguishable from those found at the crime scene. It was a gross overstatement of what the analysts could determine, and they knew it. But none of them was ever charged with a crime, and none has offered a public accounting for the lives destroyed by their testimony.

Bitemark Analysis

At least 34 former prisoners have now been exonerated from convictions based on bitemark analysis, as CLN reported. [CLN, Dec. 2020, p.45.] Like hair and fingerprint analysis, it is a method of identification that relies on observing and matching patterns found in evidence to the physical characteristics of a suspect – in this case finding a “match” between a bitemark and a suspect’s teeth. The problem, according to a 2023 report by the National Institute of Standards and Technology (“NIST”), is that the methodology rests on three key presumptions, all of which have now been disproven.

First, bitemark analysts insist that your bitemark is like your fingerprint – unique to you. But NIST found no evidence to support the idea that “human dentition is unique at the individual level.” Different people can leave very similar bitemarks. Moreover, whatever uniqueness that your bitemark does have cannot be “accurately transferred to human skin.” Not all bites leave marks, and when they do, “human skin can change the appearance of a bitemark over time depending on the rate and amount of swelling at the site, healing, and skin elasticity.” Finally, the report concluded, there is no solid evidence that “identifying characteristics can be accurately captured and interpreted by analysis techniques.” Bitemark Analysis: A NIST Scientific Foundation Review, NIST (Mar. 2023).

The unraveling of bitemark “science” began with an earlier 2009 report from the National Research Council (“NRC”) of the National Academy of Sciences (“NAS”), which determined that bitemark and hair analysis – even fingerprint analysis – are less-than-reliable forensic techniques. As that report summarized, “no forensic method other than nuclear DNA analysis has been rigorously shown to have the capacity to consistently and with a high degree of certainty support conclusions … ‘matching’ … an unknown item of evidence to a specific known source.” Strengthening Forensic Science in the United States: A Path Forward, NAS/NRC (2009).

Exonerations from convictions based on bitemark analysis followed. But the debunking of bitemark analysis has not overturned every conviction obtained with testimony from bitemark analysts. In the case of Alabama prisoner Charles McCrory, even Dr. Richard Souviron, the analyst who testified at his 1985 murder trial, has since recanted. Then already famous for helping convict Florida serial killer Ted Bundy in 1979, Souviron “testified that alleged bitemarks on the victim had been made at or about the time of death and were consistent with dental impressions taken from McCrory,” as the Supreme Court of the United States (“SCOTUS”) later recalled, adding: “The jury convicted.”

But in an affidavit filed in McCrory’s 2020 bid for postconviction relief, Souviron admitted that “it would be unreliable and scientifically unsupported for me or any forensic odontologist to offer individualization testimony that Mr. McCrory was the source of the teeth marks, as I testified in 1985.” As a result, he said, “I therefore fully recant my testimony.” It was a remarkable admission: the expert whose testimony had helped convict Ted Bundy – and whose credibility had sent McCrory to prison – now acknowledged that the entire discipline lacked scientific support.

Alabama courts found circumstantial evidence to support the conviction anyway – a witness reported seeing McCrory’s truck outside when the murder took place – and denied the petition for retrial. SCOTUS then declined to issue a writ of certiorari to hear McCrory’s appeal. In a statement accompanying the denial on July 2, 2024, SCOTUS noted several obstacles facing any defendant seeking to challenge a conviction based on “science” that has since been discredited.

First, “because science evolves slowly rather than in conclusive bursts,” it is difficult to say when a defendant knew or should have known about the new evidence, presenting a tolling problem for statutes of limitations. Second, courts in Alabama and many other states discount “impeachment evidence,” which is evidence that “would merely have given the jury reason to disbelieve the expert’s evaluation” at the original trial. Third, state courts “have held that changed science evidence is merely cumulative of other evidence or fails to point affirmatively to innocence.” SCOTUS declined to consider whether this amounted to a constitutional violation, noting that state legislatures could step in with relief for the wrongly convicted, as some states have already done. McCrory v. Alabama, 144 S. Ct. 2483 (2024).

Other Discredited Techniques

Joining hair and bitemark analysis on the trash heap of discredited forensic science are several others relying on feature comparison and pattern-matching techniques, including shoe print analysis and bloodstain pattern analysis. Tracy Harpster’s 911 call analysis should join them, which is likely why the cops using it are so keen to keep it a secret from defense attorneys.

Each of these techniques relies on observation to spot similarities to, or variations from, an expected pattern. But a hallmark of pseudoscientific forensic techniques is that there is little agreement what that expected pattern looks like. Det. Matthews wrote in his notes that Logan showed little emotion during her 911 call and kept repeating that her son had been ill. To him this indicated that she was not remorseful about his death and was in fact trying to build a case for an alternative explanation – one other than that she killed him. But Jayden had been sick; wouldn’t his mom be just as likely to fret that she could have done more to get him timely medical help? Couldn’t the lack of emotion in her voice betray shock?

These forensic techniques also suffer from a lack of any standardized points of comparison, as well as any population studies to quantify the characteristics and their prevalence. If bitemark analysis works by identifying “unique” features found in a suspect’s teeth, on which teeth and where exactly are they found? If MHCA works by matching the chemical composition of crime scene hair to that of a suspect’s hair, where are the studies showing how likely it is to find that chemical signature on any given head? Without a database of these characteristics, it is impossible to calculate the probability of a match.

The PCAST report stressed that something of scientific value may be salvageable from these techniques with further study to quantify the probability of detecting a given characteristic. Called “black-box studies,” these examinations ignore the internal process of the forensic method and look only to see if a given input reliably produces a given output.

In footwear analysis, for example, an examiner inputs his observation of a given shoe’s “randomly acquired characteristics” (“RACs”), such as “cuts, nicks, and gouges” that the shoe acquired “in the course of use.” At the conclusion of the forensic process, the output purports to show a probability estimate of finding any given RAC on any given shoe of the same model and size. But without those black box studies to provide a statistical baseline, the report said, “claims for ‘identification’ based on footwear analysis are breathtaking – but lack scientific foundation.”

“The entire process – from choice of features to include (and ignore) and the determination of rarity – relies entirely on an examiner’s subjective judgment,” the report concluded. “Under such circumstances, it is essential that the scientific validity of the method and estimates of its reliability be established by multiple, appropriate black-box studies.”

The NAS 2009 report was only somewhat less harsh in its evaluation of bloodstain pattern analysis. “Scientific studies support some aspects of bloodstain pattern analysis,” the report allowed, such as whether “the blood spattered quickly or slowly.” But experts were being qualified primarily by experience, not scientific training, and many of their analyses were found to “extrapolate far beyond what can be supported.” Because “the complex patterns that fluids make when exiting wounds are highly variable,” the report stressed that “many experiments must be conducted to determine what characteristics of a bloodstain pattern are caused by particular actions during a crime and to inform the interpretation of those causal links and their variabilities.” Since those studies had not been performed, the report concluded, the “uncertainties associated with bloodstain pattern analysis are enormous.” Strengthening Forensic Science in the United States: A Path Forward, NAS/NRC (2009).

The November 2023 NIJ report found identification errors in 27% of blood spatter analyses that were reviewed. That’s an error rate higher than what was found for hair comparison analysis (20%) or fingerprint analysis (18%) but less than the error rates for shoe or footprint analysis (41%). Most striking was bitemark analysis, where 73% of the examinations reviewed contained identification errors – a failure rate so catastrophic it approaches the reliability of a coin flip. The Impact of False or Misleading Forensic Evidence on Wrongful Convictions, NIJ (Nov. 2023).

With error rates this high, wrongful convictions are inevitable. A stark example of both the power and fallibility of blood spatter analysis is provided by the case of Dewayne Dunn, who was convicted of fatally bludgeoning his neighbor Angel “Ching Ching” Torres outside their apartment building in Elkhart, Indiana, in September 2008. Dunn’s live-in partner, Letha Sims, told cops that the men got into an argument that escalated when Dunn bashed open the door with a baseball bat, and Torres, who was drunk, chased him outside onto the second-floor balcony. Hearing the sound of someone falling down the stairs, she rushed outside to find Torres face-down on the parking lot below, the bat underneath him. First responders arrived and attempted to stop his bleeding, but he died in a hospital without regaining consciousness.

Pathologist Dr. Scott Wagner determined that Torres had been beaten after he fell, which state cop Dean Marks confirmed by analyzing the pattern of blood spatter at the scene. At trial, when Dunn’s attorney pointed out that Torres landed on the bat that Dunn supposedly beat him with, and his blood was not found anywhere on Dunn except his shoes, prosecutors assured jurors that Wagner and Marks were not wrong and that there must have been another murder weapon which had not been found. The jury was convinced and voted to convict Dunn of first-degree murder in 2011. He was sentenced to 58 years in prison.

After the state Court of Appeals upheld the verdict on direct appeal, Dunn filed a petition for a new trial, arguing he received ineffective assistance of counsel when his attorney failed to call expert witnesses to refute the prosecution’s theory of the case. In support of his claim, a second forensic pathologist, Dr. Thomas Sozio, contested Wagner’s determination regarding Torres’ cause of death. But in April 2017, the Elkhart County Circuit Court said that this was merely another expert opinion and legally insufficient to merit a new trial.

The following year, Dunn filed a habeas petition pro se in the U.S. District Court for the Northern District of Indiana, which appointed him counsel, attorney Michael Ausbrook of the Indiana University Maurer School of Law. He successfully argued that Dunn needed an expert witness to point out glaring inconsistencies in the prosecution’s case – not only the suspect pathology report but also contamination of the crime scene by medics and other people photographed as they walked through Torres’ blood, under a heavy downpour, too. After the District Court granted the habeas petition and the prosecution appealed, the U.S. Court of Appeals for the Seventh Circuit affirmed on August 12, 2022.

First, there were inconsistencies between the evidence – the absence of blood spatter on Dunn’s shorts, for example – and the prosecution’s theory of the case, which the state court accepted. But “[i]f, as the state argues, the weapon was so saturated by blood that it caused the blood spatter when swung,” the Court said, “the absence of evidence of blood spatter tied to Torres on Dunn’s clothes diminishes the argument that he wielded such a weapon.” Moreover, the testimony of the second pathologist, Dr. Sozio, “was critical in this case to creating reasonable doubt, because it countered the scientific evidence by the state and gave the jury reason to doubt that Torres’s injuries resulted from a fall.”

The Court rejected the state court’s conclusion that Dunn failed to show by a preponderance of the evidence that the result of his trial would have been different with better counsel. Citing the SCOTUS ruling in Williams v. Taylor, 529 U.S. 362 (2000), the Court said that Dunn instead needed to demonstrate only a “reasonable probability that … the result of the proceeding would have been different.” The state court’s denial of Dunn’s request for a new trial would typically be afforded deference on appeal under 28 U.S.C. § 2254(d), as amended by the Antiterrorism and Effective Death Penalty Act (“AEDPA”), Pub. L. 104–132, 110 Stat. 1214. But since the ruling contradicted that SCOTUS precedent in Williams, it met one of two exceptions to the AEDPA. Accordingly, the decision of the District Court to grant Dunn habeas relief was affirmed. Dunn v. Neal, 44 F.4th 696 (7th Cir. 2022).

Elkhart County then decided not to refile the charges against Dunn, and the case against him was dismissed in November 2022. His suit for damages against the County and its officials was filed in the District Court under 42 U.S.C. § 1983 in July 2024. Claims for immunity by Dr. Wagner, Marks, and other defendant officials have been denied, and the litigation remains pending. Dunn is represented in his suit by attorneys with Loevy & Loevy in Chicago.

Whatever scientism a particular cop decides to indulge, Dunn’s case, like Logan’s, shows that there are questions competent prosecutors should be asking about it, the same questions that competent defense counsel must ask, if prosecutors do not. Of course, a defense attorney will not get the chance to ask if the suspect agrees to a plea bargain before arraignment and discovery, as the overwhelming majority of suspects do. But waiting for courts and prosecutors to be sufficiently embarrassed by wrongful convictions to initiate reform seems unwise and futile. Over 3,700 exonerations have not yet been enough. How many more will it take?

Overcoming Limitations of Defense Counsel

If waiting on courts and prosecutors is futile, then reform must begin with the defense bar, which too often buckles under shoddy forensic science. When Dunn asked the District Court for a new trial, his counsel testified to his own ineffectiveness. He should have called experts to refute Wagner and Marks, he said, admitting that he was too focused instead on his own theory of the case – that Torres was drunk and tussling over the bat when he fell (1) from a rickety balcony; and (2) landed on top of the bat that Dunn supposedly then beat him with (before the prosecution changed its theory to include a second unfound weapon); while at the same time somehow (3) Dunn ended up with no blood spatter on his clothes. Dunn v. Warden, 2020 U.S. Dist. LEXIS 243264 (N.D. Ind. Dec. 23, 2020).

The record in Jessica Logan’s case is less well developed, but it’s not hard to deduce something similar happened there. Focused on his own theory of the case, her attorney cross-examined Det. Matthews and stumbled into his analysis of the 911 call. The hapless attorney no doubt realized that he couldn’t object to testimony he had just solicited. But he owed it to his client to vigorously question the detective about Harpster’s course and the way he applied its teachings. The attorney may not have had time to find the same discrediting studies that ProPublica later uncovered. But persistent questioning could have exposed the method’s weakness: What was the checklist of “guilty cues” from Harpster’s training? Did Matthews apply all of them, or only some? Where were the population studies showing the probability that a given response indicates guilt? And how was a determination made within the first minute of hearing the call any different from a hunch?

For defense attorneys dealing with junk forensic science, a better example was offered by the team from the Western Michigan University Cooley Law School Innocence Project, which in 2017 helped state prisoner LeDura Watkins win exoneration from his 1976 murder conviction. As the U.S. Court of Appeals for the Sixth Circuit recalled in a suit filed over his wrongful conviction, the state’s case against Watkins, who was just 19 at the time of his arrest, “hinged on an acquaintance with a grudge and a single hair” found on clothing worn by the victim, schoolteacher Yvette Ingram.

Detroit Police Department Technician Ronald Badaczewski testified that it was a match to a hair sample from Watkins. By the time attorneys with the Innocence Project filed for a new trial for him decades later, the hair was no longer available for DNA testing to disprove that testimony. Instead, they marshalled studies done over the intervening decades, disproving the underlying theory that a hair match made with certainty was even possible. Arguing that this was new evidence not available at the time of Watkins’ original trial, the attorneys said that it warranted revisiting his conviction.

Bolstering their claim was a September 2016 report to then-President Barack Obama from the PCAST on the state of the science in several types of forensic methods. In defense of the MHCA technique used to convict Watkins, the DOJ had presented several arguments and supporting evidence. But the PCAST concluded that none of it provided “a scientific basis for concluding that microscopic hair examination is a valid and reliable process.” The PCAST findings struck at the heart of the Daubert framework: MHCA had no known error rate, no standardized methodology, and no genuine acceptance in the broader scientific community. It should never have been admitted in the first place.

In fact, the DOJ report had included a test in which one examiner failed to replicate the results of another’s testing, forcing the second to manually change his co-worker’s incongruous findings. As the PCAST noted dryly, “Ex post facto reclassification of errors is generally not advisable in studies pertaining to validity and reliability.” In response, the DOJ attempted to frame those misidentifications as “not an ‘error’ but simply a ‘limitation of the science.’” Once more, the PCAST called that “disingenuous,” adding: “When an expert witness tells a jury that a hair found at the scene of a crime is microscopically indistinguishable from a defendant’s hair, the expert and the prosecution intend the statement to carry weight.” The DOJ’s semantic gymnastics revealed more than it concealed: when the science fails, defenders of the status quo reach for euphemism.

Because the DOJ was also unable to offer any information “about the proportion of individuals with similar characteristics” in the general population, moreover, an identification made with MHCA was “scientifically meaningless” because “it has no probative value, and considerable potential for prejudicial impact.”

“In short,” the PCAST concluded, “if scientific hair analysis is to mean something, there must be actual empirical evidence about its meaning.” The same report, it is important to note, also said that bitemark analysis was “clearly scientifically unreliable at present.” Report to the President – Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods, PCAST (Sep. 2016).

At his release in 2017, Watkins, then 61, was one of the longest-serving U.S. prisoners ever exonerated. After he filed suit over his wrongful conviction, the U.S. District Court for the Eastern District of Michigan denied qualified immunity to defendant officials with Wayne County and the city of Detroit, a decision the Sixth Circuit affirmed in 2021. Watkins v. Healy, 986 F.3d 648 (6th Cir. 2021). The following year, Novi attorney Wolf Mueller told the Detroit News that he had secured a $3.35 million settlement for Watkins, who also dropped his appeal to a state court’s dismissal of his claims under the Michigan Wrongful Imprisonment Compensation Act, MCL § 691.1751 et seq.

For Watkins, compensation came after 41 years. For the defense bar, his case offers a template: when the science is junk, say so loudly, repeatedly, and with evidence.

Misplaced Faith in Cross-Examination

Saying so loudly, however, requires institutional support that defense attorneys rarely receive. After the PCAST issued its report in 2016, then-Attorney General Loretta Lynch declined to adopt its recommendations to rein in the use of suspect forensic science. Apparently, she agreed with a contemporaneous statement from the National District Attorneys Association, which expressed faith in judges to act as “gatekeepers” for expert forensic testimony during Daubert hearings, as well as faith in defense attorneys to beat back spurious claims with effective cross-examination. Writing in the Yale Law Journal, Texas appellate attorney Adam B. Shniderman took exception to both assumptions. Adam B. Shniderman, Prosecutors Respond to Calls for Forensic Science Reform: More Sharks in Dirty Water, 126 Yale L.J. F. 348 (2016).

“Both the NRC Report and the PCAST Report cast serious doubts on the ‘foundational validity’ … and ‘validity as applied’ … of several feature comparison methods, including bitemark, shoe print, fingerprint, and firearm/toolmark analysis,” Shniderman declared. “Given this, how can we, as Attorney General Lynch suggests, ‘proper[ly] use’ unvalidated techniques and unsubstantiated testimony?”

The Daubert framework, as discussed earlier, places judges in the role of scientific gatekeepers. But gatekeepers who cannot understand what they are guarding against are no gatekeepers at all. In its Daubert ruling, SCOTUS recommended five factors for judges to consider in assessing the reliability of scientific testimony – that it is based on a theory which is (1) falsifiable or testable and (2) subjected to peer review, with (3) a known or potential error rate and (4) sufficient standards and controls so that it (5) is generally accepted by the scientific community. But as Shniderman pointed out, the 2009 NAS/NRC report found that the process breaks down under use by “judges and lawyers who generally lack the scientific expertise necessary to comprehend and evaluate forensic evidence in an informed manner.”

Corroborating this was a 2001 survey of 400 state court judges, 48% of whom “felt they had been inadequately prepared in their education to handle the types of scientific evidence they faced on the bench,” Shniderman noted. The same researcher “found that an overwhelming majority of the judges surveyed could not correctly demonstrate a basic understanding of two of the Daubert criteria: falsifiability and error rates.” The judges offered responses that indicated they confused the “falsifiability” of a technique with a “false” result, which is not the same thing at all. Moreover, very few of them seemed to understand “that an error rate has two components – false negatives (missing a true match) and false positives (wrongly declaring a match).”

Further complicating this is what Shniderman called “a systemic pro-prosecution bias on the bench.” Partly this reflects the fact that so many judges were once prosecutors themselves. “[R]e-election and retention pressures” also serve to “systematically disadvantage criminal defendants” in those states with judicial elections, Shniderman said, citing a 2015 Brennan Center for Justice study. The result is that “criminal defendants tend to lose admissibility challenges to forensic evidence” and often find defense experts “excluded from court,” sometimes on grounds that border on the absurd.

An example was offered from the 2001 burglary trial of James Hyatt in New York, when the expert calling out junk science was discredited as a junk scientist himself. Foreshadowing the doubts that the NAS/NRC would express eight years later, defense expert Simon Cole, Ph.D., was called to offer his opinion on the unreliability of fingerprint examination methods, as Hyatt’s defense attorney attempted to counter fingerprint evidence offered by the prosecution. Cole was a visiting Cornell University professor who had “published a book on the subject of fingerprint evidence and several peer review and magazine articles on the subject.” Nevertheless, the Supreme Court for Kings County concluded that “what Dr. Cole has offered here is ‘junk science’” and precluded his testimony in the case. People v. Hyatt, 2001 N.Y. Slip Op. 50115(U) (Sup. Ct. 2001). The irony was sharp: Cole’s critique of fingerprint analysis would be vindicated by the NAS report eight years later, and his scholarship is now widely cited by the very scientific community the court claimed to be protecting.

Moving on to what he called a misplaced faith in cross-examination, Shniderman cited a 2006 Boston University Law Review article by Distinguished Law Professor David L. Faigman of the University of California Hastings College of Law (renamed UC College of the Law, San Francisco in 2023) to note that “lawyers generally lack training in scientific methods and usually struggle to articulate scientific concepts.” Faigman observed, only half in jest, that “nothing puts law students to sleep faster than putting numbers on the board.”

Given this, Shniderman concluded that it is simply unrealistic to expect defense attorneys who are “overburdened with larger than recommended case-loads … to subject forensic experts to meaningful cross-examination that would highlight the potential methodological flaws, lack of scientific validity, and possibility for procedural errors.” And even if that were to happen, it would be equally unrealistic to expect the testimony would be understood by the average juror.

The adversarial system, in short, assumes a fair fight. When one side wields the imprimatur of “science” and the other lacks the training to challenge it, the fight is over before it begins.

The Road Forward

Writing in the Virginia Journal of Criminal Law, M. Chris Fabricant, then Strategic Litigation Director at the Innocence Project, and Tucker Carrington, then Director of the Mississippi Innocence Project, called out “the failure of courts and litigants to distinguish between magic and science,” as well as “the judicial system’s continuing reflexive reliance on deeply flawed, scientifically invalid precedent to support the admissibility of false and misleading evidence.” M. Chris Fabricant & Tucker Carrington, The Shifted Paradigm: Forensic Science’s Overdue Evolution from Magic to Law, 4 Va. J. Crim. L. 1 (2016).

As their initial example, they offered the case of Han Tak Lee, who served 25 years of a life sentence in a Pennsylvania prison for starting a fire in a cabin at a retreat in the Poconos that killed his mentally ill daughter in 1989. Based on fire science, experts testified that Lee must have used over 60 gallons of fuel as an accelerant to create the conflagration, after murdering his daughter. In support of the second conclusion, the state offered not evidence but the absence of it, calling an expert to testify that the lack of smoke in Ji Yun Lee’s lungs meant that she died before the fire started. No fuel or fuel cans were found to support the first conclusion, but the expert’s detailed fuel estimate calculation was accepted by the court and presented to the jury, which apparently credited it in light of the other exculpatory evidence.

Lee filed a federal habeas petition, and in 2014, the U.S. District Court for the Middle District of Pennsylvania accepted a recommendation to grant it from a magistrate who concluded “that much of what was presented to … [the] jury as science is now conceded to be little more than superstition.” Han Tak Lee v. Tennis, 2014 U.S. Dist. LEXIS 110766 (M.D. Pa. Aug. 7, 2014). Gallingly, the state appealed, insisting that the evidence was still sufficient to preclude overturning Lee’s guilty verdict. Fortunately, the U.S. Court of Appeals for the Third Circuit disagreed and affirmed the lower court ruling. Lee v. Superintendent Houtzdale SCI, 798 F.3d 159 (3d Cir. 2015).

The absence of smoke in the lungs could be explained by a quick-burning “flashover” fire rather than murder, the Court said. What prosecutors called “inconsistencies” in Lee’s recollection of the event were minor, too, and easily explained by the added difficulty of translating interrogatories and answers between English and his native Korean. Like Jessica Logan’s flat affect, Lee’s calm demeanor after the fire also seemed suspicious to prosecutors, but the Court said that it could be chalked up to “cultural stoicism” which was “misconstrued as nonchalance.”

To Fabricant and Carrington, a case like Lee’s “necessarily calls into question the force and legitimacy of precedent as basis to introduce purportedly scientific evidence.” When scientific advancement exposes “the failure of courts and litigants to distinguish between magic and science” in an earlier case, the response by the judicial system cannot be “continuing reflexive reliance on deeply flawed, scientifically invalid precedent to support the admissibility of [more] false and misleading evidence.”

Troublingly, that is precisely what happens – most recently in the case of Jimmie Chris Duncan. He remains imprisoned in Mississippi for the 1993 murder of his girlfriend’s toddler, based largely on bitemark analysis by a pair of experts. But discredited testimony from Drs. Michael West and Steven Hayne has led to nine exonerations in other cases.

One of those exonerated, former state prisoner Levon Brooks, was convicted in 1992 and freed in 2009, but the 1999 ruling by the state Supreme Court that affirmed his conviction included an affirmation that still stands in the state: “[b]ite mark identification evidence is admissible in Mississippi.” Brooks v. State, 748 So. 2d 736 (Miss. 1999).

“This kind of self-serving, court-facilitated pseudo-jurisprudence,” Fabricant and Carrington wrote, “not only facilitates trial courts’ wholesale admission of flawed evidence, but also insulates such decisions from appellate review, no matter how legally indefensible and intellectually dishonest.”

As they noted, there is at the University of Pennsylvania Carey Law School a statue of Hsieh-Chai, a creature in Chinese mythology said to be gifted with the ability to discern guilt. On its base is inscribed a reminder: “Slow and painful has been man’s progress from magic to law.” No one knows just how slow and just how painful like the hundreds of Americans now exonerated from convictions based on flawed forensics and flawed forensic scientists. But for the rest of us, here’s a clue: Despite its dubious distinction as the most thoroughly debunked type of forensic “science,” bitemark evidence has never been declared inadmissible in a U.S. court.

This is ridiculous. It is also the reality we live in. Which leaves it primarily up to defense attorneys, despite their caseloads and educational limitations, to arm themselves with the necessary knowledge and legal precedents to stand between the next Jessica Logan and a murder conviction based on pseudoscience like 911 call analysis.

They must at least become familiar with the 2018 DOJ guidelines for the language experts are permitted to use when testifying. Those guidelines prohibit experts from testifying, unless required by law, with a “reasonable degree of scientific certainty” or “reasonable scientific certainty.” Experts may not use the words “individualize” or “individualization” to say or imply that two prints came from the same person “to the exclusion of all others.” They may not draw conclusions from prints with “a numerical or statistical degree of probability.” The number of career examinations performed by the analyst is not permitted in testimony. And experts must never call conclusions “perfect” or “mistake-free,” nor testify that they have a “zero error rate.” Uniform Language for Testimony and Reports, DOJ (2018).

Another tool to help jurors is an instructional video to frame the forensic evidence they are about to see. As CLN reported, one such video developed and tested by the NIJ in 2023 helped jurors who watched it to correctly distinguish lower-quality evidence in the mock trial that followed. [CLN, Jan. 2024, p.36.] Importantly, the video also did not lower their overall faith in forensic science as a criminal justice tool.

Those who are not defense attorneys can also actively campaign for more enlightened judges in jurisdictions with judicial elections. Waiting on courts to figure it out, however, is apparently not a viable option, as another anecdote from Fabricant and Carrington made clear.

A Mississippi prisoner convicted of a 1981 sexual assault was connected to the crime scene by a single hair. The MHCA that made the match was performed in an FBI lab by Michael Malone – the same analyst whose fraudulent testimony in Washington, D.C. triggered the review that eventually exposed the unreliability of hair comparison evidence. After the DOJ Inspector General flagged his suspect work in 1997, it took years to connect him to the Mississippi case. When the DOJ finally wrote to advise prosecutors in 2001, the letter also requested “any other information [they] may have related to the … case to determine if Malone’s laboratory work was material to the conviction.” The response from the district attorney’s office arrived back at the DOJ eight months later: “This is a 20 year old case with all record files having been previously destroyed. No determination to your request can be made.”

The file was gone. The hair evidence was gone. And somewhere in Mississippi, a man who may be innocent remained in prison, convicted on testimony that had since been exposed as worthless. Somewhere in Illinois, Jessica Logan serves year six of her 33-year sentence. The science was never there. The system grinds on.  

Additional sources: BBC News, Center on Wrongful Convictions, Northwestern Pritzker School of Law, Encyclopaedia Britannica, FIU News, Human Rights Center, IFLScience, Journal of Mississippi History, KKC, NBC News, National Association of Criminal Defense Lawyers, Office of Legal Policy, PBS News, Psychology Today, Scientific American.

Subscribe today

Already a subscriber? Login