by Michael Dean Thompson

In forensic technology, the term “black box” has gained prominence. It describes a system whose inner workings remain opaque—an output emerges, but how it is produced eludes the user. Such black-box algorithms underpin artificial intelligence (“AI”) tools that identify suspects from video footage, guide police patrols and investigations, and match DNA samples. Increasingly, these systems rely on complex and opaque algorithms to assess whether someone’s DNA appears in a degraded sample. That was precisely the situation in a case that unfolded in Southern California.

In March 2016, police pulled over Alejandro Sandoval for a traffic violation. During the stop, officers claimed two plastic bags containing 80 grams of methamphetamine slipped from beneath his armpits. Convinced the evidence would exonerate him, Sandoval’s defense team requested DNA testing on the bags.

The evidence yielded a mere 92 picograms (“pg”) of DNA, roughly two-thirds the minimum needed for a full profile, according to Applied Biosystems, the maker of the testing kits. Most of that DNA traced to a woman; less than 8 percent—6.9 pg—came from a man, an amount roughly equivalent to a single human cell’s content (about 6 pg). This trace quantity fell well below the threshold where “stochastic effects”—random fluctuations that erode testing reliability—become a concern. Deeming it unfit for manual analysis, the analyst turned to a probabilistic genotyping system (“PGS”) to estimate the odds of Sandoval’s involvement.

This result would be a statistical projection regarding the probability of exclusion versus inclusion, but prosecutors found the initial results murky and unhelpful. They pressed analysts to rerun the test with adjusted thresholds for clearer findings; the analysts refused. A second program was employed, yielding sharper but still unfavorable results. William C. Thompson, a professor emeritus of criminology, law, and society at the University of California, Irvine, was brought in to offer insight into the probabilities involved.

While both programs excluded Sandoval, each of them—STRmix and TrueAllele—diverged sharply about the degree of his likelihood ratio of being excluded compared to his likelihood of being included. The first program, STRmix, reached the conclusion that the mixture was 5 to 24 times more likely to exclude the suspect. However, TrueAllele was much more emphatic, giving a likelihood ratio of a 16.7 million times greater chance of him not being a contributor (exclusion). Thompson reviewed the results and published his findings as a peer-reviewed paper in the Journal of Forensic Sciences in 2023, igniting a broader debate about such technologies.

Prosecutors attempted to have the DNA evidence thrown out when it failed to provide the answers they wanted. Rather than dismiss the charges, they opted to suppress the evidence of his innocence. Sandoval eventually capitulated and pleaded guilty to the drug charges and was sentenced to 90 days in jail plus probation. Prosecutors got their conviction, despite the DNA results.

TrueAllele and STRmix guard their proprietary code closely and resist calls for transparency from researchers seeking to examine their methods. Mark Perlin, CEO and chief scientific officer of Cybergenetics, the company that makes TrueAllele, is a staunch defender of the closed-source, black-box software’s use in forensic contexts. Perlin attacked Thompson’s paper, claiming that it contains “120 mistaken or otherwise problematic assertions.” Thompson countered that Perlin’s accusations, including a claim that he named Sandoval in the original study (he did not), were “demonstrably false.”

Thompson recounted the backlash to Undark: “They wrote a letter that accused me of all kinds of scientific misconduct and claimed I was using their proprietary data and made all kinds of wild allegations, hoping to get the article suppressed.”

Perlin and his allies cast Thompson as “a lawyer masquerading as a scientist,” a jab at his past legal work. To Perlin, critics like Thompson—along with academics and advisory groups pushing for independent review—aren’t scientists but “advocates.” Researchers, in turn, insist they seek only to understand how these tools produce their results.

Regulators are taking note. In January 2025, the Senate Judiciary Committee held hearings on these technologies, focusing on the legal and ethical implications. The law remains in flux, with courts issuing rulings that challenge developers to justify their opaque systems. Meanwhile, an untold number of convictions may hinge on these enigmatic algorithms.  

Sources: Undark; Journal of Forensic Sciences

Subscribe today

Already a subscriber? Login