Shreya Sharma, a Third year student at National Law University, Odisha, analyzes Prosecuter’s Fallacy, its use and misuse and the role of Bayes Theorem in determining odds of a case. A brief overview of Defense Attorney’s Fallacy along with the landmark judgments has been encompassed to comprehend the role of Statistics in Justice Delivery.
Statistics is more commonly used in the field of law than one may imagine. In criminal cases, lawyers often play on the probability of the case and plant seeds of doubt in the minds of judges and jury to win the case. One error is interpretation of such probability is the Prosecutor’s fallacy. This paper talks about Prosecutor’s fallacy, its use and misuse, and the role of Bayes Theorem in determining the odds of a case. It also briefly discusses an alternate error called Defense Attorney’s fallacy and lists cases in which these errors can be seen. This paper concludes on the dilemmas faced by the justice system regarding probabilities where juries are concerned and suggestions to deal with the same.
The conviction of an accused is often determined by probabilities. In civil cases, judgment is given based on whose side the balance of probabilities tilts. However, in the case of criminal lawsuits, no conviction is done unless the accused is ‘guilty beyond doubt’. This stems from the Black Stone ratio which affirms “Better that ten guilty persons escape, than that one innocent suffer”
As forensic science advances and legal technology becomes a concrete terminology in the courthouse, the rise in the use of scientific evidence is a given. A few decades ago, this evidence was given increasingly through statistics but more recently, statistical methods for weighing evidence are being blocked. This is due to an amalgamation of reasons including inability of jury and masses to understand complex probability equations, lawyers’ inability to quantify subjective evidence and high probability of misuse of statistics which causes fear of its future suppression. Polygraph test is an example of Prosecutor’s fallacy and due to its unreliability; is inadmissible in a court of law. Nevertheless, legal commentary on the issue appears divided between those who argue that statistical evidence may have an exaggerated impact on the jury and those who argue that statistical evidence is likely to be underutilized.
Prosecutor’s fallacy says that the value of A, given B[P(A/B)] is equal to the value of B given A [P(B/A)]. This is called conditional probability. However, usually this is not the case. To explain, this means that the probability that a man has 2 legs and 2 arms is equal to the probability that anything that has 2 arms and 2 legs is a man. This of course is untrue. As simple as this example may seem, real life is much more complicated and often how a case is presented becomes particularly important in determining its outcome. Prosecutor’s fallacy is an exaggeration of the guilt of the accused and the prosecutor’s specious strategy is capable of leading to serious lapse in judgment. Mathematically, it based on the Bayes Theorem. Bayes’ Theorem is a way of finding a probability when we know certain other probabilities. It provides a principled way for calculating a conditional probability of events. Although Bayes Theorem is the right theoretical method for evaluating probability, non-mathematicians still consider it impenetrable. The fact that one can expect only the simplest of Bayesian reasoning to be grasped by the laymen, limits its acceptance in the court of law.
Defense Attorney’s Fallacy: A Divergence
While associative evidence in itself should not be given too much importance in case of weak prior evidence, there is no reason for it to be dismissed completely. Defence attorneys have sometimes argued that the probability of a match, no matter how rare, is irrelevant since it only shows that the defendant and lawbreaker are a part of the same large group of people. This is conveniently called the Defence Attorney’s Fallacy. But what this misjudgement fails to take into account is that the large group of people are not all suspects. Being a suspect in itself increases the credibility of this test, however slightly. As a Bayesian study shows, the associative proof significantly narrows the category of individuals who are or may have been witnesses, thus refusing to exclude the perpetrator, and is thus strongly probative (showing proof). It is important to make sure that if the tendency towards Prosecutor’s fallacy is slighted, the favour is not toward Defence Attorney’s fallacy instead.
People v Collins
The first legal case in which we saw the issue of Prosecutor’s fallacy was People v. Collins. Malcolm Collins and his wife Janet Collins were accused of being guilty of robbery. A mathematician was called as expert witness and the probability of their being guilty was calculated based on distinctive characteristics of their appearance. However, the prosecution had no other evidence that the defendants had committed the crime. The supreme court of the state (of California) reversed the judgment which convicted Collins’ and held that their trial by mathematics was a distortion of the role played by the jury and unfairly disadvantaged defense attorney in a that it mandates reversal lest there is a failure in delivery of justice. According to the court, doing the trial by mathematics invaded what is the right and duty of jury and confused them based on unfounded assumptions. “The reasoning that probability of 1 in 12 million that the defendants have the distinctive characteristics if they were innocent is the same as the probability that the defendants were innocent if they had these distinctive characteristics was 1 in 12 million” is wrong and this flawed logic is an example of Prosecutor’s fallacy.
R v Clark
Sally Clark was convicted of murdering her two children, who were 11 and 8 weeks old at the time of death. Certain pathological findings during post mortem made the Home office pathologist suspicious of the deaths. There were no witnesses to the murder of the two infants and the evidence against her therefore was primarily through medical experts called in by the prosecutor. The expert witness observed that the probability of two SIDS (Sudden Infant Death Syndrome) in a family was 1 in 73 million. She was convicted on this basis. However the judgment was later overturned with the introduction of new evidence and she was set free in 2003. A more accurate approach to finding out the truth may have to use Bayes theorem to also calculate the probability of innocence so that presentation of facts is not tilted in the favour of prosecution. Here prosecutor’s fallacy made her a victim of a horrible miscarriage of justice and the traumatic experience may be the cause of her death in 2007 by acute intoxication.
People of the state of California v. OJ Simpson
O J Simpson was a star NFL player and actor accused of the murder of his ex-wife and her friend. In his case called People of the State of California v. Orenthal James Simpson, the main evidence was DNA and motive based, as there were no witnesses to the murder and no murder weapon found. The defense argued that blood type found would match 10,000 people in the city of residence and murder, Los Angeles. Although this may be true, this argument does not take into account the motive, prior recurrence of domestic violence and other evidence against Simpson and the means to kill of the accused. This is a clear example of Defence Attorney’s fallacy and is just as misleading as Prosecutor’s fallacy. O J Simpson was finally convicted and sentenced to 33 years in prison in 2008. This further shows that probabilities should not be taken at face value.
Bayes theorem is a good tool to prove associative evidence and maybe used to support a case. However, only relying on probabilities when no other evidence is given is a precarious task and utmost care should be taken to clear bias and take forensic evidence with a grain of salt. Countries where judicial system demands only judges and no jury, it is easier to bring in expert evidence which may help further the case quickly but judges have been especially hesitant to use probabilities where a jury is present. Justice should be blind to prejudice and fair in its dealing. For that, it is important that expert testimony should not be permitted to compel juries to add statistical values to assumptions arising from non-scientific evidence presented at court.
Shreya Sharma is a Third Year Student at National Law University, Odisha
William Blackstone, Commentaries on the Laws of England (1753).
Alexander Volokh, Guilty Men, University Of Pennsylvania Law Review(1997).
FentonN, Improve statistics in court, NATURE, (2011), https://doi.org/10.1038/479036a.
Ray Johns, The Prosecutor’s Fallacy, TOWARDS DATA SCIENCE (Jun 13, 2019), https://towardsdatascience.com/the-prosecutors-fallacy-cb0da4e9c039.
Wai-Ching Leung, The Prosecutor’s Fallacy – A Pitfall in Interpreting Probabilities in Forensic Evidence (2002).
Jason Brown Lee, A Gentle Introduction to Bayes Theorem for Machine Learning, MACHINE LEARNING MASTERY, (Dec 4, 2019), https://machinelearningmastery.com/bayes-theorem-for-machine-learning/.
People v Collins, Californian Reporter .
R v Clark,Court of Appeal (Criminal Division) (2000).
Dershowitz A M, Reasonable doubts: the O.J. Simpson case and the criminal justice system, New York: Simon & Schuster (1996).
IMPORTANT – Opinions expressed in this article are the sole responsibility of the author and do not necessarily reflect the views of IJOSLCA.