Unlock the Editor’s Digest for free
Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.
The writer is a science commentator
For those interested in the legal use of scientific and mathematical evidence, the Lucy Letby case is grimly compelling. Letby, a former neonatal nurse is serving a life sentence for murdering seven premature babies at a Cheshire hospital.
Since then, and with a statutory public inquiry into the events beginning this week, theories have circulated alleging an unsafe conviction. It would be wrong for this column to attempt to relitigate the case. It would also be wrong, however, to gloss over known shortcomings in how scientific and statistical evidence is gathered and presented in court. Several eminent statisticians have raised such concerns of late, while making no claims as to Letby’s guilt or innocence.
As a 2022 report by the Royal Statistical Society showed, those shortcomings particularly matter when health workers, who deal daily with sickness and death, are in the dock. That report, prompted by several miscarriages of justice and entitled Healthcare Serial Killer or Coincidence?, showed that statistical interpretations can be simultaneously persuasive and misleading. The society will hold a meeting on the issue next week and has contacted the Thirlwall Inquiry outlining concerns. “The RSS has been working hard to encourage the better use of statistics in legal cases . . . and to reduce the potential for bias leading to miscarriages of justice,” Sarah Cumbers, its chief executive, told me.
Safeguards must be put in place that allow judges and jurors to fully understand the numbers, charts and other technical evidence put before them. One option for strengthening statistical due diligence, suggested by Warwick university statistician Jane Hutton, is for clusters of hospital deaths to be analysed by the Health Security Agency. Such measures could minimise the risk of a wrongful conviction while reassuring grieving families that a conviction or acquittal is robust.
Correlation can be confused for causation, the RSS report warns, because “seemingly improbable clusters of events can often arise by chance without criminal behaviour”. Faced with multiple hospital deaths, investigators must weigh the relative likelihood of two improbable alternatives: a chance cluster or a serial killer. The first challenge, then, is to assess whether murder has happened at all.
Duty rosters might suggest a link between one person and suspicious incidents. Such a chart featured in the Letby case. But, the RSS cautions, shift patterns alone cannot prove misconduct (the Letby prosecution offered other evidence). Medical staff looking after the sickest patients, for example, can reasonably expect more adverse events on their watch.
Gathering evidence after identifying a suspect introduces the risk of confirmation bias. Even well-meaning investigators can end up unconsciously elevating information that reinforces a theory while discounting pointers to the contrary. There might be so-called confounding variables at play too. A controversial New Yorker article highlighted staffing and plumbing issues at Letby’s hospital.
The framing of probabilities, meanwhile, can powerfully shape perception. A lottery winner with a one-in-a-million chance of hitting the jackpot, is more likely to be congratulated on their luck than accused of rigging the draw, despite having a vastly larger chance (999,999 out of a million) of not winning. Somehow, we grasp that millions play every week, and it is quite likely that someone, somewhere, will land the booty by chance.
By analogy, there are millions of health workers globally and an unlucky few might be linked to death clusters. We struggle to ascribe this to bad luck, preferring a causal explanation. “If misconduct by the medical professional appears to be the only plausible alternative explanation,” the report points out, “then people might be tempted to conclude that the probability of misconduct is overwhelming (999,999 chances in [a] million)”. That line of reasoning is known as the prosecutor’s fallacy.
Flawed statistics contributed to the wrongful 1999 conviction of Sally Clark for murdering her two sons; jurors were told there was only a one in 73mn chance of the babies dying naturally. The numbers were wrong; Clark was later cleared. Lucia de Berk, a Dutch paediatric nurse, was convicted of four murders in 2003 and acquitted on appeal in 2010.
Other potential solutions include employing independent statisticians in complex cases; educating lawyers and judges on statistical pitfalls; and guiding juries on how to read the numbers. Change must come: the current cluster of expert concern is too great to be coincidence.