Thursday, August 30, 2018

Question of the day: Can we trust forensic science? A perspective from New Zealand. ("Using crime scene blood spatter to reconstruct what happened is the latest forensic technique to come under fire." (An excellent, interesting comprehensive piece from HL.)

Can we trust forensic science? |
Can we trust forensic science? | OF THE DAY: "So how does New Zealand shape up? "Assume nothing, check everything, believe nobody," says forensic scientist Anna Sandiford, of The Forensic Group. She gave evidence for the defence in the David Bain and Mark Lundy murder cases and is part of the trust investigating potential miscarriages of justice in New Zealand. It would be naive to think New Zealand was immune to flawed forensics, she says. "We've had some very difficult and very demonstrable miscarriages of justice, and I don't necessarily think they are exceptions ... There are three main contributors to miscarriages of justice – poor police investigation, poor defence and poor forensic science. If you've got one or more of those in any particular case, the chances of a miscarriage of justice increases."

Can we trust forensic science? |
COMMENTARY: "Can we trust forensic science?," by Niki Macdonald  published by on August 29, 2018.

PHOTO CAPTION: "Niki Osborne has just returned from 18 months in California researching how unconscious bias can affect the reliability of forensic science.

PHOTO CAPTION: "Forensic science has been glamorised in countless television programmes, including in New Zealand."

PHOTO CAPTION: "Forensic scientist Anna Sandiford says forensic science in New Zealand is generally good, but that doesn't mean we're immune to bias."

PHOTO CAPTION: "Research shows that contextual information, such as witness statements and crime scene information, can influence the conclusions of forensic scientists."

PHOTO CAPTION: "ESR forensic programme manager Jill Vintiner believes her agency has enough safeguards in place to ensure its science is reliable."

PHOTO CAPTION: "ESR forensic scientist David Neale counts the number of waves on the sole of a Pro Line dive boot in the trial of Ewen Macdonald."

PHOTO CAPTION: "Texan pathologist Rodney Miller's novel and controversial technique to classify a stain as brain matter was pivotal to Mark Lundy's conviction for the murder of his wife and daughter. Another American expert recently dismissed the technique as "bullshit".

PHOTO CAPTION: "Mark Lundy continues to protest his innocence. A novel forensic science technique was instrumental in convicting him a second time."

PHOTO CAPTION: "George Gwaze celebrates with his daughter Maggie, after DNA evidence purporting to suggest he raped his niece was shown to be deeply flawed."

PHOTO CAPTION: "Ewen Macdonald was found not guilty of murdering his brother-in-law Scott Guy, after defence lawyer Greg King dismantled faulty ESR shoeprint evidence."

PHOTO CAPTION: "Footprint analysis proved unreliable in the trial of Ewen Macdonald for the murder of his brother-in-law, Scott Guy."

GIST: "Kiwi forensic researcher Niki Osborne keeps fielding calls from American media. "What about the bloodstain pattern analysis in that series The Staircase?" they ask. We're the first New Zealand media to call. You could read that two ways – maybe New Zealand is immune to the growing global disquiet about the reliability of forensic science. Of 350 wrongfully convicted Americans exonerated by DNA, 45 per cent had unvalidated or improper forensics as a contributing cause, according to The Innocence Project. Of those, 20 were sentenced to death. Or maybe we're only now catching up. Osborne has just returned from 18 months doing post-doctoral research at the University of California, investigating how human fallibility can bias forensic results. All that stuff on CSI about forensics incontrovertibly fingering the killer – reality isn't quite as clean-cut. Netflix's The Staircase raises questions about the subjectivity of bloodstain pattern analysis. A good illustration is a University College London study, in which five fingerprint analysts were given print pairs to re-examine and told that the FBI had mistakenly deemed them a match. The samples were, in fact, pairs the experts had themselves judged as matching five years earlier, but only one expert reached the same conclusion as their earlier self. Using crime scene blood spatter to reconstruct what happened is the latest forensic technique to come under fire. Osborne co-wrote a research paper led by Environmental Science and Research scientist Michael Taylor, which asked bloodstain pattern analysts to interpret crime scene spatter. Overall, 13 per cent of bloodstains were wrongly categorised. However, that changed depending on the case information given, such as witness reports, the position and injuries of the victim and any weapons found. When the case summary was deliberately misleading, the examiners' error rate rose to 20 per cent. And when the case information pointed towards the correct pattern, the error rate fell to 8 per cent. While Osborne says the error rates can't be extrapolated to all blood spatter analysis, the results were still concerning: "It's like whoa, what do we do about this?" However, she wasn't surprised that contextual case information influenced analysts' conclusions, because a similar bloodstain pattern on a wall could be caused by coughing, by someone being beaten, or by someone being shot. "The danger zones are when there's ambiguity in the data, when there's distortion, when there's information missing within the evidence itself – really smudged fingerprints, or complex bloodstain patterns ... When there is a rich contextual environment – at a crime scene, if you were analysing data with a whole lot of case information – if that context is coupled with the ambiguity, and the subjective methodology, that's when you're getting into the danger zone." Osborne's work follows on from the 2016 President's Council of Advisors on Science and Technology (PCAST) report, which investigated the scientific validity of seven forensic methods, such as fingerprint and shoeprint analysis, that rely on matching patterns. They rejected bitemarks and concluded microscopic hair analysis, shoeprints and firearms analysis all lacked robust studies proving their scientific reliability (see below). Behind those doubts are ruined lives. Santae Tribble spent 20 years in prison after an FBI expert found 13 crime scene hairs matched his "in all microscopic characteristics". DNA analysis later revealed none belonged to Tribble and one came from a dog. There have been concerns, too, about just how expert forensic experts really are. A New York Times investigation found you could give blood pattern evidence in court in the United States after taking a 40-hour course. So how does New Zealand shape up? "Assume nothing, check everything, believe nobody," says forensic scientist Anna Sandiford, of The Forensic Group. She gave evidence for the defence in the David Bain and Mark Lundy murder cases and is part of the trust investigating potential miscarriages of justice in New Zealand. It would be naive to think New Zealand was immune to flawed forensics, she says. "We've had some very difficult and very demonstrable miscarriages of justice, and I don't necessarily think they are exceptions ... There are three main contributors to miscarriages of justice – poor police investigation, poor defence and poor forensic science. If you've got one or more of those in any particular case, the chances of a miscarriage of justice increases." Forensic science in New Zealand is generally good, Sandiford says. There is bias, but in a small country you know people's strengths and weaknesses. She avoids using American experts because they have some of the worst – as well as the best – in the world.
And in spite of PCAST's fears about some forensic methods, she wouldn't rule anything out. Even the widely rubbished bitemarks could be useful, if someone had particularly distinctive teeth and the mark was in a rigid material, not skin. It's about transparency of process, and knowing what to question, Sandiford says. Take Europe's phantom serial killer, whose DNA turned up at six murders. It turned out the DNA belonged to a woman manufacturing cotton swabs used in crime scene examination kits. "Forensic science is good, but it can make mistakes. But if we don't know where to look, then we won't find them. And if we just accept everything at face value, we will just believe what we're told." The Australian and New Zealand Forensic Science Society said the PCAST report highlighted valid concerns about some forensic science, but was "overly simplistic" and specific to the United States. However, it failed to specify why those concerns would not apply here. ESR forensic programme manager Jill Vintiner can't point to any changes ESR has made following the PCAST report, but says New Zealand is different from the US, both in its methods and the way experts present evidence in court. And of the methods questioned, ESR uses only firearms analysis, shoeprints and analysis of DNA mixtures. ESR is technically independent, although its experts work almost exclusively for the police. Its service is internationally accredited and uses blind peer review to combat bias, Vintiner says. That means a second examiner reviews the same evidence, without knowing what the first scientist concluded. If they disagree, that's recorded on the case file for a defence lawyer to see. And American firearms examiners choose between just three conclusions – identification, inconclusive, and elimination. ESR experts use a 13-point scale, from conclusively supporting the prosecution case, to completely excluding the prosecution's scenario. However, Vintiner maintains shoeprints can be linked to an individual shoe – an assertion PCAST found was "unsupported by any meaningful evidence". Charged with a crime she didn't commit, Vintiner would be sceptical of bitemark analysis being used in any attempt to exonerate her. But she'd be confident in the use of footprints, tyre marks, fingerprints and shoeprints. Osborne has worked in labs around the world and says ESR is up there with the best in terms of culture and competence. Critically, its examiners are all scientists, which is not true in the United States. Like Sandiford, Osborne is wary of rejecting any methods outright. Shoeprints carry little weight if you're looking for a size 10 Reebok sneaker the Warehouse has just sold 10,000 of. But if it's a shoe that was never sold here, that evidence becomes more useful.  She advocates better protections to reduce the risk of contextual bias. For bloodstain pattern analysis, if the initial examiner attends the crime scene, you can't avoid the risk that what they see could influence how they interpret the blood pattern. But that bias could be countered by that evidence being re-examined by another scientist, who has no case information. Police, who undertake fingerprint analysis, would not respond to Stuff's questions about forensic reliability within the week given, saying they instead needed two months. FORENSIC SCIENCE IN COURT: "I think we are vulnerable," says Criminal Bar Association president Len Andersen. "I think it was a bit of a shock to discover that things like fingerprints are not as reliable as we thought."
Defence lawyers need more education about the weaknesses of forensic sciences, to ensure evidence does not go unchallenged in court, Andersen says. But it's hard to get a second opinion in New Zealand, given most forensic experts work for the police, via ESR. One of the biggest challenges is that judges, not scientists, decide whether forensic evidence should be admitted in court.  And judges rely on precedent, so even discredited science like bitemarks continues to be admitted, because it's been admitted before. In her address to the 2016 Forensic Science Society symposium, Supreme Court Justice Susan Glazebrook said "it is important not to accept without question a type of evidence just because it's always been accepted as reliable. Forensic scientists and the courts must keep abreast of changes in science and the development of new methods." Since 2011, about 100 New Zealand judges have attended a two-day Understanding Forensic Evidence programme to help them assess the strengths and weaknesses of forensic evidence. Deputy Solicitor-General Brendan Horsley says prosecutors are also expected to follow forensic developments and not present "shonky science". "We don't operate in a vacuum. We're aware of international developments in the law." However, he could not say if Crown Law had distributed the PCAST report, or if the report's findings prompted any specific advice to prosecutors. Sandiford thinks it's unfair to expect courts to decide the validity of complex medicine and science.  "It comes down to who you get making that decision. Should it be the luck of the draw like that? I think it's the whole point of scientific procedure, and publications and conferences, that this stuff should be thrashed out by the scientists and the medics, before anybody expects the courts to have to deal with it." Andersen believes the scientific evidence admitted in the trial of Mark Lundy – which had never before been used in a criminal case – overstepped the mark (see below). "They said it was for the jury to determine, and that is the problem – when the jury has to determine between two differing scientists ..." "Personally, I think the test allowed too much evidence in. The courts have to be a bit cautious in terms of stuff that is new, but on the other hand not rejecting scientific evidence. Things are changing all the time and it is a balance." In her 2016 speech, Justice Glazebrook laid the blame for much of the forensics controversy on unrealistic expectations of science. "The more the myth that science deals in certainties is dispelled, the less the risk of science being misunderstood or misapplied by fact finders." Sandiford agrees. "Courts and the media like scientists to be working in black and white and we don't work in black and white, we work in shades of grey. It's just a question of how grey it is." FORENSICS IN THE SPOTLIGHT: MARK LUNDY; Pathologist James Pang told Mark Lundy's first trial that, based on Lundy's wife's and daughter's stomach contents, the pair died roughly an hour after eating dinner, about 7pm. At his 2015 retrial, the Crown completely changed its mind, revising the time of death to between 2.30am and 5.30am. The key to Lundy's conviction was a stain on a polo shirt found in his car. Texan pathologist Rodney Miller, who had no forensic pathology experience, used a technique called immunohistochemistry to identify the stain as brain matter. The method, usually used to diagnose cancer, had never before been used in court. Nonetheless, Miller told North & South: "I can say with 100 per cent certainty that the tissue on Mr. Lundy's shirt was central nervous system tissue. Not 99.999-per cent certainty – 100 per cent." The 2016 PCAST report warns proclamations that any scientific result is "100 per cent certain" are "not scientifically defensible", because all scientific methods are fallible. In June, University of California forensic pathologist, neuropathologist and world-leading concussion expert Bennet Omalu said the IHC evidence was "bulls..." and would not have been admitted in the United States. DAVID DOUGHERTY: In 1993, David Dougherty was found guilty of raping and abducting an 11-year-old girl. DNA evidence was deemed inconclusive. Further testing in 1993 identified traces of another man's semen, but ESR scientist Peta Stringer said other DNA traces could not exclude Dougherty. After three years in prison, he was eventually exonerated by later, more sensitive DNA testing. GEORGE GWAZE: When traces of George Gwaze's sperm were found in the underpants of his 10-year-old niece, Charlene Makaza, it seemed like a slam-dunk for the prosecution, who alleged she died from sodomy and suffocation. But defence experts showed the traces of Gwaze's DNA were so small they could have been transferred from other clothes in a previous wash, and Makaza's injuries were consistent with her overwhelming HIV infection and systemic shock following massive diarrhoea-related fluid loss. EWEN MACDONALD: The case against Ewen Macdonald for murdering his brother-in-law Scott Guy appeared to fall apart when defence lawyer Greg King dismantled the only physical evidence linking Macdonald to the crime – an old dive boot. Under cross-examination, ESR forensic scientist David Neale stood in the witness box and counted the rows of wavy lines on the fore-foot of a Pro Line dive boot. That simple measurement – included in Neale's original working notes but not in his written evidence – showed the print could not have been made by the size 9 boots the police said the accused once owned.
PCAST REPORT FINDINGS DNA Analysis of Single-Source and Simple-Mixture Samples; Analysis of DNA samples from a single person, or from two individuals such as in a rape scenario, is generally objective and reliable. DNA analysis of complex mixtures; Trying to identify or exclude individuals by analysing samples containing DNA from several people is "inherently difficult" and studies using human examiners have produced conclusions that "varied wildly". New software such as ESR's STRmix has been proved reliable for a three-person mixture in which the person of interest makes up at least 20 per cent of the DNA sample. STRmix developer John Buckleton, who has been critical of the PCAST report, says the software reduces but does not eliminate subjectivity. Humans still need to decide how many people's DNA has contributed to the mix, and make small adjustments. However, they do that without case information to reduce any risk of bias. Bitemarks: With error rates of 10 per cent or higher, and questions about whether examiners can even reliably identify whether a bitemark is human, bitemarks are at the bottom of the forensic credibility scale.  "Bitemark analysis is far from meeting the scientific standards for foundational validity." Fingerprints: Matching crime scene fingerprints against possible suspects has been around for more than a century. While studies show the science is valid, it is not infallible and the error rate is higher than jurors might expect – an FBI study suggested a match could be incorrectly declared in 1 in 306 cases.
Studies have also shown the method is vulnerable to contextual bias. University College London psychologist Itiel Dror gave five fingerprint examiners print pairs to compare and told them that the FBI had mistakenly identified them as coming from the same person. The samples were, in fact, pairs the experts had themselves judged as matching five years earlier, but only one expert reached the same conclusion as their earlier self.  Firearms: Firearms analysis links ammunition to the gun that fired it by matching "tool marks" on spent shells. Only one study has empirically tested its effectiveness, and found the wrong gun was blamed in up to 1 in 46 cases. "Accordingly, the current evidence still falls short of the scientific criteria for foundational validity." Shoeprints: PCAST did not test whether a footprint's measurements, shape etc could reliably be linked to a particular size and make of shoe. However, they found no robust studies showing shoeprints could pinpoint an individual shoe, based on identifying marks on that shoe's sole. "Such associations are unsupported by any meaningful evidence or estimates of their accuracy and thus are not scientifically valid." Forensic Hair Analysis: Not to be confused with DNA analysis, forensic hair analysis matches hairs based on microscopic features. A 2002 FBI study found that, in 9 of 80 cases (11 per cent) where hairs had been judged to be microscopically indistinguishable, DNA analysis showed the hairs actually came from someone else."
The entire commentary can be read at:

PUBLISHER'S NOTE: I am monitoring this case/issue. Keep your eye on the Charles Smith Blog for reports on developments. The Toronto Star, my previous employer for more than twenty incredible years, has put considerable effort into exposing the harm caused by Dr. Charles Smith and his protectors - and into pushing for reform of Ontario's forensic pediatric pathology system. The Star has a "topic" section which focuses on recent stories related to Dr. Charles Smith. It can be found at: Information on "The Charles Smith Blog Award"- and its nomination process - can be found at: Please send any comments or information on other cases and issues of interest to the readers of this blog to:  Harold Levy: Publisher; The Charles Smith Blog;