---------------------------------------------------
COMMENTARY: "Can we trust forensic science?," by Niki Macdonald published by Stuff.co on August 29, 2018.
PHOTO CAPTION: "Niki Osborne has just returned from 18 months in California researching how unconscious bias can affect the reliability of forensic science.
PHOTO CAPTION: "Forensic science has been glamorised in countless television programmes, including in New Zealand."
PHOTO CAPTION: "Forensic scientist Anna Sandiford says forensic science in New Zealand is generally good, but that doesn't mean we're immune to bias."
PHOTO CAPTION: "Research shows that contextual information, such as witness statements and crime scene information, can influence the conclusions of forensic scientists."
PHOTO CAPTION: "ESR forensic programme manager Jill Vintiner believes her agency has enough safeguards in place to ensure its science is reliable."
PHOTO CAPTION: "ESR forensic scientist David Neale counts the number of waves on the sole of a Pro Line dive boot in the trial of Ewen Macdonald."
PHOTO CAPTION: "Texan pathologist Rodney Miller's novel and controversial technique to classify a stain as brain matter was pivotal to Mark Lundy's conviction for the murder of his wife and daughter. Another American expert recently dismissed the technique as "bullshit".
PHOTO CAPTION: "Mark Lundy continues to protest his innocence. A novel forensic science technique was instrumental in convicting him a second time."
PHOTO CAPTION: "George Gwaze celebrates with his daughter Maggie, after DNA evidence purporting to suggest he raped his niece was shown to be deeply flawed."
PHOTO CAPTION: "Ewen Macdonald was found not guilty of murdering his brother-in-law Scott Guy, after defence lawyer Greg King dismantled faulty ESR shoeprint evidence."
PHOTO CAPTION: "Footprint analysis proved unreliable in the trial of Ewen Macdonald for the murder of his brother-in-law, Scott Guy."
GIST: "Kiwi forensic researcher Niki Osborne keeps fielding calls from American media. "What about the bloodstain pattern analysis in that series The Staircase?" they ask. We're the first New Zealand media to call. You could read that two ways – maybe New Zealand is immune to the growing global disquiet about the reliability of forensic science. Of 350 wrongfully convicted Americans exonerated by DNA, 45 per cent had unvalidated or improper forensics as a contributing cause, according to The Innocence Project. Of those, 20 were sentenced to death. Or maybe we're only now catching up. Osborne has just returned from 18 months doing post-doctoral research at the University of California, investigating how human fallibility can bias forensic results. All that stuff on CSI about forensics incontrovertibly fingering the killer – reality isn't quite as clean-cut. Netflix's The Staircase raises questions about the subjectivity of bloodstain pattern analysis. A good illustration is a University College London study, in which five fingerprint analysts were given print pairs to re-examine and told that the FBI had mistakenly deemed them a match. The samples were, in fact, pairs the experts had themselves judged as matching five years earlier, but only one expert reached the same conclusion as their earlier self. Using crime scene blood spatter to reconstruct what happened is the latest forensic technique to come under fire. Osborne co-wrote a research paper led by Environmental Science and Research scientist Michael Taylor, which asked bloodstain pattern analysts to interpret crime scene spatter. Overall, 13 per cent of bloodstains were wrongly categorised. However, that changed depending on the case information given, such as witness reports, the position and injuries of the victim and any weapons found. When the case summary was deliberately misleading, the examiners' error rate rose to 20 per cent. And when the case information pointed towards the correct pattern, the error rate fell to 8 per cent. While Osborne says the error rates can't be extrapolated to all blood spatter analysis, the results were still concerning: "It's like whoa, what do we do about this?" However, she wasn't surprised that contextual case information influenced analysts' conclusions, because a similar bloodstain pattern on a wall could be caused by coughing, by someone being beaten, or by someone being shot. "The danger zones are when there's ambiguity in the data, when there's distortion, when there's information missing within the evidence itself – really smudged fingerprints, or complex bloodstain patterns ... When there is a rich contextual environment – at a crime scene, if you were analysing data with a whole lot of case information – if that context is coupled with the ambiguity, and the subjective methodology, that's when you're getting into the danger zone." Osborne's work follows on from the 2016 President's Council of Advisors on Science and Technology (PCAST) report, which investigated the scientific validity of seven forensic methods, such as fingerprint and shoeprint analysis, that rely on matching patterns. They rejected bitemarks and concluded microscopic hair analysis, shoeprints and firearms analysis all lacked robust studies proving their scientific reliability (see below). Behind those doubts are ruined lives. Santae Tribble spent 20 years in prison after an FBI expert found 13 crime scene hairs matched his "in all microscopic characteristics". DNA analysis later revealed none belonged to Tribble and one came from a dog. There have been concerns, too, about just how expert forensic experts really are. A New York Times investigation found you could give blood pattern evidence in court in the United States after taking a 40-hour course. So how does New Zealand shape up? "Assume nothing, check everything, believe nobody," says forensic scientist Anna Sandiford, of The Forensic Group. She gave evidence for the defence in the David Bain and Mark Lundy murder cases and is part of the trust investigating potential miscarriages of justice in New Zealand. It would be naive to think New Zealand was immune to flawed forensics, she says. "We've had some very difficult and very demonstrable miscarriages of justice, and I don't necessarily think they are exceptions ... There are three main contributors to miscarriages of justice – poor police investigation, poor defence and poor forensic science. If you've got one or more of those in any particular case, the chances of a miscarriage of justice increases." Forensic science in New Zealand is generally good, Sandiford says. There is bias, but in a small country you know people's strengths and weaknesses. She avoids using American experts because they have some of the worst – as well as the best – in the world.
PHOTO CAPTION: "Niki Osborne has just returned from 18 months in California researching how unconscious bias can affect the reliability of forensic science.
PHOTO CAPTION: "Forensic science has been glamorised in countless television programmes, including in New Zealand."
PHOTO CAPTION: "Forensic scientist Anna Sandiford says forensic science in New Zealand is generally good, but that doesn't mean we're immune to bias."
PHOTO CAPTION: "Research shows that contextual information, such as witness statements and crime scene information, can influence the conclusions of forensic scientists."
PHOTO CAPTION: "ESR forensic programme manager Jill Vintiner believes her agency has enough safeguards in place to ensure its science is reliable."
PHOTO CAPTION: "ESR forensic scientist David Neale counts the number of waves on the sole of a Pro Line dive boot in the trial of Ewen Macdonald."
PHOTO CAPTION: "Texan pathologist Rodney Miller's novel and controversial technique to classify a stain as brain matter was pivotal to Mark Lundy's conviction for the murder of his wife and daughter. Another American expert recently dismissed the technique as "bullshit".
PHOTO CAPTION: "Mark Lundy continues to protest his innocence. A novel forensic science technique was instrumental in convicting him a second time."
PHOTO CAPTION: "George Gwaze celebrates with his daughter Maggie, after DNA evidence purporting to suggest he raped his niece was shown to be deeply flawed."
PHOTO CAPTION: "Ewen Macdonald was found not guilty of murdering his brother-in-law Scott Guy, after defence lawyer Greg King dismantled faulty ESR shoeprint evidence."
PHOTO CAPTION: "Footprint analysis proved unreliable in the trial of Ewen Macdonald for the murder of his brother-in-law, Scott Guy."
GIST: "Kiwi forensic researcher Niki Osborne keeps fielding calls from American media. "What about the bloodstain pattern analysis in that series The Staircase?" they ask. We're the first New Zealand media to call. You could read that two ways – maybe New Zealand is immune to the growing global disquiet about the reliability of forensic science. Of 350 wrongfully convicted Americans exonerated by DNA, 45 per cent had unvalidated or improper forensics as a contributing cause, according to The Innocence Project. Of those, 20 were sentenced to death. Or maybe we're only now catching up. Osborne has just returned from 18 months doing post-doctoral research at the University of California, investigating how human fallibility can bias forensic results. All that stuff on CSI about forensics incontrovertibly fingering the killer – reality isn't quite as clean-cut. Netflix's The Staircase raises questions about the subjectivity of bloodstain pattern analysis. A good illustration is a University College London study, in which five fingerprint analysts were given print pairs to re-examine and told that the FBI had mistakenly deemed them a match. The samples were, in fact, pairs the experts had themselves judged as matching five years earlier, but only one expert reached the same conclusion as their earlier self. Using crime scene blood spatter to reconstruct what happened is the latest forensic technique to come under fire. Osborne co-wrote a research paper led by Environmental Science and Research scientist Michael Taylor, which asked bloodstain pattern analysts to interpret crime scene spatter. Overall, 13 per cent of bloodstains were wrongly categorised. However, that changed depending on the case information given, such as witness reports, the position and injuries of the victim and any weapons found. When the case summary was deliberately misleading, the examiners' error rate rose to 20 per cent. And when the case information pointed towards the correct pattern, the error rate fell to 8 per cent. While Osborne says the error rates can't be extrapolated to all blood spatter analysis, the results were still concerning: "It's like whoa, what do we do about this?" However, she wasn't surprised that contextual case information influenced analysts' conclusions, because a similar bloodstain pattern on a wall could be caused by coughing, by someone being beaten, or by someone being shot. "The danger zones are when there's ambiguity in the data, when there's distortion, when there's information missing within the evidence itself – really smudged fingerprints, or complex bloodstain patterns ... When there is a rich contextual environment – at a crime scene, if you were analysing data with a whole lot of case information – if that context is coupled with the ambiguity, and the subjective methodology, that's when you're getting into the danger zone." Osborne's work follows on from the 2016 President's Council of Advisors on Science and Technology (PCAST) report, which investigated the scientific validity of seven forensic methods, such as fingerprint and shoeprint analysis, that rely on matching patterns. They rejected bitemarks and concluded microscopic hair analysis, shoeprints and firearms analysis all lacked robust studies proving their scientific reliability (see below). Behind those doubts are ruined lives. Santae Tribble spent 20 years in prison after an FBI expert found 13 crime scene hairs matched his "in all microscopic characteristics". DNA analysis later revealed none belonged to Tribble and one came from a dog. There have been concerns, too, about just how expert forensic experts really are. A New York Times investigation found you could give blood pattern evidence in court in the United States after taking a 40-hour course. So how does New Zealand shape up? "Assume nothing, check everything, believe nobody," says forensic scientist Anna Sandiford, of The Forensic Group. She gave evidence for the defence in the David Bain and Mark Lundy murder cases and is part of the trust investigating potential miscarriages of justice in New Zealand. It would be naive to think New Zealand was immune to flawed forensics, she says. "We've had some very difficult and very demonstrable miscarriages of justice, and I don't necessarily think they are exceptions ... There are three main contributors to miscarriages of justice – poor police investigation, poor defence and poor forensic science. If you've got one or more of those in any particular case, the chances of a miscarriage of justice increases." Forensic science in New Zealand is generally good, Sandiford says. There is bias, but in a small country you know people's strengths and weaknesses. She avoids using American experts because they have some of the worst – as well as the best – in the world.
And
in spite of PCAST's fears about some forensic methods, she wouldn't
rule anything out. Even the widely rubbished bitemarks could be
useful, if someone had particularly distinctive teeth and the mark was
in a rigid material, not skin. It's
about transparency of process, and knowing what to question, Sandiford
says. Take Europe's phantom serial killer, whose DNA turned up at six
murders. It turned out the DNA belonged to a woman manufacturing cotton
swabs used in crime scene examination kits. "Forensic
science is good, but it can make mistakes. But if we don't know where
to look, then we won't find them. And if we just accept everything at
face value, we will just believe what we're told." The
Australian and New Zealand Forensic Science Society said the
PCAST report highlighted valid concerns about some forensic science, but
was "overly simplistic" and specific to the United States. However, it
failed to specify why those concerns would not apply here. ESR
forensic programme manager Jill Vintiner can't point to any changes ESR
has made following the PCAST report, but says New Zealand is different
from the US, both in its methods and the way experts present evidence in
court. And of the methods questioned, ESR uses only firearms analysis,
shoeprints and analysis of DNA mixtures. ESR is technically independent, although its experts work almost exclusively for the police. Its
service is internationally accredited and uses blind peer review to
combat bias, Vintiner says. That means a second examiner reviews the
same evidence, without knowing what the first scientist concluded. If
they disagree, that's recorded on the case file for a defence lawyer to
see. And
American firearms examiners choose between just three conclusions
– identification, inconclusive, and elimination. ESR experts use a
13-point scale, from conclusively supporting the prosecution case, to
completely excluding the prosecution's scenario. However,
Vintiner maintains shoeprints can be linked to an individual shoe – an
assertion PCAST found was "unsupported by any meaningful evidence". Charged
with a crime she didn't commit, Vintiner would be sceptical of bitemark
analysis being used in any attempt to exonerate her. But she'd be
confident in the use of footprints, tyre marks, fingerprints and
shoeprints. Osborne
has worked in labs around the world and says ESR is up there with the
best in terms of culture and competence. Critically, its examiners are
all scientists, which is not true in the United States. Like
Sandiford, Osborne is wary of rejecting any methods outright.
Shoeprints carry little weight if you're looking for a size 10 Reebok
sneaker the Warehouse has just sold 10,000 of. But if it's a shoe that
was never sold here, that evidence becomes more useful. She
advocates better protections to reduce the risk of contextual bias. For
bloodstain pattern analysis, if the initial examiner attends the crime
scene, you can't avoid the risk that what they see could influence how
they interpret the blood pattern. But that bias could be countered by
that evidence being re-examined by another scientist, who has no case
information. Police,
who undertake fingerprint analysis, would not respond to Stuff's
questions about forensic reliability within the week given, saying they
instead needed two months. FORENSIC SCIENCE IN COURT: "I
think we are vulnerable," says Criminal Bar Association president Len
Andersen. "I think it was a bit of a shock to discover that things like
fingerprints are not as reliable as we thought."
Defence
lawyers need more education about the weaknesses of forensic sciences,
to ensure evidence does not go unchallenged in court, Andersen says. But
it's hard to get a second opinion in New Zealand, given most forensic
experts work for the police, via ESR. One
of the biggest challenges is that judges, not scientists, decide
whether forensic evidence should be admitted in court. And judges rely
on precedent, so even discredited science like bitemarks continues to be
admitted, because it's been admitted before. In
her address to the 2016 Forensic Science Society symposium, Supreme
Court Justice Susan Glazebrook said "it is important not to accept
without question a type of evidence just because it's always been
accepted as reliable. Forensic scientists and the courts must keep
abreast of changes in science and the development of new methods." Since
2011, about 100 New Zealand judges have attended a two-day
Understanding Forensic Evidence programme to help them assess the
strengths and weaknesses of forensic evidence. Deputy
Solicitor-General Brendan Horsley says prosecutors are also expected to
follow forensic developments and not present "shonky science". "We don't operate in a vacuum. We're aware of international developments in the law." However,
he could not say if Crown Law had distributed the PCAST report, or if
the report's findings prompted any specific advice to prosecutors. Sandiford thinks it's unfair to expect courts to decide the validity of complex medicine and science. "It
comes down to who you get making that decision. Should it be the luck
of the draw like that? I think it's the whole point of scientific
procedure, and publications and conferences, that this stuff should be
thrashed out by the scientists and the medics, before anybody expects
the courts to have to deal with it." Andersen
believes the scientific evidence admitted in the trial of Mark Lundy
– which had never before been used in a criminal case – overstepped the
mark (see below). "They
said it was for the jury to determine, and that is the problem – when
the jury has to determine between two differing scientists ..." "Personally,
I think the test allowed too much evidence in. The courts have to be a
bit cautious in terms of stuff that is new, but on the other hand not
rejecting scientific evidence. Things are changing all the time and it
is a balance." In
her 2016 speech, Justice Glazebrook laid the blame for much of the
forensics controversy on unrealistic expectations of science. "The
more the myth that science deals in certainties is dispelled, the less
the risk of science being misunderstood or misapplied by fact finders." Sandiford
agrees. "Courts and the media like scientists to be working in
black and white and we don't work in black and white, we work in shades
of grey. It's just a question of how grey it is." FORENSICS IN THE SPOTLIGHT: MARK LUNDY; Pathologist
James Pang told Mark Lundy's first trial that, based on Lundy's wife's
and daughter's stomach contents, the pair died roughly an hour after
eating dinner, about 7pm. At his 2015 retrial, the Crown completely
changed its mind, revising the time of death to between 2.30am and
5.30am. The
key to Lundy's conviction was a stain on a polo shirt found in his car.
Texan pathologist Rodney Miller, who had no forensic pathology
experience, used a technique called immunohistochemistry to identify the
stain as brain matter. The method, usually used to diagnose cancer, had never before been used in court. Nonetheless, Miller told North & South: "I
can say with 100 per cent certainty that the tissue on Mr.
Lundy's shirt was central nervous system tissue. Not 99.999-per cent
certainty – 100 per cent." The
2016 PCAST report warns proclamations that any scientific result is
"100 per cent certain" are "not scientifically defensible", because all
scientific methods are fallible. In June, University of California
forensic pathologist, neuropathologist and world-leading concussion
expert Bennet Omalu said the IHC evidence was "bulls..." and would not have been admitted in the United States. DAVID DOUGHERTY: In
1993, David Dougherty was found guilty of raping and abducting an
11-year-old girl. DNA evidence was deemed inconclusive. Further testing
in 1993 identified traces of another man's semen, but ESR scientist Peta
Stringer said other DNA traces could not exclude Dougherty. After three
years in prison, he was eventually exonerated by later, more sensitive
DNA testing. GEORGE GWAZE: When
traces of George Gwaze's sperm were found in the underpants of his
10-year-old niece, Charlene Makaza, it seemed like a slam-dunk for the
prosecution, who alleged she died from sodomy and suffocation. But
defence experts showed the traces of Gwaze's DNA were so small they
could have been transferred from other clothes in a previous wash, and
Makaza's injuries were consistent with her overwhelming HIV infection
and systemic shock following massive diarrhoea-related fluid loss. EWEN MACDONALD: The
case against Ewen Macdonald for murdering his brother-in-law Scott Guy
appeared to fall apart when defence lawyer Greg King dismantled the only
physical evidence linking Macdonald to the crime – an old dive boot.
Under cross-examination, ESR forensic scientist David Neale stood in the
witness box and counted the rows of wavy lines on the fore-foot of a
Pro Line dive boot. That simple measurement – included in Neale's
original working notes but not in his written evidence – showed the
print could not have been made by the size 9 boots the police said the
accused once owned.
PCAST REPORT FINDINGS DNA Analysis of Single-Source and Simple-Mixture Samples; Analysis
of DNA samples from a single person, or from two individuals such as in
a rape scenario, is generally objective and reliable. DNA analysis of complex mixtures; Trying
to identify or exclude individuals by analysing samples containing DNA
from several people is "inherently difficult" and studies using human
examiners have produced conclusions that "varied wildly". New software
such as ESR's STRmix has been proved reliable for a three-person mixture
in which the person of interest makes up at least 20 per cent of the
DNA sample. STRmix developer John Buckleton, who has been critical of
the PCAST report, says the software reduces but does not
eliminate subjectivity. Humans still need to decide how many
people's DNA has contributed to the mix, and make small adjustments.
However, they do that without case information to reduce any risk of
bias. Bitemarks: With
error rates of 10 per cent or higher, and questions about whether
examiners can even reliably identify whether a bitemark is human,
bitemarks are at the bottom of the forensic credibility scale. "Bitemark analysis is far from meeting the scientific standards for foundational validity." Fingerprints: Matching
crime scene fingerprints against possible suspects has been around for
more than a century. While studies show the science is valid, it is not
infallible and the error rate is higher than jurors might expect – an
FBI study suggested a match could be incorrectly declared in 1 in 306
cases.
Studies
have also shown the method is vulnerable to contextual bias. University
College London psychologist Itiel Dror gave five fingerprint examiners
print pairs to compare and told them that the FBI had mistakenly
identified them as coming from the same person. The samples were, in
fact, pairs the experts had themselves judged as matching five years
earlier, but only one expert reached the same conclusion as their
earlier self. Firearms: Firearms
analysis links ammunition to the gun that fired it by matching "tool
marks" on spent shells. Only one study has empirically tested its
effectiveness, and found the wrong gun was blamed in up to 1 in 46
cases. "Accordingly, the current evidence still falls short of the scientific criteria for foundational validity." Shoeprints: PCAST
did not test whether a footprint's measurements, shape etc could
reliably be linked to a particular size and make of shoe. However, they
found no robust studies showing shoeprints could pinpoint an individual
shoe, based on identifying marks on that shoe's sole. "Such
associations are unsupported by any meaningful evidence or estimates of
their accuracy and thus are not scientifically valid." Forensic Hair Analysis: Not to be confused with DNA analysis, forensic hair analysis matches hairs based on microscopic features. A
2002 FBI study found that, in 9 of 80 cases (11 per cent) where hairs
had been judged to be microscopically indistinguishable, DNA analysis
showed the hairs actually came from someone else."
The entire commentary can be read at:
https://i.stuff.co.nz/science/106163245/can-we-trust-forensic-science
PUBLISHER'S NOTE: I am monitoring this case/issue. Keep your eye on the Charles Smith Blog for reports on developments. The Toronto Star, my previous employer for more than twenty incredible years, has put considerable effort into exposing the harm caused by Dr. Charles Smith and his protectors - and into pushing for reform of Ontario's forensic pediatric pathology system. The Star has a "topic" section which focuses on recent stories related to Dr. Charles Smith. It can be found at: http://www.thestar.com/topic/
---------------------------------------------------------------------
https://i.stuff.co.nz/science/106163245/can-we-trust-forensic-science
PUBLISHER'S NOTE: I am monitoring this case/issue. Keep your eye on the Charles Smith Blog for reports on developments. The Toronto Star, my previous employer for more than twenty incredible years, has put considerable effort into exposing the harm caused by Dr. Charles Smith and his protectors - and into pushing for reform of Ontario's forensic pediatric pathology system. The Star has a "topic" section which focuses on recent stories related to Dr. Charles Smith. It can be found at: http://www.thestar.com/topic/
---------------------------------------------------------------------
