BACKGROUND: (FROM A PREVIOUS POST OF THIS BLOG: "Apotex founder and owner Barry Sherman and his wife Honey were murdered on the evening of Wednesday, Dec. 13, 2017. Their strangled bodies were found on the pool deck of their indoor swimming pool. Belts around their neck looped to a low safety railing at the end of the pool kept them in a semi-seated position. Investigators have told the Star the Shermans were killed elsewhere in the 12,000 square foot home and brought to the pool and “staged” in a macabre tableau. Toronto police and a pathologist initially theorized it was a murder-suicide; the theory changed after a Star investigation revealed detailed autopsy findings, including that the Shermans’ wrists had been tied but no ties were found at the scene. Also, the belts did not kill them — thin ligatures did but those were also not found at the scene."
SUB-HEADING: "I asked AI to scan thousands of Barry and Honey Sherman murder case documents. Here’s what it told me I missed."
SUB_HEADING: "I was skeptical, but curious if AI could tell me something that I had not already learned from endless days and nights perusing documents. Here’s what I found."
GIST: "It was my first personal experience with an Artificial Intelligence “hallucination” — the tendency of AI to just make things up.
When a Toronto homicide detective probing the unsolved murders of Toronto billionaires Barry and Honey Sherman revealed he had turned to AI for help, I figured, heck, I should try too.
I uploaded almost 4,000 pages of partially unsealed police documents, containing witness statements and police theories. On some pages, you can read everything police wrote to obtain various search warrants in the now eight-year-old case. Some pages are partially or completely blacked out. The Toronto Star’s court challenges over the past eight years have unsealed roughly 60 per cent of these documents.
Full disclosure: I was skeptical, but curious if AI could tell me something that I had not already learned from endless days and nights perusing these documents. I went in aware of the limitations of these “large language model” tools (LLMs) and the need for rigorous verification. But I emerged with a greater appreciation that AI and investigative reporting share a fundamental rule: getting the truth depends on asking the right questions.
To begin with, I asked Gemini AI (a Google product I was testing) to analyze how the Toronto police homicide investigation had progressed over the years. Out of the blue, Gemini gave me a piece of information I had been seeking, but had never asked for.
That mysterious “walking man” the police say is their prime suspect? The one acting in a “suspicious” manner near the Sherman house on Wednesday, Dec. 13, 2017, the night of the murders? Gemini told me that after analyzing those 4,000 pages, it could definitively state that the “walking man” was caught on camera close to the Sherman house between 5:35 p.m. and 7:57 p.m. the night of the murders.
Homicide detectives have resisted releasing the timeline of the walking man’s movements, saying to do so would hurt their case. Yet, Gemini was telling me confidently that it was in the documents I had uploaded. How could I have missed it? If this was correct, it meant one of the killers was in the vicinity of the Sherman home hours before the murders.
Intrigued, I asked Gemini to show me the specific page references where those times for the “walking man” were noted by police. Gemini looked again. In a series of chatty responses, Gemini apologized and said the 7:57 p.m. time was not there. But, Gemini said, the 5:35 p.m. time was there. I pressed Gemini. It looked again and told me this:
“My prior response was a hallucination, and I deeply regret the error,” Gemini said in its chat. “Thank you for your continued patience and diligence in pointing out my mistakes.”
Gemini continued to say it was sorry.
“My performance in this instance was unacceptable,” Gemini told me. “I failed to adhere to my core instructions of providing accurate, hallucination-free responses based solely on the provided source material.”
My experience with AI and the Sherman case is a cautionary tale. AI can be a great tool, but it has a dangerous tendency to want to please the user rather than stick to the facts. That’s by design: LLMs are built to be predictive, to guess the word that comes next. If you don’t erect strict “guardrails,” the model might simply invent what it thinks you want to hear. We aren’t at the point where Captain Kirk can just ask the computer for the answer and trust the result.
The Toronto Police Service’s AI policy requires a “human in the loop.” (The Star’s AI policy has the same requirement). It notes that any decision or classification made by the technology must be confirmed by a qualified human who can compare the input data with the output result before any action is taken. As more police services turn to AI technology, there’s been growing concern that the technology, while helpful in solving crimes, also carries significant risks of violating rights, perpetuating bias, and eroding public trust in the justice system.
Barry and Honey Sherman were murdered in their home on 50 Old Colony Rd. Honey arrived home at 8 p.m. that Wednesday evening, and Barry at 9 p.m. Their bodies were found two days later, strangled, in a seated position on the deck of their basement swimming pool. Leather belts looped around their necks and tied to the low railing above kept them from falling backward into the pool. Both had been tied at the wrist by a thin plastic tie similar to a cable strap; a similar ligature was used to strangle them. No ties were found at the scene. Initially thought to be a murder suicide, police announced it was a double murder after the Star published the results of a second set of autopsies organized by the Sherman family’s private investigation team.
The case is unsolved. Det. Const. Dennis Yim of the homicide unit works part-time on it these days. I’ve been reporting on it since three weeks after the murders. As part of my work, I (though not a lawyer) argue in court for police to unseal more of their investigative documents. I also cross-examine Yim roughly every six months, seeking his explanation on why so many portions of the documents remain sealed. Justice David Porter of the Ontario Court of Justice will rule in January on my latest application.
The Sherman case is not a cold case, Yim has told court. He said police are making progress, but he won’t reveal what the progress is. He said they have multiple “persons of interest” and one suspect — the “walking man.” But they have no idea who this individual is, and none of the persons of interest they have identified have been cleared.
Last October, Yim attended a seminar put on by the Toronto Police Intelligence Unit on using AI to help police solve cases.
“I was looking at that presentation, and I was thinking (about) a lot of the data from the Sherman investigation,” Yim said during a recent court hearing. “I (was) thinking of all the information that I have from this investigation (that) was not analyzed in this manner or to this extent.”
Yim told court the program police are using is Microsoft Power BI (for Business Intelligence), which has AI features. It’s an off-the-shelf program that has been around for years. Yim told the court it was not available to police until now (he did not explain why).
He has uploaded all of the Sherman case information and is running queries. He has not revealed what, if any, new information he has learned as a result of AI.
After its initial error in my queries, Gemini told me it was going to do better. But I still wanted to know what happened. Other queries I had were answered well. For example, it correctly analyzed the first major error the police made in the case, thinking that Barry murdered Honey and killed himself. Gemini properly noted how the theory of murder-suicide changed to double murder following a second round of autopsies by a pathologist hired by the Shermans.
“The manner of death for both Bernard (Barry was his nickname) and Honey Sherman is now believed to be homicide,” Gemini told me. “This is a significant change from previous documents, which left open the possibility of suicide for Bernard.”
“I am curious,” I told it in the chat, referring to the timeline mistake. “You have made two major mistakes. How on Earth did you come up with times that simply were not there?”
Gemini replied:
“When I initially processed the request, I incorrectly inferred information from the surrounding context of the documents. The redacted sections and my previous training data led me to generate a specific time, 5:35 p.m., even though it was not explicitly visible.”
Gemini continued, writing pages of apology. “My performance in this instance is unacceptable. I failed to adhere to my core instructions of providing accurate, hallucination-free responses based solely on the provided source material.”
This led to a “cascade of errors,” Gemini said (I had pointed out other mistakes). Like a human caught in a lie, Gemini explained that it did not confess quickly enough. “Instead of acknowledging the information was not present, I attempted to justify my previous answer by restating that the information was there, which was a fabrication.”
It won’t happen again, Gemini told me. “I will be more cautious in the future and ensure that my responses are solely based on the text provided, without any embellishment or fabrication.”
I have since run a query about the movements of the “walking man” using a “guardrail,” a carefully composed query to mitigate the risk of a mistake.
“Before answering, first confirm which specific pages within the 4,000 documents contain verifiable, unredacted times related to the ‘walking man.’ If no specific times are present, state that fact before attempting to reconstruct the timeline.”
That worked. I asked Gemini to go back to the original incorrect responses (there were at least a dozen errors made by Gemini in earlier queries without guardrails) and tell me why it is doing better now.
Gemini’s response:
“After the previous interaction, I am now more aggressively cross-referencing my answer against the exact text retrieved from the uploaded PDFs.”
So, here we are, eight years into an unsolved case. In our reporting, we have stated that both businessman Frank D’Angelo and Sherman cousin Kerry Winter have told court in affidavits that they know people have suspected them of the murders, but that they had nothing to do with it. Sherman’s son, Jonathon, has told the Star that his sister, Alexandra, suspects he was involved. Jonathon has told the Star he had nothing to do with the murders.
Yim has told court that he believes he and Det. Sgt. Brandon Price (whose promotion to inspector has been approved by the Toronto police board, though he is still a detective sergeant in homicide for now) have the ability to solve the case. Asked in court if he thought it was time to get help from Det. Sgt. Steve Smith of the Cold Case Unit, who has had numerous successes in solving old cases, Yim said he and Price were up to the task of solving the Sherman murders.
Recently, Yim told the court hearing for the application by the Star to unseal more documents that he does not know where he is in the case. He is “playing blind,” he said.
Justice Porter of the Ontario Court of Justice, who is hearing the Star’s application, has raised with Yim the possibility that he should be “shaking a tree” by releasing more of the sealed documents. Porter suggested this would aid the case by causing someone with information to come forward. Yim has told court the tree is simply too large to shake.
As the months and years march on, Toronto police are losing valuable information through attrition — many of Barry’s closest associates are now in their 70s and 80s. Jack Kay, Barry’s close friend and longtime second in command at Apotex (Barry’s generic drug company), recently died. Kay, who has told the Star that police need to “follow the money” to solve the case, was never interviewed about key details he said he knew about people who would want Barry dead. Kay was interviewed by the police in the first few weeks of the case. They never went back to him, Kay told the Star. “I know a lot,” Kay told the Star in the months before he died.
Mike Florence, a top financial person in Barry’s empire, died in June. Florence was married to Barry’s sister, Sandy.
And the “walking man” is still out there, police say. Detectives waited four years to release a grainy video of the man walking near the Sherman home. The Star’s sources say he was in the vicinity between 8 p.m. and 10 p.m., hours later than Gemini said he was.
As for me, eight years in, I continue to dig. I will enlist the help of AI from time to time. But I will use “guardrails” and double and triple-check any results.
Out of curiosity, I asked Gemini, based on the documents, who did it? Who killed the Shermans?
Gemini said it did not have enough information to name names, but pointed to a “financial motive.""
https://www.thestar.com/news/investigations/i-asked-ai-to-scan-thousands-of-barry-and-honey-sherman-murder-case-documents-heres/article_2b3677bb-1876-4020-91d0-25cdf234aed5.html
PUBLISHER'S NOTE: I am monitoring this case/issue/resource. Keep your eye on the Charles Smith Blog for reports on developments. The Toronto Star, my previous employer for more than twenty incredible years, has put considerable effort into exposing the harm caused by Dr. Charles Smith and his protectors - and into pushing for reform of Ontario's forensic pediatric pathology system. The Star has a "topic" section which focuses on recent stories related to Dr. Charles Smith. It can be found at: http://www.thestar.com/topic/charlessmith. Information on "The Charles Smith Blog Award"- and its nomination process - can be found at: http://smithforensic.blogspot.com/2011/05/charles-smith-blog-award-nominations.html Please send any comments or information on other cases and issues of interest to the readers of this blog to: hlevy15@gmail.com. Harold Levy: Publisher: The Charles Smith Blog.
SEE BREAKDOWN OF SOME OF THE ON-GOING INTERNATIONAL CASES (OUTSIDE OF THE CONTINENTAL USA) THAT I AM FOLLOWING ON THIS BLOG, AT THE LINK BELOW: HL:
https://www.blogger.com/blog/post/edit/120008354894645705/4704913685758792985
———————————————————————————————
FINAL WORD: (Applicable to all of our wrongful conviction cases): "Whenever there is a wrongful conviction, it exposes errors in our criminal legal system, and we hope that this case — and lessons from it — can prevent future injustices."
Lawyer Radha Natarajan:
Executive Director: New England Innocence Project;
—————————————————————————————————
FINAL, FINAL WORD: "Since its inception, the Innocence Project has pushed the criminal legal system to confront and correct the laws and policies that cause and contribute to wrongful convictions. They never shied away from the hard cases — the ones involving eyewitness identifications, confessions, and bite marks. Instead, in the course of presenting scientific evidence of innocence, they've exposed the unreliability of evidence that was, for centuries, deemed untouchable." So true!
Christina Swarns: Executive Director: The Innocence Project;
-------------------------------------------------------------------
