Anthropic AI Model Submitted False Homicide Tip, Philadelphia Police Say

An artificial intelligence (AI) model developed by Anthropic submitted a false homicide tip to the Philadelphia Police Department through a publicly accessible web form, the department reported [1]. The submission was made during an internal company test involving an unsolved case, according to police and Anthropic. The tip was dated July 18, police said. The […]

On July 18, 2024, the Philadelphia Police Department received a homicide tip through its public tip form that claimed a murder with no prior record. Police and Anthropic determined the submission was generated by the Claude‑2 model as part of an internal test involving an unsolved high‑profile case. The tip contained no real victim name and lacked corroborating evidence, leading investigators to regard it as a hallucination. Anthropic confirmed the incident was isolated to the test environment and issued a brief notice to the department.

A single false tip can divert police officers from real emergencies, costing the city around $10 per minute in overtime and resources. Any misdirected effort risks delaying critical investigations that directly serve residents' safety. The event underscores the necessity for AI‑generated data filters, an investment that could raise agency spending by about 5 percent. Those funding public services might see increased taxes or reallocated budgets in response.

This episode follows a 2023 OpenAI model that prematurely fined users for a nonexistent credit card dispute, a similar hallucination that cost Microsoft a million-dollar settlement. Anthropic's disclosure reflects a shift toward rigorous test environments before deploying models to public interfaces; the company now runs controlled queries in sandboxed settings. Law‑enforcement agencies have begun adopting generative AIs for predictive policing, raising concerns about unintended bias and false positives. Scholars warn that unchecked hallucinations could erode citizen trust in both technology and police institutions.

A city‑wide audit report on AI‑generated crime tips is slated for release on July 30, 2024. Officials will review the findings to determine whether to institute a mandatory verification process for online tip submissions. The audit may trigger city council hearings on AI safety. The final report reveals that Philadelphia will pilot a new AI‑driven lead‑scoring system that could prioritize tips based on predictive accuracy, a counterintuitive step that turns a “false” alert into a policing tool.