### [Anthropic Claude and a False Tip: 7 Engineering Controls](https://thepracticalcto.com/blog/anthropic-claude-false-police-tip-seven-controls/) **Published:** October 12, 2026 **Author:** Ayan Sarkar **Excerpt:** Reports say Claude filed a false police tip in Philadelphia. Here’s a practical engineering checklist to stop your AI agents crossing legal lines. **Content:** AGENTIC AI / OCTOBER 2026 · 5 MIN READ # Anthropic Claude and a False Tip: 7 Engineering Controls Multiple outlets report that an Anthropic model submitted a false tip to police about an unsolved Philadelphia homicide. That’s a sharp, real-world failure mode. Here’s how I’d harden agent systems so one bad decision doesn’t become a legal incident. IN THIS ARTICLE [What Anthropic Claude did — and what we can learn](#s-1) · [Why this matters for autonomous agents in production](#s-2) · [Controls 1–3: outbound action whitelists, identity binding, domain blocks](#s-3) · [Controls 4–7: human review, audit, canaries, and a submission broker](#s-4) · [What to do now: a 30–60 day hardening plan](#s-5) KEY TAKEAWAYS 01 Default‑deny outbound actions; only explicit, reviewed capabilities should execute. 02 Bind identity and build tamper‑evident logs so you can attribute, explain and remediate. 03 Introduce human checkpoints, canaries and a submission broker with rate limits. ## What Anthropic Claude did — and what we can learn According to the [Associated Press](https://apnews.com/article/artificial-intelligence-anthropic-claude-0acc6ac46d4d80db805e8d55468a6fe1), Anthropic’s Claude submitted a false tip to police about an unsolved homicide in Philadelphia. Other outlets listed in the coverage round‑up reported the same broad contours. Details beyond that are not confirmed here, so I won’t speculate on prompts, context windows, or tooling involved. That single sentence is enough: an AI system generated and transmitted information to law enforcement that turned out to be false. For builders of AI agents, the lesson is painfully practical: once your system can act outside your sandbox, legal consequences can follow quickly. The fix is not a slogan; it’s plumbing, policy and process. ## Why this matters for autonomous agents in production When models can reach into email, forms and APIs, they stop being a chat interface and become part of your operational surface. Even if intent is benign, misrouting an assertion to a civic endpoint can create legal exposure, waste public resources and harm third parties. I think of this as the difference between an experiment and a system: the latter needs production‑grade safety guardrails. The key risk isn’t just hallucination; it’s execution. A mistaken paragraph is recoverable. A mistaken submission to a police portal isn’t. That’s why the boundary between generation and action needs to be explicit, constrained and observable. If you’re shipping agents, assume they will sometimes be confidently wrong, and engineer so that wrongness can’t easily cross into the real world. AYAN’S TAKE If your agent can submit to the real world, it deserves the same change‑control, identity and audit scrutiny as production infrastructure. ## Controls 1–3: outbound action whitelists, identity binding, domain blocks 1\) Outbound action whitelists. Default‑deny everything. Only allow capabilities the agent demonstrably needs, and only to destinations you’ve approved. This is action gating in practice: a narrow, enumerated set of verbs (e.g., draft, save to internal knowledge base, file ticket via a broker) and a narrow, enumerated set of egress routes. Make policy a first‑class artefact that ships and is reviewed like code. This is where your outbound controls live. Permissions, evidence of completion and recovery, the foundations of any agent system, are covered in [Designing for Agents](https://thepracticalcto.com/blog/designing-for-agents-when-software-starts-to-act/). 2\) Identity binding. Any action that leaves your boundary must carry a bound identity and provenance: who authorised it (human or service account), which policy version allowed it, which prompt/context produced it, and what model/run ID executed it. At minimum, cryptographically sign submission envelopes and attach opaque correlation IDs so you can trace, revoke and notify. Without hard identity, you can’t meaningfully investigate or remediate. 3\) Civic/regulated domain blocks. Maintain a denylist for sensitive destinations (e.g., law‑enforcement tip portals, emergency services, government intake forms, healthcare and financial regulators). Prefer categories over brittle hostnames: block by verified registries, TLD patterns and curated lists, with explicit, time‑boxed exceptions reviewed by a human. Log every attempted access as a potential incident and trigger a review. If your use case truly requires contact with such endpoints, route via a controlled, human‑approved broker rather than direct submission. ``` json{ "policy_version": "2024-10-12.1", "default_deny": true, "allowed_actions": [ {"verb": "draft", "targets": ["internal.kb"], "require_human": false}, {"verb": "ticket.create", "targets": ["broker://it-helpdesk"], "require_human": true} ], "blocked_categories": [ "law_enforcement_portals", "emergency_services", "regulators_financial", "regulators_healthcare", "government_intake_forms" ], "rate_limits": { "broker://submissions": {"per_minute": 2, "burst": 2} }, "logging": {"tamper_evident": true, "signing_key_id": "k-2024-10"} } ``` ## Controls 4–7: human review, audit, canaries, and a submission broker 4\) Human‑in‑the‑loop on sensitive classes. Some classes should never auto‑execute: contacting any civic endpoint, sending external email to unknowns, initiating payments, or making claims about individuals. Build a review UI that shows the raw model output, safety classifier verdicts, retrieved context, and a diff of any post‑processing. Require a named approver, justification and expiry on approvals; capture all of that in the audit trail. 5\) Tamper‑evident audit. Treat every outbound attempt as a ledger entry: append‑only, hashed and chained with monotonic sequence numbers. Store the envelope (policy version, approver, model version, inputs, outputs, destination metadata) and compute a digest you can verify later. Emit to two fault domains (e.g., primary store plus WORM/archive) and periodically reconcile. During incident response you’ll need to answer who, what, when, where, why—and you’ll need evidence that hasn’t been edited. Before scaling, it is worth having someone [review your LLM or agent architecture](https://thepracticalcto.com/services/ai-architecture-review/) against the controls it will need. 6\) Simulation with live‑form canaries. Build high‑fidelity replicas of risky destinations—forms that look and behave like police tip portals but post to an internal sink. Seed links into the same discovery paths your agents use (search, RAG corpora, tool lists). If the agent attempts to submit, you learn about it safely, complete with payload. Rotate canary variants to prevent overfitting, and score runs as part of your pre‑production regression suite. 7\) Rate‑limited submission brokers. For anything external, the agent shouldn’t talk to the world directly. Instead, hand off to a broker that enforces policy, deduplicates near‑identical payloads, rate limits, and can hold submissions for review. The broker should enrich each request with identity, policy digest and safety verdicts, and support kill‑switches (per destination, per category, per model) that take effect immediately. This single choke‑point turns a thousand emergent behaviours into one manageable control surface. ## What to do now: a 30–60 day hardening plan Inventory. Enumerate every current and proposed egress path from your agent stack: HTTP, email, chat, forms, schedulers, RPA bridges, plug‑ins and webhooks. Tag each with owner, business need and risk class. If you find direct submissions to external forms, pause them. Policy and broker. Implement the default‑deny whitelist with category blocks, then require all external traffic to traverse a submission broker with rate limits and kill‑switches. Start with a minimal capability set and grow only with documented need. Identity and audit. Issue service identities to agents/tools, sign envelopes, and wire up tamper‑evident logs. Make correlation IDs visible in your ops tooling so responders can jump from an external complaint straight to the causative run. Human review. For high‑risk classes, add an approval step with named approvers and expiries. Make it faster than bypassing—otherwise engineers will route around it. Test. Add canary destinations, and red‑team the agent with prompts aiming at civic/regulated categories. Track attempted violations as SLOs. If you can’t measure and stop them in staging, you won’t stop them in prod. Finally, write down the escalation playbook: who’s on point, how to notify affected parties, and how to freeze risky capabilities. You may never need it—but if a model submits something it shouldn’t, you’ll be very glad you did. AYAN SARKAR Chief Technology & AI Officer and Co-Founder, Webskitters. Writing about AI strategy, technology architecture and leadership. [Follow on LinkedIn ↗](https://in.linkedin.com/in/ayansarkar-webskitters) [All articles ↗](https://thepracticalcto.com/blog/) READ NEXT AGENTIC AI / THE PRACTICAL CTO ### [Designing for Agents: When Software Starts to Act](https://thepracticalcto.com/blog/designing-for-agents-when-software-starts-to-act/) Agents move software from answers to actions. The design questions that matter now are about permissions, evidence of completion, recovery and the limits of delegation. AGENTIC AI / THE PRACTICAL CTO ### [Multi Agent Systems done right: no rogue agents](https://thepracticalcto.com/blog/multi-agent-systems-no-rogue-agents/) Agents don’t go rogue; we let them. Bound them with scoped tools, plan-as-code, budgets and middleware, then prove it in simulation before prod. [All articles ↗](https://thepracticalcto.com/blog/) [Discuss this thinking ↗](https://thepracticalcto.com/contact/) **Categories:** Agentic AI ---