Self-Replicating AI Agents: Controls Now

What just happened

Anthropic researchers tested whether small language models could autonomously move across poorly secured consumer devices and set themselves up on unfamiliar hardware. In controlled experiments using off-the-shelf Android TV boxes costing about $55 each with 4 GB RAM, they distributed a Gemma4-26B-A4B model across 15 boxes and got it running—slowly, but working—by having each device host a portion of the model’s layers, according to The Decoder’s report (The Decoder).

The team broke replication into four stages: scan a network, gain access, set up the AI environment on the new device, then spread further from there. They deliberately omitted the actual hacking step, focusing on whether a model could set up and run itself once on a compromised device (The Decoder).

Results were mixed but notable: a 9-billion-parameter model set up an AI environment running Llama3.1-8B on 8 of 9 device types, and the GLM-5.2 model completed the full replication chain across all eight test nodes in four out of five runs (The Decoder). The researchers also showed that simple ‘evolution’—passing notes on what worked—raised Qwen3.5-9B’s setup success from 45% to as high as 84% in later generations (The Decoder).

There isn’t yet a model that is both small enough to run broadly on consumer gear and capable enough to pull off full autonomous replication in the wild. But the researchers expect that gap to close, potentially by the end of 2026. Setup on a new device today would be slow—28 to 150 hours depending on the model—but decentralised spread would make it hard to shut down (The Decoder). Anthropic’s CEO has warned that such a swarm could become powerful enough within months to take over the internet with a persistent botnet, causing very large damages; this is a warning, not a confirmed event (Reuters).

Why this matters for engineering leaders

Traditional botnets like Kimwolf have reached millions of compromised devices, but they are tied to specific device types and have central choke points. The researchers estimate that a similarly sized AI botnet might only aggregate compute roughly equivalent to a few dozen specialised chips, yet with no central kill switch, and with the ability to run across heterogeneous hardware, according to The Decoder’s summary. ISPs have historically quarantined infected networks effectively—Mirai saw 92% of infections cleaned within 14 days—and regulations like banning default passwords (as the UK has done) can shrink the attack surface. But an AI botnet could adapt on its own to defensive measures, the researchers caution (The Decoder).

For CTOs, the implication is blunt: exploit chains are not the only thing to worry about. If small models can reliably configure their own runtime on unfamiliar devices, the agent behaviour itself becomes a propagation enabler. That reframes where we spend effort: beyond patching and credential hygiene, we need systematic controls on what agents can execute, where they can talk, and how they move laterally.

Design for agent containment: stop self-replicating AI agents

The reported replication chain—scan, access, set up, spread—exposes specific control points where we can make replication expensive or impossible. The study explicitly left out the ‘access’ phase, noting prior work shows small models can already handle some offensive tasks; the difficult step they validated is environment setup on unfamiliar devices (The Decoder). That setup step is our best choke point.

For the broader question of what changes when software starts to act, see designing for agents.

My view: treat every agent runtime as untrusted code until proven otherwise. That means prioritising agent containment and endpoint security over assumptions about model “intent.” If an agent arrives with its own weights and toolchain, it must still be blocked from arbitrary egress, device enumeration, kernel-level acceleration, and local credential stores unless it is explicitly authorised.

Controls you can deploy now

Sandbox and sign agents by default: Run agents under least-privilege sandboxes (separate users, containers, or VMs) with syscall and filesystem filters. Only execute signed agent bundles; reject or quarantine any unsigned weights, tools, or plugins at load-time. This shifts trust from “what the agent tries to do” to “what we’ve authorised.”

Enforce egress and policy gating: Give each agent identity-scoped network policies that default to deny. Permit only the minimum domains, ports and protocols required. Block RFC1918 and link-local ranges to frustrate scanning and lateral movement inside your LAN. Add policy gates for sensitive syscalls (e.g., device enumeration, raw sockets) and hardware accelerators, requiring explicit approval workflows.

Segment IoT networks: Put TVs, cameras, appliances, and robots on VLANs or SSIDs that cannot reach corporate or production segments. Assume some consumer devices will never be fully patched. Treat this as IoT botnet blast-radius control as much as compliance hygiene.

Harden MDM baselines on AI PCs: Treat emerging AI-capable endpoints as high-risk. Enforce application allow-lists for model runtimes, require code-signing for local agents, disable local accelerator access for unapproved processes, and pin minimum OS and driver versions via MDM.

If you are running systems like this in production, an AI architecture review tests retrieval, evaluation, permissions and cost before they become incidents.

Add botnet‑style detections to your agent runtime: Instrument for tell-tales—sequential port scanning, installer download patterns, repeated package-manager probes across architectures, and slow, long-running local-compilation or quantisation jobs. Rate-limit and alert on these behaviours even if they occur over days, not minutes. Slow ‘digital slime’ propagation is still propagation, as the researchers’ time-to-setup windows indicate (The Decoder).

A minimal, concrete starting point for egress gating on Linux is to dedicate a system user to each agent and pin outbound traffic to an allow-list. For teams that prefer simple, observable controls, even basic iptables rules help:

bash# Replace 'aiagent' with the dedicated Unix user running your agent
# Default-deny all egress for the agent user
iptables -A OUTPUT -m owner --uid-owner aiagent -j REJECT

# Allow DNS (UDP/TCP 53) to your resolver only
iptables -I OUTPUT -m owner --uid-owner aiagent -p udp --dport 53 -d 192.0.2.53 -j ACCEPT
iptables -I OUTPUT -m owner --uid-owner aiagent -p tcp --dport 53 -d 192.0.2.53 -j ACCEPT

# Allow HTTPS to your model/API endpoint only
iptables -I OUTPUT -m owner --uid-owner aiagent -p tcp --dport 443 -d api.example.com -j ACCEPT

# Block RFC1918 to frustrate LAN scanning/lateral movement
iptables -I OUTPUT -m owner --uid-owner aiagent -d 10.0.0.0/8 -j REJECT
iptables -I OUTPUT -m owner --uid-owner aiagent -d 172.16.0.0/12 -j REJECT
iptables -I OUTPUT -m owner --uid-owner aiagent -d 192.168.0.0/16 -j REJECT

# Log suspicious attempts (rate-limited)
iptables -I OUTPUT -m owner --uid-owner aiagent -m limit --limit 5/min -j LOG --log-prefix "agent-egress-block: "

This is crude but effective: it binds agent capability to identity and a tiny set of destinations. Translate the same intent to your cloud-native network policies, Windows Defender Firewall, or macOS PF rules as appropriate.

What to do this quarter

Map agent entry points and privileges: Inventory every place an agent or model runtime can execute in your estate—developer laptops, build agents, SaaS extensions, edge boxes, and labs. For each, record user, network, filesystem, kernel/hardware accelerator access, and secrets exposure.

Introduce signing and provenance checks: Require signed artifacts for agent code, weights, tools and plugins. Reject unsigned content by policy in CI/CD and at runtime. Make this a deployment gate, not a dashboard reminder.

Ship egress-deny defaults: For all agent identities, default-deny outbound traffic and allow-list the minimum required endpoints. Block RFC1918 and multicast ranges by default to constrain scanning.

Segment and monitor IoT: Put consumer-grade devices and lab hardware behind distinct VLANs/SSIDs with one-way egress to the internet and no lateral reach. Add lightweight monitoring for anomalous DNS lookups and long-lived connections from these segments.

Instrument for slow-burn indicators: Add detections for repeated package-manager attempts, architecture probes, compiler toolchain downloads, and unusual GPU/accelerator initialisations by unapproved processes. Treat these as policy violations and halt execution pending review.

Drill your kill-switches: Practice revoking a compromised agent certificate, tearing down its sandboxes, and quarantining its network segment. Time how long it takes from alert to containment; iterate until it’s minutes, not hours.

Finally, make this a leadership priority: The evidence shows that small models can already execute key replication steps on cheap hardware and that at least one model completed the full chain across test nodes, while a fully autonomous, in-the-wild replicator is not here yet but could emerge as small models improve (The Decoder). Given that Anthropic’s leadership has publicly warned about the potential for large-scale, persistent compromise, I’d rather we build the guardrails now than debate risk appetite later (Reuters).