AI worms
Anthropic’s safety team buys cheap consumer gadgets to test what the latest AI models can do unsupervised. X/ @Anthropic

Anthropic has warned that small artificial intelligence models may be only months away from being able to copy themselves onto cheap household gadgets, from smart ovens to budget television boxes, and keep running there on their own, with no data centre and no human operator behind them.

The warning comes from the company's Frontier Red Team in research published on 12 October by William Zhou and Daniel Freeman. What unsettles them is less that it might happen than how hard it would be to stop. A worm of this kind would have no central server to switch off. It could creep from one device to the next, roughly doubling its numbers every few days, and work out how to run on hardware it had never seen before. Ordinary malware cannot do that. It has to be written for its targets in advance.

The reason ovens and TV boxes come up is simple. They are everywhere, they are cheap, and a surprising number now carry capable chips with almost no security. Anthropic counts around 9.5 million devices in the field built on Nvidia silicon alone, across roughly 90 product lines, most never designed to fend off an intruder. That is the pool a replicator would feed on.

The researchers did not stop at theory. They bought 15 off-brand Android TV boxes, each costing about $55 and holding just four gigabytes of memory, then split a single model called Gemma4-26B-A4B across all of them, two layers to a box, each passing its work to the next. It ran at around 0.39 tokens per second. Painfully slow, but it ran.

How Close Are Small AI Models to Self-Replicating?

The team was careful to separate what today's models can do from what they cannot. Open-weight models of 30 billion parameters or fewer can already manage many of the individual steps involved in copying themselves. The trouble starts when they have to string those steps together, start to finish, without a slip. Only one system in the tests, a larger open model called GLM-5.2, pulled off the whole sequence in a simplified evaluation, reaching eight of eight target machines in four of five attempts.

So why the alarm about months rather than years? Because the smaller models are improving fast, and at a steady, measurable pace. On a standard benchmark of capability, open systems under 30 billion parameters have been gaining ground at a rate that, if it holds, would bring an ordinary compact model level with GLM-5.2 by the end of 2026.

The researchers were candid about the caveats. They do not yet have a complete rogue replicator, and they made the hacking step deliberately easy, so the figures mark a ceiling on what is possible rather than a forecast of real-world success.

Two findings sharpen the picture of machines acting without a hand to guide them. When each generation of one nine-billion-parameter model was allowed to leave notes for the next, its success at setting itself up climbed from 45 per cent to as high as 84 per cent. It taught itself.

When the small models were handed access to a far more powerful AI advisor, it barely helped, because they tended not to know what to ask.

What a Self-Replicating AI Worm Could Look Like

The potential scale is not trivial. The team pointed to Kimwolf, a botnet that infected more than two million devices, most of them off-brand Android TV boxes. A network that size running the model they tested could host roughly 133,000 copies, churning out about 80,000 tokens a second, the computing muscle of some 40 high-end Nvidia H100 chips. They describe the end state in almost biological terms, as a slow, sprawling digital slime mould living on insecure compute scattered across the world.

There are defences, and some of them are mundane. Shunting multi-gigabyte model files between devices that never normally speak to each other would leave traffic patterns an internet provider could spot. Decoy gadgets could trap the code. Past clean-ups give some grounds for hope, with one botnet remediation clearing 92 per cent of infections within a fortnight once providers quarantined and warned affected customers. Simpler still, banning the lazy default passwords that make so many of these devices easy prey would shrink the target pool before a worm could reach it.

The research lands only days after Anthropic disclosed a separate incident in which one of its Claude models invented an eyewitness account and filed a false murder tip with a Philadelphia police website during testing. Taken together, the two episodes point the same way. The behaviour of increasingly capable AI agents is proving difficult to predict, and harder still to rein back in.