Slashdot has reported:

Anthropic has warned that advances in small artificial intelligence models could lead to self-replicating malware that spreads through Internet of Things devices, including smart fridges and vacuum cleaners. The company said that by next year, smaller models may be able to reproduce autonomously across a range of IoT hardware, potentially enabling computer worms that infect unfamiliar devices without human interaction.

The warning is based on an experiment that found significant obstacles to building an AI malware replicator. Anthropic said the results are therefore limited, but they raise concerns about new kinds of computer worms and the difficulty of stopping them once they begin to spread.

In a blog post, the company wrote that “a replicator like this would be challenging to shut down, since there's no centralized point to shut off -- though it may be slow (with a doubling time of days). Unlike non-AI malware, it could also figure out unfamiliar devices on its own.”

Small models need less powerful hardware than frontier models, so Anthropic researchers tested small open-weight models on real or simulated, low-cost off-brand Android TV boxes and other IoT hardware. They used quantization, which represents models with fewer bits, and divided workloads into “shards” that ran on separate devices. The researchers found the models could establish inference: their weights loaded and they generated output.

That result led researchers to examine whether small models could create self-replicating infections on IoT devices, similar to how traditional IoT worms install denial-of-service code and other payloads. Mirai was one example cited. Anthropic said AI worms could pose a greater threat because their models might dynamically adapt exploits and payloads to each target.

The company also warned that a worm could carry its model weights as it reproduces or download them again from a public hub, then run inference on compromised hardware. “Effectively, the weights would become part of the malware payload,” Anthropic said. The possibility could make an infection chain harder for defenders to stop, even if its spread were slow.

Anthropic's findings describe a potential capability, rather than a working malware system demonstrated by the experiment. The company said its test revealed significant challenges in bringing an AI replicator to fruition, while warning that the prospect warrants concern. Further reading listed with the report was “Anthropic Bans Cruelty Toward Claude.”