LLMs Are an Inside Threat

There was another “my LLM deleted production” incident (via).  In this particular case, the agent trawled the filesystem to locate another credential with the rights to delete the database and all backups.

This demonstrates an LLM in the role of a “motivated attacker."  When faced with obstacles, it doesn’t simply halt; it inspects the error, adapts its approach, and overcomes the obstacles.  Small comfort that the filesystem outside the workspace is “read only,” if the damage is done beyond the filesystem boundary.

For the foreseeable future, I’ll continue to run the (mandated) LLM tool in a VM.  It’s all alone with the code in a mock production image, and aws-vault, documents, emails, D-Bus, and the GUI are all hidden away on the other side of the hypervisor.