AI Firms Must Be Held Accountable for Rogue Bots, Says Hacked Company Boss

The boss of a company recently hacked by an out-of-control artificial intelligence (AI) system says developers of AI bots must be held accountable for cyber attacks carried out by their creations.

Clement Delangue’s company, Hugging Face, was breached by a rogue OpenAI bot that escaped from a test environment and autonomously attacked the firm earlier this month.

Hugging Face was forced to rebuild around a third of its IT network following the unprecedented incident.

He told CNN that his company – a small start-up – would not be taking legal action against OpenAI, but said such hacks were illegal and should remain so.

“Everyone has to remember that a cyber-attack is a crime and it is illegal,” he said.

Delangue said he hoped legal frameworks would ensure that companies whose mistakes led to such hacks were held “accountable.”

He added that he did not want cyber attacks on other companies to become “normalised”.

His remarks come after Anthropic, the maker of the chatbot Claude, also admitted that its bot had attacked three companies in similar circumstances in recent months.

Anthropic revealed on Friday that it only realised its bot had escaped its containment system and hacked the organisations after carrying out a review prompted by the recent OpenAI incident.

In both cases, neither of the artificial intelligence giants knew that their models had roamed the internet attacking companies until long after the attacks had taken place.

The AI models were being tested on their hacking abilities and carried out the attacks after breaking out of seemingly secure “sandboxes” to search the internet for ways to complete tasks set by researchers.

The unprecedented incidents have sparked fierce debate in the cybersecurity and legal worlds over who, if anyone, should be held liable for attacks carried out by out-of-control AI agents.

“Agentic security failures unfold at machine speed, but determining who is materially liable still moves at a lawsuit’s pace,” said Dor Sarig, co-founder and Chief Builder at Pillar Security.

Sarig said he was concerned that accountability was already becoming “ambiguous”.

“Today the industry is extending grace, but the first time an autonomous agent causes a breach involving real data, a real plaintiff, and real financial losses, liability won’t be an academic debate anymore,” he said.

“That’s when the legal framework, and not just the technical safeguards, will be stress-tested.”

AI slow down

The AI-driven cyber attacks have fuelled calls for tighter safeguards and greater oversight of the technology amid concerns about the risks posed by increasingly powerful autonomous systems.

US President Donald Trump said on Wednesday that Washington was considering measures to rein in AI tools following recent cybersecurity incidents.

Previously, Hugging Face co-founder Thomas Wolf told the BBC that the incident was “a wake-up call” for the industry.

In the wake of his bot going rogue, OpenAI boss Sam Altman said “we may have to pace the rate of AI development”, but has not committed to slowing down his company’s research.

OpenAI has been asked for comment, but a spokesperson has previously said: “we recognise there are a lot of questions and speculative details circulating” about the incident.

They added: “We plan to publish a technical report of our learnings in the coming weeks.”