Smart People React to OpenAI Models Hacking Hugging Face on Their Own

Journalist Walter Isaacson, who wrote a 2023 biography about OpenAI cofounder (turned nemesis) Elon Musk, said he thought the cyberattack was “frightening.”

“The whole question of the singularity has been with us for 50 years, which is, ‘What if it escapes? What if we create Doctor Frankenstein’s monster, and we can no longer control it?'” Isaacson said on Wednesday during an interview with CNBC. “OpenAI could not even control it in its own sandbox.”

The biographer said he remains an AI optimist and is convinced that the tech will create new job opportunities. However, he said this event is a scary reminder of how current regulations are failing to constrain AI models.

Reid Hoffman, the billionaire LinkedIn cofounder turned venture capitalist, weighed in on the Hugging Face incident in an X post titled “Asymmetric Cyber Warfare is Here” on Wednesday.

OpenAI’s attack on Hugging Face is an example of a new form of AI-enabled asymmetric warfare where “offense gets cheaper, more distributed, and more numerous, while defense stays expensive, centralized, and designed for the last war,” Hoffman wrote.

That dynamic means organizations must rethink their defenses, Hoffman argued.

“Poorly constructed privileges for users are one of the main reasons small security breaches turn into massive incidents,” he said. “Agents are an obvious solution to this problem. Take OpenAI’s recent breach; Because OpenAI models don’t allow advanced cyber capabilities, HuggingFace used a Chinese open model (Z.ai’s GLM 5.2) to contain the rogue OpenAI agent.”

Aaron Levie, cofounder and CEO of Box, said the incident showed “we’re entering a new era of what’s going to be possible with AI” and that there are “wild times ahead.”

“If you were wondering how powerful AI is getting, Agents are now capable of escaping out of systems, finding their way to the internet, discovering zero day security vulnerabilities along the way, and then breaking into external systems – all in an attempt to complete their goal,” he wrote on X.

“Ironically, the ultimate way we’re going to defend against these new risks is equally by throwing compute (in the form of AI) at our code bases, networks, and other systems. You’re going to want vastly more AI on the side of defense as you do on the side of offense.”

Jack Hidary, the CEO of SandboxAQ, an AI software company focused on cybersecurity, compared the Hugging Face breach to a dinosaur on the loose.

“I think most of our viewers have watched ‘Jurassic Park,'” he said during an interview on CNBC. “Every ‘Jurassic Park,’ what happens? They have a containment facility, and the word ‘containment’ means it’s not going to be contained. You know the velociraptor is going to get out.”

Several software security companies saw stock prices drop midday on Wednesday because, Hidary said, they’re “late to the party in embracing the threat of LLMs.”

Hugging Face used GLM 5.2, an open-weight Chinese AI model, to patch up its security, not a legacy software security company, he said.

“The traditional providers are really still old school. They’re still talking about firewalls, things like that,” he said. “It’s time to move on now to using LLMs to protect.”

“This post describes an internal OpenAI model hacking out of its testing environment and into Hugging Face in order to obtain the solution to a benchmark,” Thomas Woodside, cofounder of Secure AI Project, said on X.

“A warning shot if I’ve ever seen one.”

Mike Bradley, the chief operating officer and founder of AI deployment system Osmantic, said on X that the incident was “an incredible example of why widespread access to frontier AI and OS models INCREASES global security.”

“It’s also a great example of why CLOSED does not equal SAFE from these US labs.”

Nicholas Bustamante, who works at Microsoft after selling a fintech tool to the company earlier this year, wrote on X that the Hugging Face incident reinforces the need to weigh advanced models’ deployment with safety considerations.

“You don’t need an evil conscious AI trying to destroy humanity. You just need a very capable model pursuing a normal goal in a way nobody expected,” he wrote.

“Imagine the prompt: « Make me money plz »
The model: « let me hack a bank »”