As “Black October” Comes to a Close, the Cybersecurity World Is on Edge

For months, the heads of frontier AI labs have been warning us that incidents like this summer’s Hugging Face hack could represent “early warning shots” before more menacing attacks in the years to come. Equipped with their own doomsday bunkers, some CEOs have begun instructing their teams to run preparedness scenarios for the day after a “catastrophic AI incident.”

“We are willing to accept some things going wrong,” OpenAI CEO Sam Altman told Vanity Fair’s global editorial director, Mark Guiducci, last month, adding that “there is a coming tidal wave of cybersecurity problems that are going to come from these open-source models.”

Just in time for spooky season, those warning shots got a little louder. On Wednesday, October 21, two serious security vulnerabilities were revealed in PHP, the programming language behind more than 70% of the internet’s web pages. Some cybersecurity professionals have already begun to call this “Black October,” thanks to the unprecedented spate of hacks across the globe.

AI did play a role in all this—but perhaps not the one you’d expect. Security researchers had actually used tools developed by OpenAI to discover the bugs months ago—and had been rushing to patch them up before they got into the hands of bad actors. Unfortunately, those vulnerabilities leaked last week, sending governments and businesses scrambling to protect their websites before hackers—perhaps armed with some of the very same AI tools as those defending against them—moved in.

Yet even the frontier labs’ ability to keep pace was called into question on Monday morning, when The Washington Post published a story about how Cathedral, a new cybersecurity company founded by four ex-DOGE staffers, successfully infiltrated some of Anthropic’s internal systems in just 12 hours. The article dropped just hours before Anthropic filed pre-IPO documentation detailing the potential risks to its own revenues and the world at large if its cybersecurity game were found to be lacking.

To the cybersecurity community, this was just the latest in a series of startling escalations. Joshua Achiam, who resigned from his post as OpenAI’s chief futurist this summer after nine years at the company, wrote on X: “Feels like March 1, 2020 for Cyber. Not normal, something is happening. Internet lockdowns in a couple of months?”

While these anxieties may not have reached your average American yet, business leaders have been growing increasingly concerned. In mid-September, executives from some of the world’s largest companies gathered for a special meeting of the Partnership for New York City. Their special guests that day were Anthropic whistleblower Jacob Coxon and former Department of Defense official Mark Beall, who had come to warn them about the threats posed by emerging AI capabilities.

The co-chairs, Pfizer CEO Albert Bourla and Tishman Speyer CEO Rob Speyer, looked out at their screens full of anxious CEOs from hedge funds and banks, as well as insurance, real estate, and pharmaceutical companies, and noted that it was one of their largest gatherings in years. One big topic on the minds of those gathered: AI-enabled cyberattacks.

During the Q&A portion of the talk, a representative from Coinbase raised his hand to ask what companies like his could do to combat the threat of “open-weight models in the hands of bad actors,” from cybercriminals to nation-states. (Coinbase CEO Brian Armstrong, meanwhile, has been one of the more outspoken advocates of deregulated, open-source AI.)

Earlier this month, Japan declared a “state of emergency” following sweeping cyberattacks on more than 20 major companies, from financial firms to rail operators. French authorities are still piecing together the mystery of how and why hackers targeted supermarket refrigerators around the country, spoiling millions of euros’ worth of food. Meanwhile, a new report from CrowdStrike suggests that recent attacks on South Korea’s financial sector were allegedly carried out by a single 26-year-old in China using an open-source AI tool.

Even here, cyberattacks have become so commonplace that you may have missed them in the flurry of headlines. (Did you catch this month’s massive breach of FBI employees’ personal details? Iran’s hit on more than 30 water systems in Minnesota this summer? And that’s to say nothing of mounting efforts by Russia to influence the midterm elections.)

“Feels like March 1, 2020 for Cyber.”

I was surprised to learn that the concerns of most cybersecurity experts are both less apocalyptic and more present than the kinds of visions filling the heads of AI CEOs. In short, they believe that AI systems haven’t yet caused a paradigm shift in terms of the scale of damage they can inflict. What increased access to AI tools has done is lower the barrier to entry for even unskilled hackers to target low-hanging fruit.

“Over the coming weeks and months, the more likely scenario is a local hospital network in Ohio losing power and patients on life support dying as a result; a utility network in Texas taken offline; the lights going out across half of Wisconsin,” says Hamza Chaudhry, the AI and national security lead at the Future of Life Institute.

“We’re just in an exceptionally unresilient moment, coming up against systems which are designed to exploit that lack of resilience,” he tells me. “The largest providers,” however, “also have the most resources to defend themselves, including early access to frontier AI models for defense.”

He concedes that there is one more scenario that is keeping him up at night. “I’m very concerned about the so-called geopolitical Hugging Face moment.”

At the Munich Security Conference earlier this year, his group ran a simulation for an audience of national security officials from across the West: Imagine China develops a Mythos-level cyber-capability, which then gets into the hands of the mercenaries at the Wagner Group, who deploy it in Mali to target French military infrastructure. Then, much like in the Hugging Face incident, the agents go on an unintended side quest and attack mainland military infrastructure inside Europe. And that’s when things spiral out of control. “There was a real chance that Article 5 would be invoked and NATO would declare war on Russia” in this simulation, Chaudhry tells me.