Connexion
AI Is Changing Everything About Hacking. No One Knows What Happens Next
For much of the last month, many of the cybersecurity experts I talk to and follow have been posting like they are very scared. Many have been warning of an AI-powered “Black Swan” event, have been alarmed by the number of vulnerabilities suddenly discovered, and of the apparent power of AI agents and large language models more broadly to find and exploit previously unknown bugs.
The rhetoric has ranged from worried-but-measured (things are about to get weird), to run-of-the-mill advice like update your software and make sure you have two-factor authentication on, to vagueposting about AI-powered attacks, to things like ‘the internet was nice while it lasted,’ to outright prepper-level terror.
To give you a sense of what I’m seeing, Matthew Green, a Johns Hopkins professor and one of the most respected cryptographers in the world, posted “we’re in the part of the movie where you need to throw whatever you can in the path of the zombies and hope it slows them down,” and “so many people discovering that the whole Internet is dry tinder and the only thing keeping it from burning is the fact that teenagers haven’t found the matches yet,” Green is alluding to the fact that hackers and researchers driving AI tools and agents are getting better at finding and exploiting all sorts of bugs in important systems. I’ve also seen numerous AI employees, as mentioned, go full prepper, suggesting people pull money out of their bank accounts, disconnect from the internet, purchase shelf-stable food, hoard water, put money in gold and cash, and purchase hand-crank emergency radios.
And all those posts came before people using AI-powered tools found vulnerabilities in PHP, the programming language that powers much of the internet. In the last few days, major vulnerabilities or issues have been found in eMudhra (an Indian entity that provides website security certificates), Node.js (an alternative to PHP), Gitlab (a software development platform), Plex (media server software), Railway (a cloud provider), OpenCode (a coding agent platform), and several other companies and services, though, rightfully, none of these has gotten as much attention as the vulnerabilities in PHP, called PHP2Shell, which researchers from Wiz, Calif, and OpenAI discovered using AI.
Matthew Prince, the CEO of Cloudflare, a service that has wide insight into the general health of the internet, posted, “In the 10 hours before [PHP2Shell] was publicly announced, across the Cloudflare network we saw thousands of attempts to exploit it. That jumped to tens of thousands in the 45 minutes post announcement, hundreds of thousands within 2 hours, and millions now.”
So much is happening, and so quickly, that it has been difficult for me — someone who has been reporting and editing stories on security and hacking for a decade — to fully understand what’s going on, to have a sense of what could happen next, or predict where this is going. And I think that that is largely where we are at the moment, as I am writing this.
But it has become clear that AI, broadly defined, has fundamentally changed both offensive hacking and defensive cybersecurity. This is not because AI has become sentient or because human beings have lost control of it or because of “swarms” of “rogue agents,” but because AI is proving something that we have always known: much of the software that runs the internet, our devices, our accounts, and our lives is vulnerable, or out of date, or has security flaws that human beings never noticed, or was installed by someone at a company who left the company six years ago and has been replaced four times and no one knows how to find them now or how to update it. This doesn’t mean that all of the vulnerabilities that have been found with the help of AI have been obvious or trivial; skilled hackers and researchers, driving AI tools, seem to be better than humans alone at finding vulnerabilities. But, critically, AI and large language models have made it extremely easy to essentially brute force the internet; if every piece of software is a door, AI makes it exceptionally easy to turn turn the knobs of every door to see if it’s unlocked, even on buildings that human beings had never really bothered to try to open. What we are learning at an alarmingly fast pace is that not only are some of the locks easy to pick, in some cases the door is wide open or there’s a giant hole in the wall. The recent spate of people using AI to hack their internet of things devices to run new software shows that, in some cases, it takes essentially no expertise to actually use AI to hack something.
Meanwhile, we've seen the rise of "abliterated" AI models that are designed to not say no and open weights models that can be used for pentesting and potentially offensive hacks without the guardrails that exist in many off-the-shelf commercial models.
In a blog post explaining the PHP vulnerability discovery, OpenAI said the findings highlight “how AI can uncover security flaws with internet-scale impact, but the same capabilities can also help attackers rapidly develop offensive tools and exploits.” OpenAI basically argues that it did all of this research ethically, and that its discovery opened a “Defender’s Window” that allowed governments and critical infrastructure to prepare for the disclosure of the bug. ‘Researcher finds bug, ethically discloses it to company, company fixes it’ is the preferred course of action in cybersecurity, but AI is shortening the window for all of this, is leading more people to do more research more easily, and is making both bug finding and bug fixing much faster. Whether those fixes are actually deployed by individual companies any faster is another, much more complicated question. The answer in many cases is, probably, no.
AI is also making all of this much more unpredictable. The security company Chainguard was criticized for breaking the embargo on the PHP vulnerability disclosure because it posted fixes for the vulnerabilities several hours before it was officially disclosed. Critics, including the PHP Foundation, said this could have let attackers figure out how to exploit the issues before website owners had a chance to protect their systems. In a LinkedIn post, Chainguard CEO Dan Lorenc said that his company independently found the same bugs by typing “find bad vulnerabilities in PHP” into a frontier AI model. Amazon, meanwhile, said that it saw exploit attempts before the official disclosure. These facts raise questions about whether the bug was exceptionally easy to find, whether the AI models that originally found the vulnerabilities incorporated that information into the model somehow therefore making it easier for someone else to find, or whether there was a leak along the way from someone connected to the disclosure process. Any option is concerning.
After the PHP2Shell vulnerabilities were discovered, some infosec experts started calling this “Black October,” and highlighted the fact that the National Security Agency, the Prime Minister of Japan, and various governments around the world told people to patch their websites and services immediately, and to take down services that couldn’t be patched. 4chan, several Wikimedia services, and Archive.org were all temporarily taken offline so that they could be updated.
As The Atlantic has already suggested, one of the biggest impacts of PHP2Shell could be the fact that many historic websites can’t or won’t be updated, may be proactively pulled offline, might eventually be hacked, and might never come back. The link rot could be horrendous, and a lot of history could be lost.
For such a high-profile incident, most of the actual impacts thus far have been related to the proactive patching or removal of potentially insecure websites, and we have not seen immediate, widespread exploitation of the bug. Various low-effort, LLM-developed exploits have been dropped online; many of them will not work. A blog post called “There isn’t an easy exploit for this PHP vulnerability,” written by Elias Taylor, quickly went viral and was dissected on Hacker News, especially by people pointing out that the website eliastaylor.com had been recently registered and that it wasn’t clear who Elias Taylor was. 404 Media reached Taylor by Signal; he said “I am real and on a delayed train.”
“Lots of people are being tricked by AI into thinking they have an exploit when they don't, because they can't read or understand what the LLM gave them,” Taylor told 404 Media. “It doesn't appear to be anywhere near as exploitable as the other ‘stop the world’ vulnerabilities i.e log4j, heartbleed. But patch your servers, it’s still a vulnerability. Please do not paraphrase me as saying ‘there is no vulnerability’ for any reason. There is, and people should patch.”
Dan Guido, the CEO of Trail of Bits, a well-respected cybersecurity company that partners with OpenAI on the Patch the Planet initiative, posted that his company had created “a working exploit,” and that “Will Tan built it in 1 day with Codex, drawing on ~10 years of system security and vulnerability research. AI accelerates exploit development. Knowing how to turn a bug into a shell still takes expertise. AI makes that expertise more productive.”
It should also be noted that Taylor’s other blog post is called “Anything connected to the internet is going to get hacked to pieces,” which includes the lines “software engineers have spent the past 40 years writing vulnerable, dogshit software. LLMs are easily above the bar needed to break into said software.”
It remains unclear, at the moment, what the ultimate impact of all of this is going to be, or where we’ll be in one hour, one week, one year, or 10 years. But AI is changing how cybersecurity and hacking works. It is making finding and exploiting bugs faster and easier. We’re about to find out how insecure the internet really is.
