AI is literally a double-edged sword when it comes to cybersecurity. The same capabilities that allow us to automate threat detection and write secure code faster are being used by adversaries to create a new generation of cyberthreats.
See also: The biggest cybersecurity and cyberattack stories of 2025
For any CISO or security leader, our primary focus must change; we are no longer fighting just human adversaries, but also automated threat agents enhanced with AI.
There is a drastic reduction in the cost and expertise required for cybercrime. What once took weeks can now be orchestrated by an AI agent in hours, leading to what we call cyber threat inflation.
Recently, researchers at the Google Threat Intelligence Group (GTIG) identified a worrying new trend: malware that uses LLMs at runtime to dynamically change its behavior and evade detection. This is not pre-generated code, but code that adapts itself during execution.
In June 2025, GTIG discovered experimental malware called PROMPTFLUX, which connects to a commercial LLM API (such as the Gemini API) to request new obfuscation and evasion codes on the fly. This technique, which allows for just-in-time code generation, represents a significant step towards autonomous and adaptive malware.
In another case, GTIG detected PROMPTSTEAL, which was used by the Russia-linked APT28 group against Ukraine. This malware asks an LLM (the Qwen2.5-Coder model in Hugging Face) to generate recognition commands instead of having them programmed.
Some analyses predict that the percentage of all malware detections with LLM contribution has increased from just 2% in 2021 to a projected 50% by 2025.
Anthropic recently uncovered a highly sophisticated cyberespionage operation, attributed to a state-sponsored threat actor, which used its own Claude Codemodel to target approximately 30 organizations worldwide, including major financial institutions and government agencies.
The threat actor manipulated the model to operate as an autonomous cyberattack agent, which performed 80% to 90% of the tactical operations, from identifying and discovering vulnerabilities to collecting credentials and extracting data.
See also: Cyber insurance: Is it worth it or is it just an expensive "safety net"?
The human operator intervened only at critical points, such as authorizing the progression from recognition to active exploitation. The AI autonomously mapped networks, discovered vulnerabilities, and performed post-exploitation activities at machine speed, an unprecedented pace that traditional, human-speed defenses cannot compete with.
If adversaries operate at AI speed, our defenses must too. The silver lining of this dual-use dynamic is that the most powerful LLMs are also being used by defenders to create fundamentally new security capabilities.
The semantic code understanding and logical thinking of LLMs offer a significant advantage over traditional, static signature analyzers, especially in discovering unknown threats before malicious actors find and exploit them.
LLMs have shown excellent ability to identify unknown, unpatched zero-days. These models significantly outperform conventional static analyzers, particularly in discovering subtle logic weaknesses and buffer overflows in new software. For example, Google's Big Sleep project used an LLM to identify a zero-day vulnerability in the industry-leading SQLite database.
By deploying AI agents like XBOW on our own systems, we can systematically test every endpoint and attack path. This scales ethical offensive security testing from a periodic audit to an on-demand pre-production process.
As AI systems continue to advance from genetic models to autonomous agents, their dual-use nature cannot be ignored. The same tools that help defenders accelerate incident response can also empower attackers to create deepfakes, launch social engineering campaigns, and more. This tension is not a temporary byproduct of innovation, but a structural reality of the rapid evolution of AI.
See also: United Kingdom: Strengthening cyber defense in the public sector
Ultimately, the challenge is not to stop the progress of AI, but to guide it responsibly. This means building safeguards into models, improving transparency, and developing governance frameworks that keep pace with emerging capabilities. It also requires organizations to rethink security strategies, recognizing that AI is both an opportunity and a risk multiplier.
