Cybercriminals are increasingly leveraging Generative AI (GenAI) platforms to orchestrate sophisticated phishing campaigns, which pose unprecedented challenges to traditional security detection mechanisms.
The rapid proliferation of GenAI services has created a fertile ecosystem for malicious actors, who leverage these platforms to create convincing phishing content, clone trusted brands , and automate malicious deployments . Most worryingly, it requires little to no technical expertise.
The emergence of Generative AI web services that offer capabilities such as automated web page creation, natural language generation, and chatbot interaction has fundamentally transformed the threat landscape. These platforms allow attackers to generate phishing sites in seconds, using AI-generated images and text that mimic legitimate organizations.
See also: Phishing: Noodlophile malware distribution with new “bait”
The accessibility of these tools has lowered the barriers for cybercriminals, allowing even non-specialized actors to launch convincing social engineering.
Increased use of Generative AI across all sectors
Recent telemetry data reveals a dramatic increase in GenAI adoption across industries, with usage more than doubling in six months. Palo Alto Networks found that the high-tech dominates AI usage (accounting for over 70% of total GenAI tool usage).
However, this widespread adoption has created new attack methods, as malicious actors exploit the same Generative AI platforms that legitimate users rely on to enhance productivity. Analysis of phishing campaigns reveals that AI website builders represent the most exploited category of AI services, accounting for approximately 40% of malicious use of GenAI. The researchers documented real-world examples of phishing sites created using popular website builder platforms, capable of producing functional websites within seconds.
See also: British student convicted of selling phishing kits
These platforms typically require minimal verification, often accepting any valid email address, with no phone verification or identity verification. The attack methodology involves malicious actors inserting short company descriptions into AI Prompts, which automatically generate complete web page content, including professional images, compelling company narratives, and detailed service descriptions. The crafted phishing sites typically employ a two-stage attack mechanism. Initial landing pages generally display messages such as “You have new documents” with prominent call-to-action buttons. When victims interact with these elements, they are redirected to secondary credential collection pages.
Writing assistants are also used by hackers (30%), while chatbots account for almost 11% of observed attacks. These statistics highlight the diverse range of AI platforms being used for malicious purposes.
Threats are intensifying
The explosive growth in the use of Generative AI is creating a new landscape in cyberspace, where the line between legitimate and malicious use is becoming increasingly blurred. The ability to create highly realistic content in seconds gives attackers tools that previously required technical knowledge, time, and financial resources. Today, even inexperienced users can launch campaigns large-scale phishing, taking advantage of the accessibility of these tools.
See also: Beware of Chinese fake e-commerce websites
The most worrying aspect is that traditional security mechanisms have difficulty detecting such attacks. Since the pages, text, and images are generated by AI models that perfectly mimic the style and aesthetics of real companies, phishing emails and sites become extremely convincing. This means that victims are much more likely to trust the content and provide sensitive data.
This situation poses urgent challenges for the cybersecurity industry. A new generation of defense tools, also based on AI and behavioral analysis, is needed to detect the subtle cues that distinguish legitimate uses from malicious ones. At the same time, user education remains a critical factor, as the most advanced technology cannot replace human judgment in identifying suspicious messages and websites.
🔑 Secure your passwords with Proton Pass
Password manager from Proton — end-to-end encryption, passkeys, built-in 2FA, and monitoring for leaks of your credentials.
- ✔ Encrypted storage of passwords & passkeys
- ✔ Notification if any of your passwords are leaked (Dark Web Monitoring)
- ✔ Free version — on all devices
The link is an affiliate link — SecNews may receive a commission at no additional cost to you. It does not affect the independence of our article writing.
