Security on social media platforms is taking another hit, this time from an unexpected source: Grok itself , X's built-in artificial intelligence (AI) assistant. Researchers have revealed that attackers are exploiting X's AI system to bypass security controls designed to limit the spread of malicious ads .
According to research by Nati Tal from Guardio Labs, the method developed by the threat actors is both simple and effective. Instead of embedding malicious links directly into video ads – a practice that would be blocked by X filters – the attackers hide the link in a less than discreet metadata field: the “From:” that appears below the video cards. This field does not appear to be checked with the same rigor as the main body of the content, thus offering a “window” for malicious diversions.
See also: How generative AI opens up new privacy risks
Grok's unexpected "help"
The next step is what makes the technique particularly dangerous. The attackers themselves – or unsuspecting users who fall for the trap – address Grok via comments under the ad, with questions like: “Where is this video from?” or “What is the link to the full content?”.
Grok, designed to respond naturally to user queries, parses the hidden “From:” field and returns the full link in a clickable form. Thus, the AI assistant inadvertently becomes a “middleman” that directs the audience directly to malicious websites.
The problem is exacerbated because Grok is considered an official, trusted system account. This means that each of his posts or replies gains additional authority, boosting credibility, SEO , and visibility of malicious links. In this way, the attacks gain massive traction, reaching millions of views in a short period of time.
From fake CAPTCHAs to malware
In practice, the links spread with this technique lead to all sorts of malicious places: fake CAPTCHA pages that trap the user, websites hosting data-stealing malware , or suspicious advertising chains that constantly redirect the visitor.
See also: New 'NotDoor' backdoor targets Outlook users
The problem isn't just that the platform's filters are being bypassed. Grok's involvement gives these links a sense of "legitimacy," making users even more vulnerable. A typical user would hardly imagine that the platform's own AI assistant could lead them to dangerous content.
The “Grokking” phenomenon
Tal dubbed the technique “Grokking,” noting that it is a form of abuse that exploits the very design of the AI system. It does not require advanced tools or complex exploits; instead, the method relies on Grok’s routine operation and the trust that users place in it.
The effectiveness of "Grokking" is impressive: in several cases, malicious campaigns based on this technique were launched in numbers reaching millions of impressions, which makes them comparable to legitimate viral campaigns.
What can be done?
According to Tal, addressing the problem is not impossible. He suggests specific measures such as:
- Scanning all metadata fields and not just the main content.
- Automatic filtering of links before Grok repeats them, by comparing them to block lists.
- Prohibition of displaying unverified links by AI without prior review.
However, there is no clear indication that X has adopted such changes so far. Tal says that he has already contacted Grok's team and that the engineers are aware of his report. However, the company has not made an official statement.
See also: Phishing campaign targeted Google Cloud and Cloudflare for 3 years
🔒 Protect your privacy with Proton VPN
Swiss VPN from the creators of Proton Mail — strict no-logs policy, strong encryption, and built-in NetShield that blocks ads, trackers, & malware.
- ✔ No-logs, based in Switzerland (except 14-Eyes)
- ✔ NetShield: blocks ads, trackers & malicious domains
- ✔ Covers all devices — free version available
The link is an affiliate link — SecNews may receive a commission at no additional cost to you. It does not affect the independence of our article writing.
A lesson on AI systems
The “Grokking” offers a valuable lesson in how cybercriminals adapt. It doesn’t always require zero-day exploits or sophisticated malware; often it’s enough to exploit the “blind trust” that users place in a platform or a trusted account.
This case also highlights the new responsibility that artificial intelligence companies have. An AI assistant is not just a tool to improve the user experience; it can unwittingly become a lever for the spread of threats.
In a world where AI systems are increasingly integrated into platforms, security cannot be considered a secondary issue. Instead, it must be a key criterion in their design and operation.
Source: www.bleepingcomputer.com
