OpenAI announced that it has suspended a number of ChatGPT accounts linked to Chinese state-sponsored hackers. Cybercriminals were using the company's artificial intelligence models to improve malware and create phishing content.
The October 2025 report describes the disruption of several malicious networks, as part of the company's ongoing commitment to preventing the misuse of artificial intelligence technologies by threat actors and authoritarian regimes.
Since February 2024, OpenAI has shut down over 40 networks that violated its usage policies. The company said it continues to see threat actors integrating AI into their existing strategies to increase speed and efficiency (rather than developing new offensive capabilities with the models).
Chinese hackers abuse OpenAI's ChatGPT
A key case study in the report focuses on a group called “Cyber Operation Phish and Scripts.” This group of accounts, operated by Chinese-speaking individuals, was used to help develop malware and phishing campaigns.
See also: Malicious use of Microsoft Teams to distribute malware
OpenAI’s research found that the group’s activities were consistent with cyber operations serving the intelligence needs of the People’s Republic of China (PRC). The activity is similar to that of other publicly monitored threat groups such as UNKDROPPITCH and UTA0388.
Creating malware and phishing attacks
These hackers used ChatGPT for two main functions:
1. Malware Development: They used AI to aid in development and debug tooling, with implementation details overlapping with malware known as GOVERSHELL and HealthKick. The perpetrators also explored further automation using other AI models such as DeepSeek.
2. Phishing Content Creation: The group created targeted and culturally adapted phishing emails in multiple languages, including Chinese, English, and Japanese. Their targets included Taiwan's semiconductor sector, U.S. academia , and organizations critical of the Chinese government.
OpenAI noted that the attackers used the models for “greater efficiency,” such as creating better phishing emails and reducing coding cycles (rather than creating new types of threats).
Closing accounts linked to Chinese government entities
The report also describes the shutdown of other accounts linked to Chinese government entities. These users attempted to use ChatGPT to deploy surveillance and profiling tools.
A blocked user asked for help writing a proposal for a “Uyghur-Related High-Risk Alert Model,” designed to analyze travel reservations and police records.
See also: Ransomware groups leverage remote access tools
Another example involved an effort to design a “social media detector” capable of scanning platforms such as X (formerly Twitter), Facebook, and Reddit for political, ethnic, and religious content deemed “extremist.”
🔒 Protect your privacy with Proton VPN
Swiss VPN from the creators of Proton Mail — strict no-logs policy, strong encryption, and built-in NetShield that blocks ads, trackers, & malware.
- ✔ No-logs, based in Switzerland (except 14-Eyes)
- ✔ NetShield: blocks ads, trackers & malicious domains
- ✔ Covers all devices — free version available
The link is an affiliate link — SecNews may receive a commission at no additional cost to you. It does not affect the independence of our article writing.
Other users were banned for using artificial intelligence to research critics of the Chinese government and identify funding sources accounts critical of the PRC.
In response to these findings, OpenAI disabled all accounts associated with the malicious activity and shared breach indicators with industry partners to assist in broader cybersecurity efforts.
The report highlights that the AI models themselves often acted as a security barrier, denying direct requests to create malicious code or execute exploits. The attackers were limited to creating “building-block” code snippets that were not malicious in themselves.
See also: WARMCOOKIE backdoor gains new features
OpenAI's findings show that while state-backed actors are actively experimenting with artificial intelligence, its primary use is to augment existing operations.
The company emphasized that it continues to invest in detecting and stopping such abuses to prevent the use of its tools for malicious cyber activities, fraud, and covert influence operations.
