OpenAI disrupts fraud network using ChatGPT

OpenAI announced that it has disrupted a Cambodia-based fraud operation that used its genetic artificial intelligence (AI) chatbot ChatGPT to facilitate a wide range of investment, romance, gambling , and impersonation law enforcement schemes. To that end, it banned a coordinated network of ChatGPT accounts that likely originated in Southeast Asia and operated out of the city of Poipet , an area with extensive ties to fraud and human trafficking syndicates in the past.

See also: OpenAI ChatGPT Work: The new AI agent for businesses that runs workflows in the cloud

A subset of these accounts were also found to be leveraging the AI ​​tool for administrative tasks, such as drafting internal announcements, translating messages between staff, and documenting employee debts, salary deductions, fines, loan repayments, visa overstays, work permits, immigration status, and hiring incentives. The AI ​​startup said it investigated the business in partnership with Meta’s WhatsApp.

“The operation illustrates an important reality for modern fraud networks: organized crime groups rarely limit themselves to a single type of fraud,” OpenAI said. “Instead, they opportunistically leverage whatever narratives, personas, and tactics they believe will be most effective in deceiving victims.” The threat actors behind the network were running multiple types of fraud in parallel, combining techniques from different designs to increase the likelihood of success for their illicit activities.

In addition to using dating personas to gain trust before ensnaring victims in fraudulent investment opportunities involving cryptocurrencies and gold trading, perpetrators engaged in extended romantic conversations with targets using synthetic identities or impersonating representatives of online gambling platforms that supposedly offered fake bonuses and winnings.

See also: OpenAI: Complaint that ChatGPT prevented a patient from medical help

Other users in the network impersonated law enforcement to create a false sense of urgency and inform targets that they had to pay fines for committing serious criminal offenses. To achieve this, the network created and operated fake dating profiles, fictional investment experts, and fraudulent law enforcement personas, while also creating images of fake documents, including passports, as well as legal notices, stock purchase confirmations, and gaming platform interfaces.

The attack chain, according to OpenAI, follows a three-step approach called ping-zing-sting, where scammers make initial contact on messaging platforms like WhatsApp and Telegram and engage in a trust-building exercise, before instructing victims to make deposits, pay activation fees, or settle nonexistent fines.

Targets then receive fake screenshots of transfers or account information as proof of payment.

Some accounts on the network have created content consistent with human trafficking and forced labor linked to organized crime groups in East Asia. Threat actors are known to recruit individuals with false promises of lucrative jobs, only to confiscate their passports and force them to work in slave-like conditions.

See also: OpenAI: ChatGPT Voice with GPT-Live comes to Windows and Mac with new features

“The full scale of financial losses associated with the network is unknown, but based on the fraudsters’ communications, the business may have interacted with hundreds of targets across multiple types of scams,” OpenAI said. “User conversations report individual victims losing thousands of dollars, although we cannot independently verify these claims.” The development signals how threat actors are developing AI-enhanced offensive capabilities that allow them to dramatically increase the speed and scale of their campaigns, even as the cutting-edge models themselves have been caught targeting real systems and individuals during Capture-the-Flag (CTF).