Whisper Leak: AI steals encrypted conversations

Microsoft researchers have uncovered a new side-channel attack called Whisper Leakthat can reveal the content of encrypted conversations between users and language models, even without access to the underlying text. The discovery highlights a growing blind spot in AI security, where encryption alone no longer guarantees privacy in interactions with models.

See also: Gemini: Deep Research pulls data from Gmail, Drive & Chat

The Microsoft Defender Security Research team reported that attackers can exploit large language models that use metadata, such as the sizes and timings of network packets. For example, a government agency at the ISP level, someone on the local network, or someone connected to the same Wi-Fi router could intercept encrypted traffic and use it to infer whether the user’s prompt is about a specific topic. Metadata becomes the new attack surface. Unlike traditional data breaches or model leaks, Whisper Leak exploits a side channel in network communication rather than a flaw in the encryption itself.

LLM services generate responses step by step, producing one symbol at a time instead of the entire response at once. Also, communications with AI-powered chatbots are often encrypted with HPPS over TLS (HTTPS), ensuring server authenticity and security through encryption. However, while Transport Layer Security successfully encrypts the content of communications, it leaks the size of the underlying data chunks being transmitted.

For an LLM that transmits symbol-by-symbol responses, this size information reveals patterns about the symbols being produced. Combined with timing information between packets, these leaked patterns form the basis of the Whisper Leak attack, as they leak enough information to allow for topic classification, the Microsoft Defender Security Team explained in the white paper.

See also: ChatGPT improves school information search

Microsoft researchers simulated a real-world scenario in which an adversary could monitor encrypted traffic but not decrypt it. They chose “money laundering legality” as the proof-of-concept target. For positive samples, the team used a language model to generate 100 semantically similar question variants on this topic. For negative noise samples, 11,716 unrelated questions were randomly selected from the Quora Questions Pair dataset, covering a wide variety of topics. Once complete, the collected data was trained using LightGBM, Bi-LSTM, and BERT models, evaluated on time-only, packet-size-only, or both features.

The research team demonstrated the attack on 28 popular LLMs from major providers and achieved near-perfect classification (often >98% Area Under the Precision-Recall Curve (AUPRC)) and high accuracy even at extreme class imbalance (10,000:1 noise-to-target ratio). For many models, they achieved 100% accuracy in identifying sensitive topics while recovering 5-20% of target conversations, the report noted.

The findings were shared with OpenAI, Mistral, Microsoft, and xAI, and mitigation measures were implemented to minimize the risk. To mitigate the effectiveness of cyberattacks, OpenAI and later Microsoft Azure added a random variable-length text sequence to each response. Similarly, Mistral included a new parameter called “p” that had a similar effect.

See also: Anthropic: Automatic Memory for Claude Pro and Max Users

Even if the attack doesn’t expose the exact prompt or content of a conversation, it can accurately classify the topic or intent, putting businesses at great risk. While it’s up to AI providers to address the issue, Microsoft researchers’ recommendations include avoiding conversations about particularly sensitive topics via AI chatbots when on untrusted networks, using VPN services to add an extra layer of protection, choosing providers that have already implemented mitigation, and using non-streaming models from large language model providers.