AI hallucination at SOCPAC: US-China war almost sparked by false chatbot report

A classic AI chatbot hallucination used by a Special Operations Command Pacific (SOCPAC) in Hawaii nearly led to a US-China armed conflict in the spring of 2026, when US aircraft were already flying to intercept a Chinese ship in the Middle East. The report claimed that the ship was carrying nuclear weapons components — it turned out to be completely false, the product of a model error that no one verified before it reached the operational command level.

The case was reported by CNN on September 18, citing four sources. SOCPAC and the Pentagon declined to comment, even as Defense Secretary Pete Hegseth signed the Artificial Intelligence Acceleration Strategy that accelerates the integration of productive AI into the Army's operational workflows.

See also our report on Claude Opus 5 who hacked OpenAI in 72 hours, which shows on the other hand how quickly modern models are changing the landscape.

The AI ​​hallucination chain: how the chatbot worked technically

The SOCPAC analyst asked an AI chatbot to process material related to the manifest of a Chinese ship in the Middle East. The tool intersected two fundamentally incompatible types of data: open source (OSINT) and classified signals intelligence (SIGINT) reports from government systems. The very attempt to fuse OSINT with SIGINT by a generative AI tool without a controlled source of truth is itself a high-risk architectural choice.

• Query 1: the analyst asked the chatbot to analyze the manifest, providing both open data and classified information.
• Fusion step: the model combined the two streams without reference to the original source of each claim — there was no source attribution in the output.
• Hallucinated conclusion: the chatbot concluded that the cargo included components of a nuclear program — a recording without real evidence.
• Query 2: the same analyst returned the false conclusion to the chatbot to be formatted into a standard intelligence report.
• Dissemination: the report was circulated in a chain of command in the familiar official format, falsely reinforcing the documentation.

Why the "official-looking summary" surpassed every filter

The crux of the AI ​​hallucination isn’t just that the model was wrong — it’s that the error traveled invisibly through a familiar document. The chatbot’s second use of formatting produced text that looked exactly like the product of a human analyst, with the language, structure, and punctuation that an officer would expect. The disguise as formal flow — what the English-speaking world calls anthropomorphic laundering — bypassed human filters of skepticism far more effectively than a clumsy text.

In this particular scenario, no technical controls are mentioned: no RAG with source citation, no fact-check layer, no AI-generated content watermark, no audit trail indicating the percentage of AI-generated content. A system like RAG-V , which aspires to reduce the hallucination from ~10% to 0.1%, had no place in the flow. The report reached the officers as a typical analyst document.

Which chatbot? The "unknown tool" we'll see again

The journalistic report itself leaves the most essential question open: is named . CNN explicitly notes that it was not clear whether this is a commercially available chatbot or a tool developed internally by the Ministry of Defense. Strong suspicions fall on one of the tools approved for use in NIPRNet or SIPRNet, such as CamoGPT or NIPRGPT, but without any confirmation from the services.

• CamoGPT — Army GenAI tool based on Mistral Large 2 with 123 billion parameters, model-agnostic, with enterprise-level RAG and optional connection to GPT-4 via Azure OpenAI, available on NIPRNet and SIPRNet.
• NIPRGPT — Air Force Research Laboratory platform designed for non-classified work on NIPRNet.
• Task Force Lima — Pentagon initiative for controlled adoption of productive AI, with specialized anticipation of hallucination risks.
• Impact Level 5 CUI — the rating level at which tools like CamoGPT are allowed to handle Controlled Unclassified Information.
• Unidentified commercial chatbot — a possibility that would be considered a serious violation of classified data handling policies if confirmed.

The critical problem: OSINT + SIGINT in a single model

Concurrently feeding open and classified sources into an LLM is not just an information security issue — it is a fundamental architectural risk to the very quality of inference. When the model combines two types of data without a clear provenance chain, statistical correlation replaces logical causation. See also our recent analysis of industrial-scale AI model abuse documented by Anthropic, which shows how the same technical weaknesses become attack surfaces when models are put into operational use.

In practice, an LLM that is asked to “compose” a ship manifest does not do research — it does statistical pattern completion. If its training corpus contains examples of Chinese ships that have been linked to nuclear programs in the past (e.g. Iranian ones), the model will tend to produce similar conclusions whenever the combination of words in the prompts is similar. SIGINT pieces do not provide ground truth — they provide additional statistical signals that the model “interprets” with the same probabilistic logic.

This architecture reproduces and highlights the very classic pathology of confirmation bias: when the analyst is looking for confirmation of a pre-existing hypothesis, the unverified AI chatbot is the ideal amplifier. The fact that the same analyst reused the model to format the false conclusion in a formal report means that the tool “legitimized” its own hallucination — a classic closed-loop AI failure mode, where the AI ​​hallucination is self-validated through its formatting.

The discovery: why they examined it a few hours ago

CNN reports that officers reviewed the report “in greater depth” shortly before the operation began. The exact mechanism of discovery is not disclosed — no specific inspection process is mentioned, nor is it what triggered the alert. The assessment of the SecNews technical team, combined with public evidence, is that it likely preceded a corroboration failure: when an officer asked for the source of the nuclear claim or the primary SIGINT excerpts, there was no match in the database.

• Provenance gap: the AI-generated report had no traces back to verifiable SIGINT identifiers.
• Cross-source mismatch: the allegations did not correspond with other standard intelligence flows from other services.
• Cargo specifics: the technical characteristics of the "nuclear equipment" probably did not withstand basic subject-matter expert scrutiny.
• Timing pressure: verification was done only because the operation was approaching, not as part of a standard intelligence review flow.

What should have been done: multi-layered technical defense

The case describes with near-laboratory precision how a flawed AI deployment bypasses human filters. The SecNews technical team identifies six architectural choices that could have stopped the chain:

1. Domain separation: separate models for NIPRNet and SIPRNet, with OSINT–SIGINT fusion prohibited in a single prompt.
2. Mandatory RAG with source pinning: each assertion in the output is accompanied by a verifiable pointer to the original source.
3. AI-generated content watermarking: each generated document is marked at the metadata level and visibly in the text itself as AI-assisted.
4. Human-in-the-loop with two-analyst rule: no intelligence product with an AI source is released without verification by a second analyst who did not use AI.
5. Structured output validation: automatic check that reported SIGINT identifiers correspond to actual records before any distribution.
6. Continuous audit trail: log of all AI queries with raw prompts and outputs, so that post-hoc auditing can assign responsibilities.

Frequently asked questions

Which specific chatbot was used? It has not been publicly named. CNN reports that it has not been confirmed whether it is a commercial or government tool. Possible candidates include CamoGPT, NIPRGPT , or another tool on NIPRNet or SIPRNet.

What does AI hallucination technically mean in this context? LLM produced a claim (a payload of nuclear components) that did not exist in any real data source — it simply combined statistical patterns from the training data with the input data, without reality checking.

Could it happen with carefully designed AI? Much less likely. Techniques like RAG with strict source citation, structured output validation, and mandatory human-in-the-loop with two-analyst rule dramatically reduce the chance of a false conclusion traveling all the way to operational order.

What does it mean for Greek and European public administration?

In Greece and the EU, the same pattern can already be seen in flows where public organizations use productive AI for rapid document synthesis — from legal notices to operational analysis reports. AI hallucination does not distinguish between military and civilian use: whenever a model is called upon to combine incompatible data types without a clear chain of origin, the risk is structurally the same. The imperative is explicit rules for audit trails, mandatory reviews, and clear labeling of AI-generated sections in every official document.

The incident is not an isolated incident — it is indicative of the current reality for integrating productive AI into high-stakes workflows. As the speed of adoption outpaces the maturity of technical controls, the next incident is not a matter of “if” but “when” — and next time they may not stop the business in time. Also see our analysis of using ChatGPT safely without leaking personal data, as an example of how fundamental controls for using AI apply at both the business and individual levels.