Cybersecurity researchers have discovered an indirect prompt injection vulnerability in artificial intelligence assistant , which could allow attackers to intercept source code and inject untrusted HTML into its responses, which could be used to redirect victims to malicious websites.
See also: GitLab vulnerabilities allow bypass of security checks
GitLab Duo is an artificial intelligence (AI) programming assistant that helps users write, review, and edit code. It is based on Anthropic 's Claude models and was first introduced in June 2023.
However, as Legit Security found , GitLab Duo Chat had an indirect prompt injection vulnerability, which allows attackers to “ steal source code from private projects, modify code suggestions displayed to other users, and even exploit confidential, undisclosed zero-day vulnerabilities .”
Prompt injection refers to a class of vulnerabilities common in systems that allow malicious users to manipulate large language models (LLMs) to give unwanted or misleading responses.
Indirect prompting is more complex, as the malicious instructions are not given directly, but are embedded in another context, such as a document or web page, which the model is asked to process.
See also: GitLab fixes HTML Injection flaw leading to XSS attacks
Recent studies have shown that large language models (LLMs) are also vulnerable to “jailbreak” attack techniques, which allow AI chatbots to be tricked into producing harmful or illegal information, bypassing their ethical and operational constraints — essentially making complexly worded prompts unnecessary.
Additionally, prompt leakage (PLeak) methods can be used to inadvertently reveal the predefined system commands or instructions that the model is intended to follow.
The latest findings from the Israeli cybersecurity in the software supply chain sector show that a hidden comment, placed anywhere in merge requests, commit messages, issue descriptions or comments, as well as in the source code, was enough to leak sensitive data or embed HTML in GitLab Duo responses.
After responsible notification on February 12, 2025, the issues were addressed by GitLab.
See also: New critical GitLab vulnerability allows arbitrary execution of CI/CD pipelines
As LLMs (large language models) take on an increasingly active role in the software lifecycle, it is critical for organizations to adopt secure-by-design practices, identify potential sources of malicious content, and strengthen input filtering and control mechanisms. In addition, continuous monitoring and evaluation of the model's interactions with user data and code environments is essential to prevent leaks of sensitive information.
Source: thehackernews
🔒 Protect your privacy with Proton VPN
Swiss VPN from the creators of Proton Mail — strict no-logs policy, strong encryption, and built-in NetShield that blocks ads, trackers, & malware.
- ✔ No-logs, based in Switzerland (except 14-Eyes)
- ✔ NetShield: blocks ads, trackers & malicious domains
- ✔ Covers all devices — free version available
The link is an affiliate link — SecNews may receive a commission at no additional cost to you. It does not affect the independence of our article writing.
