Cybersecurity researchers have uncovered a serious security flaw in the Vanna.AI library, which can be exploited for remote code execution via injection techniques.
The vulnerability, which is recorded as CVE-2024-5565 (CVSS score: 8.1), is related to direct injection into the “ask” function, allowing the execution of arbitrary commands, according to security firm JFrog.
Read about: Fortra FileCatalyst Workflow: PoC exploit for SQLi vulnerability
Vanna is a Python- based machine learning library that allows users to communicate with the SQL database to gather information through queries (also known as prompts) that are translated into equivalent SQL queries using a large language model (LLM) .
The rapid development of genetic artificial intelligence (AI) models in recent years has highlighted the risks of exploitation by malicious actors, who can bypass security mechanisms with harmful techniques.
A prominent category of these attacks is injection, a type of AI jailbreak that bypasses the protective measures of LLM providers, allowing the production of offensive, harmful, or illegal content or instructions that violate the intended purpose of the application
These attacks can be indirect, such as when a system processes third-party data (e.g., incoming emails or editable documents) to trigger a malicious payload that leads to an AI jailbreak. They can also appear as a many-shot jailbreak or multi-turn jailbreak (also known as Crescendo), where the operator starts with harmless dialogue and gradually steers the conversation towards the forbidden target.
See also: Botnet exploits vulnerability in Zyxel NAS devices
This approach can be further extended to perform a new jailbreak attack, known as Skeleton Key.
“This AI jailbreak technique works through a multi-turn strategy to force the model to ignore its guardrails,” said Mark Russinovich, chief technology officer at Microsoft Azure. “Once the guardrails are ignored, the model cannot distinguish malicious or unauthorized requests from others.”
Skeleton Key differs from Crescendo in that, when the jailbreak is successful and the system rules are changed, the model can generate answers to questions that would otherwise be prohibited, regardless of the ethical and security risks.
"When the Skeleton Key jailbreak is successful, the model understands that its instructions have been updated and will comply with the new instructions for producing any content, regardless of how much it violates the AI's original responsible instructions," Russinovich added.
"Unlike other jailbreaks like Crescendo, where models must be queried indirectly or through code, Skeleton Key puts models into operation where the user can directly request tasks. Furthermore, the model's output appears completely unfiltered, revealing the full extent of its knowledge or ability to produce the requested content."
Read also: Boolk hackers distribute new BMANAGER malware
🔒 Protect your privacy with Proton VPN
Swiss VPN from the creators of Proton Mail — strict no-logs policy, strong encryption, and built-in NetShield that blocks ads, trackers, & malware.
- ✔ No-logs, based in Switzerland (except 14-Eyes)
- ✔ NetShield: blocks ads, trackers & malicious domains
- ✔ Covers all devices — free version available
The link is an affiliate link — SecNews may receive a commission at no additional cost to you. It does not affect the independence of our article writing.
The latest findings from JFrog, also independently revealed by Tong Liu, show how early injections can have serious consequences, particularly when linked to command execution.
CVE-2024-5565 exploits the fact that Vanna facilitates text-to-SQL generation to create SQL queries, which are then executed and graphically presented to users via the Plotly graphing library.
This is achieved through an “ask” function – e.g., vn.ask (“Who are the top 10 customers by sales?”) – which is one of the key API endpoints for creating and executing SQL queries against the database.
This tactic, combined with Plotly's dynamic code generation, creates a security hole that allows a threat actor to submit a specially crafted prompt with an embedded command, which will be executed on the system.
“The Vanna library uses a prompt function to visualize results to users. However, it is possible to modify the prompt via direct injection, allowing arbitrary Python code to be executed instead of the programmed data visualization,” JFrog said.
“Specifically, allowing external input to the library's 'request' method with 'visualization' set to True (default behavior) leads to remote code execution.”
Following a responsible disclosure, Vanna issued a detailed guide, warning users that the Plotly integration can be exploited to generate arbitrary Python code. It is recommended that those who exploit this feature do so in a sandbox, for security reasons.
“This discovery demonstrates that the risks of widespread use of GenAI/LLM without proper governance and security can have dramatic implications for organizations,” said Shachar Menashe, senior director of security research at JFrog.
See more: Azure Service Tags flagged as a security risk
“The dangers of direct injection are not yet widely known, but they are easy to implement. Companies should not consider pre-prompting as a foolproof defense mechanism. Instead, they should implement more robust mechanisms when connecting large language models (LLMs) to critical resources, such as databases or dynamic code generation.
Source: thehackernews
