Data poisoning threatens the integrity of Artificial Intelligence (AI) models. For CISOs, AI rarely reduces complexity, but rather adds to their already busy agenda. In addition to traditional security priorities, they must now address new AI-related risks, such as when AI solutions are used unsupervised for business purposes, models are manipulated, and new regulations are not adhered to.
See also: Nova Sonic: Amazon's new AI voice model
One of the most pressing challenges is shadow AI: the use of AI tools and models without oversight from IT or security. Just as shadow IT creates blind spots in enterprise environments, shadow AI brings unsupervised risks, such as data leaks, insecure integrations, and threats from unverified third-party providers and services.
Even if CISOs have transparency about their use of AI and know what data is being used, which AI models are being used for what purpose, they still face a wide range of threats. A central issue is where these models are hosted. Most are cloud-based and often publicly accessible.
This creates new attack surfaces, such as command injection, data poisoning, and adversarial attacks. Even without direct manipulation, Large Language Models (LLMs) are vulnerable to illusions and often produce inaccurate or misleading results.
Data poisoning – both in training data and in AI models themselves – has increased. Many people still don’t have a clear understanding of the implications of this problem. This can be compared to a self-driving car that has been trained on poisoned data and therefore misinterprets a stop sign as a speed limit sign. This could lead the vehicle to accelerate instead of stopping, thereby endangering the lives of passengers, pedestrians and other road users.
See also: Vulnerabilities found in open-source AI and ML models
This critical threat has gone largely unnoticed for a long time, but it could be one of the most dangerous threats in the AI landscape. In a world where AI is increasingly influencing decisions that affect people’s lives, the lack of attention to this issue is extremely concerning.
Expert William L. Scherlis from Carnegie Mellon University defines the threat of integrity in the context of AI as follows: “In the context of modern AI based on neural networks, including Machine Learning (ML) and genetic AI, integrity risks refer to the potential for attacks that lead to erroneous results.”
Or more simply: Even a small hole can sink an entire ship. Manipulated AI models can lead to incorrect assumptions and decisions, and even the deterioration of the mental health of their users.
Incorrect results from LLMs and machine learning can also lead to financial losses. To properly train a model, huge amounts of data are required. On the other hand, only a few datasets are enough to manipulate large language models. Research shows that as little as 0.001 percent of corrupt data introduced into the training data of an AI model is enough to cause incorrect results.
See also: Opera launches paid AI browser Neon
To prevent integrity attacks, CISOs must incorporate the principles of Secure Programming and Security by Design early in AI software development projects. This includes not only checks to ensure compliance with GDPR, the EU AI Act, and ISO/IEC 42001. Measures must also be taken to prevent model drift and maintain model reliability over time. Equally important are the requirements for the transparency, explainability, and interpretability of AI models. Only in this way can the decisions of AI systems be understood and justified.
🔒 Protect your privacy with Proton VPN
Swiss VPN from the creators of Proton Mail — strict no-logs policy, strong encryption, and built-in NetShield that blocks ads, trackers, & malware.
- ✔ No-logs, based in Switzerland (except 14-Eyes)
- ✔ NetShield: blocks ads, trackers & malicious domains
- ✔ Covers all devices — free version available
The link is an affiliate link — SecNews may receive a commission at no additional cost to you. It does not affect the independence of our article writing.
