The Nightshade “Data Poisoning Tool” is a tool, created by a team at the University of Chicago. It can add undetectable pixels to images to help creators protect their work from AI image generators, effectively poisoning (destroying) the AI’s training data
See more: Generative AI: Will it cause "catastrophic" cyberattacks?
This tool has been developed as a response to large companies such as OpenAI and Meta, which are facing lawsuits for copyright infringement. These lawsuits claim that the corresponding AI tools have stolen personal works of art without the necessary consent.
According to a report by MIT Technology Review, Professor Ben Zhao and his team from the University of Chicago created Nightshade. The Nightshade “Data Poisoning Tool” aims to support creators against Artificial Intelligence (AI) companies that use artists’ work to train AI models without their permission and without compensating them. The tool is currently undergoing peer review, and the team has already tested it on a recent Stable Diffusion and an AI tool that the researchers built from scratch, with very promising results.
The team hopes that the tool, which also leverages Glaze, another tool created by the same team, could be used to secure and protect images/content shared online, in order to prevent potential malicious uses and future breaches by artificial intelligence models, such as DALL-E, Midjourney, and Stable Diffusion.
The «poison» essentially changes the way machine learning tools interpret data from online sources, so that it sees and reproduces something entirely different. The altered pixels are invisible to the human eye, but can be fully controlled when AI research models recognize them.
Using this tool, artists who want to share their work online while still keeping protected can upload to Glaze and allow Nightshade to apply its AI Poison technology. According to Zhao, Nightshade and Glaze will be free to use, with Nightshade open source for additional improvements. If enough people start using it to protect their content from AI models, it will motivate larger companies to recognize and reward original artists.
«The more people use it and create their own versions of it, the more powerful the tool becomes», says Zhao. «Datasets for large artificial intelligence models can consist of billions of images, so the more poisoned images that can be fed into the model, the greater damage AI will cause».
See more: ShellTorch: Vulnerabilities allow code execution attacks on AI servers
“Poisoned data samples can mislead models and train them incorrectly. For example, they can make them believe that images of hats are actually cakes and images of bags are toasters. Poisoned data is very difficult to remove, requiring tech companies to find and delete every corrupted sample.” Nightshade will not only contaminate the word “dog” that has been trained on AI , but also all similar concepts such as “puppy,” “husky,” “greyhound,” and “wolf.”
Zhao acknowledges that there is a possibility that people could misuse the Nightshade “Data Poisoning Tool” for malicious actions, however, they would need thousands of poisoned samples to cause real damage.
Source: petapixel.com
