The security that accompany the Pickle format have come back to the fore with the discovery of a new “hybrid machine learning (ML) model exploit technique” called Sleepy Pickle.
According to Trail of Bits, the attack method leverages the ubiquitous format used to package and distribute machine learning (ML) models in order to destroy the model itself, posing serious risks to the supply chain of an organization's customers.
See also: Malware campaign targets Windows, Android and macOS
"Sleepy Pickle is a stealthy and innovative attack technique that targets the ML model itself rather than the underlying system," said security researcher Boyan Milanov.
Although pickle is a widely used serialization format by ML libraries like PyTorch, it can be used to execute arbitrary code simply by loading a pickle file (i.e., during deserialization).
“We recommend loading models from trusted users and organizations, relying on signed grants and/or loading models from [TensorFlow] or Jax formats with the automatic conversion mechanism from_tf=True,” Hugging Face in its statement.
Sleepy Pickle works by injecting a payload into a pickle file using open source tools like Fickling and then delivering it to a target host via one of four techniques: adversary-in-the-middle (AitM) attack, phishing, supply chain compromise, or exploiting a system vulnerability.
"When the file is deserialized on the victim's system, the payload executes and modifies the model, introducing backdoors, inspecting the data, or corrupting the processed data before returning it to the user," Milanov said.
See also: Distribution of SSLoad malware via PhantomLoader loader
In other words, the payload inserted into the pickle file containing the serial ML model can be abused, altering the behavior of the model, violating its weights or the input and output data it processes.
In a hypothetical attack scenario, this approach could be used to create harmful results or misinformation that could have devastating consequences for user safety (e.g., urging them to drink bleach to treat the flu), steal user data under certain conditions, and indirectly attack users through manipulated news article summaries with links leading to phishing pages.
Trail of Bits stated that Sleepy Pickle can be used by threat actors to maintain covert access to ML systems in a way that evades detection, since the model is compromised when the pickle file is loaded into the Python process.
Read more: Life360 reports extortion attempt after Tile data breach
This is also more effective than directly uploading a malicious model to Hugging Face, as it can dynamically modify the model's behavior or output without having to entice targets to download and execute them.
🔒 Protect your privacy with Proton VPN
Swiss VPN from the creators of Proton Mail — strict no-logs policy, strong encryption, and built-in NetShield that blocks ads, trackers, & malware.
- ✔ No-logs, based in Switzerland (except 14-Eyes)
- ✔ NetShield: blocks ads, trackers & malicious domains
- ✔ Covers all devices — free version available
The link is an affiliate link — SecNews may receive a commission at no additional cost to you. It does not affect the independence of our article writing.
“With Sleepy Pickle, hackers can create pickle files that are not ML models, but can destroy local models if they are loaded together,” Milanov said. “The attack surface is therefore much broader, because control over any pickle file in the target organization’s supply chain is enough to attack their models.”
“Sleepy Pickle demonstrates that advanced model-level attacks can exploit lower-level supply chain vulnerabilities, through the connections between underlying software and the final application.”
Read more: Warmcookie malware is promoted through fake job offers
Source: thehackernews
