VMware observed performance drops in ESXi VMs on Linux kernel 5.19 due to the Retbleed security mitigation.
VMware warns that ESXi VMs running on Linux kernel 5.19 may experience a performance drop of up to 70% when Retbleed mitigations are enabled compared to Linux kernel version 5.18 .
Specifically, VMware observed regressions in ESXi virtual machines of up to 70% in compute, 30% in networking, and 13% in storage.
From VMware's testing, it became clear that the sudden and very significant degradation was caused by the introduction of measures to mitigate the "Retbleed" vulnerability.
“After the split between kernel 5.18 and 5.19, we identified the root cause as the activation of the IBRS for the spectre_v2 with commit 6ad0ad2bf8a6 (“x86/bugs: Intel retbleed vulnerability report),” VMware explains.
See also: Retbleed speculative execution attack bypasses Retpoline solution
VMware found that disabling the Retbleed via the kernel boot parameter “spectre_v2=off” restored Linux VM performance to version 5.18 levels, confirming that the fixes are the sole reason behind the performance drop.
However, disabling mitigation would be considered a security risk, as systems would be vulnerable to cyberattacks on certain CPU.
Retbleed is a speculative execution attack discovered in July 2022 that can exploit return instructions in the CPU to extract sensitive information.
Examples of data that Retbleed can leak include items contained in kernel memory, such as root password hashes , as shown in the video below.
Speculative execution is a feature that improves performance in modern processors by allowing CPUs to perform calculations before they are requested, reducing the time required for their completion.
This performance-enhancing capability has negative consequences from a security perspective because it enables side-channel attacks.
A notable case of this is Spectre, which was mitigated with the Retpoline, a software-based solution that had minimal impact on performance.
Retbleed ,in fact, is not only a bypass of the Retpoline fix, but it abuses the mitigation by targeting return instructions to insert branch targets into the kernel address space.
See also: Quantum ransomware: Attack affects 657 healthcare organizations
Unfortunately, Linux's mitigation of Retbleed in kernel version 5.19 had a detrimental effect on performance, which could lead to a wide range of business issues in production systems and cloud infrastructures.
Retbleed affects Intel Core processors from the 6th generation (Skylake – 2015) to the 8th (Coffee Lake – 2017) and AMD Zen 1, Zen 1+ and Zen 2 processors released between 2017 and 2019, which are still ubiquitous in server systems.
With such a performance drop, many system admins who believe that Retbleed is more of a theoretical than a real threat to their systems will be open to taking a risk by disabling mitigations.
At present, the Linux kernel development team has not discussed the huge performance impact nor promised to review the mitigations and implement a better solution, so the situation remains dangerous.
Source: bleepingcomputer.com
