Last week, Apple released a detailed review of Vision Pro and visionOS data privacy.
While it was arguably something the company could have made available at launch, it helps explain exactly what the spatial computer collects from our environment and sends to third-party apps and more.
Read more: Vision Pro now has the TikTok app built-in
Privacy, shmivacy: Why is it important?
Concern is growing among security about the development of mixed reality technology, such as the Apple Vision Pro, due to the potential privacy risks it may pose. While consumers often focus on the price of these devices, security experts are concerned about potential privacy breaches. The device is equipped with multiple sensors and cameras, which can collect sensitive data, from users' physical location to their health information. This can lead to problems such as targeted advertising, behavioral profiling, and even the disclosure of sensitive medical information.
Privacy details of the new Vision Pro environment
Although seemingly real, the environments in Apple Vision Pro are created using a combination of camera and LiDAR data to provide a near-real-time view of the user’s space. In addition, visionOS uses audio ray tracing to simulate the behavior of sound waves as they interact with objects and surfaces. Apps overlay these scenes or, in some cases, create their own environments.
With the release of the new Vision Pro privacy overview, we now have a better understanding of what environmental data is being sent and shared with apps from the headset.
Plane Analysis: Detects flat surfaces near which virtual 3D objects, also known as “Volumes” by Apple, can be placed, allowing users to interact with virtual objects within their physical environment.
Scene reconstruction: Scene reconstruction involves creating a polygonal mesh that accurately represents the outline of objects within the user's physical space. This mesh helps virtual objects align correctly with physical objects in the user.
Image anchoring: This feature ensures that virtual objects remain anchored in their intended positions relative to real-world objects, even when the user moves. WSJ’s Joanna Stern demonstrated this technology early on in a video posted to X, where she appears to place multiple timers over objects boiling on a stove.
See related: Apple Vision Pro: Is it better to wait for the 4th generation model?
Object Recognition: Apple says it uses technology to detect “objects of interest in your environment.” In general, this technology is used by Vision Pro to identify objects in your space.
By default, apps on Vision Pro can’t access ambient data. However, for more realistic experiences, third-party developers may need to access this data. This process is similar to granting permission to access Photos or Camera on an iPhone. A full space on Vision Pro can be used to leverage ambient data and deliver experiences.
For example, Encounter Dinosaurs asks for permission to access your environment so that “dinosaurs” can invade your physical space. By granting access to ambient data through an app, the app can create a map of your world using a scene grid, detect objects in your environment, and determine the location of specific objects in your environment, Apple says.
See also: How to turn Apple Vision Pro on and off?
However, any app that accesses information about your environment can only do so within five meters of your location. This is because immersion data, such as shadows and reflective areas, are no longer visible beyond that distance.
Source: 9to5mac.com
