As technology continues to evolve at a staggering pace, Apple's latest innovation, the Vision Pro, emerges as a beacon of advancement in the realm of immersive computing. Drawing parallels to the futuristic landscapes portrayed in "Ready Player One," the Vision Pro promises to blur the lines between reality and virtuality, ushering in a new era of interactive experiences. With its sleek design and cutting-edge features the Vision Pro has captured the attention of immersive computing enthusiasts the world over.
The following is insight based on more then 336 hours of experience with the Apple Vision Pro. At no point did I consider returning it. There is a lot of promise, but as a first iteration, there are obvious flaws.
The headset features a high-resolution stereoscopic display with pancake lenses that provide a wide field of view. It uses eye tracking and hand gestures to allow users to control the device without any controllers. The onboard M2 chip provides powerful processing capabilities
Using the Vision Pro takes some getting used to: finding a comfortable position for your hand that doubles as the selector, adjusting the straps so it doesn't feel like a tablet is hanging off your face (which it is), and for those like me, learning to look directly at what you're clicking on. Inside the immersive Vision Pro interface, the eyes serve as the pointer, trackball, and selector. Essentially, your eyes look at an item, such as a menu bar, and then you use a single hand to pinch the thumb and index finger to click. This mechanism often leads to amusement for onlookers, as users instinctively reach out to touch or poke the object, accustomed to physical interaction being the primary interface for computing.
With the mouse out of the way, attention turns to the GUI. One of the most impressive features of the Vision Pro is its ability to map the user's environment in 3D. This allows digital objects to be anchored in space and interact with real objects. Users can collaborate with others in shared virtual spaces or even have virtual meetings with realistic avatars. Application windows remain anchored within a physical room. Imagine taking a browser window, like Safari, and placing it over your TV.
My office area is set up with multiple windows allowing more monitors and work area then would be physically available on a typical laptop. The screenshot here illustrates these static windows as they are laid out in my office. Since they are fixed in a set location, I can move into my living room, which has a different set of windows. This is where the Vision Pro's spatial computing truly shines.
The dining room has two PDF books open, and the living room serves as the entertainment center for watching movies.
Once you become better acquainted with the new interface, it's time to introduce the game changer: a keyboard. Yes, you can use a Bluetooth keyboard with the Vision Pro, and for productivity, this is a must. To provide some context, it takes me approximately 80 seconds to search for "Red team" in a Safari browser without a keyboard, compared to just 18 seconds with one. This is because I've come to realize I seldom look at what I'm clicking on. Additionally, a keyboard is necessary for better interaction with emulation tools like Kasm and Vmware, which will be covered later.
Speaking of Bluetooth, being in the Apple ecosystem means AirPods, Magic Keyboards, and other iOS devices work seamlessly. However, devices such as Bose QuietComfort headphones require an additional step to connect. As of writing this, they can be paired as a hearing aid in the accessibility settings: Settings > Accessibility > Hearing Aid.
While third-party peripherals won't provide the spatial immersive sound, they do enhance the overall immersion.
Apple's dedication to providing a personalized auditory experience is evident in their embrace and adaptation of individualized Spatial Audio. Although this concept isn't entirely novel, Apple's distinctive integration of hardware and software creates an unparalleled experience. The landscape of spatial audio undergoes a slight transformation with Apple's approach. Instead of relying on external devices such as the iPhone for tasks like head tracking and audio customization, the Apple Vision Pro utilizes its technical capabilities.
This built-in, on-device tracking leads to a more immersive and precise spatial audio experience, particularly considering the proximity of the tracking sensors to the user's ears and eyes. Unlike speakers that encase the ears or utilize bone conduction technology, Apple's implementation results in audio leakage. If you prefer a more intimate audio session, you may find it necessary to pair with AirPods. The proximity tracking brings a particular issue if you're pacing while listening to a WebEx, the speaker will remain fixed in their location. That fixed speaker application can be virtually placed in another physical room through the walls muffling the sound.
In full immersion mode with the headset properly fitted I was able to work 11 hours comfortably. However, with a battery limited to around two hours, the vision had to be connected to an outlet reducing some of the more expansive multitasking. Apple provided several settings to be immersed in, the mountains, the moon, and Joshua tree national park with more coming soon. In this environment it becomes easy to lose track of time, making the automatically synchronized sunsetting essential to knowing when it is in the physical world.
The Apple Vision pro is missing one key tool that most hacker, coders, and IT enthusiast cant live without ... virtualization. If you are also in this boat fear not, we found a couple work arounds to get the most out of virtualization while your using the Vision Pro. [Remote servers]
Emulation tools provide the ability to work on other operating systems, VPN access, and attack boxes. What follows are some of those emulation tools used:
Kasm Workspaces - a docker container streaming platform that enables the delivery of browser-based desktops, applications, and web services. Getting the software installed is fairly straight forward: create an AWS S3 bucket or GCP server (Free trial level), make sure to set at least 30GB for workspaces, install Kasm Workspaces following the GitHub instructions, and finally start the server. Kasm has many useful docker images in the catalog, like vscode, Kali, and parrot. Custom docker images can also be created and streamed.
Tailscale - a mesh VPN (Virtual Private Network) service designed to provide secure network connections between devices and networks. It allows users to easily set up secure connections between their devices, regardless of their location, using a decentralized approach.
Much like the OASIS in "Ready Player One," the Vision Pro transcends the limitations of traditional computing by offering users a seamless blend of virtual and physical realities. Equipped with state-of-the-art augmented reality (AR) capabilities and a high-resolution display, the Vision Pro transports users to immersive virtual environments with unparalleled fidelity. Whether it's pentesting a website or collaborating in virtual meetings, the Vision Pro empowers users to navigate a world where the boundaries between real and virtual are increasingly blurred. Unlike the dystopian future depicted in the film, where humanity seeks refuge in virtual reality, the Vision Pro remains firmly grounded in enhancing real-world experiences. Allowing users to seamlessly transition between virtual and physical environments, ensuring that productivity and connectivity are never compromised.
It's the movie presentation you read most about, especially Disney's immersive theater bringing you to places like the Avengers Tower to watch movies.
Movie's may get the attention of most reviewers and it does make air travel entertaining. Being able to play battleship virtually with a friend complete with flying missiles is another fun feature and a place to become more familiar with the environment.
Apple released other games designed for the device among the most fun is Builder Journey a Lego based game.
PowerPoint is the standard for creating presentations along with the coach feature this software has helped my talks, but now take a look at Keynote. Apple Keynote brings you to a theater, or a board room to practice presenting. In time this could be like going to a Toastmasters meeting to present a speech improving the quality of talks immensely.
Not all of Apple's massive library of Ipad applications have been ported to spatial computing but still work well. Discord is one such application, though it can be difficult to use with eye tracking, a keyboard improves the useability.
Apple created Personas which is what they are calling the avatars used to represent the user. It is a near cartoonish visualization with a rendered tongue, very accurate expressions, a bit of creepiness. You can see that the initial setup requires the best lighting to get a better rendering.
Looking ahead, there are several areas where the Vision Pro could undergo enhancements to better serve a wider audience
Cost: Exploring cost-effective deployment options and subscription models to make the Vision Pro more accessible to a wider audience.
Performance Tuning: Optimizing hardware to allow local virtualization will remove the need to use a browser to access other operating systems.
Content Development: Improved applications with better spatial computing interactions will make multitasking much easier. Virtualization will eliminate the need for external services and using a browser to access other operating systems.
Weight Reduction: Currently the device weighs as much as an Ipad and is not comfortable after hours of working.As components use lighter materials so to does the device weigh less.
Battery Improvements: Battery Improvements are one of the most common issues with all devices. The Vision Pro battery currently allows me to operate about 2 hours before requiring a recharge.
Light Sensitivity: Improvements on the camera may help with how well the vision pro works in low light, continuous errors are displayed when the lighting is dimmed. This is inconvenient in places like an airplane when the captain dims the lights.
.png)