Envisioning VisionOS Future: My Top Picks of What I Hope Apple Unveils at This Week's WWDC Event

Essential Strategies for Developing Visually Stunning Applications in the Era of Apple’s VisionOS - Insights

Jason Hiner/ZDNET

As the calendar turns from 2023 to 2024, Apple is getting closer to releasing its shockingly expensive and entirely experimental Vision Pro headset . Just this week, Apple announced that the iPhone 15 Pro and Pro Max can capture spatial video for the Vision Pro .

You could say that Apple’s vision of artificial reality is getting closer to real reality. And yes, I am pretty proud of that sentence.

Also: I tried Apple Vision Pro and it’s far ahead of where I expected

Recently, Apple also started to release design guidelines for developers. One that caught my eye is the company’s “Q&A: Spatial design for visionOS .” This document provides some best practices for developers creating apps for VisionOS.

Since VisionOS is a very different beast from all of the other Apple OS implementations, particularly in the user interface department, I thought it would be interesting to share with you some of those best practice recommendations.

Newsletters

ZDNET Tech Today

ZDNET’s Tech Today newsletter is a daily briefing of the newest, most talked about stories, five days a week.

See all

That way, if you’re a developer, you can start thinking about how to build your apps. And if you’re a user considering purchasing a set of these solid gold goggles, you’ll have a better feel for how designers are constructing their apps and what your experience might be.

Let’s dig in, shall we?

Disclaimer: This post includes affiliate links

If you click on a link and make a purchase, I may receive a commission at no extra cost to you.

1. Gradual immersion

Apple recommends you gradually introduce users to your app, starting with a window view within the augmented reality environment.

That way, they’re not suddenly dropped into a whole new world, but they can slowly get their “VR legs” and feel they have some level of control over their immersion level, at least until they get comfortable with their environment.

Also: Two breakthroughs made 2023 tech’s most innovative year in over a decade

2. Identify your key moments

Let’s say you’re a developer who has an existing app for the iPad. Let’s say you want to bring said iPad app into VisionOS. The iPad app, by its nature, is flat and rectangular. But VisionOS allows users to see all around. Is there anything in your app that clearly lends itself to VRification?

Also: Inside VisionOS: 17 things developers need to know right now

Apple uses its Photos app as an example. A key moment for Apple’s Photos app might be showing a panoramic photo by wrapping it around the user completely. Panoramas are a long-standing feature of the Photos app, but spatially wrapping the pano is something that would showcase the mixed-reality environment.

In a writing app, VRification might involve going from a typical windowed user interface for most writing to a focus mode that shuts out the outside world and lets your users concentrate, by providing a calming background and perhaps some calming music.

3. Adjust UI elements to accommodate the new environment

Some UI elements translate directly from iPadOS to VisionOS, but others aren’t an exact match. Apple has a spatial design guidelines video that’s well worth watching to come up to speed.

Also: Would you believe a VR headset outsold AirPods during Black Friday? It happened

One area to pay particular attention to is when you use hover effects. When you hover over a button in a flat UI, you just need to see some highlighting. But when you hover over – or in front of – something in VR space, you’ve got the entire room to work in. So pay attention to how things look and feel, and try to avoid inducing vertigo by rapidly moving objects and buttons.

4. Utilize grid systems where appropriate

Apple says that apps designed with the iPadOS grid system will translate very nicely to the Vision Pro. The grid works well for designing windows. Point sizes translate well from the iPad to Vision Pro.

But as you deal with more 3D-centric objects, grid systems break down. Once you’re moving in 3D space, be careful about where you place your controls. For example, it can be somewhat confusing if you have nearby controls for a faraway object.

5. Incorporate sound design strategically

Sound – that is, audio -- design plays a crucial role in spatial computing. Sound may not be as critical for flat UI experiences, but we humans rely on sound – and the spatial awareness that sound provides – to orient ourselves in our real-world environment. So keep in mind that sound not only makes your environment more interesting, it can serve as an orientation tool as well.

To that end, even if you’re building a non-entertainment app that would normally not focus on sound, consider how sound elements, no matter how subtle, might be used for user spatial orientation while in VR space.

Also: Apple’s Vision Pro: A concept prototype with this enormous potential

And while you’re at it, allow your users to control the sound, setting its volume and even turning it off. Give the sound cues that may be helpful, but give users control over how those cues are applied. Here’s a great resource from Apple on sound design .

6. Focus on key details, but don’t overwhelm

Apple recommends always rendering a ground plane so users feel grounded. If you’re putting someone inside a theatre, provide key details (like a stage, lighting, and curtains), but don’t get carried away trying to reproduce the cola stains in the carpet or that unique crackle that comes from walking over scraps of petrified popcorn. If you’re rendering an outdoor landscape, show slowly moving clouds but keep things subtle.

Of course, if you’re putting people into a game where the whole idea is to overwhelm them, then go for it. But Apple is much more interested in using VR and AR as an augmented tool, rather than purely as a gaming platform. And in that context, it’s probably wise to avoid making your customers barf up their lunches inside their $3,500+ head-mounted uncanny valley in a can.

7. Use the VisionOS simulator

There is, in fact, a VisionOS simulator available inside Xcode, Apple’s development environment. Here’s a guide to all of the Xcode simulators . And here’s a guide to interacting with your app inside the Xcode VisionOS simulator.

So there you go. Seven helpful tips for keeping your users’ insides inside.

Also: The iPhone 15 Pro can officially capture spatial video for the Apple Vision Pro

What do you think? Are you building apps for VisionOS? Are you planning on buying a Vision Pro? Let us know in the comments below.

You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter on Substack , and follow me on Twitter at @DavidGewirtz , on Facebook at Facebook.com/DavidGewirtz , on Instagram at Instagram.com/DavidGewirtz , and on YouTube at YouTube.com/DavidGewirtzTV .