<aside> đź’ˇ This is an in-depth case study for the design of Project Crane, design language for hand interactions in XR
For more details and info, you can contact me at [email protected]
To read this case study in dark mode, press shift+cmd(ctrl)+L
</aside>
Table of Content
What are the best input devices for future XR devices? Turns out for the era of the phone, it is your hand. What if your hand is still the best controller in this altered reality? How should the interfaces and interactions be evolved to accommodate our hands?
Let’s be honest: people get confused when they put on a headset. Ironically, many of the current VR/AR interfaces are actually re-using a lot of traditional GUI elements.
We have already used them for years, and still get confused when we see them again in a headset, with a big and bulky controller. Although it is not the end of the day using 2D/2.5D interfaces in XR, they are not the best practice for the “end game”. Using a pointer to aim at a virtual screen several meters from you can be exhausting. To design for a volumetric medium, we have to look at existing volumetric interaction patterns.
As a VR headset evangelist, I have seen this behavior on so many of the first-time users: when they see those virtual 6dof objects in VR, all of them - like no exception - try to use their hands to touch things and forget about the controllers in their hands.
This is actually an attempt to perform volumetric interaction.
However due to the current tech limit, usually the virtual world doesn’t respond well to such inputs. Instead, we’ve been given controller tutorials immediately and forced to learn how to use them to mimic our hands.
What if, on the boarding of a headset, the world can respond to your baby grab? And maybe even use it as the main interaction pattern of the whole system? Wouldn’t it be amazing?
It has been proved that for mobile devices, our hands are the best input devices in most cases. What if it is also true for XR?
So how will these fancy mysterious volumetric interactions be like? It turns out, we are already doing them every day. To open a drawer, to unfold a paper, to browse through vinyl collections in a record store.
These interactions can be found in classic industrial design objects. All of these handles, bars, stacks are “inviting” us to use them. Our body reacts naturally to their invitation without going through any user manual.