By Admin User

Summary: Good user experience design is imperative for smoother adoption of Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR). The UX focus of wow-factor experiences for VR/AR/MR must include careful attention to design of presence, narrative, engagement and emotion.    


Virtual Reality (VR) offers an immersive experience requiring a HMD (Head Mounted Display). The software is powered on a mobile phone or from a new VR-optimized PC. Users feel like they are "there" and the experience is in first person. Note: You have to experience this to realize how profound of a shift it is. First person user experience can change perception in fundamentally profound ways.  

Augmented Reality (AR) offers mobile (tablet and phone) experiences that layer interfaces, objects and information onto the real-world. Headset versions for AR or VR/AR blending (switch between modes) are emerging. AR's new cousin definition is "MR". Users augment their existing real world with floating or transparent interfaces. 

Mixed Reality (MR) is a new definition that captures 3D object integration into AR scenes, with a headset. The MR definition is needed since traditional AR overlays using GPS primarily, whereas MR (Microsoft Hololens and now on devices with Google's Project Tango) does motion tracking, spatial depth sending and spatial mapping "to actual scale" eg. edge detection can make a "hologram" appear to fall off an actual table edge or slide down an actual wall. Gestural interactions also become part of the mixed reality experience. 

VR/AR and MR are evolutionary and revolutionary 

In this previous post we discussed evolutionary vs revolutionary design. What makes VR/AR both revolutionary and evolutionary is that it dates back to the near 50 year old vision of modern computing. In 1968, Ivan Sutherland created the first VR hardware, he called "The Sword of Damacles" (see image below). The vision was that computers would help you look beyond (2D) into a virtual world. So VR's potential and modern AR/MR manifestations have always been about extending computing experiences beyond boring screens and into immersive sensory-rich 3D informational environments. Modern AR evolved out of the mobile computing revolution and MR extends it even further. 

Fun fact: At Experience Dynamics we feel a special affinity to this evolution when in our infancy 18 years ago, we started with VR usability and collaborative 3D environments. Like the rest of the 90's VR industry, we went into deep freeze and just started defrosting fuly last year. Experience Dynamics founder, Frank Spillers was active in the VR community in the UK in the 1990's.

VR already comes with wow-factor as the video illustrates above. Early reactions to the new Occulus Rift, Playstation VR and HTC Vive are also generating jaw-dropping experiences. AR is also gaining fast traction with impressive starts from Google's Project Tango and Microsoft's Hololens. Hololens extends AR into the revolutionary new category of Mixed Reality, with the ability to do things like 'holoportation'--3D telepresence in real-time. Other MR headsets will include Magic Leap, Meta 2 and ODG.

First Virtual Reality headset 1965

Image above: Computing pioneer Ivan Sutherland 1965 "Sword of Damacles" first Virtual Reality headset.

UX challenges in VR/AR/MR

VR usability research actually dates back to the early 1990's. These immersive technologies are not without usability issues. Even much of VR film-making, while impressive and impactful, lacks basic VR usability requirements such as immersive narrative

Much is known, but it needs to be updated and translated. For example, simulator sicknesss, while important, is not the driving usability issue for VR today. VR developers are becoming more savvy to reducing Sim Sickness and hardware pushing a required 90 frames-per-second (Occulus Rift and HTC Vive) removes much of the old 90's VR issues by design. UX design issues need to keep pace with hardware developments. 

All the big hardware issues that enable good VR UX are having their big breakthroughs in 2016 including: Motion tracking offering the ability to explore a space at "room-scale"; 3D sound offering binaural or spatially detectable sound, haptic or touch-like input (controllers are shipping with 2016 VR kits and gloves will mark the first wave of input); foveal rendering (eye gaze tracking that renders where users look); walking around in room-scale VR with either motion tracking or floors that move. 

What of UX of VR? VR UX/UI requires a careful understanding of what we already know and what we are now discovering to be the design priorities. At Experience Dynamics we are currently mapping and will share UX/UI best practices for VR/AR and later this year and throughout 2017. UX issues for 3D Interaction Design include (this is not an exhaustive list):

  • Playtesting/ User testing: If you pay attention to acclaimed VR developers and game studios (with games like Fantastic Contraption or Job Simulator) they do a lot of observing / playtesting with users. Just as with any user interface-- this is a must for VR/AR/MR. (Note: We are currently updating our usability labs with the latest hardware this year to offer full VR/AR/MR playtesting/ user testing). 
  • Presence: Sustaining presence is the key to good VR UX. It's less of an issue in AR/MR but it cross over in terms of engagement. Understanding what sustains and breaks presence is critical. This ties in largely to how your experience is designed with your hardware. Controllers and motion tracking (in 2016 generation hardware) add intense realism and presence, for example.
  • Narrative: Narrativity is huge for immersive interfaces. Storyline and immersive narrative is a deal-breaker (especially in VR). Since these technologies activate the imagination, story is the (invisible) interface. 
  • Engagement: The goal is to keep users motivated, intrigued, surprised (surprise is the wow-factor in emotion design research). Without relevancy, cool new technology won't grow legs as fast as the hardware wants it to. 
  • Emotion: Creating compelling emotions is an overarching requirement in new immersive or semi-immersive experiences. Emotion is generated by many factors including all of the ones listed above but can be activated by good use of 3D sound in VR, motion tracking and social presence (eg. Occulus is adding VOIP for real-time chatting this year). Note: While games are a big focus at the moment, lots of interesting stuff is happening with big potential for social VR or MR experiences like telepresence. 

Augmented Reality and Mixed Reality have fewer UX challenges because they are building on close to a decade of current generation mobile technology and AR development. They are also familiar and accessible through mobile devices. MR offers a new interaction paradigm in the 3D object integration and spatial detection of real objects. The challenge, as for VR, will be user engagement and how to sustain relevancy and synch with users (beyond a demo or first look).  

Conclusion: Immersive technologies signal an exciting crossroad for Human-Computer Interaction.  Adoption of immersive UX experiences will be defined by how well developers and designers adhere to the lessons from pulling off good UX during the prior Web and Mobile phases. That message was clear: You won't be able to sustain "wow" experiences without paying attention to your users, testing with them and designing for their brains (not yours).