A model for understanding the virtual reality experience

September 2019

n the surface a virtual reality experience seems easy to define; take a headset, pop it over your head and fire up an application. Your various senses are then bombarded with stimuli and boom; you are now in ‘Virtual Reality’.

However when you scratch the surface you realise there is a lot more going on under the hood that is working to make the magic happen. Over the last two years I’ve been on a quest to try and understand what elements come together to form a virtual reality experience and have filtered down the various theories, research and opinions into a model which I’m going to outline for you in this article. I should start by highlighting that there are quite a few models and theories that attempt to understand the virtual reality experience, some that support one another yet come at it from slightly different angles, others that are a little more in conflict with each other and I welcome you to entertain all of them in your own research.

The following diagram attempts to take into account the various aspects that make up a virtual reality experience stringing them together in one concise and self-supporting model.

Experience
Starting at the core of the model with experience, be it in ‘reality’ or ‘virtual reality’, experience is never an objective state of affairs. Our experience is made up of our awareness of our surroundings and environment and is usually clouded with various biases that make that experience highly qualitative and therefore highly personal in nature. Understanding this is critical to understanding and designing the virtual reality experience.

Perception
The interface between what we experience and the reality we find ourselves in is achieved through various sensory stimuli, be it through sight, sound, touch or smell & taste. These senses work together in what is known as ‘multimodal perception’ and hand in hand with our perception of time & space form the construct of reality that we as humans are experiencing at any given time. That construct is then subjected to various filters unique to the individual that could include social, cultural or personal conditioning resulting in a perception of one’s reality that is highly unique and qualitative to the individual.

VR System
The VR system is the hardware interface we are all so familiar with from the HTC Vive to the Oculus Rift all the way down to the Google Cardboard. The VR system looks to hijack our sensory perception, robbing us of sensory awareness of the ‘real world’ and instead attempts to plug us into a ‘virtual’ one. A VR system is made up of user input, from head tracking to hand tracking, button presses all the way through to voice. And output which updates the presented model of the virtual reality based on these inputs; for example does an object move when touched, will a character acknowledge my voice, can I hear the car whizzing past my left ear.

Currently VR systems are what are getting most attention in the VR industry as we see new system iterations emerge on a regular basis as well as brave attempts at pioneering into unchartered sensory stimuli. It’s important to highlight at this point it’s an easy hole to fall down in the belief that newer, faster, more powerful VR systems are the answer to better VR experiences. VR systems, as the model above suggests, are only part of the overall VR experience.

The VR Application
The VR System acts as a platform for the VR application, or in other words, the software running on the hardware. Every VR application is governed by three basic principles; immersion, presence and narrative.

Immersion Immersion is defined by the immersive fidelity of the VR application. Immersion is the reality that is being built up around the user, in other words everything external to the user. How immersive an experience is will depend on various factors (Jerald 2016);

  • Representational Fidelity: To what degree is the VR experience successfully conveying and displaying the digital world around the user in a congruent manner.
  • Interactive Fidelity: To what degree does the physical world react to the users actions in correspondence with the user’s expectations.
  • Experiential Fidelity: To what degree is the experience successfully representing the creator’s desired intentions.

Interestingly a virtual world does not need to strive for photorealism in design to achieve a high level of immersion. In fact striving for photorealism increases the risk of falling into the Uncanny Valley. A VR experience can be very abstract, for example very cartoony, yet still achieve a high level of immersion by being loyal to the above mentioned points.

Presence
Presence is defined by the sensation of feeling grounded in the virtual world. A great definition I’ve come across in the past is presence is “not knowing but feeling” you are in a virtual space. Presence occurs when the user suspends their disbelief and accepts the VR experience as real. It can be argued that presence is part of immersion or even a by-product of immersion but I prefer to separate the two with a clear distinction of presence being the internal state the user is feeling while immersion is the external world the user is experiencing. Presence can be made broken down into four components (Kent Bye 2016);

  • Emotional Presence: Does the user feel connected to the VR experience? Do they feel connected to the characters that embody that world?
  • Embodied Presence: Does the user feel their body or physical self is present with them in the VR experience? Do they feel their virtual body belongs to them?
  • Social Presence: Does the user feel they can convincingly interact with either AI characters or other users in the VR experience? Am I being acknowledged as present in the space?
  • Active Presence: Does the user feel they have influence over the VR world and its inhabitants? Do they have any agency (ability to change it)?

Narrative
And finally we have narrative. Storytelling is one of the most basic forms of human interaction and we have always looked to use technology to augment our storytelling throughout history. However storytelling in VR is rewriting our understanding of traditional narrative vehicles because now the user has the ability to not only direct how they experience a given narrative but also potentially influence its outcome.

Unlike passive mediums such as film, where the story and how you experience it remains the same for whomever watches it, a VR experience allows the user to experience the given narrative with a certain degree of freedom. This freedom results in an experience becoming highly personal to the user. Although two people try the same VR experience, what they take away from it could be quite different. And as a result a VR experience becomes a dynamically engaging experience unique and personal to each user who tries it.

Conclusion
In summary by bringing together the key components of virtual reality design into a harmonious and self-supporting model we can see how each component plays a critical role in sustaining the illusion of presence and immersion throughout the narrative journey within a VR experience and how that experience is translated down to (hopefully!) a unique, personal and resonating experience for the user.

References:

“The VR Book – Human-Centred Design for Virtual Reality” – 2016 –  Jason Jerald PhD
“Richard Skarbez on the two components of Presence” – Voices of VR Podcast – Kent Bye
“Devon Dolan on the Four different types of stories in VR” – Voices of VR Podcast – Kent Bye
“Rob Morgan on Narrative Design for VR” – Voices of VR Podcast – Kent Bye