Summary
Virtual reality (VR) is a computer-mediated experience in which users are immersed in a synthetic environment that can simulate or entirely replace the real world. This page covers the foundational concepts, history, technical challenges, and ethical considerations of VR as they relate to game design and interactive media. The primary source is Steinicke’s Being Really Virtual (2016), a research monograph that forecasts VR’s trajectory over a 15-year horizon.
For the game-design treatment of presence and immersion (control types, agency types, the synergy chain), see presence-and-immersion. This page focuses on the technology and research side of VR that underpins those design concepts.
Defining VR
Frederick Brooks defines VR by three required features:
- Real-time rendering with head-coupled perspective — the displayed view updates as the user’s head moves, creating the impression of looking around a 3D space
- Real space — concrete or abstract 3D virtual environments (not flat screens or 2D interfaces)
- Real interaction — the user can directly manipulate virtual objects, not merely observe
This definition distinguishes VR from other interactive media. A video game on a flat screen provides interaction but not head-coupled perspective. A 360° video provides head-coupled perspective but not real interaction. VR requires all three simultaneously.
The reality–virtuality continuum
Paul Milgram’s reality–virtuality continuum (1994) frames reality and virtuality as endpoints of a continuous scale:
Reality ←——— Augmented Reality ——— Augmented Virtuality ———→ Virtuality
(virtual augments real) (real augments virtual)
- Augmented Reality (AR): Virtual content overlaid on the real world (e.g., Pokémon GO, Microsoft HoloLens)
- Augmented Virtuality (AV): Real-world elements brought into a virtual environment (e.g., video pass-through in VR headsets)
- Mixed Reality (MR): The entire area between the two endpoints — any combination of real and virtual
AR and VR are not competing paradigms but coexisting points on one spectrum. Kurzweil’s three display modes map onto this: head-directed display (HUD), VR overlay display (AR), and VR blocking display (full VR).
VR history: from Sensorama to Oculus
Pioneers (1960s)
- Morton Heilig: Sensorama (1962) — The first multisensory VR system. An arcade-style cabinet with stereoscopic 3D display, vibrating seat, scent producer, and wind. Heilig envisioned the “cinema of the future” but could not commercialise it. “The Sensorama may have been too revolutionary for its time.”
- Comeau & Bryan: Headsight (1961) — The first fabricated head-mounted display (HMD). A helmet with a CRT element and magnetic tracking linked to a remote camera for telepresence. Lacked computer-generated imagery.
- Ivan Sutherland: The Ultimate Display (1965) and Sword of Damocles (1968) — Sutherland’s essay envisioned “a room within which the computer can control the existence of matter.” His 1968 prototype — a periscope-like apparatus displaying wireframe 3D models overlaid on the real world — was actually the first AR display.
- Thomas Furness III: Super-Cockpit (1968) — VR flight simulators for the U.S. Air Force. Furness earned the title “grandfather of VR” and later founded the HIT Lab (1989).
- Myron Krueger: Artificial Reality (1969) — Computer-mediated environments responding to full-body gestures via video cameras. His VideoPlace (1985) introduced the pinch/spread gesture that Apple later adopted for the iPhone.
Coining the term (1980s)
- Jaron Lanier: VPL Research (1985) — Lanier coined and popularised the term “virtual reality.” VPL was the first company to sell VR goggles (EyePhone) and data gloves. The company later went bankrupt when its patent collateral was foreclosed.
The 1990s bust
The early 1990s saw enormous mainstream interest driven by films like The Lawnmower Man (1992). Companies like W Industries (Virtuality Group) produced arcade VR machines with stereoscopic HMDs and networked multiplayer. But the technology was clunky: low-polygon visuals, high latency, heavy headsets, and high prices. VR faded from public consciousness, and “the death of VR” became a standard narrative.
The 2016 revival
The smartphone era inadvertently created the milieu for VR: high-density display panels, MEMS gyroscopes, accelerometers, and powerful mobile GPUs — all at consumer prices. The Oculus Rift DK1 (2013) was the first consumer HMD to outperform expensive professional displays in field of view, weight, and overall experience at a fraction of the cost. Facebook’s $2.3 billion acquisition of Oculus VR (2014) signalled mainstream investment. By 2016, HTC Vive, PlayStation VR, Samsung Gear VR, and Google Cardboard had all entered the market.
| Metric | nVisor SX60 (2010) | Oculus Rift CV1 (2016) |
|---|---|---|
| FOV (diagonal) | 60° | 110° |
| Resolution per eye | 1280×1024 | 1080×1200 |
| Weight | 2.2 lbs | 1.04 lbs |
| Cost | $22,000 | $599 |
The Long Nose of Innovation
Bill Buxton’s “Long Nose of Innovation” (2008) observes that every commercially disruptive technology has a long prior history of low-amplitude incremental innovation — typically at least 15 years from basic university research to commercial success. The high-visibility “disruption” moment is preceded by decades of quiet refinement.
Three examples:
- Mouse: Engelbart demo (1968) → Xerox Alto (1973) → Xerox Star (1981) → Apple Macintosh (1984) → Windows 95 ubiquity (1995). ~30 years from invention to mass adoption.
- Multi-touch: Capacitive touchscreen (1965) → Multi-touch table at U. Toronto (1982) → Krueger’s VideoPlace gestures (1985) → Apple iPhone (2007). ~42 years.
- Head-mounted displays: Headsight (1961) → Sutherland’s Sword of Damocles (1968) → VPL EyePhone (1985) → Virtuality machines (1991) → Oculus Rift (2013). ~52 years.
Design implication: Any technology that will have a significant commercial impact in 15 years is probably already 15 years old in research labs. Studying current VR research reveals what consumer VR will look like in the next generation.
Five technical challenges
Steinicke identifies five fundamental challenges that VR must overcome to achieve widespread adoption:
1. Cybersickness
When visual motion cues (from the HMD) conflict with vestibular and proprioceptive cues (from the body), users experience nausea, disorientation, and discomfort — collectively called cybersickness or simulator sickness. Measured via Kennedy’s Simulator Sickness Questionnaire (SSQ). The problem is most severe during locomotion that the user’s body does not physically perform.
2. Unlimited locomotion in limited space
Real walking is the most natural and presence-enhancing locomotion technique, but physical tracking spaces are small. Solutions include:
- Omnidirectional treadmills — expensive, not consumer-accessible
- Walking-in-place — gesture-based; less presence than real walking
- Redirected walking — imperceptibly manipulating the virtual camera so users unknowingly walk curved paths in the real world while perceiving straight paths in VR (see below)
3. Missing realistic haptic interaction
Users cannot touch or feel virtual objects. Current solutions (controller vibration, force-feedback gloves) are crude approximations. The “rubber hand illusion” shows that visual-tactile synchrony can create illusory ownership of virtual limbs, but generalising this to full-body haptics remains unsolved.
4. Inadequate self-representation
Users need convincing virtual bodies (avatars). Research on the Proteus Effect (Yee & Bailenson) shows that avatar appearance changes user behaviour — both inside VR and after leaving it. Participants embodied in aged avatars allocated more money for retirement; participants using a Superman flight metaphor became more prosocial in real life. This means avatar design is not cosmetic — it has measurable psychological consequences.
5. Isolated social user experience
Early VR is solitary. Users wearing HMDs cannot see the people around them, make eye contact, or read body language. Social VR (multi-user shared virtual spaces) requires solving avatar fidelity, network latency, and the uncanny valley of social interaction simultaneously.
Redirected walking
Steinicke’s own research demonstrates that humans can be physically redirected in VR without noticing, because vision dominates proprioception and vestibular sensation when they disagree.
Detection thresholds (2AFC psychophysical experiments):
- Rotation gains: Users can be turned physically 49% more or 20% less than the perceived virtual rotation without detecting the manipulation. PSE (point of subjective equality) at 0.96, meaning users slightly underestimate virtual rotations.
- Translation gains: Walked distances can be downscaled by 14% or upscaled by 26%. PSE at 1.07, meaning users underestimate virtual walking distances by ~7%.
- Curvature gains: Users can be redirected onto circular arcs with radius ≥22m while believing they walk straight. Within a 40m×40m space, users could walk unlimitedly in any direction.
Cognitive demands: Only at curvature gains above the detection threshold (radius <10m) does redirected walking significantly impair verbal and spatial working memory performance. Below thresholds, the cognitive cost is negligible.
Practical implication: In actual VR applications (where users are focused on tasks, not attending to manipulation), substantially larger gains are tolerable. Curvature gains up to radius ~3.3m are noticeable but not overly distracting.
The Graphics Turing Test
An extension of the original Turing Test to visual display quality. A user explores a scene — either real or computer-generated — and must determine which it is. The test is passed if the user cannot distinguish virtual from real better than chance.
McGuigan (2006, cited by Steinicke) estimates ~518 TeraFLOPS sustained (~1 PetaFLOP peak) rendering is required. Given that high-end GPUs reached 1–10 TeraFLOPS in 2016, and computing performance roughly doubles every 18 months (yielding ~1000× improvement in 15 years), photorealistic real-time VR that passes the Graphics Turing Test should be achievable by the early 2030s.
The evolution of Lara Croft from a few hundred polygons (1996) to tens of thousands of polygons nearly indistinguishable from a real person (2014) illustrates this trajectory within games specifically.
VR in science fiction
VR has been a recurring theme in speculative fiction, and these works have directly influenced VR researchers:
- Plato: Allegory of the Cave (360 BC) — Prisoners see only shadows of reality; the first articulation of the simulation concept
- Galouye: Simulacron-3 (1964) — A simulated city for marketing research; inhabitants are unaware they’re simulated; adapted as World on a Wire (1973) and The Thirteenth Floor (1999)
- Gibson: Neuromancer (1984) — Coined “cyberspace”; hackers jack directly into virtual networks
- Stephenson: Snow Crash (1992) — The “Metaverse” as a shared VR successor to the Internet, populated by avatars
- The Wachowskis: The Matrix (1999) — All of perceived reality is a VR simulation; the “red pill” as metaphor for choosing truth over comfortable illusion
- Cline: Ready Player One (2011) — The OASIS as a VR society where virtual life is preferable to a dystopian reality
These works consistently ask the same question Steinicke’s book poses: if VR becomes indistinguishable from reality, does the distinction still matter?
Ethics of VR
Madary & Metzinger’s code of ethical conduct (2016)
The first systematic code of VR ethics, covering five areas:
- Ethical experimentation — No VR experiment should have foreseeable consequences of serious or lasting harm
- Informed consent — Must explicitly state that immersive VR can have lasting behavioural influences, including unknown risks
- Clinical applications — VR therapy must be developed in close collaboration with physicians
- Dual use — “Torture in a virtual environment is still torture”; VR military/interrogation applications raise serious concerns
- VR and the Internet — Privacy, data security, and behavioural data (eye tracking, emotional responses) require protection
Three rules for VR usage (Steinicke’s lab)
Inspired by Asimov’s Laws of Robotics:
- Humans (and animals) must not be seriously harmed due to VR — Non-debatable. Applies to both research and consumer use. Note: cybersickness is acknowledged but not yet fully preventable.
- Avatars must not be seriously harmed, except where Rule #1 would be violated — Because avatar embodiment creates real psychological effects (Proteus Effect, rubber hand illusion), harming a user’s avatar can harm the user. Humans always have priority over avatars.
- Immersion must not be concealed — Users must always know they are in VR and must have access to the “red pill” (the ability to exit VR at any time). As display technology advances toward indistinguishable contact lens displays, maintaining this rule becomes increasingly critical and technically challenging.
The Simulation Argument (Bostrom, 2003)
Steinicke discusses Bostrom’s trilemma: at least one of the following is almost certainly true: (1) human civilisations go extinct before reaching posthuman capability, (2) posthuman civilisations are not interested in running ancestor simulations, or (3) we are almost certainly living in a simulation. The Golden Ratio and Fibonacci patterns in nature are cited as potential “signs” of a simulation, though Steinicke acknowledges these may be confirmation bias.
The Singularity and VR’s future
Kurzweil’s Law of Accelerating Returns shows that computing performance per dollar has grown exponentially through five successive paradigms: mechanical calculators → relays → vacuum tubes → transistors → integrated circuits. Each paradigm replaced the previous when it ran out of steam.
By ~2023, individual computer performance was projected to surpass a single human brain; by ~2045, all human brains combined. Whether “computational speed equals intelligence” is debatable, but the trajectory implies that the technical barriers to the Ultimate Display will be overcome within a generation.
Steinicke’s concluding vision: a generation of “immersive natives” will grow up in a world where seamless merging of physical reality and digital information is “totally natural and unremarkable.” For these users, proximity will no longer determine where and with whom they spend their time, and physical reality will no longer constrain what they do or want to be.
Evidence
Content drawn primarily from Steinicke, Being Really Virtual (Springer, 2016). Experimental data on redirected walking from Steinicke et al. (2008, 2010, 2014) — psychophysical studies with 12–16 participants using 2AFC methods in controlled lab environments. Ethics framework from Madary & Metzinger, “Real Virtuality” (2016). Computing projections from Kurzweil, The Age of Spiritual Machines (1999) and The Singularity Is Near (2006).
Related
- presence-and-immersion — Game design treatment of presence and immersion; updated with Slater’s VR research framework
- flow — VR presence is a precondition for deep flow in immersive experiences
- game-feel — The “real interaction” requirement of VR connects to Swink’s real-time control building block
- dark-patterns — VR-specific ethical concerns (embodiment, concealed immersion) complement game monetisation ethics
- overview-ethical-game-design — Broader ethical context for responsible design
- music-in-games — Spatial audio is critical for VR presence; 3D sound design extends the principles covered there
- source-being-really-virtual