Hardware Technology

In this chapter:

Exercise: discuss with Jurriaan Mulder
VR Input and Systems
VR Optics & Displays
Gesture Recognition
Eye-Tracking
Hardware Prototyping

VR Technology: how it works

VR Input and Systems

article 1: Lecture 'VR Input and Systems.'

Taught by Bruce Thomas on August 10th 2017 at the University of South Australia. Slides by Mark Billinghurst.

article 2: Most important sensors in VR (Magnetometer, Accelerometer, Gyroscope)

VR Optics & Displays

The primary subject of virtual reality is simulating the vision. Every headset aims to perfect their approach to creating an immersive 3D environment. Each VR headset puts up a screen (or two - one for each eye) in front of eyes thus, eliminating any interaction with the real world. Two autofocus lenses are generally placed between the screen and the eyes that adjust based on individual eye movement and positioning. The visuals on the screen are rendered either by using a mobile phone or HDMI cable connected to a PC.

article 3: The Basics of VR Technology: frame rate (GPU processing) - screen refresh rate (rendering) and FOV

How Lenses for Virtual Reality Headsets Work

Google's Foveated Rendering

article 4: Google is Developing a VR Display With 10x More Pixels Than Today’s Headsets

Eye-tracking unlocks "foveated rendering," a technique in which graphical fidelity is only prioritized for the tiny portion of the display your pupils are focused on.

Google's Light Field Display Technology

![Light Field Display Tech](https://cdn.vox-cdn.com/thumbor/H9CGoho_H9M3h1en5n23BuaIo_I=/0x0:661x300/1200x800/filters:focal(201x88:305x192):no_upscale()/cdn.vox-cdn.com/uploads/chorus_image/image/59030915/ShuttleOddityLoop_Compressed_15fps.0.gif)

"Because these places were filmed with a high-resolution prototype camera that reproduces some of the key cues we use to understand depth in the real world, it felt more like actually being there than anything I’ve experienced with any other live-action VR. Which is to say it was pretty damn cool." - Google's light field technology

article 5: VR is still a novelty, but Google’s light-field technology could make it serious art

article 6: STEAM VR "Welcome to Light Fields"

Light Field Technology: the Future of VR and 3D Displays, Gordon Wetzstein, Stanford Computational Imaging Group

Motion Parallax in Stereo 3D: Model and Applications (SIGGRAPH Asia 2016) - side-by-side

Learning-Based View Synthesis for Light Field Cameras

CAVE

CAVE2

The CAVE® is a multi-person, room-sized, high-resolution 3D video and audio environment invented at EVL in 1991. Graphics are projected in stereo onto three walls and the floor, and viewed with active stereo glasses equipped with a location sensor. As the user moves within the display boundaries, the correct perspective is displayed in real-time to achieve a fully immersive experience.

In the early 1990s, Daniel Sandin, along with his colleagues Thomas DeFanti and Carolina Cruz-Neira, developed the CAVE System (Cave Automatic Virtual Environment) to project interactive, computer-generated 3-D imagery and audio into a physical space defined by multiple projection screens and a surround-sound system. The immersive nature of CAVE was intended as an allusion to Plato's Cave, evoking the shadowy presence of the representation of reality. The CAVE System also returns full circle to the earliest attempts at virtualization and multisensory experience, as practiced in the prehistoric caves of Lascaux, seventeen thousand years earlier. read more

Example: CAVE® - A Virtual Reality Theater - 1992

SIGGRAPH Showcase 1992 and CAVE® Documentation - Part 2

article 7: CAVE VR Systems explained coursera

Gesture Recognition

Sensor Based Gesture Recognition

Sensor based gesture recognition is achieved by the use of input devices which utilize accelerometers, gyroscopes and various other micro-electro-mechanical systems for movement measurement and processing. These sensors control and improve the acceleration along three axes as well as improve the rotating movements of the objects in a virtual environment. These sensors are oft accompanied by illumination and optical proximity sensors. AudioCubes, Wii Remotes, Taclim shoes VR or the Myo armband’s are examples of applications which make use of sensor based gesture recognition.

Glove Based Gesture Recognition

Glove based gesture recognition is the implementation of this technology which makes use of a glove-like device very commonly made use of in virtual reality environments. The gloves, wired with multiple inertial as well as magnetic tracking devices can provide data input to a computer about the rotation motion and the position of the hands. These gloves can even detect the bending of fingers to a high accuracy for close replication of hand movements.

Make your own Glove Controller

Hardware: tearing down components

To Hold or To Wear

I. HTC Vive

article 1: Teardown HTC Vive PRO

Teardown HTC Vive

II. Oculus

III. Sony Playstation

Teardown Sony Playstation VR

IV. GearVR

Repair Samsung Gear VR

VI. Windows Mixed Reality

Teardown Acer Windows Mixed Reality

VII. Microsoft Hololens

Teardown Microsoft Hololens

VIII. Gloves, Shoes & Mocap suits

Hardware prototyping: HMD's & Controllers

Predecessor prototype of the HTC Vive

Game and VR Controllers have been designed extensively and are wonderful pieces of knowledge transfer in Product Design, Electronics, Software Engineering, Ergonomics and Visual Information Design. In order to get a good feel about the evolution of these products, it is good to have a look at the expert’s design & development processes. Here, you will find two of these: HTC Vive and Touch VR Oculus.

article 2: The Secret Prototypes of Valve’s VR Lab (HTC Vive)