Audio & Sound

In this chapter:

• Design for Audio Experience

• Spatial audio

• Binaural audio

• FMOD, Wwise, Auro3D and Google Spatial Audio Toolkit for Unity

Design for Audio Experience

3D audio

Sound amplifies presence in experiences.

article 1: Musical Theory For Sound Design

article 2: Interactive Music

article 3: Sound Design Methods for VR Storytelling - Danny Piccione, Penrose

Notes article 3: VR, a mix of production techniques

• Podcast: full dependence on audio for imagination.

• Film & TV: viewer looking with two speakers in front of them, lineair timeline, beforehand decided. (Full predictability)

• Theater: audio guidance, look in direction where sound is coming from. (Concept of spatialisation)

• Video Games: non-lineair sound triggering methods, no fixed timeline, player's movement, interactions with things. (Non-predictability, beforehand preparing the sounds in scenarios)

• Surround sound set: sound from any direction. (Orchestrate sound)

• Virtual Reality: full 3D experience, location-based spatialisation, sounds playing at once, user dictates what they are hearing.

Sound attenuation zones & spatialisation

• Distance of sounds within the design of the experience and positioning sounds within space.

Environment sound

• Base layer provides a feeling of the space that you are in. (Ambience, 4 points 90 degrees around the user in space)

Spatial audio

Unity audio general

Unity VR Audio Spatializers

Unity FMOD

article 4: FMOD Audio for Unity 5: Survival Shooter

It’s possible to create sound libraries in Fmod that can be called with parameters in the Unity script. You can set distance in the Fmod that Unity is communicating with when the player is moving.

Fmod: 1. New Event 2. Right-mouse click on sound line -> Add 3. Create master bank 4. Assign event to master bank 5. Save project 6. Master bank -> build Unity: 1. SteamVR Camera (head) -> Add component -> FMOD Studio Listener 2. Sound source object -> Add component -> FMOD Studio Event Emitter 3. Check assign event to master bank

Unity Wwise

Wwise Spatial Audio is a family of built-in spatial audio features complemented by in-house and 3rd party plug-ins that seamlessly integrate with the Wwise production workflow. Wwise Spatial Audio specializes in sound propagation, virtual acoustics, and spatially informed audio rendering.

Unity Wwise: VR Game Development and 3D Audio Implementation Demo Reel Wwise playlist: Tutorial 101-1 to 101-16 Wwise Audio Lab Wwise: immersive audio in virtual reality

Auro3D in the context of Wwise

Auro-3D demonstration in the city of Amsterdam

Auro-3D® is the next generation three-dimensional audio standard. It provides a realistic sound experience unlike anything before. By fully immersing the listener in a cocoon of life-like sound, Auro-3D® creates the sensation of actually 'being there'.

Google 3D audio

Google spatial audio toolkit

Resonance Audio is a multi-platform spatial audio SDK, delivering high fidelity at scale. This powerful spatial audio technology is critical to realistic experiences for AR, VR, gaming, and video.

5D audio in VR

article 5: 5D Audio in VR - Expanding 3D audio to enhance VR experiences using higher dimensions of sound

5D includes ALL possible outcomes for ALL events. Quantity = Length, Width, Height, Time, Choice/Probability How can you use the latest neural and auditory system studies to hack the player's brain and choices?

article 6: Sound of One Hand