ETD

• engagement
• flow
• immersion
• music
• plausibility
• presence
• recognition
• sound
• sound design
• videogames
• virtual reality

This thesis explores the nature, perception, and function of sound as a central element in shaping coherent and immersive interactive experiences, both in virtual reality and in traditional videogames. Starting from a physical and perceptual description of sound, the work examines how acoustic vibrations travel through different media and are processed by the human auditory system. It analyses the biological and psychological mechanisms that allow humans to transform mechanical pressure waves into meaningful experiences, highlighting the complexity of hearing as a perceptual system that collaborates with vision to construct our understanding of reality.

The first part of the research focuses on the physical and cognitive foundations of sound perception. It introduces fundamental acoustic properties such as frequency, amplitude, and timbre, explaining how they relate to the sensations of pitch, loudness, and tone color. The text then examines how sound contributes to spatial awareness and orientation through binaural hearing, which enables humans to localize sound sources and perceive environmental depth. The chapter also explores the Auditory Scene Analysis (ASA), a cognitive process by which the brain separates and organizes complex soundscapes into distinct perceptual streams, a mechanism that underlies our ability to interpret and navigate the auditory world.

The second part of the thesis connects these perceptual principles to interactive digital environments. It discusses how sound, beyond its acoustic and emotional dimensions, plays a fundamental role in maintaining the coherence and usability of interactive systems. In videogames and virtual reality, sound does not merely accompany the visual experience but defines it: it provides feedback, establishes spatial relationships, and guides the user’s focus. The study draws attention to the Response-As-If-Real (RAIR) phenomenon — the user’s natural reaction to virtual stimuli as if they were real — and explores the concepts of Place Illusion (PI) and Plausibility (Psi), as defined by Mel Slater. These concepts are used to explain how sound, in coordination with visual and sensorimotor cues, sustains the illusion of presence and realism in virtual environments.

The third section addresses the emotional and expressive dimensions of sound. It investigates how auditory elements influence mood, tension, and empathy, acting as emotional amplifiers that guide the player’s psychological engagement. Particular attention is given to music, which, through rhythm, harmony, and melody, not only reinforces immersion but also contributes to narrative and affective coherence.

Finally, the thesis presents a series of case studies that apply theoretical frameworks by R. Murray Schafer and Michel Chion to the analysis of sound design in interactive experiences. Schafer’s ecological model, which distinguishes between field, ground, and signal, and Chion’s taxonomy of cinematic sound, including diegetic, non-diegetic, acousmatic, and on-screen categories, are adapted to the context of videogames and virtual reality. Through these analytical tools, the study investigates how soundscapes are constructed to support both spatial realism and narrative intention.

By integrating perspectives from acoustics, psychology, and media theory, this work aims to demonstrate that sound is not a secondary component of interactive media but a structural and cognitive pillar that shapes perception, usability, and emotional resonance. Understanding how sound functions in virtual and digital spaces reveals its essential role in defining the coherence, presence, and expressive power of modern interactive experiences.