New and flexible formats for immersive 3D audio
Sound and music playback systems have developed from mono and stereo to horizontal (2D) surround formats. The latter fixed-channel approach has certain shortcomings, and newer formats, such as Ambisonic and, in particular, Dolby Atmos, are gaining popularity. These formats extend to 3D audio by including height. Additionally, they decouple spatial authoring from reproduction by encoding the spatial audio scene into an intermediate format that later can be decoded for multiple playback formats. 3D audio thus can be reproduced in large venues such as cinemas or concert halls, in domestic environments, or binaurally on headphones, mobile devices, and in extended reality (XR). Many major streaming platforms for film, TV and music now support Dolby Atmos, and in the coming years, spatial audio might become the preferred format for distributing sound and music, similar to how stereo replaced mono in the early 1970s.
3D audio in the arts
Spatial audio has a long history in film, music, and sound art. Examples of early pioneering work include the FantaSound system developed for Disney’s Fantasia in 1940-41 and the audio-visual multimedia enhanced Phillips Pavilion in 1958 by the Le Corbusier office, which featured compositions by Varèse and Xenakis distributed to ca. 350 loudspeakers. While there is a rich tradition of exploring spatial audio in music and sound arts, the film industry has taken the lead in developing commercially distributable formats for spatial audio.
Between sound and music
From modernity onwards, the distinction between music and sound has blurred. Arts and culture have undergone a sonic turn that conceptualises sound art as a practice situated between and beyond music and visual arts. Compositional material is increasingly preoccupied with the sonic domain against which music earlier on defined itself: noise, silence, and non-musical sound. In classic film there is a narrative divide between diegetic sound and non-diegetic music. Diegetic sound exists within the world where the film takes place, while music is positioned outside that world. The late musicologist and film scholar Kulezic-Wilson suggested a tendency for this divide to blur as music approaches sound design and vice versa in integrated soundtracks; “sound design is the new score”. This tendency is part of a more general musicalisation of all parts of the film and is in part technologically driven or afforded, as the same editing software is used for music and sound design. Similar tendencies are theorised in sound design for games.
Axis thinking
Through artistic research, this project explores new creative and expressive possibilities along three axes that jointly define a multidimensional space with diverse artistic practices and possibilities.
The first axis spans from real space to virtual reality, with several means of artistic expression within sonic arts, film, games, and XR distributed along this axis. The key is that the same 3D audio formats can be used.
A second axis maps how various artistic genres are positioned along a continuum between music and sound. Some may be “pure music,” others “pure sound,” but the line often blurs in experimental music and sonic arts, as well as in film and game sound.
A third axis connects the fields of sonic arts with sound and music in film, games, and XR. The same or similar editing software is used in both fields, and this research project cleaves open a productive space for cross-disciplinary exchange, enriching both fields.
Artistic Research
The methodology is within artistic research, emphasising research in and through artistic practice. A series of artistic and creative works and studies will be carried out, individually and in collaboration, mapping the space of expressions along axes as discussed above. Examples are sound and audio-visual installations, film in the expanded field, projects exploring audio-only augmented reality such as soundwalks, and explorations of how artistic works, when developed, shared, and distributed, can transition between real and virtual (binaural) space. How can VR be used for the development and documentation of sound installations?
Implicit is exploring, developing, and exchanging methods and perspectives on aesthetics for composition and sound design in 3D audio. Additionally, the research project aims to develop new tools for Ambisonic composition and design according to the needs of the various artistic projects. Artistic explorations and outcomes will be documented, and the project will lead to published expositions of artistic research that share and reflect on working methods and insights, emphasising interdisciplinarity.
