HEY, WHERE’S THE MUSIC?
Sound design affects users’ perception of immersion, enjoyment, and even privacy.
This is the latest chapter in my research series on what makes a welcoming social VR experience. Social Virtual Reality platforms are working to generate engaging experiences that draw people in and motivate them to return. It led me to wonder — what are the core elements what make humans feeling comfortable and stimulated and turns them into repeat visitors? I gave demos of social VR platforms to nine experts of spatial & social experience design and then asked them for their impressions. Read the study’s introduction here. You can also read part 2 on Owning the Narrative, part 3 on Who is this Space for? part 4 Place Creates the Rules of Behavior, part 5 Context is Comforting, part 6 New people want to be hosted in Social VR, part 7, Retail in VR: What Makes a Good Shopping Experience, part 8 The Importance of Detail in Virtual Spaces, and part 9 Recommendations on VR Environmental Design (Part 1), and part 10 Recommendations on VR Environmental Design (Part 2).
Sound design can single-handedly make the difference between a good experience or not. Humans are alert to sound and still often unconscious of how it’s affecting them. These experts from architecture, retail, interior design, and hospitality all consider sound design an integral part of space creation.
Sound design affects users’ perception of immersion, enjoyment, and even privacy.
The right sound increases a scene’s believability
A person’s sense of sound can overpower other sensory information. Take a look at this quote from an architect whose sense of immersion was more informed by sound than sight:
It’s pretty poor graphics but I actually get the sense that I’m in Mexico, from the sound… the sound helps a lot.
Based on this type feedback, investing resources in sound might provide a better ROI than other types of improvements in an experience. Make sure the sound is coming from the right place and distance in the model to enhance the experience.
Music helps people’s engagement and enjoyment
One person reported peak enjoyment when the music and the surrounding scene were a good fit:
“The warmest feeling I had was in the last world, where I heard music.
In contrast, when the sound and the scene had a mismatch the experts noted it right away:
Going to the dance party room felt empty and weird because there was no music. Sound makes so much difference…
Visiting a VR nightclub was something that I requested many of my participants to do as part of their demo. However, if people arrived in off-peak hours, the music was gone and the entire world felt uninviting to people:
I’m not hearing anything music-wise or otherwise that would entice me to go into this club vs another club, if I’m walking down the street with my friends.
And from another expert:
They should have good music, especially in the dance one. It felt like going into a closed warehouse.
It’s a question for creators — for online worlds where there’s a global audience and a range of timezones — who is the music for? When will the sound be on?
Sound gives spatial cues. One expert was extremely particular about the type of sound he craved in VR.
There isn’t a lot of audio right now. No spatial cues. If I’m in a VR space and fully immersed, I want to hear stereo panning as I’m moving around, not just when someone is coming into the space, so that I have a sense that I’m really in there. When I first popped in and I heard a voice on my left, but now as I’m moving around, there’s nothing. That is taking away from the experience.”
Because sound conveys information about space, it can signal a user’s privacy or security.
Sound can designate if you’re in a public or a private space
Sound shapes a user’s experience. Sound can be used to manipulate a person’s sense of privacy due to the information it conveys about the space. Often because sound gives cues about distance from others, and what a space has been designed for.
For example, if you are eating at a loud restaurant, you may be more revealing in your conversation with your dinner partner because you are confident the people at the next table can’t hear you. Conversely, if you are eating a restaurant where the tables are very close together, you may keep your conversation less personal.
One expert visited a house that had a pool in a fenced backyard. He imagined how the sound design would differ from the different locations around the house:
If you were having a conversation with someone and someone could hear from out there, you lose that sense of privacy.
In his mind, he wanted the sound in VR to work the way it does in the physical world:
The sound from outside would be dampened and quieter.
The topics of privacy and intimacy are very closely linked together and more education is needed to let users new to headsets understand spatial audio and who exactly can hear them.
Close conversation is a huge potential for VR, especially to let us connect with loved ones who are geographically far away. I conducted a separate study with Hubs by Mozilla where once pairs of people felt confident they were in a private VR environment, then they opened about more personal topics with each other.
Takeaways for XR creators
While all of these interviews were based on demos of social virtual reality (VR) platforms, I believe all of these insights could apply to Augmented Reality (AR) as well.
- Take notice of the sound design in spaces that are familiar to you in the coming week. It’s unlikely that you’re in restaurants, buildings, or stores that are silent. If you slow down and pay attention, you might discover some details that surprise you.
- Get a music creator’s perspective on VR. Kent Bye’s Voices of VR interview with Torkam Ji about sound healing and quantum harmonics expanded my beliefs about the possibilities for music.
- Let the music take control.
