1-774-573-7265 alex@alexandre-cote.com

The Basics of Game Music Interactivity

The Basics of Game Music Interactivity

Game music has evolved exponentially over the years, moving from beeps and bloops of Space Invaders to the lush orchestra of games like Uncharted and Halo. But even though there are 30 plus years separating these games, there is one unifying musical element that links them together; music interactivity.


 

Reactive Music!

Check out this video about Portal 2's interactive music!

Check out this video about Portal 2’s interactive music!

Now what is this music interactivity you speak of? Music interactivity refers to how the soundtrack adapts to player input and varying game states. Why this is this important? At their core, games are just another storytelling medium, much like film and TV. However, the unique feature of video games is that the story is nonlinear. When you watch a film, every event happens exactly the same way every time. This is not the case in games. Every person’s play-through of a game is slightly different. Maybe the player doesn’t go into that cave, or encounter that character, or engage in a battle. Since these choices are in the player’s hands, it makes it difficult for the music team to predict the player’s actions, and as a result, nearly impossible to write a linear score.

So composers and sound designers developed systems within the game engine for music to adapt to player choice, and music interactivity was born. Over time, game designers have developed three main ways to create scores that respond to player input, hence the name interactive. These three methods are known as branching, layering, and crossfading, and are still used today. However, some audio teams choose to blend the techniques to create a more intricate and musical score.


 

Branching

Very useful technique, often combined with others for intricacy.

Very useful technique, often combined with others for intricacy.

First let’s talk about branching. Let’s say that we’re exploring some pretty hills, and the music is providing us a warm ambience. Suddenly, we enter a dark cave, and the game engine says: “Oh the player has entered the dark scary cave, and we should switch the music cues to reflect the scary cave.” If we were using branching for interactivity, the music would wait until the pretty hill music has finished before transitioning to the cave music. This is great because we get to hear the complete musical gesture, and it’s not interrupted by the change in environment. However, this method brings problems as well. The music doesn’t react right away, so it’s hard to sync to anything to the visuals. As a result, branching tends to work better with shorter musical phrases.

Crossfading

Very common in games! Listen to Skyrim!

Very common in games! Listen to Skyrim!

Let’s take the same scenario and using crossfading instead. Now when we enter the cave, the hill music immediately fades out as the cave music fades in and we’re instantly immersed in the scary cave music. Crossfading allows the composer to sync more easily with the rapid changes the player may make, and it let’s you switch instrumentation and mood completely. However, it is sometimes less musical because it may crossfade in the middle of a melody (you can hear examples of that in my Hearthstone video.) Sometimes composers will write short transition pieces to help get the player from one mood to the other. These 10-second pieces help bridge the gap between a high intensity action track and the chiller explore track. In cases like Hearthstone, they use a sound effect to cover the crossfade. Both are equally effective, it just depends upon the needs of your game. Crossfading is still commonly found in games and is pretty easy to hear if you listen for it.

Layering

 

Great in stealth and horror games and easy to implement!

Great in stealth and horror games and easy to implement!

The final method is layering. Both Red Dead Redemption and the Dead Space franchise makes extensive use of this technique, as do many stealth games. Let’s say we’re sneaking around in Splinter Cell. While undetected, we just hear an eerie, light suspense track to keep us hyped. Then a guard hears something, but doesn’t know where we are. To account for this, the music fades some percussion on top of the light suspense track, making it a little bigger and signifying that the danger has increased. And if we’re discovered, the full orchestra will come in blaring to amplify the new danger we’re facing.


 

Middleware & Moving Forward

All of these methods can be found in games today, but often companies will use an audio middleware tool such as Wwise or FMOD. These allow the audio team to create intricate interactivity by blending the techniques above. For example, you can set branch points within the middle of your track on various beats of the music, allowing you to write a longer pieces of music and still keep the musical phrase together.

Music interactivity allows the composer to create a more cinematic experience to what the player is seeing on screen, and adds depth to story and gameplay. Try turning all the sound off except the music next time you play a game, and just listen!

Submit a Comment

Your email address will not be published. Required fields are marked *