Shifting Perspective in FPS Sound Design

"Operation Blood Storm", a Call of Duty Infinite Warfare campaign level I was responsible for completing sound design and mixing, had an interesting 40 second scripted event finale. Although this section of the campaign featured giant shooting space lasers and dramatic depressurization set pieces, this scripted ending was primed for novel use of a shifting perspective. Visually we remain in the cockpit for the entirety of the scene, but there was an opportunity to attenuate near sounds to focus on events outside the cockpit. Surprisingly, this is something we as listeners do daily. Check out the scene below. I included the gameplay lead-in to give the scene some context.

 

The above busy scene with voice over, exposition, distant events, and constant near field danger presented unique audio challenges. The scene begins with a crippling hit on your pilot's "Jackal" (a SATO airspace fighter). This transitions the player instantaneously from free range flight combat gameplay to an on-rails finale. Though scripted, the player maintains limited head movement control throughout the scene. Once your pilot, Reyes, stabilizes the Jackal and radios for help, the "Olympus Mons" has a dramatic entrance into the scene, hurling out of control over head towards a distant cruiser, the "Retribution". The collision and subsequent explosion are an obvious focal point not only because of the dramatic visuals, but because the scene benefits from being organized into before and after the explosion.

The first 6 seconds of the event's audio are focused on in-cockpit sounds. The impact of the projectile, ship diagnostics and alerts, Jackal shakes and vibrations, and fire. As the ship stabilizes I decided to help transition the player's ears from inside the ship, to listening outside. The ominous fanfare we hear of the Olympus Mons approaching behind us is mixed heavily to rear speakers in surround sound to give us the first directional sound we hear in the scene. I use this opportunity to drop out Jackal vibrations and cockpit sounds.

Our ears naturally attenuate to accommodate our focus as we navigate daily life. Much like our eyes focus on near field objects and relax to take in a large landscape, our ears do the same. This processing of specific parts of auditory information in the brain is called "selective auditory attention". We experience this when we are in a loud restaurant and tune out ambient conversations and kitchen noises to focus on our dinner guest's joke. We can further push the conversation into the background and focus on the lyrics of the song playing over the restaurant's speakers, then quickly localize the sound of a broken glass behind us, and finally back to hear our dinner guest's punch line. The changing focus of our ears can lessen sensitivity to other unrelated sounds. Rather than presenting the player with all the auditory information in the scene simultaneously, shifting our focus lets me simulate selective hearing with mix and sound design to create an immersive experience.

Once our cockpit is silenced by the Olympus Mons, the giant damaged cruiser cutting into the scene overhead, we hear brief radio contact before our focus is shifted completely away from our cockpit. The audio of the scene is now exclusively focused on the unfolding distant events. Many sound designers might choose to continue ship vibrations, alarms, and fire, but I felt there was little new auditory information to communicate inside of our ship. Continuing nearfield audio might interfere with the priority collision and explosion we want to hear and feel. It is important that we hear the Olympus Mon's bottom engines fire as Ethan attempts to "pull up" avoiding the ship directly in his path. Next, a rapid succession of small explosions report as the Olympus Mons collides with the Retribution, the other damaged cruiser. The entire audio perspective is distant and focused outside of our ship despite us being heavily damaged and on fire. And we'll need a stylistically smooth and practical way to transition back to nearfield events in preparation for the rest of the scene. The first invisible wave of the nuclear-like explosion mutes the entire mix and even cuts out Ethan's radio line briefly. This initial punch of the explosion is more felt than heard.

If you watch carefully, the player's Jackal shakes at the same time we see the bright flash of a final explosion. Then the blue pulse wave of the explosion is triggered. Barring a discussion on the fictional technology of this universe, and the acoustical universe that communicates sound in space, this initial and nearly instantaneous tremor indicates that there are layers to the explosion radiating out at different speeds.

Following this initial flashing explosion is a second blue shock wave. I used this second energy wave that impacts our fighter to bring the mix back to nearfield for the remainder of the scene. After all, the wave visually bridges the gap between far and near. Our Jackal is shaken and pushed by the shockwave. More cockpit alerts fire off as waves of atmosphere pulse over our rapidly descending ship. The mix has returned to the space in our small cockpit. Our ally Salter enters screen right in her Jackal to survey our damage while we pass out from rapid descent. There is a lot of information packed in this 37 seconds, and the goal was to keep the scene focused, immersive, and flowing.

This there-and-back-again idea was a fun audio approach to enhance the scene. It enabled me to organize compound events, focus the player, and push us into different parts of the scene's space without leaving our cockpit. Ideally, the player won't be aware of what sounds are pushed out of the mix, because their eyes and ears are locked onto the priority stimulus. This is no different than us consciously spying on another table's whispers, listening to a glass break in the kitchen, traveling up into the chorus of a radio pop song, and cutting off background for a conversation all while seated at our restaurant table. Our dinner party is the space battle and our seat is in the cockpit. 

Film mixes that typically juggle huge music scores and vital dialogue, benefit from sound design focusing on one or two sounds at a time. And a television show's dialogue mix can aggressively duck backgrounds. But mimicking selective auditory attention is a technique that is particularly effective in first person perspectives when sensorial focus changes. Awareness of how you listen will help you make intelligent and creative choices when mixing for immersion.