Sunday, 15 November 2015

A Discussion of Adaptive/Interactive Music Systems in Video Games.

Tomb Raider Legend, Legend of Zelda: Ocarina of Time, Final Fantasy VII, Ballblazer and Metal Gear Solid: A discussion of adaptive music systems in Video Games.

Professionals and Academics alike agree that one of the main problems that can arise when working in audio for video games has always been and still is the technical restraints, as the amount of ‘physical memory’ that is allocated to audio can ‘directly impact’ the final output quality of your audio (Marks, 2009).

Marks (2009) expresses further that in the past, audio for video games has often been an afterthought to game developers, with resources being focused on non-sound related content. Fortunately this has changed and sound and music is now considered ‘an important aspect of gaming’ according to Marks (2009).

Throughout the years, the more tech-savvy composers and sound designers have worked towards establishing and creating systems of less repetitive audio for their projects. An example of an attempt at non-repetitive design is Ballblazer (LucasArts, 1984).
    In Ballblazer (LucasArts, 1984) Langston used a method that he later coined as the ‘riffology algorithm’. This was a selection of ’32 eight-note melody fragments’, which were played in an order dictated by the computer. Parameters such as ‘how fast to play it, how loud to play it, when to omit or elide notes’ were also all dictated by the computer using dynamically weighted choices. This algorithm allowed for a variation in the melody, which is then further modulated through variation in tempo, velocity, rhythm and expression with each individual play-though of the melodic fragment (1986).

This idea was later defined by Collins (2008) as ‘variable melody’ and this idea based in algorithmic generation for audio content in a video game -‘using the computer itself to improvise musically’- was an early attempt at non-repetitive design/generative music in video games.

Collins (2008) also states that as memory and disk space allowed, such as with 8-bit and 16-bit, games ‘tended toward direct hard cutting between cues’, this allowed for more variety as rather than one piece playing throughout the whole game, multiple pieces could be used for a variety of different purposes. Legend of Zelda: Ocarina of Time (Nintendo, 1996) ‘makes frequent use of this type of segue’, as when Link moves form one area into another, one piece of music will fade out completely as another fades in. This ties into the idea of non-repetitive music systems as it is a technique referred to by Phillips (2014) as Horizontal Re-sequencing and by Stevens (2015) as the Transitional Approach. These ideas are demonstrated in the link below (00:00 – 00:15).

The Transitional Approach (Stevens, 2015) is also used in video games of the same generation, such as Final Fantasy VII (Squaresoft, 1997) as demonstrated below (00:00 – 01:30).

Legend of Zelda: Ocarina of Time (Nintendo, 1996) also makes use of an established technique known as Vertical Layering (Phillips, 2014) or the Parallel Approach (Stevens, 2015), in which during specific situations or encounters inside of a game, different pieces of music will be faded in and out to match the on screen actions of the player. This is demonstrated in Hyrule Field, in which during the day there is a piece of theme music, while at night, there is no music, except for a piece of music when enemies are encountered, as demonstrated in the video below (4.15-5.15).

The Parallel Approach (Stevens, 2015) is also demonstrated in games such as Metal Gear Solid (Konami, 1998) when Snake, the protagonist is noticed by the cameras in the level, and a piece of music begins to play, where as there was silence to begin with as demonstrated in the link below (14:00 – 14:22).

Moving onto a more recent example of non-repetitive and more interactive music systems, Tomb Raider: Legend (Eidos Interactive, 2006) is an interesting example of non-repetitive design in video game audio, as it makes use of Micro-Scoring, which Troels Brun Folmann defined at GDC 2006 as ‘making the music work in a context’ and having a ‘contextual relationship between the music and the game’.

Folmann (2006) later explained that Micro-Scoring is a ‘compositional methodology which is really about understand scoring for units, sub-units and transitions between units, while using the complexity and context of the game to score to’. This relates to the non-linear nature of gameplay and how video game audio needs to relate closely to this and be able to adapt as the players behaviour does.

Units and sub-units to Folmann (2006) are what you are left with if you ‘take a score and break it down into smaller and smaller and smaller components,’ as you can ‘suddenly start to … put them together in a variety of different ways’ as the ‘game progresses’.
    This means that ‘when the player does something, the score changes’ and an example of this is that ‘when pillars are breaking’ they have there own short scores. This short leitmotif is then played over the background underscore as the player reaches this point of interaction. These ‘units’ have transitions written so that they can flow smoothly between each other and other sub-units

The following example is a piece of music played in full and then three times with the leitmotif for when Lara dies happening in different places throughout the piece (52:44 – 53:37).

This methodology can be considered to be somewhere between Ornamental Form and the Transitional Approach (Stevens, 2015) as the piece that plays when Lara dies, is a musical stinger (unit) which is triggered in real-time over the underscore. However, a transition does occur as the underscore then ends.

This ties in closely with current pieces of audio middleware such as FMOD Studio (Firelight Technologies, n.d.) and Wwise (Audiokinetic, n.d.), which work on a similar principle of taking chunks of music and writing short transitions for between them, as well as triggering pieces of sound design or stingers in real-time over a piece of music.

The Transitional Approach is again used in another section of the game where depending on the gear and relating speed in which Lara is driving, the music adapts in real-time to match the pace, by transitioning between units of differing intensity (44:00 – 45:25).

The ideology behind both Micro-Scoring and audio middleware clearly demonstrates the role of unpredictability in the nature of audio for video games, and how writing in small pieces of music means that sound and music in a game can never be completely repetitive, as it will be different with every play through in a similar way to how actions in real-life sound different with every time an action is carried out. This also demonstrates that although these terms are fairly recent, that this ideology has always been in the mind of composers in video game, as with technical evolution of hardware and software has come in tandem with the evolution of musicality and technicality within musical composition and soundtracks.


Audiokinetic (2015) Wwise (Version 2015.1.3 build 5488)  [Software] Audiokinetic.

Collins, K. (2008) Game Sound, An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. Massachusetts Institute of Technology: MIT Press.

Darth Qui-gon Jinn (2012) Legend of Zelda-Ocarina of time- Great Deku Tree - Part 2 [Online video], 25th April. Available from:<> [Accessed October 21st 2015].

Eidos Interactive (2006) Tomb Raider: Legend  [Playstation 3] Eidos Interactive.

Firelight Technologies (2015) FMOD Studio (Version 1.07.02) [Software] Firelight Technologies.

Folmann, T, B. (2006) Tomb Raider Legend – Scoring a Next-Generation Soundtrack. GDC Vault. [Online] [Accessed on 12th October 2015]

GoingCrazy201 (2007) Final Fantasy VII - Walkthrough Part 13 [Online video], 20th July. Available from: <> [Accessed November 15th 2015].

Guy A. Person (2013) Ballblazer (Atari 8-Bit) Theme - Song of the Grid (NTSC) [Online video], 5th January. Available from: <> [Accessed October 20th 2015].

Konami (1998) Metal Gear Solid [Playstation 1] Konami.

LucasArts (1984) Ballblazer [Commodore 64] LucasArts.

MahaloVideoGames (2011) Legend of Zelda: Ocarina of Time Walkthrough - Hyrule Field [Online video], 5th April. Available from: <> [Accessed November 15th 2015].

Nintendo (1996) Legend of Zelda: Ocarina of Time [Nintendo 64] Nintendo.

Marks, A. (2001) the complete guide to Game Audio For Composers, Musicians, Sound Designers, and Game Developers. 2nd ed., Oxon: Focal Press.

Phillips, W. (2014) A Composer's Guide To Game Music. 1st Ed. Massachusetts Institute of Technology: MIT Press.

RabidRetrospectGames (2014) Metal Gear Solid Walkthrough Part 1 No Commentary Gameplay Lets Play [Online video], 30th June. Available from: <> [Accessed November 15th 2015].

Squaresoft (1997) Final Fantasy VII [Playstation 1] Squaresoft. 

Stevens, R. & Raybould, D. (2011) The Game Audio Tutorial. Burlington: Focal Press.

Stevens, R. (2015) A quick intro to music in Wwise. Course Handout.


Post a Comment