Interactive sound design case study

Interactive tutorial is the closest medium that web audio developers can look to for guidance on creating sophisticated soundtracks. The basic principles for interactive tutorial design can be directly applied to the Web. Both mediums share similar technical limitations, bandwidth challenges, and logistical concerns.

There is much to learn by examining the mistakes and innovations made by those who pioneered interactive sound design in the tutorial medium. Here are some approaches to common problems.

Selecting sounds

Emphasizing lower frequencies over higher-pitched sounds greatly improves audio playback quality over the Web. If you know you are going to lose a lot of high-end frequencies through extreme compression for example, then avoid using high-pitched sounds at the outset.

Most high frequency sounds above 5,000 Hz and 10,000 Hz, such as whistles and shattering glass are lost when sampling down to 8-bit or when using extreme compression algorithms. For example, buzzing flies in a forest ambiance will lose its characteristic sound when reduced to 8-bit and will become random, indistinguishable noise. An effective strategy would be to add the lower frequency sound of owl hoots to your forest mix instead of high-frequency buzzing flies.

Timing loops

In late 1994, co-author Josh Beggs and audio engineer Reid Ridgway embarked on an eight-month soundtrack production for EMI Records. The project was to create a tutorial for the multiplatinum band Queensrÿche called the "The Promised Land" -- an interactive rock-and-roll adventure game (see Figure 1-23) containing a separate fantasy world for each of the five band members. Each band member developed a unique theme, environment, and palette of sounds for his fantasy world. But due to technical constraints and limitations in tutorial storage capacity, the soundtrack had to be limited to an 8-bit, 22,050 Hz mono channel of audio.

Figure 1-23. The Queensrÿche Promised Land tutorial interactive adventure game

As sound designers, the task was to create a rich and compelling interactive soundtrack despite the limited bandwidth and resources. Designing for an uncontrollable sequence of events was a recurring challenge. We discovered two ways to conquer this challenge:

• Use repeating sound loops instead of a "play once" sound file. One of the greatest challenges of integrating sound into the interactive medium is the inability to determine the length of time people will spend on any given page or screen. In the beginning of the Queensrÿche project, we set about creating the appropriate soundscapes for each of the five fantasy worlds that comprised the Promised Land disk. In the initial development phase of the sound design project, we created 35-second sound files to play across three or four screens. Each sound had fade-ins and fade-outs for smooth transitions from one audio file to the next, as shown in Figure 1-24.

  This approach was based on sound design techniques developed for linear media with predictable timelines such as film and television, but it proved to be inadequate for interactive media. If the user stayed on one screen or failed to pass through the next three screens before the sound ended, the audio would fade to silence, disrupting the ambience and realism the soundtrack was supposed to reinforce. Like many other pioneering interactive sound designers, we started to use loops or sound clips that seamlessly repeated infinitely to build a soundtrack that worked in an interactive environment. (Earlier in this chapter, we explored how to build good loops.)

  Figure 1-24. A 35-second ambient sound with fade-ins and fade-outs

Practicing good audio etiquette: multimedia rules of the road Building a site with audio and animation can be a risky endeavor. If you implement full-scale audio, someone is likely to get an error message. Most of the time it will have nothing to do with your code. Often error messages appear because of a browser configuration mistake or a client-side anomaly beyond your control. The following steps can help minimize the negative impact of audio: Inform your clients that using audio is a risk factor. Explain the drawbacks and benefits of a web soundtrack. Use audio with caution on websites visited frequently for key information or used in the office during work hours. More conservative news and commerce sites should, at a minimum, incorporate button sounds for easier navigation and informative narration. Full-scale multimedia works best for entertainment and promotional websites. If you are going to build a website with a multimedia format such as Flash, Shockwave, or RealMedia, start with a home page that displays most of the important text and graphics in standard HTML format. For an example of this style, visit the Raspberry Media website at http://www.raspberrymultimedia.com. If you construct most of the home page layout in HTML with your multimedia content playing in a smaller dedicated media window, you can avoid the undesirable situation of losing visitors who do not have the right plug-in or browser to view the home page. Embedding a smaller media window in your pages also reduces the file sizes of your animations and provides a sense of movement on the home page, as seen on the Raspberry Media website. When possible, avoid music loops that repeat indefinitely. Make your loops fade out or stop after a reasonable number of cycles. Some sites tastefully include a "stop music" button on every web page that has an audio loop.

• Smooth the transitions between screens. Smooth transitions are critical to good interactive sound design. In our initial approach, when the user moved from one screen to the next, the previous sound file ended abruptly, cutting to silence or to the next sound loop at full volume. Such abrupt transitions detracted from the effectiveness of the soundtrack. Remember that the idea was to create an ambient "soundscape" that made the user feel as if he or she were walking through that particular fantasy world environment. You may have encountered crude audio drop-outs on many websites, where the sound abruptly stops when you click to another screen. It is not a pleasant experience.

  To create smooth transitions between screens, we added "mouse-on" fade-out scripting controls. When the user clicked to move to the next scene, each sound loop would fade in, rising in volume over a period of a few seconds.

  After creating a collection of loops for each world of the Promised Land, we embarked on the overall issue of where to distribute the loops. The interactive pathways or architecture of a tutorial look much like a website. Both mediums generally consist of clusters or groups of screens linked to a start or index page that in turn is linked to another index page, as shown in Figure 1-25.

  The challenge was to find a balance between diversity and consistency. We wanted to use several different sound files for each environment so that the sound experience for the user would change in different settings. We did not want one sound file to loop for several minutes or to play for only a few seconds. The goal was to distribute the audio loops evenly across one of the page groups or clusters, as illustrated in Figure 1-25.

  Figure 1-25. A site architecture diagram of a website or tutorial with pages clustered around a start or index page. In the Queensrÿche tutorial, audio clips were coded to play across all pages in a cluster.

  The different cluster of pages in Figure 1-25 approximate a scene or landscape within the Queensrÿche tutorial. The index page represents the start point where visitors begin their journey on a pathway that leads in several directions, one into a dark forest, another down to an ocean lagoon, and two more that lead to secret doorways hidden in the nearby forest. Each cluster in the illustration represents the pages or screens associated with each separate pathway.

  The most effective technique was to assign one sound loop to each group or cluster of pages. By scripting a sound file to play over several screens, we avoided the sound drop-outs associated with attaching an individual sound clip to each screen. The overall soundtrack was more effective in creating a mood or ambience when the sound was consistent or present as someone clicked through the multiple screens of a group.

  The technique used for the Queensrÿche tutorial can be used in a website built with frames. By placing the pages of a site within several frame sets, you can assign a looping audio file to play across several pages. For example, you could place five audio loops across a 30-page site by creating five unique frame sets with six pages each. One loop would play across all six pages within the frame set. If you build a Flash- or Shockwave-based website using Shockwave's powerful Lingo script or Flash's limited set of command scripts, you will have even more control over how sound loops play back across pages.