Creating a Real-Time Sound Filter Architecture

Sound filters are simple audio processors that can modify existing sound samples, usually in real time. Sound filters, also known as digital signal processors, are used in audio production all the time—for example, to add distortion to a guitar or an echo to a voice. Sound filters can make your game more dynamic. You might have a level in your game in which the player is wandering around a cave. In that case, you could add an echo to any sounds played. Or, if a rocket is whizzing past the player, you could make the sound shift from the left to right speaker. In this section, you'll create a real-time sound filter architecture and create two filters: an echo filter and a simulated 3D sound filter. First, you'll create the architecture. It is fairly straightforward—all you need is a simple class that can modify sound samples in a buffer. The SoundFilter class, in Listing 4.3, accomplishes this task. It's an abstract class that you can extend to create sound filters and has three important methods:

  • filter(byte[] buffer, int offset, int length) Filters an array of samples.
  • getRemainingSize() Gets the remaining size, in bytes, that this filter can play after the sound is finished. For example, an echo would play longer than the original sound.
  • reset() Resets the filter so that it can be used again on a different sound.

Because SoundFilter is an abstract class, it won't actually filter anything. Subclasses do the filtering work.

Listing 4.3 SoundFilter.java
package com.brackeen.javagamebook.sound;
/**
 An abstract class designed to filter sound samples.
 Since SoundFilters may use internal buffering of samples,
 a new SoundFilter object should be created for every sound
 played. However, SoundFilters can be reused after they are
 finished by calling the reset() method.
 Assumes all samples are 16-bit, signed, little-endian.
 format.
 @see FilteredSoundStream
*/
public abstract class SoundFilter{
 /**
 Resets this SoundFilter. Does nothing by default.
 */
 public void reset() {
 // do nothing
 }
 /**
 Gets the remaining size, in bytes, that this filter
 plays after the sound is finished. An example would
 be an echo that plays longer than its original sound.
 This method returns 0 by default.
 */
 public int getRemainingSize() {
 return 0;
 }
 /**
 Filters an array of samples. Samples should be in
 16-bit, signed, little-endian format.
 */
 public void filter(byte[] samples) {
 filter(samples, 0, samples.length);
 }
 /**
 Filters an array of samples. Samples should be in
 16-bit, signed, little-endian format. This method
 should be implemented by subclasses. Note that the
 offset and length are number of bytes, not samples.
 */
 public abstract void filter(
 byte[] samples, int offset, int length);
 /**
 Convenience method for getting a 16-bit sample from a
 byte array. Samples should be in 16-bit, signed,
 little-endian format.
 */
 public static short getSample(byte[] buffer, int position) {
 return (short)(
 ((buffer[position+1] & 0xff) << 8) |
 (buffer[position] & 0xff));
 }
 /**
 Convenience method for setting a 16-bit sample in a
 byte array. Samples should be in 16-bit, signed,
 little-endian format.
 */
 public static void setSample(byte[] buffer, int position,
 short sample)
 {
 buffer[position] = (byte)(sample & 0xff);
 buffer[position+1] = (byte)((sample >> 8) & 0xff);
 }
}


SoundFilter objects may contain state data, so a different SoundFilter object should be used for every sound played. To make it simple, allow the SoundFilter class to assume all sound is 16-bit, signed, little-endian format. The term little-endian refers to the byte order of the data. Little-endian means the least significant byte is stored first in the 16-bit sample; big-endian means the most significant byte is stored first. Signed data means the samples have a signed bit, so they range from -32768 to 32767 instead of from 0 to 65536. Signed little-endian samples are the default for the WAV format. Java Sound likes its data in bytes, so you must convert those bytes to 16-bit signed shorts to work with them. The SoundFilter class provides this with some static methods for getting and setting samples from a byte array: getSample() and setSample(). Now you need an easy way to apply a SoundFilter to your sounds. You could apply the filter directly to your array of samples, but this would be a good idea only if you want the filter to be permanent because it would modify the original samples. Some filters, such as a 3D sound filter, rely on specific game elements that change in real time, so you need to apply your filters in real time, keeping the original sound intact. Because the SimpleSoundPlayer class plays sounds from an InputStream, you can create a new InputStream subclass that applies a SoundFilter. The FilteredSoundStream class in Listing 4.4 does just that. Besides a SoundFilter, it takes an InputStream as a source (you'll use a ByteArrayInputStream or a LoopingByteInputStream, as before). This means that every byte read from the input source is passed through the SoundFilter to the sound player, resulting in a real-time filter that doesn't modify the original sound.

Listing 4.4 FilteredSoundStream.java
package com.brackeen.javagamebook.sound;
import java.io.FilterInputStream;
import java.io.InputStream;
import java.io.IOException;
/**
 The FilteredSoundStream class is a FilterInputStream that
 applies a SoundFilter to the underlying input stream.
 @see SoundFilter
*/
public class FilteredSoundStream extends FilterInputStream {
 private static final int REMAINING_SIZE_UNKNOWN = -1;
 private SoundFilter soundFilter;
 private int remainingSize;
 /**
 Creates a new FilteredSoundStream object with the
 specified InputStream and SoundFilter.
 */
 public FilteredSoundStream(InputStream in,
 SoundFilter soundFilter)
 {
 super(in);
 this.soundFilter = soundFilter;
 remainingSize = REMAINING_SIZE_UNKNOWN;
 }
 /**
 Overrides the FilterInputStream method to apply this
 filter whenever bytes are read
 */
 public int read(byte[] samples, int offset, int length)
 throws IOException
 {
 // read and filter the sound samples in the stream
 int bytesRead = super.read(samples, offset, length);
 if (bytesRead > 0) {
 soundFilter.filter(samples, offset, bytesRead);
 return bytesRead;
 }
 // if there are no remaining bytes in the sound stream,
 // check if the filter has any remaining bytes ("echoes").
 if (remainingSize == REMAINING_SIZE_UNKNOWN) {
 remainingSize = soundFilter.getRemainingSize();
 // round down to nearest multiple of 4
 // (typical frame size)
 remainingSize = remainingSize / 4 * 4;
 }
 if (remainingSize > 0) {
 length = Math.min(length, remainingSize);
 // clear the buffer
 for (int i=offset; i<offset+length; i++) {
 samples[i] = 0;
 }
 // filter the remaining bytes
 soundFilter.filter(samples, offset, length);
 remainingSize-=length;
 // return
 return length;
 }
 else {
 // end of stream
 return -1;
 }
 }
}


The FilteredSoundStream class extends the FilterInputStream class, which is an abstract class designed to filter an InputStream. It needs only one method, read(), to do its work. As mentioned before, when a filter such as an echo is played, the echoes are often played back long after the original sound has finished playing. In FilteredSoundStream, if the SoundFilter still has some sound remaining in it, the read() method takes care of this by clearing the source buffer to all zeroes (making it silent) and then filtering that silent buffer. Finally, after everything is done, it returns -1 to signify the end of the stream. We've talked a lot about an echo filter. Now you will actually create one.



   
Comments