Using Java's Serialization API for Game State Persistence

As a games developer, if you plan to rely on the Serialization API, you have no choice but to learn how most of the framework works in detail. Unlike some APIs, in which you can restrict yourself to learning how to exploit just the tip of the API iceberg, the Serialization API doesn't lend itself to such a short-cut approach. The next few sections give you an in-depth look at serialization techniques and pitfalls.

Introduction to Serialization

The core function of Java's Serialization API is to transform objects into byte streams, and vice versa. The terms used are serialization and deserialization, respectively. The Serialization API was added to the Java Platform back in the Stone Age days of the switch from Java 1.0 to Java 1.1 (around the end of 1996). The API's main raison d'être at the time was as an enabling technology to support Remote Method Invocation (RMI—that is, distributed objects) and Java's reusable component architecture, JavaBeans. You'll use the API for a far more exciting goal: game state persistence. For any object to persist—that is, be saved from the inevitable death of the JVM it lives in—all that needs to happen is for that object to be transformed into an output byte stream and for this byte stream to be transmitted to some external storage entity that is independent from the current JVM, such as a file or a network socket. (The remainder of the chapter assumes you are familiar with Java's basic I/O stream APIs.) The net effect is object persistence.

Serialization: The Basics

The Serialization API lives in the java.io package; at the heart of the API are the Serializable interface and two special stream classes capable of serializing and deserializing objects that implement the Serializable interface: ObjectOutputStream and ObjectInputStream. If you are familiar with java.io's venerable DataOutputStream and DataInputStream classes, you can think of ObjectOutputStream and ObjectInputStream as enhanced versions of these classes that can also read/write objects, in addition to reading/writing primitive data values and Strings. Before exploiting these stream classes, though, let's see what behavior the Serializable interface expects from any objects that want to take part in any serialization/deserialization round-trip:

public interface Serializable {
 // empty !
}


The Serializable interface specifies no behavior whatsoever. Unlike the vast majority of interfaces, Serializable requires no methods to be implemented. It also does not define any constants. The interface exists purely to tag classes as being serializable. But what exactly does this mean? Can you just load up all your Java source files and sprinkle implements Serializable willy nilly over all your classes? Most definitely not. The reasons why this is not possible are uncovered in increasing levels of detail as you progress through the remainder of this chapter. NOTE The Serializable interface has a closely associated subinterface called Externalizable. You can find out more about it in the Java Object Serialization Specification, available online on Sun's Java website, java.oracle.com.


A Simple Serialization Example

Let's just see how simple the Serialization API can be at its simplest (be warned, it can get pretty hairy at its most complex). Listing 15.1 shows how an example program serializes an everyday StringBuffer object and writes it out to a file.

Listing 15.1 SaveStringBuffer.java
import java.io.*;
class SaveStringBuffer {
public static void main (String[] args) {
 StringBuffer sb = new StringBuffer("Beam me up!");
 try {
 FileOutputStream fos = new FileOutputStream("sb.ser");
 ObjectOutputStream oos = new ObjectOutputStream(fos);
 oos.writeObject(sb);
 oos.close();
 } catch (IOException ioException) {
 System.out.println(ioException);
 }
}
}


The sharp end of this example is the writeObject() call on an ObjectOutputStream object. The stream object does all the hard work of serializing the argument object: It analyzes, via core reflection, the list of instance fields making up the StringBuffer and converts the values of these fields into a byte stream. If you have a quick peek at the source code for StringBuffer, you'll find that every StringBuffer consists of a char[], an int, and a boolean; these fields are read out of the object and streamed to an OutputStream. In this example, you send the resulting byte stream to hard disk via a terminus FileOutputStream. If you run this program, you'll find a new file called sb.ser in the current directory, which contains the serialized StringBuffer. To prove this claim, you can run the following complementary program to reload, deserialize, and print your StringBuffer (see Listing 15.2).

Listing 15.2 LoadStringBuffer.java
import java.io.*;
class LoadStringBuffer {
public static void main (String[] args) {
 StringBuffer sb = null;
 try {
 FileInputStream fis = new FileInputStream("sb.ser");
 ObjectInputStream ois = new ObjectInputStream(fis);
 sb = (StringBuffer) ois.readObject();
 ois.close();
 } catch (IOException ioException) {
 System.out.println(ioException);
 } catch (ClassNotFoundException classNotFoundException ) {
 System.out.println(classNotFoundException );
 }
 System.out.println("Welcome back aboard, Sir! : " + sb);
}
}


Running this program produces the following output on the console, confirming that sb.ser contains a persistent StringBuffer object:

Welcome back aboard, Sir! : Beam me up!


Although it is marginally more complex than its saving counterpart, the heart of this LoadStringBuffer example is the readObject() call on an ObjectInputStream object. readObject() simply does the reverse of writeObject(): It deserializes a byte stream into a copy of the original object that was serialized. Because readObject() is declared as returning an Object, you will always need to cast the return value to some more specific subtype—in this example's case: StringBuffer.

Serialization: The Rules

For any class to be Serializable, it should be possible for its objects' states to be extracted as a sequence of bytes—that is, a byte stream. Object state means the collection of values of an object's nontransient instance fields (by definition, transient fields are ignored by the serialization engine). Any primitive fields (boolean, char, byte, short, int, long, float, and double) can always be transformed into bytes without any problems, but reference fields that are not null at runtime can be serialized only if the objects referred to by the references also implement the Serializable interface. This latter rule is applied recursively, so any objects referred to, either directly or indirectly, by some root object to be serialized need to be serializable, in turn. In practice, this means that you can declare, or tag, as being serializable any of your classes that declare only primitive fields, without having to worry about the truth of such a declaration. The resulting class will always be serializable. On the other hand, any classes that declare references to other objects require very careful analysis to determine whether such classes can be declared serializable. When a class declares any reference fields, you need to check one by one whether the types of the objects referred to are serializable. If any type referred to is not serializable, your class cannot normally be declared serializable (but you'll see later how to possibly get around this problem via customization of the serialization process). Only if all references refer to serializables can the declaring class also be declared serializable. Later in this chapter, we return to this fundamental nature of serializability.

Core Classes Support for Serialization

Given that most nontrivial classes do declare references, and given that these references often refer to core library types such as String, Point, Color, and so on, you need to be aware of the serializability landscape of the core library's classes. In particular, it is handy to know which classes in the java.lang and java.util packages are serializable. Tables 15.1 and 15.2 contain this information for the 1.4 release of the Java 2 platform.

Table 15.1. JDK 1.4 java.lang Classes and Their Serializable Status

Class Name

Serializable?

java.lang.Boolean

Screenshot

java.lang.Byte

Screenshot

java.lang.Character

Screenshot

java.lang.Class

Screenshot

java.lang.ClassLoader

 

java.lang.Compiler

 

java.lang.Double

Screenshot

java.lang.Float

Screenshot

java.lang.InheritableThreadLocal

 

java.lang.Integer

Screenshot

java.lang.Long

Screenshot

java.lang.Math

 

java.lang.Number

Screenshot

java.lang.Object

 

java.lang.Package

 

java.lang.Process

 

java.lang.Runtime

 

java.lang.RuntimePermission

Screenshot

java.lang.SecurityManager

 

java.lang.Short

Screenshot

java.lang.StackTraceElement

Screenshot

java.lang.StrictMath

 

java.lang.String

Screenshot

java.lang.StringBuffer

Screenshot

java.lang.System

 

java.lang.Thread

 

java.lang.ThreadGroup

 

java.lang.ThreadLocal

 

java.lang.Throwable

Screenshot

Table 15.2. JDK 1.4 java.util Classes and Their Serializable Status

Class Name

Serializable?

java.util.AbstractCollection

 

java.util.AbstractList

 

java.util.AbstractMap

 

java.util.AbstractSequentialList

 

java.util.AbstractSet

 

java.util.ArrayList

Screenshot

java.util.Arrays

 

java.util.BitSet

Screenshot

java.util.Calendar

Screenshot

java.util.Collections

 

java.util.Currency

Screenshot

java.util.Date

Screenshot

java.util.Dictionary

 

java.util.EventListenerProxy

 

java.util.EventObject

Screenshot

java.util.GregorianCalendar

Screenshot

java.util.HashMap

Screenshot

java.util.HashSet

Screenshot

java.util.Hashtable

Screenshot

java.util.IdentityHashMap

Screenshot

java.util.LinkedHashMap

Screenshot

java.util.LinkedHashSet

Screenshot

java.util.LinkedList

Screenshot

java.util.ListResourceBundle

 

java.util.Locale

Screenshot

java.util.Observable

 

java.util.Properties

Screenshot

java.util.PropertyPermission

Screenshot

java.util.PropertyResourceBundle

 

java.util.Random

Screenshot

java.util.ResourceBundle

 

java.util.SimpleTimeZone

Screenshot

java.util.Stack

Screenshot

java.util.StringTokenizer

 

java.util.Timer

 

java.util.TimerTask

 

java.util.TimeZone

Screenshot

java.util.TreeMap

Screenshot

java.util.TreeSet

Screenshot

java.util.Vector

Screenshot

java.util.WeakHashMap

 
As you can see from Tables 15.1 and 15.2, not all standard java.lang and java.util classes are serializable. This probably comes as a bit of a surprise, but there are valid reasons for this patchy support which we touch upon in a minute. Crucially, Object itself does not implement the Serializable interface. This implies that any direct subclasses of Object are not serializable unless explicitly tagged (and, hence, designed) as such. If you modify the earlier SaveStringBuffer example to attempt saving a plain Object:
oos.writeObject( new Object() );


then running this modified version would produce the following runtime exception:

java.io.NotSerializableException: java.lang.Object


NotSerializableException often competes with NullPointerException for your gray hairs when you're trying to debug nontrivial serialization aspects of any Java app. Looking at other classes in the java.lang package, you can see that most bread-and-butter classes are serializable: All wrapper classes are—String, StringBuffer, and even Throwable. Because the Throwable class is serializable, all exceptions, regardless of which package they are declared in, and whether checked or not, are also serializable. In the java.util package, you can see that most classes are serializable, except the abstract skeleton implementations of the Collection and Map interfaces. Note that both interfaces Collection and Map could theoretically have extended Serializable (remember, in Java, multiple inheritance can be achieved only via interfaces), thus forcing all collection classes to be serializable; however, Sun apparently decided against this possibly too restrictive approach. Classes that do not declare themselves as serializable normally have a perfectly valid reason for doing so, typically because serialization simply does not make sense for the class. Classes consisting only of static methods (Math and Collections, for example) never produce any objects, while other classes with very intricate dependencies on underlying JVMs (Thread and System, for example) do not support serialization for complex technical and class-specific reasons. NOTE The companion website includes a console utility called IsSerializable that you can use to interactively determine the serializability of any fully qualified class. If no command-line argument is passed to this utility, it lists the serializability of all classes in the 1.4 java.lang package.


Serialization: The Pitfalls

Java's built-in support for object serialization can be a great help when it is working transparently for you. Unfortunately, more than a few pitfalls are waiting to shatter that transparency if you don't arm yourself with an understanding of the trickier sides of serialization.

No Deserialization of Unknown Objects

If you imagine for a second that you just left your desk for a coffee break between running your earlier SaveStringBuffer example and running LoadStringBuffer, and if you also imagine that some joker substituted behind your back your sb.ser for another sb.ser containing a serialized ButYouDontHaveSuchAClassAnywhere-OnYourClassPath object, you can see why ObjectInputStream.readObject() also forces you to deal with a possible ClassNotFoundException. Not surprisingly, the deserialization engine hiding behind readObject() needs to have access to the classes of any objects it is asked to reconstitute. If these classes are not already present in the JVM or are not loadable via the current ClassLoader, deserialization fails. ClassNotFoundExceptions are not uncommon when working on RMI apps, but they should not be a problem in standalone games that rely on conventional (that is, default) ClassLoader behavior. After all, serialized objects normally are deserialized by exactly the same game program that serialized them in the first place.

Nonserializable Object Graphs

If you modify the SaveStringBuffer example to save a HashSet object containing the following mixed bag of elements:

Set bag = new HashSet();
bag.add("Hello");
bag.add(new Object());
bag.add(new Integer(2003));
oos.writeObject(bag);


then running this new version again produces that all-too-common runtime exception when dealing with serialization:

java.io.NotSerializableException: java.lang.Object


Although HashSet implements the Serializable interface and, therefore, should be serializable at face value, a HashSet's runtime content might not be. In this case, the Object included in the bag isn't serializable. When you ask an ObjectOutputStream to serialize a single object and that object refers to other objects, and those other objects, in turn, refer to more objects, and so on, then all of those objects (that is, the entire object graph) are serialized. If any object in this graph is not itself serializable, the entire operation fails with a NotSerializableException. Screenshot illustrates this very common serialization pitfall.

Screenshot The entire object graph must consist of Serializable objects. For example, any image that is part of an object graph to be serialized will cause serialization of the entire graph to fail.

Java graphics 15fig01.gif


The HashSet containing a plain Object is a good example of the more common scenario of wanting to serialize a container object that is polluted, as far as serialization is concerned, by one or more elements that are not serializable.

Serializing Arrays

Let's check the precise method signatures for ObjectOutputStream.writeObject() and ObjectInputStream.readObject():

public final void writeObject(Object obj)
 throws IOExceptionpublic final Object readObject()
 throws ClassNotFoundException, IOException


You should see what, at first sight, might look like sloppy parameter and return value typing: You probably wonder why these methods use a Serializable instead of an Object, as follows

public final void writeObject(Serializable aSerializable)
 throws IOException public final Serializable readObject()
 throws ClassNotFoundException, IOException


After all, only Serializable objects are accepted, respectively produced, by these methods. The existing method signatures can be explained by the quasi-exception to the rule: arrays. Arrays are full-fledged objects in Java; given that any <array> instanceof Serializable test always returns true, passing arrays to the serialization engine poses no problems as long as every array element is serializable, too. This caveat is essentially the same as the one that applies to serializing container objects.

Class Evolution and Versioning

Another serialization pitfall lies in attempting to use serialization for long-term persistence of objects. By "long-term," we mean any stretch of time long enough to accommodate changes to the classes that spawned the serialized objects. Let's say you wrote a chess game that allows any game in progress to be saved. The following classes would be at the heart of your implementation:

public class ChessGameState implements Serializable {
private ChessBoard theBoard;
private ChessPlayer white;
private ChessPlayer black;
}
class ChessPlayer implements Serializable {
private String name;
}
class ChessBoard implements Serializable {
private ChessPiece[][] squares;
}


Saving the game in this chess scenario boils down to serializing the singleton ChessGameState object and storing this serialized object to hard disk. Now imagine your v1.0 chess game has become a popular success (maybe because you're using the classic Internet marketing tactic of giving your product away for free in the hope of building critical mass market share). The success of MyChess v1.0 enables you to attract venture capital to fund the development of an improved v2.0 version for which you intend to charge hard cash. On the to-do list for v2.0 is fixing one v1.0 bug: In v1.0, you forgot to implement the 50 moves rule when one player is left with nothing but his king. Fixing this should be trivial. To address this rather embarrassing bug, you add an int field and associated logic to ChessGameState:

public class ChessGameState implements Serializable {
private ChessBoard theBoard;
private ChessPlayer white;
private ChessPlayer black;
private int fiftyMovesCounter;
}


But now, regression testing of MyChess v2.0 unearths a problem. Restoring a saved v1.0 game crashes with the following runtime exception:

java.io.InvalidClassException: ChessGameState; local class incompatible: stream classdesc serialVersionUID = 8698504323191665099,
local class serialVersionUID = 3813626932283615031


The deserialization engine detected that the v2.0 ChessGameState class present in the JVM is not the same v1.0 ChessGameState class from which the object was serialized in the past. In other words, the deserialization engine noticed and objected to the fact that the fiftyMovesCounter field was added. To avoid this type of class evolution problem, you need to ensure that your evolving classes remain serialization version–compatible. What does this mean? Serialization version–compatible classes are classes that, as far as serialization is concerned, are identical. This equality is evaluated via the intermediary of 64-bit class signature numbers called serialVersionUIDs. For output (that is, serialization), these class signatures either are calculated on the fly by the serializing engine or can be explicitly declared in a serializable class by the programmer:

private static final long serialVersionUID = 3786198910865385080L;


Upon input (that is, deserialization), the serialVersionUID embedded in the serialized object stream is compared with the serialVersionUID of the class of the object. If the two differ, deserialization fails. The serialVersionUID for a class is calculated by essentially taking the fully qualified class name and all the declared nontransient instance fields for the class and forming a value similar to an object's runtime hash code. So adding, removing, or changing an instance field alters the value of a computed serialVersionUID and leads the deserializer to throw an InvalidClassException. To get around this problem, you can explicitly tag a class with a static serial VersionUID field that stays constant across class changes. That way, any changes that would otherwise change a computed serialVersionUID will not result in an InvalidClassException during deserialization. As long as any changes to the class are compatible as far as serialization goes, long-term persistence of objects of evolving classes can be made to work. Serialization-incompatible class changes include these:

  • Deleting or changing the type of instance fields
  • Changing the name of an instance field (watch out when using obfuscation)
  • Changing the position of the class in the inheritance tree
  • Making incompatible changes to the logic of any private writeObject() or readObject() methods

Serialization-compatible class changes include these:

  • Adding instance fields
  • Changing the access scope of any field

This list of changes consists of only the most common changes. The full list is beyond the scope of this tutorial but can be analyzed in the Java Object Serialization Specification.

Overriding Default Serialization Engine Behavior

ObjectOutputStream serializes object graphs into a byte stream formatted according to a serialization stream protocol. The default protocol is essentially little more than a binary representation of object field values and a clever way of dealing with object types and references. From a game programmer's point of view, this format might be problematic in that the protocol's simplicity also results in data transparency. Serialized objects can easily be hacked by cheating players blessed with some Java knowledge. For example, typical saved games record the player's current score, how many lives a player has left, and what kinds of objects or powers the player has so far accumulated. These attributes form juicy hacking targets (to a cheat, that is). To foil hackers and cheats, persistent objects need be made as opaque as possible. To this end, the serialization API allows its default behavior to be overridden. This is not done by extending the ObjectOutputStream and ObjectInputStream classes and overriding their public writeObject() and readObject() methods, as you might expect (as you saw earlier, the methods cannot be overridden because they are final). Instead, the API employs nongeneric techniques that we will now explore.

Adding Private writeObject() or readObject() Methods

Let's say you have writeObject() and the following game state classes:

class GameState implements Serializable {
private Player player;
}
class Player implements Serializable {
private short livesLeft;
}


Saving your game's singleton GameState object using default serialization would leave your game open to fairly trivial hacking of a player's lives left status. What you need is extra data armor to protect the serialized livesLeft fields from being trivially altered with a file editor (typically a hex editor). One approach supported directly by the serialization engine is to implement in your vulnerable Serializable class private methods that write and read the fields of your class manually, without the help of the serialization engine. This then opens the door to writing field values in an obfuscated, or even strongly encrypted, format. These complementary private methods must be declared using the following signatures:

private void writeObject(ObjectOutputStream s) throws IOException private void readObject(ObjectInputStream s) throws IOException


Listing 15.3 shows how you would writeObject() and enhance the Player class to exploit this technique.

Listing 15.3 GameState.java
import java.io.*;
public class GameState implements Serializable {
private Player player = new Player(3);
public static void main (String[] args) {
 try {
 FileOutputStream fos = new FileOutputStream("game.ser");
 ObjectOutputStream oos = new ObjectOutputStream(fos);
 oos.writeObject( new GameState() );
 oos.close();
 FileInputStream fis = new FileInputStream("game.ser");
 ObjectInputStream ois = new ObjectInputStream(fis);
 System.out.println( ois.readObject() );
 ois.close();
 } catch (Exception exception) {
 System.out.println(exception);
 }
}
public String toString() {
 return "GameState[ " + player.toString() + " ]";
}
}
class Player implements Serializable {
private short livesLeft;
/*package*/ Player(int lives) {
 livesLeft = (short) lives;
}
private void writeObject(ObjectOutputStream s) throws IOException
{
 System.out.println("Encrypting lives left...");
 s.writeByte(- livesLeft);
}
private void readObject(ObjectInputStream s) throws IOException {
 System.out.println("Decrypting lives left...");
 livesLeft = (short) - s.readByte();
}
public String toString() {
 return "Player[" + livesLeft + "]";
}
}


When running GameState's main(), you can see from the console output that the additional writeObject() and readObject() methods were called not by our own code, but by the serialization engine. The serialized GameState object (containing one serialized Player object) now has the player's livesLeft variable stored, not in what cryptologists call plain text, but in an encrypted form. To keep this example as short as possible, the "encryption" used here is nothing more than some bits-massaging negation of the value (using the unary minus operator). In a real-life game, you might want to give your game's hackers some tougher encryption to chew on.

Explicitly Specifying Which Fields Should be Serialized

Adding private writeObject() and readObject() methods takes you a big step away from the default serialization engine's behavior because the methods essentially take over total responsibility for the serializing and deserializing of your particular class's objects. For classes that contain lots of fields, most of which do not require encryption or obfuscation, this all-or-nothing approach undermines some of the benefits of using the serialization API in the first place. You need a way to use the default mechanism for most of your fields, but deal with the minority of likely hacking targets in a customized way. Java's Serialization framework lets you deal with this problem by explicitly specifying which fields should be handled via the default mechanism. Reminiscent of the serialVersionUID field approach, this is done by defining in your Serializable class another magic private final static field, called serialPersistentFields, as follows:

class Player implements Serializable {
private String playerName;
private short livesLeft;
private static final ObjectStreamField[] serialPersistentFields = {
new ObjectStreamField("playerName", String.class)
};
 // .. Player methods here.
}


As you can see, this serialPersistentFields field must be typed as a one-dimensional array of ObjectStreamField objects. This array defines which instance fields, present and declared in the current class, should be subjected to the default serialization mechanism. Each ObjectStreamField array element object defines a field by way of its declared name and type; field scope and any other field attributes are unnecessary and are therefore writeObject() and objectignored. When you include the serialPersistentFields field in a Serializable class, you also need to alter the way your writeObject() and readObject() methods are implemented by invoking ObjectOutputStream.defaultWriteObject() and ObjectInputStream.defaultReadObject(), respectively:

private void writeObject(ObjectOutputStream s) throws IOException
{
 s.defaultWriteObject();
 System.out.println("Encrypting lives left...");
 s.writeByte(- livesLeft);
}
private void readObject(ObjectInputStream s) throws IOException {
 s.defaultReadObject();
 System.out.println("Decrypting lives left...");
 livesLeft = (short) - s.readByte();
}


To summarize, by mixing the following two techniques:

  • Adding a private writeObject() or readObject() duo to your Serializable class
  • Explicitly specifying which fields need to be serialized using the default mechanism

You can tune the way your objects are serialized on a field-by-field basis instead of on the often too coarse level of entire classes. This flexibility enables you to encrypt any sensitive fields that form part of your game's persistent state.



   
Comments