Caching to Improve Performance

Data caching is an important architectural issue in many J2EE apps, especially as web apps tend to read data much more often than they update it. Caching is especially important in distributed apps, as without it the overhead of remote invocation is likely to prove far more of a problem. However, it's valuable even in collocated web apps such as our sample app, especially when we need to access data from persistent stores. Thus, caching is more than just an optimization; it can be vital to making an architecture work. Caching can offer quick performance wins and significantly improve throughput by reducing server load. This will benefit the entire app, not just those use cases that use cached data. However, caching can also lead to complex issues of concurrent code and cluster-wide synchronization. Do not implement caching (especially if such implementation is non-trivial) without evidence that it is required to deliver adequate performance; it's easy to waste development resources and create unwarranted complexity by implementing caching that delivers no real business value.

Caching Options

The sample app requires a moderate amount of heavily-accessed reference data. We will definitely want to cache this data, rather than run new Oracle queries every time any of it is used. Let's consider the caching options available to this type of app, and some common issues surrounding caching. Caching will deliver greater performance benefit the closer the cache is to the user. We have the following choices in the sample app (moving from the RDBMS, from which data is obtained, towards the client):

JSP tags and caching filters have the disadvantage that caching will benefit only a web interface. Both these types of caches don't "understand" the data they cache, only that they may hold the information necessary to respond to an incoming request for content. This won't be a problem for the sample app's initial requirements, as no other interfaces are called for. However, another caching strategy, such as cached Java objects in the web tier, might be required if we need to expose a web services interface. On the positive side, "front caching" doesn't care about the origin of data it caches, and will benefit all view technologies. It doesn't matter whether data was generated using XML/XSLT, Velocity, or JSP, for example. It will also work for binary data such as images and PDF.

Whatever caching strategy we use, it's important that we can disable caching to verify that we aren't covering up appalling inefficiency. Architectures that rely on caching to conceal severe bottlenecks are likely to encounter other problems.

A Caching Strategy for the Sample app

To implement a successful caching strategy in our sample app, we need to distinguish between reference data (which doesn't change, or changes rarely) and dynamic data, which must always be up-to-date. In this app, reference data, which changes rarely, but should be no more than one minute out of date, includes:

The most heavily requested dynamic data will be the availability of seating for each type of seat for each performance of a show. This is displayed on the "Display Show" screen. The business requirements mandate that this availability information can be no more than one minute out of date, meaning that we can only cache it briefly. However, if we don't cache it at all, the display show screen will require many database queries, leading to heavy load on the system and poor performance. Thus we need to implement a cache that is capable of accommodating relatively frequent updates. We could handle caching entirely by using "front caching". The "Welcome" and "Display Show" screens could be protected by a caching filter set to a one minute timeout. This approach is simple to implement (we don't need to write any code) and would work whatever view strategy (such as JSP or XSLT) we use. However, it has serious drawbacks:

These problems can be addressed by implementing caching in Java objects in the web tier. There are two more issues to consider before finalizing a decision: updates from other apps and behavior in a cluster. We know from the requirements that no other processes can modify the database (The one exception is when an administrator updates the database directly; we can provide a special internal URL that administrators must request to invalidate app caches after making such a change). Thus, we can assume that unless our app creates a tutorialing, any availability data it holds is valid. Thus, if we are running the app on a single server and caching data in Java objects, we never need to requery cached data: we merely need to refresh the cached data for a performance when a tutorialing is made. If the app runs in a cluster, this optimization is impossible, unless the cache is cluster-wide; the server that made the tutorialing will immediately reflect the change, but other servers will not see it until their cached data times out. We considered the idea of sending a JMS message that will be processed by all servers when a tutorialing is made, getting round this problem. However, we decided that caching with a timeout offered sufficiently good performance that the overhead of JMS message publication was hard to justify.

Such data synchronization issues in a cluster of servers are a common problem (they apply to many data access technologies, such as entity beans and JDO). Ideally we should try to design our sample app so that it can achieve optimum performance on a single server (there is no requirement that it ever will run in a cluster if it's fast enough) yet be configurable to work correctly in a cluster. Thus we should support both "never requery" and "requery on timeout" options, and allow them to be specified on deployment, without the need to modify Java code. As the one minute timeout value may change, we should also parameterize this.