Ajax calls are just like any other HTTP request that might be used to build a web page. However, due to their dynamic nature people often overlook the benefit of caching them.

Rule 14 of High Performance Web Sites states:

Make Ajax Cacheable

Make sure your Ajax requests follow the performance guidelines, especially having a far future Expires header.

The rest of this blog post covers two important facts that will help you understand and effectively apply caching to Ajax requests.

Fact #1 : Ajax Caching Is The Same As HTTP Caching

The HTTP and Cache sub-systems of modern browsers are at a much lower level than Ajax’s XMLHttpRequest object. At this level, the browser doesn’t know or care about Ajax requests. It simply obeys the normal HTTP caching rules based on the response headers returned from the server.

If you know about HTTP caching already, you can apply that knowledge to Ajax caching. The only real difference is that you may need to setup response headers in a different way to static files.

The following response headers are used to make your Ajax cacheable:

• Expires: This should be set to an appropriate time in the future depending on how often the content changes. For example, if it is a stock price you might set an Expires value 10 seconds in the future. For a photograph, you might set a far futures Expires header because you don’t ever expect it to change. The Expires header allows the browser to reuse the cached content for a period of time and avoid any unnecessary round-trips to the server.
• Last-Modified: It’s a good idea to set this so that the browser can use an If-Modified-Since header in a conditional GET request to check its locally cached content. The server would respond with a 304 status code if the data doesn’t require an update.
• Cache-Control: If appropriate, this should be set to ‘Public’ so that intermediate proxies and caches can store and share the content with other users  It will also enable caching of HTTPS requests on Firefox.

Of course, this doesn’t apply if you use the POST method in your Ajax requests, because POST requests are never cached. You should always use the POST method if your Ajax request has side effects, e.g. moving money between bank accounts.

We’ve setup a Ajax caching demo that shows these headers in action. In HttpWatch, you can see that we’ve set all three of these headers in the Ajax response:

If you click on the ‘Ajax Update’ button at regular intervals, the time only changes approximately once a minute because the Expires header is set to one minute in the future. In this HttpWatch screenshot you can see that repeated clicks of the update button cause Ajax requests that read directly from the browser cache and result in no network activity (i.e. the value in the Sent and Received columns is zero bytes) :

The final click at 1:06.531 does result in an Ajax request that requires a network round-trip, because the cached data is now more than one minute old. The 200 response from the server indicates that a fresh copy of the content was downloaded.

Fact #2: IE Doesn’t Refresh Ajax Based Content Before Its Expiration Date

Sometimes Ajax is used at load time to populate sections of a page (e.g. a price list). Instead of being triggered by a user event such as a button click, it is directly called from the Javascript that runs when the page is loaded. This makes the Ajax call behave as if it were a request for an embedded resource.

As you develop a page like this, it is tempting to refresh the page in an attempt to update the embedded Ajax content. With other embedded resources such as CSS or images, the browser automatically sends the following types of requests depending on whether F5 (Refresh) or Ctrl+F5 (Forced Refresh) is used:

1. F5(Refresh) causes the browser to build a conditional update request if the content originally had a Last-Modified response header. It uses the If-Modified-Since request header so that server can avoid unnecessary downloads where possible by returning the HTTP 304 response code.
2. Ctrl+F5 (Forced Refresh) causes the browser to send an unconditional GET request with a Cache-Control request header set to ‘no-cache’. This indicates to all intermediate proxies and caches that the browser needs the latest version of the resource regardless of what has already been cached.

Firefox propagates the type of refresh down to any Ajax request that is made during the loading of the page and will therefore update any Ajax derived content as if it were an embedded resource. This screen shot of the HttpWatch plugin-in for Firefox shows the effect of refreshing our Ajax Caching demo page:

Firefox ensured that the Ajax request was issued as a conditional GET. The server responds with a 304 in our demo if the cached data is less than 10 seconds old or a 200 response with the updated content if it is out of date.

In Internet Explorer, the load-time Ajax request is treated as though it is unrelated to the rest of the page refresh and there is no propagation of the user’s Refresh action. No GET request is sent to the server if the cached Ajax content has not yet expired. It simply reads the content directly from the cache, resulting in the (Cache) result value in HttpWatch. Here’s the effect of F5 in IE before the content has expired:

Even with Ctrl+F5, the Ajax derived content is still read from the cache:

This means that any Ajax derived content in IE is never updated before its expiration date – even if you use a forced refresh (Ctrl+F5). The only way to ensure you get an update is to manually remove the content from the cache. In HttpWatch, you can do this using the Tools menu:

An often overlooked aspect of web performance tuning is the effect of using the HTTPS protocol to create a secure web site. As applications move from the desktop onto the web, the need for security and privacy means that HTTPS is now heavily used by web sites that need to be responsive enough for every day use.

The tips shown below may help you to avoid some of the common performance and development problems encountered on sites using HTTPS:

Tip #1: Use Keep-Alive Connections

Whenever a browser accesses a web site it must create one or more TCP connections. That can be in lengthy operation even with normal unsecured HTTP.  The use of Keep-Alive connections reduces this overhead by reusing TCP connections for multiple HTTP requests. The screen shot below from HttpWatch shows the TCP connection time over HTTP is approximately 130 milliseconds for our web site when it is accessed from the UK:

Using HTTPS, the lack of Keep-Alive connections can lead to an even larger degradation in performance because an SSL connection also has to be setup once the TCP connection has been made. This requires several roundtrips between the browser and web server to allow them to agree on an encryption key (often referred to as the SSL session key) . The corresponding connection time to the same server using HTTPS is nearly four times longer as it includes the HTTPS overhead:

If a HTTPS connection is re-used the overhead of the both the TCP connection and SSL handshake are avoided.

Some web browsers and servers now allow the re-use of these SSL session keys across connections, but you may not always have control over the web server configuration or the type of browser used.

Tip #2: Avoid Mixed Content Warnings

In a previous blog post we talked about the confusing and annoying dialog that is displayed by default if a secure page uses any HTTP resources:

To stop this warning dialog interrupting the page download you need to ensure that everything on the page is accessed over HTTPS. It doesn’t have to be from the same site but it must use HTTPS. For example, the addition of HTTPS support allows the Google Ajax libraries to be safely loaded from secure pages.

Tip #3: Use Persistent Caching For Static Content

If you follow Tip #2 then everything that your page needs, including images, CSS and Javascript, will be accessed over HTTPS. You would normally want to persistently cache static content like this for as long as possible to reduce load on the web site and improve performance when a user revisits your site.

Of course, you wouldn’t want to cache anything on the disk that was specific to the user (e.g. HTML page with account details or a pie chart of their monthly spending). On most pages though, nearly all of the non-HTML content can be safely stored, shared and re-used.

There seems to be some confusion over whether caching is possible with HTTPS. For example, Google say this about Gmail over HTTPS:

You may find that Gmail is considerably slower over the HTTPS connection, because browsers do not cache these pages and must reload the code that makes Gmail work each time you change screens.

Although, 37Signals acknowledge that in-memory caching is possible, they say that persistent caching is not possible:

The problem is that browsers don’t like caching SSL content. So when you have an image or a style sheet on SSL, it’ll generally only be kept in memory and may even be scrubbed from there if the user is low on RAM (though you can kinda get around that).

Even when you do your best to limit the number of style sheets and javascript files and gzip them for delivery, it’s still mighty inefficient and slow to serve them over SSL every single time the user comes back to your site. Even when nothing changed. HTTP caching was supposed to help you with that, but over SSL it’s almost all for naught.

In reality, persistent caching is possible with HTTPS using both IE and Firefox.

Using HttpWatch you can see if content is loaded from the cache by looking for (Cache) in the Result column or by looking for the blue Cache Read block in the time chart. Here is an example of visiting https://www.httpwatch.com with a primed cache in IE. You can see that all the static resources are reloaded directly from the cache without a round trip to the web server:

If you try this in Firefox 3.0 without adjusting your response headers you will see this instead:

The ‘200’ values in the Result column indicate that the static content is being reloaded even though the site was previously visited and a valid Expires setting was used. Unless you specify otherwise, Firefox will put HTTPS resources into the in-memory cache so that they can only be re-used within a browser session. When Firefox closes the contents of the in-memory cache is lost.

The about:cache page in Firefox confirms that these files are stored using the in-memory cache:

To allow persistent caching of secure content in Firefox you need to set the Cache-Control response header to Public:

This value moves HTTPS based content into the persistent Firefox disk cache and in the case of https://www.httpwatch.com it more than halves the page load time due to the decrease in network round trips and TCP/SSL connections:

The Public cache setting is normally used to indicate that the content is not per user and can be safely stored in shared caches such as HTTP proxies. With HTTPS this is meaningless as proxies are unable to see the contents of HTTPS requests. So Firefox cleverly uses this header to control whether the content is stored persistently to the disk.

This feature was only added in Firefox version 3.0 so it won’t work with older versions. Fortunately, the take up version 3.0 is reported to be much faster than IE 6 to 7.

Tip #4: Use an HTTPS Aware Sniffer

A network sniffer is an invaluable tool for optimizing and debugging any client server application. But if you use a network level tool like Netmon or WireShark you cannot view HTTPS requests without access to the private key that the web site uses to encrypt SSL connections.

Often organizations will not allow the use of private keys outside of a production environment, and even if you have administrative access to your web site, getting the private key and using it to decrypt network traffic is not an easy task.

HttpWatch was originally born out of the frustration of trying to debug secure sites. Viewing HTTPS traffic in HttpWatch is easy as it integrates directly with IE or Firefox, and therefore has access to the unencrypted version of the data that is transmitted over HTTPS.

The free Basic Edition shows high level data and performance time charts for any site, and lower level data for a number of well know sites including Amazon.com, eBay.com, Google.com and HttpWatch.com . For example, you can try it with HTTPS by going to https://www.httpwatch.com .

You can check SSL/TLS configuration our new SSL test tool SSLRobot . It will also look for potential issues with the certificates, ciphers and protocols used by your site. Try it now for free!

When you setup a web server there are generally two types of caching that you need to configure:

1. HTML resources are expired immediately so that any changes made to a site are quickly picked up by existing users.
2. You set everything else (e.g. images, CSS, Javascript) to expire at some distance time in the future.

This caching scheme is covered in Two Simple Rules for HTTP Caching along with some ideas about how to manage changes.

Now that HttpWatch 6.0 supports Firefox we wanted to cover some differences in the way that it handles caching compared to Internet Explorer. The use of long expiration times (item 2 above) still directly applies to Firefox but there are some subtle differences in the configuration of item 1.

In the previous post, we broke item 1) down into:

• Sometimes dynamic HTML pages need to be fetched from the server whenever they are displayed – even when the back button is used. For example, pages showing the state of a bank account or online order.
• Static HTML pages, such as contact pages, FAQs or sitemaps, can make use of caching if they have a Last-Modified response header allowing the brower to revalidate them as required

The rest of this post covers two important differences in Firefox that the affect caching of HTML pages.

1. Using no-cache Doesn’t Stop Caching in Firefox

You can prevent any caching in IE by simply setting this response header:

Cache-Control: no-cache

Pages that use this header aren’t stored in the cache and IE will always reload them from the server; even if you use the Back button to reach them.

Here’s an example in the HttpWatch online store where we show that an order has already been processed if you click on Submit button followed by the Back button:

Of course, we always redirect POST requests to a GET to avoid breaking the Back button.

However, this response header doesn’t prevent caching in Firefox. It just means that Firefox will never read the page from the cache during a normal visit unless it has been re-validated by sending a GET request. Also, if the page is reached using the Back button there’s no round-trip to the server and Firefox simple re-loads the page directly from the cache.

So how can caching be turned off in Firefox? The simple answer is that it cannot. Firefox relies on having a copy of every page in the cache for commands such as File->Save As and View Page Source. However, you can control where the page is cached and whether the cached entry can be used for display purposes.

The following response header in Firefox prevents persistent caching, by forcing the page into the in-memory cache:

Cache-Control: no-store

This header also prevents reuse of the cached version of the page and triggers an HTTP GET if the page is navigated to using the Back button.

These two header values can be combined to get the required effect on both IE and Firefox:

Cache-Control: no-cache, no-store

As shown here in the HttpWatch header tab:

UPDATE: The lack of no-cache support is limited to early versions of Firefox 3.0 and was caused by a bug. Although, no-cache alone should now work it is possible that visitors to your site will be running affected versions of Firefox.

2. If You Don’t Specify an Expiration Date Firefox May Set One for You

When IE encounters an HTTP response with no Expires header it just assumes that it can never automatically reuse the cached entry without re-validating it against the server. With the default setting of ‘Check for newer versions of stored pages’ in IE set to ‘Automatically’, it will normally do this just once per session.

This provides a reasonable way of controlling the caching of static HTML content. The user will get the latest version if they open a fresh copy of IE and the cached version of the page will be used until they close IE.

Firefox handles the lack of an Expires header differently. If there is a Last-Modified response header it uses a heuristic expiration value as specified in the HTTP 1.1 spec RFC2616:

Also, if the response does have a Last-Modified time, the heuristic
expiration value SHOULD be no more than some fraction of the interval
since that time. A typical setting of this fraction might be 10%.


The calculation is as follows:

Expiration Time = Now + 0.1 * (Time since Last-Modified)

For example, if the last change to your static HTML file was 100 days ago, the expiration date will be set to 10 days in the future. Here’s an example from the Cache tab in HttpWatch of a page that had no Expires header:

Firefox has automatically set an expiration date in 8 days time because the page has not changed for approximately 80 days.

This means that to retain control of your HTML pages, as we discussed in Two Simple Rules for HTTP Caching, you should set up a suitable Expires header value on your web server for your HTML content as well as other resources such as images and CSS files.

Conclusions

In order to ensure consistent caching behaviour with IE and Firefox you should:

• Always specify an Expires header. It will normally be set to -1 for immediate expiration of HTML pages or a date well into the future for other resources such as images, CSS and Javascript
• If you want to force a page to be reloaded, even with the Back button, then use Cache-Control: no-cache, no-store