When you setup a web server there are generally two types of caching that you need to configure:

1. HTML resources are expired immediately so that any changes made to a site are quickly picked up by existing users.
2. You set everything else (e.g. images, CSS, Javascript) to expire at some distance time in the future.

This caching scheme is covered in Two Simple Rules for HTTP Caching along with some ideas about how to manage changes.

Now that HttpWatch 6.0 supports Firefox we wanted to cover some differences in the way that it handles caching compared to Internet Explorer. The use of long expiration times (item 2 above) still directly applies to Firefox but there are some subtle differences in the configuration of item 1.

In the previous post, we broke item 1) down into:

• Sometimes dynamic HTML pages need to be fetched from the server whenever they are displayed – even when the back button is used. For example, pages showing the state of a bank account or online order.
• Static HTML pages, such as contact pages, FAQs or sitemaps, can make use of caching if they have a Last-Modified response header allowing the brower to revalidate them as required

The rest of this post covers two important differences in Firefox that the affect caching of HTML pages.

1. Using no-cache Doesn’t Stop Caching in Firefox

You can prevent any caching in IE by simply setting this response header:

Cache-Control: no-cache

Pages that use this header aren’t stored in the cache and IE will always reload them from the server; even if you use the Back button to reach them.

Here’s an example in the HttpWatch online store where we show that an order has already been processed if you click on Submit button followed by the Back button:

Of course, we always redirect POST requests to a GET to avoid breaking the Back button.

However, this response header doesn’t prevent caching in Firefox. It just means that Firefox will never read the page from the cache during a normal visit unless it has been re-validated by sending a GET request. Also, if the page is reached using the Back button there’s no round-trip to the server and Firefox simple re-loads the page directly from the cache.

So how can caching be turned off in Firefox? The simple answer is that it cannot. Firefox relies on having a copy of every page in the cache for commands such as File->Save As and View Page Source. However, you can control where the page is cached and whether the cached entry can be used for display purposes.

The following response header in Firefox prevents persistent caching, by forcing the page into the in-memory cache:

Cache-Control: no-store

This header also prevents reuse of the cached version of the page and triggers an HTTP GET if the page is navigated to using the Back button.

These two header values can be combined to get the required effect on both IE and Firefox:

Cache-Control: no-cache, no-store

As shown here in the HttpWatch header tab:

UPDATE: The lack of no-cache support is limited to early versions of Firefox 3.0 and was caused by a bug. Although, no-cache alone should now work it is possible that visitors to your site will be running affected versions of Firefox.

2. If You Don’t Specify an Expiration Date Firefox May Set One for You

When IE encounters an HTTP response with no Expires header it just assumes that it can never automatically reuse the cached entry without re-validating it against the server. With the default setting of ‘Check for newer versions of stored pages’ in IE set to ‘Automatically’, it will normally do this just once per session.

This provides a reasonable way of controlling the caching of static HTML content. The user will get the latest version if they open a fresh copy of IE and the cached version of the page will be used until they close IE.

Firefox handles the lack of an Expires header differently. If there is a Last-Modified response header it uses a heuristic expiration value as specified in the HTTP 1.1 spec RFC2616:

Also, if the response does have a Last-Modified time, the heuristic
expiration value SHOULD be no more than some fraction of the interval
since that time. A typical setting of this fraction might be 10%.


The calculation is as follows:

Expiration Time = Now + 0.1 * (Time since Last-Modified)

For example, if the last change to your static HTML file was 100 days ago, the expiration date will be set to 10 days in the future. Here’s an example from the Cache tab in HttpWatch of a page that had no Expires header:

Firefox has automatically set an expiration date in 8 days time because the page has not changed for approximately 80 days.

This means that to retain control of your HTML pages, as we discussed in Two Simple Rules for HTTP Caching, you should set up a suitable Expires header value on your web server for your HTML content as well as other resources such as images and CSS files.

Conclusions

In order to ensure consistent caching behaviour with IE and Firefox you should:

• Always specify an Expires header. It will normally be set to -1 for immediate expiration of HTML pages or a date well into the future for other resources such as images, CSS and Javascript
• If you want to force a page to be reloaded, even with the Back button, then use Cache-Control: no-cache, no-store

HttpWatch version 6.0 has been released and is now available for download.

Any customers eligible for a free upgrade to HttpWatch Professional can install the latest version using their existing license key. If you’re not sure whether your license will work with version 6.0 go to Help->Check For Updates in HttpWatch and it will show you any available updates or upgrades.

What’s New?

The major new feature in this release is support for Firefox 2.0 and 3.0 on Windows. If you install version 6.0 you’ll get an HttpWatch status bar icon in Firefox:

Clicking on the icon will open the same HttpWatch user interface that’s previously only been available in Internet Explorer:

A few other areas have also been improved to accommodate Firefox. First of all there’s a new Properties window that displays information about the browser that recorded a log file:

The comment field allows simple notes and annotations to be saved with each log file.

The automation interface has been updated to include the data from the Properties window and to allow automation of HttpWatch within Firefox:

We’ve also done some work on the way that Start times are displayed. You can now choose between time offsets, local time and GMT/UTC. The latter can be particularly useful when trying to compare log files from other monitoring tools and other locations world-wide:

And finally, HttpWatch Studio now has tabs making it easier to switch between log files:

Compatibility with Version 5.x

We’ve kept the file format unchanged from version 5.x, so you can readily exchange log files with version 6.0. Although you cannot view some of the information such as the comment and browser version in version 5.x, this data is maintained if you re-save the log file and then re-open it in version 6.0.

The automation interface maintains backwards source compatibility with interpreted script clients and binary compatibility with existing compiled clients (e.g. C#, C++, VB.Net). However, if you attempt to recompile an automation program with version 6.0, you may need to make some source code changes. For example, the New method has moved from the Plugin object onto the Firefox and IE objects.

Jake Howlett at CodeStore has written an interesting post about how he used HttpWatch Basic Edition to diagnose a customer’s performance problem:

To record the HTTP transaction log I had them install the free Basic Edition of HttpWatch. All they had to do then was start recording, open the application, open a new form, open a new document, edit the document, save the document etc and then stop recording. The result is a .HWL file which I can then open in my version of HttpWatch’s “Studio”. I can then see all headers, response code and the amount of time taken for all transactions involved.

The Professional Edition of HttpWatch allowed him to look at the customer’s log file and find the cause of the performance problem by looking at the HTTP response headers returned by the server.