There’s a post over on the Gmail blog by Wiltse Carpenter, the Tech Lead for Gmail Performance, about how they used HttpWatch and other tools to speed up the login for Gmail.

Here’s what they said about HttpWatch:

“The Httpwatch plug-in for Internet Explorer was one that proved easy to use and provided us with most of the information we needed. It really helps that we can capture and save browser traces with it too.”

To reduce the time required to login they looked for ways to minimize the number of requests and reduce the overhead of each request:

“We spent hours poring over these traces to see exactly what was happening between the browser and Gmail during the sign-in sequence, and we found that there were between fourteen and twenty-four HTTP requests required to load an inbox and display it. To put these numbers in perspective, a popular network news site’s home page required about a 180 requests to fully load when I checked it yesterday. But when we examined our requests, we realized that we could do better. We decided to attack the problem from several directions at once: reduce the number of overall requests, make more of the requests cacheable by the browser, and reduce the overhead of each request.”

If you’re tuning a web page’s performance there are two types of visitors that you need to be concerned about:

1. A new visitor to your site who won’t have any of your pages, scripts, CSS or images in their browser cache.
2. An existing user of your site who will have your cacheable pages, scripts, CSS or images in their browser cache.

Visitor type 1) is said to be in the Empty Cache state and Vistor type 2) is said to be in the Primed Cache state. Optimizing for visitors with an empty cache is important as their initial impressions of your site will be affected by how quickly its pages are loaded.

The performance of any page is never going to worse for visitors with a primed cache, but minimizing the load on your site caused by existing users through the use of effective caching will reduce your bandwidth costs and server load.

Your can manually simulate these two scenarios with HttpWatch. For example, you could run an Empty Cache test on our home page (www.httpwatch.com) using these steps:

• Open a HttpWatch in IE and click on Tools->Clear Cache or use the keyboard shortcut Alt+C. If you haven’t cleared your browser cache recently this could take a few minutes:
  
• Click on Record and go to www.httpwatch.com
• HttpWatch will then display a time chart with the page load time:
  

To run the Primed Cache test you would first need to ensure that the page has already been visited and then re-visit it in a new instance of IE. You shouldn’t re-use the same instance of IE because there may some images held in memory from the first visit to the page. To perform the Primed Cache test you would need to:

• Visit www.httpwatch.com in IE to prime the cache
• Close down IE and start a new instance
• Open HttpWatch and click on Record
• Go to www.httpwatch.com
• HttpWatch will then display a time chart with the page load time:
  

Of course, with a test like this you really want to run it automatically. The HttpWatch Automation interface (document in the HttpWatch Help file) allows you to do this in a few lines of code. Here’s the code in C# for the Empty Cache test:

// Set a reference to the HttpWatch COM library
// to start using the HttpWatch namespace
//
// This code requires HttpWatch version 6.x
//
 
using HttpWatch;                
 
namespace EmptyCacheTest
{
    class Program
    {
        static void Main(string[] args)
        {
            string url = "https://www.httpwatch.com/download/";
            Controller controller = new Controller();
 
            // Create an instance of IE (For Firefox use
            // controller.Firefox.New("") )
            Plugin plugin = controller.IE.New();                
 
            // Clear out all existing cache entries
            plugin.ClearCache();                
 
            plugin.Record();
            plugin.GotoURL(url);                
 
            // Wait for the page to download
            controller.Wait(plugin, -1);                
 
            plugin.Stop();                
 
            // Find the load time for the first page recorded
            double pageLoadTimeSecs =
                plugin.Log.Pages[0].Entries.Summary.Time;                
 
            System.Console.WriteLine( "The empty cache load time for '" +
                url + "' was " + pageLoadTimeSecs.ToString() + " secs");                
 
            // Uncomment the next line to save the results
            // plugin.Log.Save(@"c:\temp\emptytestcache.hwl");                
 
            plugin.CloseBrowser();
        }
    }
}

and here’s the Primed Cache test. Notice how we use controller.IE.New() a second time to ensure that a new instance of IE is started:

// Set a reference to the HttpWatch COM library
// to start using the HttpWatch namespace
//
// This code requires HttpWatch version 6.x
//
using HttpWatch;               
 
namespace PrimedCacheTest
{
    class Program
    {
        static void Main(string[] args)
        {
            string url = "https://www.httpwatch.com/download/";
            Controller controller = new Controller();               
 
            // Create an instance of IE (For Firefox use
            // controller.Firefox.New("") )
            Plugin plugin = controller.IE.New();               
 
            // Visit the page to prime the cache
            plugin.GotoURL(url);
            plugin.Record();
            controller.Wait(plugin, -1);               
 
            // Shutdown IE and create a new instance
            plugin.CloseBrowser();
            plugin = controller.IE.New();               
 
            plugin.Record();               
 
            // Visit the page a second time with the cache primed
            plugin.GotoURL(url);
            controller.Wait(plugin, -1);               
 
            plugin.Stop();               
 
            // Find the load time for the first page recorded
            double pageLoadTimeSecs =
                plugin.Log.Pages[0].Entries.Summary.Time;               
 
            System.Console.WriteLine( "The primed cache load time for '" +
                url + "' was " + pageLoadTimeSecs.ToString() + " secs");               
 
            // Uncomment the next line to save the results
            // plugin.Log.Save(@"c:\temp\emptytestcache.hwl");               
 
            plugin.CloseBrowser();
        }
    }
}

BTW, everything we’ve mentioned in this blog post works with the free Basic Edition of HttpWatch as well as the Professional Edition.

Web 2.0 is a term often used to describe next generation web sites that have moved beyond the simple page request->process->response cycle and are utilizing services on the web server to return data that can be rendered without making page transitions. The result is often a more responsive user interface that closely mimics a desktop application.

The technology to make HTTP calls from JavaScript embedded in an HTML page was first introduced for general use by Microsoft in IE5 in order to support Outlook Web Access way back in 1999. However, the XmlHttpRequest object was not widely used until it was adopted by Mozilla in 2002. In 2005 the programming model was given the name Ajax which has now been widely recognised and stands for Asynchronous JavaScript + XML.

From a performance perspective Ajax has two main benefits:

1. It can reduce the number of round trips by minimizing the number of page transitions in a web application
2. It can reduce the size of uploaded and downloaded data because it allows the web programmer to control exactly what is transferred. For example, if a user performs a search the data can be returned in a compact data format (e.g. JSON ) rather than HTML.

In order to illustrate how Ajax can increase the speed and usability of a site let’s first look at a traditional web site and then compare it to an Ajax example.

This is what happens on Expedia.com when we try to book a flight but we specify a partial or mispelled city code. In this case we typed in “LOND” instead of “LON” or “London”. We expectantly press the submit button hoping for a list of possible flights, but instead we’re directed to an error page:

Using HttpWatch we can see that Expedia took 4.4 seconds, 30 round trips and 156K of data (of which 41K was uploaded)  to display the error page:

Even if there are no further mistakes we will still have to make further page transitions before we get the results we’re expecting.

Let’s look at a similar example on BA.COM where Ajax has been used:

Here we did exactly the same thing – typed “LOND” instead of London. Instead of having to press submit, fetch a new choices page (with all linked page components) and drag the user through an additional form, the page used background processing to query the server and display a list of possible choices.

The beauty of this approach is that is that it didn’t require any user interaction, is very fast and it didn’t leave the search page. Here’s the Ajax request in HttpWatch:

BA.COM uses the DWR framework for Ajax to request the possible airports. The framework returns a JavaScript list used to render the content into the dropdown list control. The single HTTP call required is much smaller and faster than the Expedia model where the whole page must be fetched and rendered.

Due to the inconsistencies between the programmable object models on different browsers, most developers now choose a framework to ease JavaScript development and most provide Ajax helper functions that wrap XmlHttpRequest. Java’s DWR has RPC style support. ASP.Net’s Ajax library, Dojo, Yahoo’s YUI and Google’s GWT support a whole suite of UI and control extensions as well as offering some kind of Ajax support. Similarly, Prototype and jQuery are two extremely lightweight and popular JavaScript libraries.

Let’s say you wanted to populate a list box with city codes based on the country a user selects. In Web 1.0 you would have to submit a form making a round-trip to the server with your choice of country so that the server could render the appropriate list of choices on the resulting HTML page.

With Ajax, this can all be done in the background. A lot of services use the XML data format, but there’s no restriction on the content type that XmlHttpRequest can send or receive. It could be HTML, JSON or any other format that the page’s JavaScript can parse.  Here’s an example of using the jQuery library to populate a list control from an Ajax request:

<script type="text/javascript" src="jQuery.js"></script>
...
<h2>Products</h2>
<div id="products">(fetching product list ...)</div>
<script>
...
jQuery("#products").load("productListHTML.aspx");
...
</script>


The products <div> is initially empty, but the load() method wraps XmlHttpRequest and fetches some HTML from the server and uses it to populate the <div>.

One of the drawbacks people found with using Ajax was that they ended up breaking the back button. Clicking the Back button in some Ajax applications did not go to the previous logical operation. For example, if you were viewing the results of a search you would expect Back to take you to the search page so that you could modify the search criteria. Instead, the user would be taken to whatever happened to be the previous page in the browser’s history list.  In some cases, they may even have been the previous web site that the user had visited.

More recently though, browsers have exposed object models to help Ajax applications manipulate the browser history artificially so that they can correct the flow of operations as the user uses the forward and back buttons.