How to check HTTP Compression with HttpWatch

calendarJuly 10, 2009 in Automation , C# , HttpWatch , Optimization

HTTP compression is one of the easiest and most effective ways to improve the performance of a web site. A browser indicates that it supports compression with the Accept-Encoding request header and the server
indicates the compression type in the Content-Encoding response header.

This screenshot from the Stream tab of HttpWatch shows these headers and the compressed content being returned from the server:

Compressed Page

Here’s another screenshot of a page that is not compressed:

Page with no compression

The browser still indicated that it accepted gzip and deflate compression, but the server ignored this and returned uncompressed HTML with no Content-Encoding header.

The easiest way to check the amount of compression achieved is to use the Content tab in HttpWatch to view a ‘200 OK’ response from the server:

Compressed Content

Don’t try checking for compression on other HTTP status codes. For example, a ‘304 Not Modified’ response will never have any compression saving because no content is returned across the network from the web server. The browser just loads the content fom the cache as shown below:

Content Tab with 304

So, if you want to see if compression is enabled on a page, you’ll either need to force a refresh or clear the browser cache to make sure that the content is returned from the server. The HttpWatch Automation API lets you automate these steps. Here’s an example using C# that reports how many bytes were saved by compressing a page’s HTML:

// Set a reference to the HttpWatch COM library
// to start using the HttpWatch namespace
//
// This code requires HttpWatch version 6.x
//
 
using HttpWatch;
 
namespace CompressionCheck
{
    class Program
    {
        static void Main(string[] args)
        {
            string url = "http://www.httpwatch.com";
            Controller controller = new Controller();
 
            // Create an instance of IE (For Firefox use
            // controller.Firefox.New("") )
            Plugin plugin = controller.IE.New();
 
            // Clear out all existing cache entries
            plugin.ClearCache();
 
            plugin.Record();
            plugin.GotoURL(url);
 
            // Wait for the page to download
            controller.Wait(plugin, -1);
 
            plugin.Stop();
 
            // Find the first HTTP/HTTPS request for the page's HTML
            Entry firstRequest = plugin.Log.Pages[0].Entries[0];
 
            int bytesSaved = 0;
            if (firstRequest.Content.IsCompressed)
            {
                bytesSaved = firstRequest.Content.Size
                                   - firstRequest.Content.CompressedSize;
            }
 
            System.Console.WriteLine("Compression of '" +
                firstRequest.URL + "' saved " + bytesSaved + " bytes");
 
            plugin.CloseBrowser();
        }
    }
}

Tip: If you access a web site through a proxy you may not see the effect of compression. This is because some proxies strip out the Accept-Encoding header so that they don’t have to process compressed content. Tony Gentilcore’s excellent ‘Beyond Gzipping’ talk at Velocity 2009 described how 15% of visitors to your site will not receive compression due to problems like this. A simple way to effectively bypass proxy filtering for testing purposes is to use HTTPS if it is available. For example, try https://www.httpwatch.com if you don’t see compression on http://www.httpwatch.com.

Using Firefox, HttpWatch and Internet Explorer on Vista 64-bit

calendarMay 9, 2009 in Automation , C# , Firefox , HttpWatch , Internet Explorer , Javascript

The 64-bit version of Windows Vista is becoming increasing popular due to its ability to access more than 4 GB of physical memory. To an ordinary Windows user, there are few noticeable differences from the 32-bit version because it can run both 32-bit and 64-bit programs.

In this post we wanted to answer some of the questions you may have about Windows Vista 64-bit and the effect it has on the use of Firefox, HttpWatch and Internet Explorer (IE).

How can I tell if a program is running in 32-bit or 64-bit mode?

The Task Manager on Vista 64-bit shows 32-bit tasks with a ‘*32′ after the process name. If there’s no ‘*32′ after the name then it is running in 64-bit mode. This screen shot shows a 32-bit process called page_check running on Vista 64-bit:

32-bit Program Running on Vista 64-bit

Why does IE run in 32-bit mode by default on Vista 64-bit?

Clicking on the launch bar IE icon in Vista 64-bit starts the 32-bit version of IE:

32-bit version of IE on Vista 64-bit

A 64-bit version of IE is available but you have to explicitly select it from the Start menu:

64-bit version of IE

Microsoft has done this because the 64-bit version of IE can only load 64-bit browser extensions and most  extensions, such as the Adobe Flash player, are only available in 32-bit.

Here’s what Adobe says about 64-bit support:

Adobe Flash Player is not supported for playback in a 64-bit browser. However, you can run Flash Player in a 32-bit browser running on a 64-bit operating system.


Adobe is working on Flash Player support for 64-bit platforms as part of our ongoing commitment to the cross-platform compatibility of Flash Player. We expect to provide native support for 64-bit platforms in an upcoming release of Flash Player following Flash Player 10.

Without the Flash player, you can’t use sites like YouTube or BBC iPlayer. That’s a non-starter for many web users and the reason why 32-bit IE is the default on Vista 64-bit.

Microsoft is also putting off 64-bit support in Silverlight until other 64-bit extensions are available:

Right now our plan is to run SL in 32-bit mode (and not have a 64-bit native version). This is mostly because other browser plug-ins (and most browsers) don’t support 64-bit yet. We are looking at adding native 64-bit support in the future though.

Can I run Firefox in 64-bit mode on Vista 64?

There’s currently no official build of Firefox for 64-bit Windows as stated in a Mozilla support forum:

There are no releases or nightly builds of 64 bit Firefox for Windows from Mozilla as the earliest release to likely have 64-bit versions is Firefox 4.0. The only option you have is either roll your own or use a third-party build.

There’s little motivation for the browser vendors to go 64-bit because unlike other applications, such as memory hungry video editors or database servers, web browsers will not see significant performance benefits in 64-bit mode and there is also the problem with the lack of 64-bit extensions.

Can I use HttpWatch on Vista 64-bit?

Yes, you can use HttpWatch on Vista 64-bit but it is currently only available in 32-bit. This means that the HttpWatch plug-in can only be used in the 32-bit edition of IE. We will provide 64-bit plug-ins when Adobe, Microsoft and Mozilla provide significant 64-bit support.

The automation interface can be used from 64-bit programs without making any changes so as long as you have HttpWatch version 6.1.32 or later.

In some cases, such as accessing a .hwl file opened with the OpenLog method, you may see reduced performance in a 64-bit program. This is because the 64-bit API support in HttpWatch uses a separate WIN32 process rather than directly loading the API. To avoid this issue make sure that your client program runs 32-bit (see below).

Can I use the HttpWatch version 5.x automation on Vista 64-bit?

This will work, but you will need to ensure that your program runs in 32-bit mode. Otherwise, you will see this error when you attempt to create an instance of the HttpWatch controller object:

Unhandled Exception: System.Runtime.InteropServices.COMException (0x80040154): Retrieving
the COM class factory for component with CLSID {C4CEDB78-2B64-4703-99BE-A037A849D703}
failed due to the following error: 80040154.

If you’re seeing this error with HttpWatch version 6 make sure that you’ve installed version 6.1.32 or later.

How can I get my HttpWatch automation program to run 32-bit?

As mentioned above, you will need to ensure that a program using the HttpWatch automation API runs in 32-bit mode if:

  • You are using HttpWatch version 5.x
  • You want to improve data access performance on log files that are opened using the OpenLog method.

.Net projects (e.g. C# or VB.Net) have a default configuration setting of ‘Any CPU’. This causes them to automatically run as a  64-bit process on Vista 64-bit. To force them to always run 32-bit you’ll need to change this setting to ‘x86′ in Visual Studio:

x86build

Script based programs, such as Javascript or VBScript, that use the Windows Scripting Host will normally run in 64-bit on Vista 64-bit because cscript.exe and wscript.exe are  64-bit programs. You can force the use of 32-bit by using the script programs that are installed in the Windows\SysWow64 directory:

32bitwsh

The sample automation code that is installed with HttpWatch version 6.1.32 or later, uses the ‘x86’ setting  in C# projects and includes instructions on how to run the Javascript samples in 32-bit mode.

The Firefox Process Model

calendarFebruary 10, 2009 in Automation , Firefox , HttpWatch

One of the interesting new features in Google’s Chrome browser is the use of one Windows process per site or tab. This helps to enforce the isolation between tabs and prevents a problem in one tab crashing the whole browser.

In comparison, Firefox seems to have a simplistic process model on Windows. It doesn’t matter how many tabs or windows you open, or how many times you start Firefox  – by default you get one instance of firefox.exe:

Firefox Process Model

In Internet Explorer you can create a separate instance of the browser process just by starting another copy of iexplore.exe.

There are advantages to Firefox’s single process model:

  1. It uses less system resources per tab compared to creating multiple Windows processes.
  2. Firefox can use fast in-process data access and syncronization objects when it interacts with the history, cookie and cache data stores.

However, the lack of isolation means that if anything causes a page to crash, you’ll lose all your Firefox tabs and windows. This is mitigated to some degree by Firefox’s ability to restart the browser and reload the set of pages displayed in the previous session.

So what do you do if you are developing an add-on for Firefox or you want to run automated tests in Firefox whilst still using Firefox to browse in the normal way?

In Firefox, multi-process support is provided through the use of profiles. When Firefox is installed, you automatically get one default profile that contains user settings, browsing history, the browser cache and persistent cookies. Additional profiles can be created using the Firefox Profile Manager.

The Profile Manager is built into Firefox and is started by running this command in Start->Run:

firefox -P -no-remote

The -P flag indicates that the Profile Manager should be started and the -no-remote flag indicates that any running instances of Firefox should be ignored. If you run without this flag and you have already started Firefox, the command will simply open a new Firefox window without displaying the Profile Manager.

The Profile Manager has a simple user interface that allows you to create, delete and rename profiles:

You can start Firefox in a non-default profile by using the following command line:

firefox -P <myprofile> -no-remote

For example, if you created a new profile called AutoTest:

You could set up a shortcut like this to start Firefox in the AutoTest profile:

Each profile uses its own copy of the firefox.exe process, as well as its own settings, browser cache, history and peristent cookies. This provides more isolation than you would achieve by running multiple processes in IE. You can even separately enable or disable add-ons like Firebug or HttpWatch in each profile.

Internet Explorer’s cache and persistent cookies are maintained on a per user basis making it difficult to run separate instances with their own storage and settings. With Firefox you simply use different profiles. For example, you could use your default profile for normal browsing and have a separate profile to use for another purpose such as automated testing:

The HttpWatch automation interface in version 6.0 supports the use of profiles with Firefox. The profile name can be passed to the Attach and New methods of the Firefox plugin object. Passing an empty string indicates that you want to use the default profile.

Here’s a modified version of the page load test that we previously featured. It’s written in C# and uses a non-default profile to run the test:

// Set a reference to the HttpWatch COM library
// to start using the HttpWatch namespace
using HttpWatch;                
 
namespace EmptyCacheTest
{
    class Program
    {
        static void Main(string[] args)
        {
            string url = "http://www.httpwatch.com";
            string profileName = "AutoTest";
 
            Controller controller = new Controller(); 
 
            // Create an instance of Firefox in the specified profile
            Plugin plugin = controller.Firefox.New(profileName);                
 
            // Clear out all existing cache entries
            plugin.ClearCache();                
 
            plugin.Record();
            plugin.GotoURL(url);                
 
            // Wait for the page to download
            controller.Wait(plugin, -1);                
 
            plugin.Stop();                
 
            // Find the load time for the first page recorded
            double pageLoadTimeSecs =
                plugin.Log.Pages[0].Entries.Summary.Time;                
 
            System.Console.WriteLine( "The empty cache load time for '" +
                url + "' was " + pageLoadTimeSecs.ToString() + " secs");                
 
            // Uncomment the next line to save the results
            // plugin.Log.Save(@"c:\temp\emptytestcache.hwl");                
 
            plugin.CloseBrowser();
        }
    }
}

Ready to get started? TRY FOR FREE Buy Now