A Guide to Automating HttpWatch with PHP

calendarMarch 11, 2011 in Automation , HttpWatch

Stoyan Stefanov, the creator of smush.it, architect of YSlow 2.0 and engineer at Facebook, has written an excellent three part guide to controlling HttpWatch from PHP:

He’s even published a class that wraps the HttpWatch API and makes it easier to use from PHP.

In the rest of this blog post we wanted to follow up on a few points mentioned in Stoyan’s blog posts. These items don’t just apply to PHP. You may find them useful when automating HttpWatch using other languages such as C# or Ruby.

Hiding the IE window

The ‘A better experience in IE’ section of Automating HttpWatch with PHP shows how you can hide the IE browser window during tests by separately creating IE and attaching HttpWatch to it.

It’s also possible to do this using the Container property without having to separately create and attach to IE:

$controller = new COM("HttpWatch.Controller");
$plugin = $controller->IE->New();
$browser = $plugin->Container; // Only works with IE
$browser->Visible = false;

If you do decide to do this in order to stop windows popping up during a test please bear these points in mind:

  1. Orphaned instances of iexplore.exe will be left running in the background if your script ever terminates before calling CloseBrowser.
  2. In IE 8 and earlier, HttpWatch will not record a Render Start event because the hidden IE window does not get updated by Windows. However, the event will be recorded in Firefox 3.5+ and IE 9.
  3. Your performance measurements may not directly match user experience. It’s possible that current or future browsers may avoid certain rendering and processing actions if they detect that the output will not be in a visible window. For that reason, we recommend running tests in visible browser windows on a normal interactive desktop either on a physical machine or VM. Also, viewing a browser test through Remote Desktop is likely to have a significant negative impact on performance as the graphics and text making up the page have to be transferred over the network.

Opening the HttpWatch Plugin Window

It’s often handy to open the HttpWatch window in the browser when you are developing an automation script so that you can check that it is working as expected.

The OpenWindow method allows you to do this and specify whether you want the HttpWatch window docked or undocked. For example, here is the PHP code to open HttpWatch as an embedded window in the browser:

...
// Open docked HttpWatch window in browser
$plugin->OpenWindow(false);
...

Handling Differences Between HttpWatch Basic and Professional Editions

In part 3, Stoyan mentions using try-catch to handle the errors that occur when attempting to access data that is restricted in HttpWatch Basic Edition. While this is a valid approach, there is a risk of hiding other errors that might be occurring that are not due to the restrictions in HttpWatch Basic edition.

There are a couple of properties in the HttpWatch automation interface that help you handle the differences. The first is the IsBasicEdition property on the Controller class.

For example, here’s a high level test in PHP:

$controller = new COM("HttpWatch.Controller");
if ( $controller->IsBasicEdition )
{
    echo "\nThis test requires HttpWatch Professional Edition";
}

At a lower level, you can also check each request to see if it has been restricted using the IsRestrictedURL property:

...
if ( $entry->IsRestrictedURL)
{
    // Goes here in HttpWatch Basic Edition for URLs outside Alexa Top 20
    echo "\nSome of the properties for this request are restricted";
}
else
{
    // Goes here in HttpWatch Basic Edition for URLs in Alexa Top 20
    // or in HttpWatch Professional Edition for any URL
    echo "\nAll the properties of this request are available";
}
...

How to check HTTP Compression with HttpWatch

calendarJuly 10, 2009 in Automation , C# , HttpWatch , Optimization

HTTP compression is one of the easiest and most effective ways to improve the performance of a web site. A browser indicates that it supports compression with the Accept-Encoding request header and the server
indicates the compression type in the Content-Encoding response header.

This screenshot from the Stream tab of HttpWatch shows these headers and the compressed content being returned from the server:

Compressed Page

Here’s another screenshot of a page that is not compressed:

Page with no compression

The browser still indicated that it accepted gzip and deflate compression, but the server ignored this and returned uncompressed HTML with no Content-Encoding header.

The easiest way to check the amount of compression achieved is to use the Content tab in HttpWatch to view a ‘200 OK’ response from the server:

Compressed Content

Don’t try checking for compression on other HTTP status codes. For example, a ‘304 Not Modified’ response will never have any compression saving because no content is returned across the network from the web server. The browser just loads the content fom the cache as shown below:

Content Tab with 304

So, if you want to see if compression is enabled on a page, you’ll either need to force a refresh or clear the browser cache to make sure that the content is returned from the server. The HttpWatch Automation API lets you automate these steps. Here’s an example using C# that reports how many bytes were saved by compressing a page’s HTML:

// Set a reference to the HttpWatch COM library
// to start using the HttpWatch namespace
//
// This code requires HttpWatch version 6.x
//
 
using HttpWatch;
 
namespace CompressionCheck
{
    class Program
    {
        static void Main(string[] args)
        {
            string url = "http://www.httpwatch.com";
            Controller controller = new Controller();
 
            // Create an instance of IE (For Firefox use
            // controller.Firefox.New("") )
            Plugin plugin = controller.IE.New();
 
            // Clear out all existing cache entries
            plugin.ClearCache();
 
            plugin.Record();
            plugin.GotoURL(url);
 
            // Wait for the page to download
            controller.Wait(plugin, -1);
 
            plugin.Stop();
 
            // Find the first HTTP/HTTPS request for the page's HTML
            Entry firstRequest = plugin.Log.Pages[0].Entries[0];
 
            int bytesSaved = 0;
            if (firstRequest.Content.IsCompressed)
            {
                bytesSaved = firstRequest.Content.Size
                                   - firstRequest.Content.CompressedSize;
            }
 
            System.Console.WriteLine("Compression of '" +
                firstRequest.URL + "' saved " + bytesSaved + " bytes");
 
            plugin.CloseBrowser();
        }
    }
}

Tip: If you access a web site through a proxy you may not see the effect of compression. This is because some proxies strip out the Accept-Encoding header so that they don’t have to process compressed content. Tony Gentilcore’s excellent ‘Beyond Gzipping’ talk at Velocity 2009 described how 15% of visitors to your site will not receive compression due to problems like this. A simple way to effectively bypass proxy filtering for testing purposes is to use HTTPS if it is available. For example, try https://www.httpwatch.com if you don’t see compression on http://www.httpwatch.com.

Using Firefox, HttpWatch and Internet Explorer on Vista 64-bit

calendarMay 9, 2009 in Automation , C# , Firefox , HttpWatch , Internet Explorer , Javascript

The 64-bit version of Windows Vista is becoming increasing popular due to its ability to access more than 4 GB of physical memory. To an ordinary Windows user, there are few noticeable differences from the 32-bit version because it can run both 32-bit and 64-bit programs.

In this post we wanted to answer some of the questions you may have about Windows Vista 64-bit and the effect it has on the use of Firefox, HttpWatch and Internet Explorer (IE).

How can I tell if a program is running in 32-bit or 64-bit mode?

The Task Manager on Vista 64-bit shows 32-bit tasks with a ‘*32′ after the process name. If there’s no ‘*32′ after the name then it is running in 64-bit mode. This screen shot shows a 32-bit process called page_check running on Vista 64-bit:

32-bit Program Running on Vista 64-bit

Why does IE run in 32-bit mode by default on Vista 64-bit?

Clicking on the launch bar IE icon in Vista 64-bit starts the 32-bit version of IE:

32-bit version of IE on Vista 64-bit

A 64-bit version of IE is available but you have to explicitly select it from the Start menu:

64-bit version of IE

Microsoft has done this because the 64-bit version of IE can only load 64-bit browser extensions and most  extensions, such as the Adobe Flash player, are only available in 32-bit.

Here’s what Adobe says about 64-bit support:

Adobe Flash Player is not supported for playback in a 64-bit browser. However, you can run Flash Player in a 32-bit browser running on a 64-bit operating system.


Adobe is working on Flash Player support for 64-bit platforms as part of our ongoing commitment to the cross-platform compatibility of Flash Player. We expect to provide native support for 64-bit platforms in an upcoming release of Flash Player following Flash Player 10.

Without the Flash player, you can’t use sites like YouTube or BBC iPlayer. That’s a non-starter for many web users and the reason why 32-bit IE is the default on Vista 64-bit.

Microsoft is also putting off 64-bit support in Silverlight until other 64-bit extensions are available:

Right now our plan is to run SL in 32-bit mode (and not have a 64-bit native version). This is mostly because other browser plug-ins (and most browsers) don’t support 64-bit yet. We are looking at adding native 64-bit support in the future though.

Can I run Firefox in 64-bit mode on Vista 64?

There’s currently no official build of Firefox for 64-bit Windows as stated in a Mozilla support forum:

There are no releases or nightly builds of 64 bit Firefox for Windows from Mozilla as the earliest release to likely have 64-bit versions is Firefox 4.0. The only option you have is either roll your own or use a third-party build.

There’s little motivation for the browser vendors to go 64-bit because unlike other applications, such as memory hungry video editors or database servers, web browsers will not see significant performance benefits in 64-bit mode and there is also the problem with the lack of 64-bit extensions.

Can I use HttpWatch on Vista 64-bit?

Yes, you can use HttpWatch on Vista 64-bit but it is currently only available in 32-bit. This means that the HttpWatch plug-in can only be used in the 32-bit edition of IE. We will provide 64-bit plug-ins when Adobe, Microsoft and Mozilla provide significant 64-bit support.

The automation interface can be used from 64-bit programs without making any changes so as long as you have HttpWatch version 6.1.32 or later.

In some cases, such as accessing a .hwl file opened with the OpenLog method, you may see reduced performance in a 64-bit program. This is because the 64-bit API support in HttpWatch uses a separate WIN32 process rather than directly loading the API. To avoid this issue make sure that your client program runs 32-bit (see below).

Can I use the HttpWatch version 5.x automation on Vista 64-bit?

This will work, but you will need to ensure that your program runs in 32-bit mode. Otherwise, you will see this error when you attempt to create an instance of the HttpWatch controller object:

Unhandled Exception: System.Runtime.InteropServices.COMException (0x80040154): Retrieving
the COM class factory for component with CLSID {C4CEDB78-2B64-4703-99BE-A037A849D703}
failed due to the following error: 80040154.

If you’re seeing this error with HttpWatch version 6 make sure that you’ve installed version 6.1.32 or later.

How can I get my HttpWatch automation program to run 32-bit?

As mentioned above, you will need to ensure that a program using the HttpWatch automation API runs in 32-bit mode if:

  • You are using HttpWatch version 5.x
  • You want to improve data access performance on log files that are opened using the OpenLog method.

.Net projects (e.g. C# or VB.Net) have a default configuration setting of ‘Any CPU’. This causes them to automatically run as a  64-bit process on Vista 64-bit. To force them to always run 32-bit you’ll need to change this setting to ‘x86′ in Visual Studio:

x86build

Script based programs, such as Javascript or VBScript, that use the Windows Scripting Host will normally run in 64-bit on Vista 64-bit because cscript.exe and wscript.exe are  64-bit programs. You can force the use of 32-bit by using the script programs that are installed in the Windows\SysWow64 directory:

32bitwsh

The sample automation code that is installed with HttpWatch version 6.1.32 or later, uses the ‘x86’ setting  in C# projects and includes instructions on how to run the Javascript samples in 32-bit mode.

Ready to get started? TRY FOR FREE Buy Now