HTTP Compression

By David Nelson

Goals of HTTP Compression:

To enable your web site to save nearly 30% on bandwidth cost, as well as to improve the perceived application speed to dial-up users, transparently to the user, with minimal IT spending for implementation.

HTTP Compression Overview:

HTTP Compression is a feature of the HTTP 1.1 specification that has been around for a long time, but is not commonly used. Google uses HTTP Compression according to http://www.websiteoptimization.com/speed/tweak/compress/ . There are a couple different supported compression formats, but the most commonly used and compatible format is gzip. Internet Explorer and Netscape (along with most graphical browsers) support gzip decompression. Compression is used on all html output from web applications or event static html files. Images are already compressed and don�t need to be touched by an http compression system.

Testing Plan For HTTP Compression:

Browser Compatibility: According to research I�ve done, Netscape 4 has issues with compression resulting in occasional decompression bugs in the browser. The goal of browser compatibility would be to detect the browser inside the application, and only compress pages for IE4+, and Netscape 6+ (Netscape didn�t release a version 5). This would need to be exhaustingly tested.

Web Server CPU Overhead: According to my research, the best compression rates require a 10% cpu overhead on the web server machines to compress the html output before it is sent to the browser. This would have to be exhaustingly tested as well, and analyzed along with our current (and estimated future) web server machine cpu utilization.

Browser Decompression Overhead: This test is to determine how fast of a machine the end user needs for the browser to decompress the compressed html output quickly enough to be 100% transparent for the user. I recommend shooting for a base computer speed of 100 to 200 mhz - the Pentium 1 class chip range. This as well needs to be tested exhaustively.

HTTP Compression On Windows & IIS:

There are three paths to go in terms of implementation. One is to buy a third-party tool which plugs into the Internet Information Server software and enables http compression completely transparently outside of applications. This is good, but costs a small amount of money. A permanent license for the best tool to do this costs $300 per server. You will see an amazing ROI on this if you use a lot of bandwidth, but I don�t like the idea of spending the extra money on it. The most popular third party tool I�ve found to do this is called HTTPZip (http://www.port80software.com/products/httpzip/ ). Another benefit to buying this software from a vendor is that we would get technical support from them at no extra cost.

Another option is to do it inside the application, using an open source HTTP Compression library specifically written for ASP.NET. I�ve looked for one, and found a great one called HTTPCompressionModule (http://www.asp.net/ControlGallery/ControlDetail.aspx?Control=696&tabindex=2 ). This way you have more control of the performance and browser support, and save the $2,500+ licensing cost. The time required to implement and test using this implementation model is around 1 month of man hours, possibly less depending on how rigorously you test it.

The third option is to use IIS� built in http compression. The problem with this is that due to Netscape 4�s bugs, certain users wouldn�t be able to use your web site without errors. I don�t consider this a viable solution. However, I haven�t used IIS built in compression, so I can�t say how well it works or doesn�t work.

HTTP Compression On Linux & Apache:

I recommend using mod_gzip to implement http compression on linux and apache. You can read more about it at sourceforge: http://sourceforge.net/projects/mod-gzip/.

HTTP Compression Questions & Answers Section:

Would this solution compress all html or asp.net html out of the servers?

What I think is optimum is doing the compression as gzip at the application level for asp.net dynamic page output. There is also the option of doing it as an ISAPI filter, but I like the control of the application level.

Either way, this would put the additional overhead on a sliding scale right? The more traffic, the additional compression required.

I believe its somewhere in the neighborhood of a few milliseconds added to each request - this needs testing though for confirmation.

Is there any way to determine the de-compression stats on the client side or what browsers support it?

Yes, there's a lot of info on the web about this. Here's a good URL: http://www.webreference.com/internet/software/servers/http/compression/

So it looks like Netscape has a few issues with http compression. You could easily turn this off for netscape based http requests to avoid that problem. Most users use IE 4+, so you'd still see a considerable performance improvement by using http compression.

Any idea what percentage of customers are on a dial up?

Not really sure for your site specifically, but for U.S. surfers as a whole, about 69% ( stats collected in 2002, reported January 2003 ) are on dial up according to Nielsen Netratings. See: http://www.itworld.com/nl/ecom_in_act/01292003/

Will our customers complain regarding this if we don�t implement it?

Highly doubt it - it's an advanced performance tuning subject that your customers probably wouldn't even think about. But I'm sure they'd appreciate their sites loading faster on dial-up, and you could save money on bandwidth costs.