October 2015

Please note that republishing this article in full or in part is only allowed under the conditions described here.

HTTP Evasions Explained - Part 5 - GZip Compression

TL;DR

This is the fifth part in a series which will explain the evasions done by HTTP Evader. This part is about failures to handle gzip compression properly. Contrary to deflate compression all products I've seen are able to handle gzip compression in theory. But several major products fail if you set some special bits, invalidate the checksum, remove some bytes from the end etc. But, the browsers unpack the content anyway so we get a bypass again.

Some products can already by bypassed with the following simple HTTP response

   HTTP/1.1 200 ok
   Content-Encoding: gzip
   
   gzipped content, last 4 bytes missing

The previous article in this series was Part 4 - Double Compression and the next one is Part 6 - Attack of the White-Space.

The GZip File Format

The gzip format is defined in RFC 1952. Basically it consists of a header with several optional flags and extensions, followed by the compressed content and it ends with the validation part consisting of the checksum and the length of the uncompressed content.

Optional flags and a validation only after all the content was already unpacked and maybe processed offer a lot of ways to modify the gzip data without modifying the compressed content. Let' start with the validation part.

Validating the unpacked content - or not

Browsers need to render content as quickly as possible which means that a validation of the content only after everything is unpacked will negatively impact the time of rendering. Thus as expected all of the tested browsers (latest Chrome, Firefox, MSIE 11, Edge, Safari and Opera) will skip all or at least most of the validation:

all browsers ignore if the checksum (CRC-32) of the content is wrong
all browsers but MSIE and Edge ignore if the size (ISIZE) does not match
all browsers ignore if the length is missing (last 4 bytes) or if the checksum and the length are missing (last 8 bytes)

Contrary to that several of the firewalls fail to unpack the content in all or at least some of these cases. While there are firewalls which in this case block the content, most of the products I've seen consider the content harmless (i.e. cannot be unpacked so it cannot be bad) and deliver it unchanged to the client.

Manipulating header options in a valid way

The gzip header has several options to mark the content as text (FTEXT), add a file name (FNAME) or add some comment (FCOMMENT). No browser has problems when these options are used. But again several products fail if any options is used or some fail only on FCOMMENT. And sure enough they simply forward the content unchanged because they did not find anything bad in it.

Manipulating header options in an invalid way

The gzip header can optionally contain a checksum of the header (FHCRC). Firefox, Chrome and Opera have no problems if this checksum is wrong, contrary to several firewalls.

Also, the gzip header has some unused bits which are reserved for some later unspecified use and which must be 0 (FLG 5..7). But, Chrome and Opera don't care and accept the content even if any of the reserved bits is set. Like expected, several firewalls fail to unpack and analyze the content in this case and simply forward it unchanged to the client.

Try it for yourself

You can verify yourself if the claimed Advanced Threat Protection of your firewall can easily be bypassed with simple manipulations to a HTTP response. If you are behind some firewall able to detect malware then all you need is a browser and then follow the instructions to test against the HTTP Evader test site.