About Code Talks Research Shots

---

September 2015

Please note that republishing this article in full or in part is only allowed under the conditions described here.

HTTP Evasions Explained - Part 3 - Chunked Transfer

TL;DR

This is the third article in a series which will explain the evasions done by HTTP Evader. It covers the failure of several firewalls (and some browsers) to support the Transfer-Encoding chunked in the correct way. For example it is possible to bypass the malware analysis of several firewalls with the following innocent looking HTTP response (hint: look at the HTTP version):

   HTTP/1.0 200 ok
   Transfer-Encoding: chunked
   
   content which is not chunked

The previous article in this series was Part 2 - Deflate Compression and the next part is Part 4 - Double Compression.

What is chunked transfer

In the old days of HTTP 0.9 the response simply ended with the end of the underlying TCP connection. With HTTP 1.0 the HTTP response header was added and usually included the Content-length header to specify the size of the content. But giving the size in the HTTP header only works if the length is known up-front by the web server. This means that dynamic content either had to be buffered first to compute its final size before sending the HTTP header or that the response had to be finished old-style with the end of the TCP connection which would make further requests inside this TCP connection (i.e. HTTP keep-alive) impossible.

Thus with HTTP 1.1 the chunked transfer was added to the standard. With this transfer mode the server prefixes each chunk of content with its length and thus can start sending as soon as the first data are available. The length of each chunk is given as a hexadecimal number. The end of the response is signaled by an empty chunk, i.e. a size of 0. For example the following chunked response delivers the string "0123456789ABCDEFGH" within three chunks and a final empty chunk:

   HTTP/1.1 200 ok
   Transfer-Encoding: chunked

   b
   0123456789A
   3
   BCD
   4
   EFGH
   0
   

Note that the line delimiter is CR+LF as within the HTTP header and that each chunk ends with a CR+LF. Thus there is an empty line as the last empty chunk after the size 0.

While chunking looks simple there are several ways to modify it slightly so that firewalls have a different view of the content than the clients browser and can thus be bypassed.

Combining Transfer-Encoding chunked with Content-Length

The HTTP 1.1 standard clearly says that if both a Content-Length and a chunked Transfer-Encoding are given then the transfer mode is chunked and the Content-Length must be ignored. All browser behave this way but about 15% of the firewalls I've seen in the reports of HTTP Evader do it the other way around and thus can be bypassed with this simple technique to transfer malware:

   HTTP/1.1 200 ok
   Transfer-Encoding: chunked
   Content-length: 22

   3
   MAL
   4
   WARE
   0
   

Combining Transfer-Encoding chunked with HTTP 1.0

Since chunked transfer was only defined with HTTP 1.1 it should not be possible to use it within a HTTP 1.0 response. That is the header should simply be ignored by the browser and the browser should not interpret the body as chunked. And that's what the majority of browsers do.

But, about 20% of the firewalls tested behave differently and for some strange reason another 20% behaves differently only if additionally a Content-length header gets added. Thus with the following HTTP response about 40% of the tested firewalls can be bypassed, among them several Gartner Top NGFW:

   HTTP/1.0 200 ok
   Transfer-Encoding: chunked
   Content-length: 7

   MALWARE

Surprisingly the affected firewalls simply ignore that the response is not chunked and pass it through. There are some firewalls like Sophos UTM which also interpret the Transfer-Encoding for HTTP 1.0, but at least these firewalls sanitize the response enough so that a bypass is not possible in this case.

But even some firewalls which do not interpret Transfer-Encoding with HTTP 1.0 might be bypassed if one is using the right browser. Contrary to all the other browsers Safari does not look at the HTTP version and thus also interprets the Transfer-Encoding header within a HTTP 1.0 response. Thus, unless the firewall sanitizes the response and removes the bad header (which few do) users of Safari can be attacked by this:

   HTTP/1.0 200 ok
   Transfer-Encoding: chunked

   3
   MAL
   4
   WARE
   0
   

I've looked into this problem already 2 years ago and found IDS like Squid and Bro vulnerable too to this simply bypass. Looking at the source code of the current Snort version 2.9.7.5 this still seems to be the case.

Hiding the Transfer-Encoding header by another one

With about 20% of the firewalls one can simply hide the proper Transfer-Encoding header by adding another one which is invalid. Sometimes the order is important because only the last or the first header is considered but with most firewalls the order actually does not matter. For example with the following response one can bypass 20% of the firewalls, including several Gartner Top NGFW:

   HTTP/1.1 200 ok
   Transfer-Encoding: foo
   Transfer-Encoding: chunked

   3
   MAL
   4
   WARE
   0
   

Browser behave differently too. While Internet Explorer seems to consider only the last Transfer-Encoding header (i.e. "chunked" in this example) all the other major browsers just look if there is any Transfer-Encoding line with the value "chunked" in the HTTP header.

"chunked" vs. "xchunked" vs. "x chunked" etc

Browsers like Chrome and Internet Explorer interpret the value in the Transfer-Encoding header in a strict way and only allow the string "chunked". But Firefox also accepts the word "chunked" together with other words, e.g. "foo chunked" or "chunked foo". Safari is even worse and only checks for the string "chunked" somewhere and also accepts it inside another strings, like "this-is-not-chunked-and-I-mean-it".

Of course again there are several firewalls which don't consider the body chunked in this case but still forward this invalid header. And again this behavior is not restricted to some unknown and uncommon firewalls but includes several Gartner Top NGFW. Thus the following HTTP response makes it possible to bypass analysis in about 25% of the tested firewalls at least when used with Firefox or Safari:

   HTTP/1.1 200 ok
   Transfer-Encoding: x chunked

   3
   MAL
   4
   WARE
   0
   

and there is more

There are several ways to hide the Transfer-Encoding header itself from the firewall, like with adding spaces or other characters at unexpected places. Browsers often ignore these characters and still see the header, but other times the firewall sees a header where the browser sees none.

Apart from that there are chunk extensions and nearly 20% of the tested firewalls fail to handle these the correct way and thus can be bypassed (and again: this includes Gartner Top NGFW).

Try it for yourself

Since it should be obvious that you could not trust the marketing of the vendor you should better check yourself if there Advanced Threat Protection claims are just bragging. If you are behind some firewall claiming to detect malware then all you need is a browser and then follow the instructions to test against the HTTP Evader tool.