October 2015
Please note that republishing this article in full or in part is only allowed under the conditions described here.
HTTP Evasions Explained - Part 6 - Attack of the White-Space
TL;DR
This is part six in a series which will explain the evasions done by HTTP Evader. This part is about misusing white-space to bypass the firewall. As a simple example several products can by bypassed with line folding, which is supported by all browsers except Internet Explorer and Edge:
HTTP/1.1 200 ok
Transfer-Encoding:
chunked
malware with chunked encoding
The previous article in this series was Part 5 - GZip Compression and the next is Part 7 - Lucky Numbers.
Why white-space?
HTTP is a format designed to be text based and human readable. But for humans white-space is mainly some delimiter and they don't really care about the amount of white-space and the different kinds of horizontal and vertical white-space.
Unfortunately the HTTP standard has a different view: while there are several places were any number and different kinds of white-space is allowed, in other places only a single space character is allowed. Additionally, the end of the line is explicitly defined as \r\n even though UNIX systems traditionally only use \n as line ending. This make it possible to construct HTTP messages which look the same but actually differ in the octets and where some are valid and others not.
To deal with the resulting mess in implementations browsers use a heavily relaxed interpretation of the standard and accept different kind of white-space on several places and sometimes even accept white-space where none is allowed. Since all of this is outside the specification anyway there is no rule on how exactly to divert from the standard so all browsers do it a bit different. And since the firewalls have their own idea of robustness they do it different too. Which of course means that we will find enough cases where the firewall has a different interpretation than the browser and thus the analysis in the firewall can be bypassed.
Line folding
HTTP reuses lots of ideas from the e-mail standards. Since lines in e-mails have a limited length large header values in an e-mail could be extended over multiple lines using line folding, e.g.
Subject: This is a very
long subject which spans
multiple lines
From: ...
But, even though there is no restriction for the line length inside a HTTP header, the ability for line folding was kept in the original HTTP/1.1 standard from 1999 and was only marked as obsolete in the updated standard 2014.
From the browser only Internet Explorer and Edge don't understand line folding and Firefox, Chrome, Safari and Opera support it. Which means that the following HTTP response is seen as chunked by most browsers, but IE and Edge see only junk:
HTTP/1.1 200 ok
Transfer-Encoding:
chunked
chunked data
Unfortunately several firewalls fail to understand that the response is chunked and thus simply pass the chunked data because they cannot find the malware in it.
On the other hand the following HTTP response is considered invalid by most browsers and only IE and Edge see the intended data:
HTTP/1.1 200 ok
Transfer-Encoding:
chunked
data which are not chunked
And in this case we have other firewalls which do support line folding and understand that the data should be chunked. Unfortunately they fail to unchunk the data for analysis (because the data are not chunked) and thus often let the data simply pass through.
Wrong kind of new line
The standard only allows \r\n as new line but all browsers accept \n instead too. Most (but not all) Firewalls have no problems with it but some have problems if \n is used together with line folding. That is they properly deal with this:
HTTP/1.1 200 ok\r\n
Transfer-Encoding:\r\n
chunked\r\n
But fail in this case where only \n is used to fold the line:
HTTP/1.1 200 ok\r\n
Transfer-Encoding:\n
chunked\r\n
And while no browser supports a single \r as a new line to delimit the HTTP body from the header all but Firefox accept it as a delimiter between header fields:
HTTP/1.1 200 ok\rTransfer-Encoding: chunked\r\n \r\n chunked content
Since this behavior is surprising lots of firewalls fail to properly deal with it and can be bypassed.
Wrong place for white-space
All browsers except Edge allow some empty lines in front of the HTTP header. All of these allow also a limited amount of space and tabulator characters and Internet Explorer even allows practically any junk and not only white-space in front of the HTTP header.
Firewalls instead often expect the response to standard immediately. And if it does not they simply pass the data. Which means with the following response one can bypass several firewalls (note the space at the beginning of the first line):
\r\n
HTTP/1.1 200 ok\r\n
...
Internet Explorer and Edge also accept white-space in front of a header line, contrary to what the standard allows. While for the other browsers this looks like line folding IE and Edge don't support line folding. Since the standard does not allow any kind of white-space at the start of a header line most firewalls consider this header line invalid and either block the request (usually not), remove or sanitize the line (some) or simply pass the line in the original form (lots). Thus with the following HTTP response malware can bypass several firewalls and reach Internet Explorer or Edge (note the space in front of the header line):
HTTP/1.1 200 ok\r\n
Transfer-Encoding: chunked\r\n
\r\n
chunked data
And finally Chrome and Opera support space in front of the colon inside a header line and most firewalls fail to consider this or sanitize the header. These can by bypassed with (note the space before the colon):
HTTP/1.1 200 ok\r\n Transfer-Encoding : chunked\r\n \r\n chunked data
There is more
There are more ways to play with white-space to bypass firewalls and HTTP Evader contains a lot more ideas then I've described here. Thus if you are behind some firewall with malware detection and want to see if there is a relation between the claimed and the actual security offered by the firewall simply find out yourself by using the HTTP evader test site.