September 2015

Please note that republishing this article in full or in part is only allowed under the conditions described here.

HTTP Evader - Automate Firewall Evasion Tests

TL;DR

If you are behind a firewall (today often known marketed as IPS, NGFW or UTM) which claims to protect you from malware you might want to verify these claims. HTTP Evader provides you with a way to automatically test how your firewall deals with situations where the malware hides in rare or invalid responses from the web server. Lots of highly praised firewalls fail to detect malware in this cases, which means they fail to protect you properly. Please note that this is not about bypassing web application firewalls (WAF) which protect a web server but about bypassing firewalls which should protect the client (browser). It is also not about bypassing URL filters.

The following tests try to transfer the EICAR test virus to you using differently shaped responses of the web server. This official test virus should be detected by any antivirus solution but does not do any harm.

To find out if you are vulnerable simply point your browser to the HTTP Evader test site. Before you report any problems to your firewall vendor please read the section about false positives and verify that the detected evasion is really possible.

Disclaimer

This tool was developed and the evasions detected during my work at the research project APT-Sweeper. In this project we research new ways to protect against attacks. This includes analysis were current products fail in order to improve the situation. This research project is funded by the german IT-Security company genua gmbh and by the german ministry of education and research.

Intro

Vendors of current perimeter firewalls (i.e. NGFW, UTM, IPS, Secure Gateways) promise to be able to detect malware downloaded with the browser. They claim to use advanced threat protection mechanism, cloud intelligence and various other buzzwords in order to protect the user.

But, lots of these firewalls fail already at extracting the payload properly from the HTTP connection and thus forward the wrong data to their malware detection for analysis. This means of course that the malware gets not detected and blocked but will instead be passed to the user.

HTTP Evader is designed to help both users and firewall vendors to detect bypasses at the HTTP application level in an easy way and give the vendors the help and motivation to fix these problems in order to protect their users.

What makes these attacks here so dangerous

Firewall bypasses at the HTTP level are dangerous not only because malware can be passed to the user but also because they often leave no traces and thus get completely unnoticed. If one has a look at the source code of current IDS solutions like the commonly used Snort, Bro or Suricata one can see that they mostly assume that the attacker uses common, standard conforming HTTP responses and will treat the input accordingly. That is they will pick the parts from the HTTP response which they can easily understand and will usually silently ignore the rest. Experiences with commercial products show that they often blindly trust the HTTP layer in a similar way.

While HTTP Evader has no relation to Evader by McAfee it fits their description for Advanced Evasion Techniques because lots of products have problems detecting these evasion, even if the evasion by themselves are trivially to create for an attacker.

How does it work

HTTP Evader is a tool to automate tests for possible bypasses. It uses the EICAR test virus as this official test virus should be detected by all antivirus solutions as bad but is nevertheless harmless. HTTP Evader consists of a small custom web server which serves this test virus within different HTTP responses. Some of these responses are invalid, some or valid but strange or rarely used and some are typical responses any web server provides. Bypasses of the firewall are often possible because the interpretation of such responses differs between browser and firewall and thus the firewall sees and analyzes different data than the browser, that is we have a Semantic Gap.

A custom web server is needed since some of the responses would be sanitized by the common web servers before sending them to the client and thus would not have the desired effect. But many of the bypasses can actually be done with a standard web server and some PHP, like described in Is this URL safe? Hiding malware in plain sight from online scanners.

To start the test simply point your browser to the test site and start the "Bulk test with virus payload". This will load a site which will do a lot of different bypass tests using XMLHTTPRequest and compare the response from the test webserver to the expected response, i.e. the test virus. If the test virus passed the firewall unchanged it means an evasion might be possible. While this is true in most cases you should check for false positives (see below) before reporting the problem to the vendor. All possible evasions will be listed at the HTML page so you can use it as a summary to show the vendor. Additionally a report will be sent to the test server which includes the possible evasion and also the error message it got for the detected malware. This error message might give hints which firewall product and version was used, so that the vendor can be informed by me.

Since different browsers show a different behavior regarding invalid or rare responses you should check with all the browsers which might be in use in your environment. Chrome and Firefox behave similar in that they both can deal with doubly compressed content. Internet Explorer instead behaves differently here and Safari shows a different (and non-standard) behavior with some chunked encoding. This shows that even if an evasion is not possible with one browser it might be possible with another browser.

How to check for False Positives

False Positives mean, that an evasion got detected which is not usable in practice. In practice the malware gets downloaded from a link and in the test the malware was downloaded using a XMLHTTPRequest. But links and XMLHTTPRequest might behave differently and this behavior is again browser dependent. For example a common bypass found by this tool is to deliver the malware within an 301 redirect response, but without providing the target of the redirect inside the expected Location header. With all tested browsers the delivery with XMLHTTPRequest will be successful, but Firefox and Chrome will not allow the download of the resource using the link. Safari instead happily loads it.

This means for every evasion reported you should use the 'TRY' link offered after the description to actually try the evasion. Note that this actually tries to download the test virus. Since any antivirus product should detect this virus the download might fail since the antivirus on the local machine blocks it. In this case you either need to switch off the antivirus during the tests or take the warning shown by the antivirus as a sign that the test virus was not blocked by the perimeter firewall but only by the local antivirus. If not antivirus is blocking the download you should get the EICAR test virus, which is 68 byte long and looks like this:

  X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*

How to report Bypasses

If you found a bypass please make sure that it is not a false positive. Then make sure that the firewall is actually configured properly, because sometimes essential checks are disabled for performance reasons and sometimes this is even the case for the default configuration provided by the vendor.

If you could verify the bypass it would be the best to report it to the vendor so it can be fixed. You should provide the version of the product you use, details about the configuration and details about the bypasses. It should also point the vendor to this tool so that your findings can be easily verified. If you don't feel comfortable contacting the vendor yourself feel free to send the details to http-evader[at]noxxi[dot]de and I will try to contact the vendor.

Results: Detected Evasions

While developing this tool several bypasses were found inside Sophos UTM. All of these got reported to the vendor and most of them got fixed. This was possible since Sophos offers a Home Edition of their UTM.

I could do no tests with firewalls from other vendors since I have no access to the necessary hardware myself. Therefore I'm not able to give these vendors advanced notice when problems are detected, since everybody can use HTTP Evader to detect these problem too. But, lots of the problem I've seen are actually not new but I've written about these already years ago. To bad the vendors did not notice.

Since the tool is already online for a while (although not highly publicized) I've noticed tests with various products (mostly unknown vendor) and even attempts to tune or fix these products in order to lower the amount of possible bypasses. I was also informed of possible evasions in products of two highly praised firewall vendors ( Gartner top 10 ) but these reports still need to be fully verified. This shows, that there are several vulnerable products out there.

Evasion techniques explained

In a series of articles I will explain the simply techniques used in this tool. The following parts are already available:

What more can HTTP Evader do

Apart from testing evasions of firewalls HTTP Evader can also be used to simply test the behavior of the browser to see where the behavior differs from the standard. If there is a proxy/firewall involved one can also see if it causes changes in the behavior. While lots of firewalls only watch the traffic and simply block malicious data application layer firewalls and proxies can actually make changes to the data and thus enforce a consistent interpretation in client and firewall for security reasons. This interpretation is not always the correct one (there are often misinterpretations regarding chunked encoding) but at least they are consistent and thus can protect against bypass attempts.

The best is to head over to the HTTP Evader test site and look at the various sub-tests. There is an automatic sub-tests similar to the evasion test but using a non-malicious payload. And there are sub-tests for different kinds of unusual HTTP responses which also show that browser differs also how these responses get handled depending on the kind of the response: commonly images and simple HTML gets rendered even if the response is invalid while in some cases Javascript and downloads fail even though images succeed.

What will HTTP Evader not detect

HTTP Evader is strictly limited to evasions at the HTTP level, i.e. using invalid or uncommon HTTP responses. There are lots of other attacks this tool will not detect because they work at a different level. Typical examples are:

Relation to Evader by McAfee

HTTP Evader has no relation to Evader by McAfee but I like the underlying research from Stonesoft. But according to their Data Sheet they have only few evasions at the HTTP level and mainly provide evasions at the transport layer or for other application protocols (MSRPC, SMB). But the underlying idea of both tools is the same: evade firewalls by employing the interpretation differences between the firewall and the system which should be protected.

How to install your own test server

If you don't have access to the public test site or want to do your own private tests you can setup you own test server. The test server is written in Perl. There are only few dependencies and the source code is available at github.

Contact

You can contact me about HTTP Evader at http-evader[at]noxxi[dot]de.