Choosing and properly configuring a Web Proxy / Web Filter

The WWW (Word Wide Web), or the Internet, is the most widely used vector of attack targeting your users directly, in a way opening a tunnel from the attacker to your endpoints through all your defenses.

That is why choosing your web-filter wisely is of utmost importance. The presentations which sales engineers perform are attractive and interesting but most times they don’t understand the threats you are facing and rarely can offer a complete solution – very few are the vendors which can cover most bases in the checklist below.

Whitelisting is no longer the ‘too difficult to achieve’ methodology of keeping your users secure. It is not even the high target – it has become the minimum norm, as even whitelisted sites get compromised and you have to think how to defend against that, focus on this, rather than spread your attention on defending your users from millions of potential attack vectors by blacklisting known bad sites.

Actually it is much tougher to deal with the consequences of constant breaches and exploitation due to malvertising / exploit packs on hacked sites / phishing sites than maintaining a whitelist. Please don’t hide your head in the sand and implement a whitelisting policy – it is very difficult initially but it does pay off tremendously in terms of protection.

A simple ‘statistical run’, building a baseline of the most visited sites in a month, followed by an assessment which ones are safe and business-focused will give you a good list of sites to whitelist. Then just move further down the funnel, reviewing requests to add sites as they come in.

Evaluating a web filter before purchasing the product / service

Out of the hundreds of features / capabilities you could look for in a web filtering appliance you should focus on the following:

  1. Support – when you attend the first technical sales presentation have a list of difficult questions based on your past experience / needs. Look for the confidence and speed you receive answers with – if you keep getting the ‘we will get back to you on that’ – expect worse support in the future. Actually, always expect worse support than what you’ll get during the technical sales meetings – and base your assessment on that. Support should be able to modify settings on the fly and when needed, even tweak undocumented features. When you get promised a feature ‘which is being worked on’ – demand the exact stage at which the feature development is and demand hard dates for implementation with financial penalties if the promise is broken – if they were honest when promising it, it would not be a problem for them.
  2. The ability for users to request the unblocking of a web resource right on the page where it says “blocked”, which would then be entered in a database allowing the admin to quickly and easily check / uncheck requests. It is very important to avoid e-mail notifications on a blocked site unblock request and only focus on products which offer seamless integration of that page with the back-end administrative system.
  3. Demand a demonstration of the web filter ability to filter malicious pages – with malicious javascript on them, for example. Find new malicious URLs – for example, from http://www.malwaredomainlist.com/mdl.php – and TEST the newest entries there right during the sales demonstration to test the ability of the solution to ‘think on the fly’ without relying on a database of known bad hosts. This is very, very important.
  4. The ability to block executables within archives for certain users / groups
    – The reason behind this requirement: only system admins / designated personnel should be able to download executables or executables within archives. All other users should be denied this right – and you should have a formal process to allow employees to request a download of a certain executable, for which a request should be filed and which request should go through a security review process. No executable should enter your network without proper authorization and risk assessment.
  5. The ability to detect mismatching MIME types and block them (executable with a .jpg extension, for example).
  6. Ability to block downloads from untrusted (non-whitelisted sites).
  7. Ability to block advertisement networks and ads. Very important.
  8. Ability to block POST requests larger than X bytes for all sites except a list of users, accessing a list (whitelist) of allowed websites.
    – this requirement effectively prevents data exfiltration and / or phishing. If you block ALL post requests except a list of whitelisted sites for a list of users (a user should request access to submit POST requests to a specific site and every request should be pre-approved).
  9. Ability to block websites based on reputation and age. You should NOT!!!! Allow access to any website younger than 1 month and having no reputation or classification in the web filter database.
  10. Ability to block encrypted sessions to IP addresses and URLs not present on a pre-approved list.
    This is obvious – only pre-approved sites and hosts should be able to establish encrypted sessions with your endpoints.
  11. Ability to block based on regular expressions
  12. Ability to integrate 3rd party AV engines
    – this requirement allows for the use of multiple AV engines at once, as opposed to a single AV engine.
  13. Ability to integrate with the mobile devices managed by your company.
  14. SSL inspection should be on by default. There should be a capability of raising an alert on first SSL connection to a non-SSL whitelisted (allowed) host. SSL whitelisting should be possible.

This is a very short list – but a very important one. The final list you should be going to vendors with should include several times as big a list – just remember that many vendors might not have all of the features you request. But the above are almost mandatory.

Generally no proxy connection should be possible in a ‘transparent’ mode. All devices wanting to connect through the proxy should have it manually configured, explicitly.

No server or workstation on the internal network should be able to bypass the proxy and connect directly to the Internet.

One additional note… ask your web proxy vendor if they can prevent data exfiltration over encrypted channels and how. If their answer is unsatisfactory, you could implement your own box for that purpose following the guide here: http://resources.sei.cmu.edu/asset_files/TechnicalNote/2013_004_001_40234.pdf