Cyber Security Monitoring

If you need to work on this topic, I suggest you read the book of Anton Chuvakin – Logging and Log Management: The Authoritative Guide to Understanding the Concepts Surrounding Logging and Log Management. It has helped me tremendously in several projects, and I would like to pass on the fact that it is an incredibly useful book.

Besides that, we have a few ideas to share, which might be useful to you and are not present in the book above.

Before even starting to collect logs, you need to understand your environment. That is why audits and assessments are useful. But that is not all – you need to know how much data your devices are producing in log format every second, minute, hour, 24 hours – then you need to understand how to optimize what is logged and what is not.

It is straightforward to lose yourself in the amount of logging information generated on each of your devices, especially if you add them all at once to your SIEM (even if you add only servers/switches and firewalls and omit the desktops).

The noisiest devices in our experience are Windows systems – especially domain controllers and mail servers. Second would be your proxy servers and firewalls.

To understand how logging in a Microsoft Environment works, I suggest you read these two links:

• Advanced Audit Policy Configuration FAQ
• HOWTO: https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-r2-and-2008/dd408940(v=ws.10) 

Once you’ve read the above and understand the difference between basic and advanced audit policy configuration, move on to the next link:

• Spotting the Adversary with Windows Event Log Monitoring

It explains in detail the changes you need to make to have an optimized for detection logging environment. It also emphasizes which EventIDs you should target.

To clean up your logs and only collect what is needed, you need to know what can be discarded. To understand what is being logged, a good idea would be to export the logs from your noisiest server for a day to a CSV file, open it in Excel, and filter by EventID. It should be easy to distinguish by a percentage which events are happening most frequently and which are not.

In one case, I saw thousands of events per second with the word “Filtering engine” in them. It turns out packet logging was turned on – and every single network packet coming through the internal Windows firewall was being logged! You can imagine the number of logs generated by this server per hour and the usefulness of these logs.

Unfortunately, such a level of detail can render your SIEM and your storage incapacitated – and you should be very careful what you log and what you discard.

If you want a good start with configuring your SIEM, you could only set it to log the events in as in the book which we mentioned earlier: “Spotting the Adversary with Windows Event Log Monitoring” paper by NSA – and expand to other systems/events once you are sure you got that one right. Don’t just plug in your SIEM and all your devices into it in a shotgun manner – all your money spend on storage and SIEM will go to waste if the system is not optimized and tuned for effectiveness.

It would help if you also focused on the following events:

• 4688 Process Create (after going to Command line process auditing and enabling logging)
• 4663 File/Registry Auditing
• 4075 Service Created
• 4070 Service Changed
• 4624 User Login Success
• 5140 Share accessed

(List is taken from http://www.slideshare.net/Hackerhurricane/ask-aalware-archaeologist)

The same process can and should be repeated for all your systems/devices. Only plug a device into your SIEM solution once you fully understand the format of logging it uses, the amount/type of events, and how you could fine-tune the number of details in these logs.

You can find an excellent set of guides on configuring Windows Logging here, too.

Tools

These links will be useful during your work on optimizing your logging environment.

http://sourceforge.net/projects/syslogserverwindows/

http://sourceforge.net/projects/nxlog-ce/

LogExpert – especially this one!

One of my favorite tools to analyze logs quickly and filter them is Mandiant Highlighter.

Using open-source tools for centralized logging management

Before even thinking about buying anything, think about optimizing your logs and storage – storage is utilized very quickly and becomes expensive if not managed properly.

There are multiple log storage calculation options online – what I used back in the days were a set of spreadsheets with pre-configured EPS (events per second) per device or operating system, or one of the online tools, such as http://codepen.io/packetinspector/details/vxjbL/

Another way of obtaining proper log storage calculation spreadsheets is to ask SIEM vendors for them in a pre-sales meeting – their engineers have them and are willing to share them.

There are a few outstanding open source solutions for centralized log storage and analysis (well, not as in a SIEM, but with a perfect searching and filtering capability which is still good) for smaller budgets.

One of them – a classic – is Syslog-NG. 

One of the cybersecurity monitoring tools often used with our clients is Wazuh – it is worth checking out.

GrayLog2

Another excellent open-source log management product is GrayLog2 (https://www.graylog.org/).

It is very similar in operation and performance to the ES/Logstash/Kibana – with the difference that it is a bit simpler to set up and get up and running and that the user interface of GrayLog2 does not use Kibana – which is a weakness, as Kibana is incredibly powerful in terms of what you can do with its dashboards and custom filtering/reporting capabilities.

I will spare you the screenshots / comprehensive technical capabilities – and I’m just mentioning the projects so you could have them in your mind when comparing to commercial solutions.

Security Onion

I cannot possibly write a chapter on security monitoring without mentioning this security Linux distribution.

It is installable on your hardware – and for a small (500 endpoints) would require at least 16 gigs of RAM and many TB of storage (as much as you can / want to afford) to store all the network traffic passing through your gateway.

It operates by running all network traffic through a set of open-source intrusion detection systems. As with all of them turned on the machine would need a significant amount of RAM and CPU power as well as the maximum IO you can push it is not recommended to run it on a VM. It is also recommended that your storage is built out of high-performance disks.

The distro allows you to look back in time in a way, and if an incident happens, to extract the exact packets containing the attack. The more extended period you can store your network traffic for, the better.

https://securityonion.net/

Feeding your SIEM with external threat intelligence data

The value of having threat intelligence data fed into your SIEM can be seen when events from your IDS/IPS/Firewalls/other devices properly correlate with it. This new data will help make sense of the unknowns and generate alerts when a pattern matches a known threat indicator when unknown binaries or traffic patterns enter your network.

Since it is humanely impossible to read all the information passing through your SIEM, you would probably rely on its alerting capability whenever it detects anything suspicious. Let us set aside anomaly detection and spike detection for a moment and focus on threat intelligence as that is a very good source of alerts whenever something matches your network.

External threat intelligence feeds usually comprise of file hashes, IP addresses, hostnames, domain names, Indicators of Compromise (IOCs), matches to YARA rules.

Companies make a living from dissecting botnets, malware, and breach investigations to produce valuable information on detecting even small portions of malware based on similarity and functionality rather than hashes (so-called fuzzy hashing, for example). Then they convert that data into actionable form and sell access to it – so you could plug it into your SIEM.

Once your SIEM is fed with one or more threat intelligence feeds it could notice any similar activities, files, portions of files, accesses to suspicious networks or hosts – and alert you.

Below I will list some free and commercial threat intelligence feeds. It is your choice which ones to use, but I would focus on the commercial ones (after reading reviews from their customers and comparing them).

Commercial threat intelligence providers:

Kaspersky Security Intelligence Services:

http://www.kaspersky.com/enterprise-it-security/security-intelligence-services/

TrendMicro Security Threats Connect: http://www.trendmicro.com/us/security-intelligence/current-threat-activity/threat-connect/

Norse Intelligence Service: http://www.norse-corp.com/products/norse-intelligence-service/index.html (they also have a pretty cool (funny to look at) map of ongoing attack traffic here: http://map.ipviking.com/ )

iSightPartners ThreatScape®: http://www.isightpartners.com/products/threatscape/

VeriSign Security Intelligence: http://www.verisigninc.com/en_US/cyber-security/security-intelligence/threat-intelligence/index.xhtml

CrowdStrike Falcon Intelligence: http://www.crowdstrike.com/falcon-intelligence/

And finally, one service by Microsoft, which is focused on internal data analytics rather than external:

http://www.microsoft.com/en-us/server-cloud/products/advanced-threat-analytics/

Open Source / Free Threat Intelligence providers:

AlienVault Open Threat Exchange (OTX): https://www.alienvault.com/open-threat-exchange

Thanks to http://cyberwarzone.com/30-malicious-ip-list-and-block-lists-providers-2015/, we have a list of 31 (30 active) malicious IP/domain list providers:

1. Intel Stack
2. ATLAS AIF
3. CLEAN-MX Realtime Database
4. CYMRU
5. DShield Blocklist
6. Malc0de Database
7. OpenPhish
8. PhishTank Phish Archive
9. Project Honey Pot’s Directory of Malicious IPs
10. Scumware.org
11. Shadowserver
12. ThreatStop

You can also use these to plug into your Web Filter appliance/proxy server blacklist.