ourmon - network monitoring and anomaly detection system

download and related web pages

introduction

Ourmon is a statistically oriented open-source network monitoring and anomaly detection system. It may also be viewed as a flow collection system. Ourmon is based on promiscuous mode packet collection on Ethernet interfaces and typically uses port mirroring via an Ethernet switch. A probe collects packets deemed important and sends internally defined tuples back to a graphics display system which may or may not be on the same host. Ourmon employs more extreme aggregation than netflow. Ourmon does not collect all the packets because one principle design goal is to extract signal from noise, not store all the noise in a giant bag under the assumption that you can peruse it "later" (there is no later). Ourmon also has its own notion of tuples and although it does support a traditional flow tuple, it also uses tuples focused on IP host addresses, and even Layer 7 IRC channels. Ourmon is not shy about looking at Layer 7 data payloads.

Ourmon analyzes data using both multiple instances of the Berkeley Packet Filter, and also various hashed top talker lists and then displays the data using RRDTOOL graphs, histograms, and ASCII reports. Data is produced in near realtime every thirty seconds. Hourly reports are also produced for some of the top talker lists and summarized on a daily basis, giving approximately one week of summarized logging reports.

Ourmon is customizable and allows the user to create his or her own graphs by grouping BPF expressions (as used in tcpdump) into RRDTOOL-style graphs. A user may also create new regular expressions for tagging layer 7 data payloads in flows. Ourmon provides graphs, logging, and reports concerning various kinds of flow lists including conventional IP flows, top N syns, top N ports, ICMP and UDP errors, and scanners including 1-N IP src to IP dst, and 1-N IP src to UDP or TCP L4 ports. The latest version also allows regular expressions to be used to tag various kinds of flow outputs including traditional flows, the top_n syn tuple (TCP port report) and UDP error tuple (UDP port report) flows. This facility uses PCRE or Perl Compatible Regular Expressions. The goal is to allow the administrator to look at layer 7 information and tag hosts using P2P applications like Bittorrent, or determine which hosts are performing as an FTP server. Some application tags are provided in the current release. New ones may be created by the user.

Ourmon includes traditional flow monitoring but in point of fact defines its own internal formats for "flows" allowing flows to be more efficient and focus only on gathering information of interest to the flow tuple in question. For example, ourmon currently includes two IRC (layer 7 application) "flow" tuples, one for IRC channels and one for IRC hosts in a given IRC channel. A flow tuple for DNS queries now exists as well as another flow tuple for SSH scanning. Many flow tuples are based on the IP host address and give important clues about what a host is doing. These are useful for anomaly detection. Ourmon also provides information about coordinated attacks and various kinds of worms.

target systems:

FreeBSD and or Linux, ubuntu in particular. We run the probe on FreeBSD, although it can work on Linux.

some things ourmon can catch or do:

new features:

features:

news/papers about ourmon

documentation

support

send email to binkley.jim@gmail.com