Network session data analysis with Snort and Argus

This tip explains how to use flow/session data can complement the alert data supplied by the Snort intrusion detection system for network session data analysis.

The Snort intrusion detection system can identify suspicious and malicious activity by inspecting network traffic. Snort makes a judgment based on its analytical capabilities and notifies the operator of its decision by generating an alert. I call the output of this collect-inspect-report process "alert data."

While this is a good and necessary methodology, it has one important flaw. In most configurations, Snort is not told to report on what it sees if the traffic in question is deemed to be "normal." One might consider this aspect of Snort to be a benefit. Why generate an alert if the traffic is "normal" and not suspicious or malicious?

No alerting system can perfectly identify all suspicious or malicious activity. In many cases it's simply not possible -- especially on a packet-by-packet basis -- to identify a packet or stream as being worthy of an operator's attention. In those cases it makes sense to keep a log of the traffic. Recording traffic or characteristics of traffic for later analysis has recently been labeled retrospective network analysis (RNA), not to be confused with Sourcefire's Real-time Network Awareness. Others call recording traffic in this manner "network forensics," but that implies a degree of care and evidence handling that exceeds the methodology I present here.

When you collect data about traffic that Snort didn't consider to be suspicious or malicious, you have the opportunity to look back (hence the term "retrospective") to see what happened during an incident. How do you know to look back? Perhaps you receive a tip from law enforcement. Maybe a client reports odd activity. Or you perform a manual investigation and realize you'd like to know as much as possible about the network traffic of a certain host. In all of these situations, Snort might not have provided any clue that something was amiss.

Despite my attention to Snort in this series, I never deploy Snort as a stand-alone tool. I always supplement Snort with additional data sources. One of the most important supplementary data sources I collect is session data.

What is session data?

As I wrote in my first book, The Tao of Network Security Monitoring (Addison-Wesley, 2005): "Session data represents a summary of a conversation between two parties. [...] A session, also known as a flow, a stream or a conversation, is a summary of a packet exchange between two systems." I also defined the seven minimum elements of a session, namely:

  • Timestamp (generally when the session began, preferably including the end of the session, too).
  • Protocol.
  • Source IP address.
  • Source port (for protocols that offer ports, like TCP, SCTP and UDP).
  • Destination IP address.
  • Destination port.

Measure of the amount of information exchanged during the session (to include bytes sent by the source, bytes sent by the destination, packets sent by the source and packets sent by the destination).

I don't consider records that lack these seven basic elements to be session records. If you have a record showing only a source IP talking to a destination IP, it isn't completely worthless. However, a real session record includes the other five elements listed in addition to source IP and destination IP.

What does a session look like?

The following is a session record. I'll explain how I generated this record later.

 StartTime Proto SrcAddr Sport DstAddr Dport SrcPkts SAppBytes DstPkts DAppBytes 1204514703.744796 tcp 69.255.105.234.32903 64.233.185.83.443 10 1953 9 4049

The columns contain the following:

1204514703.744796: date in Unix time format. Using the date command we can convert to something more obvious:

p># date -r 1204514703
Sun Mar 2 22:25:03 EST 2008

tcp: the protocol of the session

69.255.105.234.32903: the source IP and source port

64.233.185.83.443: the destination IP and destination port

10: number of packets sent by the source
1953: number of bytes of application (above layer 4) data sent by the source
9: number of packets sent by the destination
4049: number of bytes of application (above layer 4) data sent by the destination

This is a bidirectional session record. In other words, it displays data sent by the source and destination in a single record. Other types of session records, such as NetFlow (not shown here) are unidirectional. That means two records are needed to describe a conversation. The first lists data from source to destination, while the second lists data from destination to source.

Getting started with Argus

Let's look at how you can collect session data using Argus 3.0, developed by Carter Bullard. Argus dates back to 1993, the days before Snort and Wireshark/Ethereal, when Tcpdump was one of the few tools a network security analyst had. For the last two years, Carter has been coding Argus 3.0, and this article uses code posted to ftp.qosient.com/dev for introductory purposes.

Argus 3.0 is a complicated and powerful tool. Those who frequent the Argus Development Mailing List are not going to learn anything new here! This article is for those who are unfamiliar with Argus but would like to try it for network session data analysis.

The following examples are run on FreeBSD 6.2:

Argus is currently packaged as a server component (argus) and client programs (argus-clients). I retrieve both using "fetch" and then extract them.

hacom:/usr/local/src# fetch ftp://ftp.qosient.com/dev/argus-3.0/argus-3.0.0.tar.gz
argus-3.0.0.tar.gz       100% of 391 kB 833 kBps
hacom:/usr/local/src# fetch ftp://ftp.qosient.com/dev/argus-3.0/argus-clients-3.0.0.rc.69.tar.gz
argus-clients-3.0.0.rc.69.tar.gz      100% of 1422 kB 1058 kBps

hacom:/usr/local/src# tar -xzf argus-3.0.0.tar.gz
hacom:/usr/local/src# tar -xzf argus-clients-3.0.0.rc.69.tar.gz

Because I like to experiment and keep multiple versions of programs like Argus, I install each program into its own directory in /usr/local. (Yes, I could use Stow.)

hacom:/usr/local/src# mkdir /usr/local/argus-3.0.0
hacom:/usr/local/src# mkdir /usr/local/argus-clients-3.0.0.rc.69

Next, I compile the Argus server package. Only an "argus" binary is created in the "bin" directory:

hacom:/usr/local/src/argus-3.0.0# ./configure --prefix=/usr/local/argus-3.0.0
hacom:/usr/local/src/argus-3.0.0# make
hacom:/usr/local/src/argus-3.0.0# make install
hacom:/usr/local/src/argus-3.0.0# ls /usr/local/argus-3.0.0/sbin/
argus

Finally, I compile the Argus clients package. A variety of tools are created in the "bin" directory:

hacom:/usr/local/src/argus-3.0.0# cd ../argus-clients-3.0.0.rc.69
hacom:/usr/local/src/argus-clients-3.0.0.rc.69# ./configure --prefix=/usr/local/argus-clients-3.0.0.rc.69
hacom:/usr/local/src/argus-clients-3.0.0.rc.69# make
hacom:/usr/local/src/argus-clients-3.0.0.rc.69# make install
hacom:/usr/local/src/argus-clients-3.0.0.rc.69# ls /usr/local/argus-clients-3.0.0.rc.69/bin/
argusbug        radark          rahisto         rapolicy        rastrip
ra              radump          rahosts         raports         ratemplate
rabins          rafilteraddr    ralabel         rasort          ratimerange
racluster       ragraph         ranonymize      rasplit         ratop
racount         ragrep          rapath          rastream        ratree

Now I'm ready to collect session data with Argus.

Starting the Argus server

Let's look at Argus 3.0's more easily demonstrated features. The help screen summarizes command-line options; I don't create a config file here:

hacom:/root# /usr/local/argus-3.0.0/sbin/argus -h
Argus Version 3.0.0
usage: argus [options] [-i interface] [filter-expression]
usage: argus [options] -r packetfile [filter-expression]

options: -A                   Generate application byte metrics.
         -b                   dump filter compiler output.
         -B             specify bind interface address.
         -c  <dir>            daemon chroot directory.
         -d                   run Argus in daemon mode.
         -e            specify Argus Identifier .
         -h                   print help.
         -F         read configuration from .
         -J                   generate packet performance data.
         -M             set MAR Status Report Time Interval (300s).
         -m                   turn on MAC Layer Reporting.
         -O                   turn off filter optimizer.
         -p                   don't go into promiscuous mode.
         -P           enable remote access on   (561).
         -r 

Note that the help page is missing the -i option to specify a sniffing interface.

Now I start the Argus server.

hacom:/root# /usr/local/argus-3.0.0/sbin/argus -A -B 10.1.13.4 -P 651 -R -u sguil -g sguil -U 256 -w /nsm/argus/argus3.test.arg -Z -i bridge0 - ip
ArgusWarning: argus[42020]: 02 Mar 08 22:10:18.647475 started
ArgusWarning: argus[42020]: 02 Mar 08 22:10:18.648282 ArgusGetInterfaceStatus: interface bridge0 is up

I tell Argus to generate application byte metrics, bind its server to IP 10.1.13.4 and port 651 TCP, to generate response time data, to run as user sguil group sguil, to capture 256 bytes of user data, where to write its output, to generate packet size data, to sniff on interface bridge0 and to watch only IP traffic.

The sockstat command shows Argus has bound a server to port 651 TCP. We'll see what that offers soon.

hacom:/root# sockstat -4 | grep argus
sguil    argus      41996 4  tcp4   10.1.13.4:651         *:*
sguil    argus      41996 6  udp4   *:*                   *:*

As Argus collects data it appears in the specified directory:

hacom:/root# ls -lh /nsm/argus/
total 3586
drwxr-xr-x  2 root   sguil   512B Mar  2 22:04 .old
-rw-r--r--  1 sguil  sguil   3.5M Mar  2 22:04 argus3.test.arg

With Argus saving data, I need to use one of its client programs to read it.

Using the Argus ra client

One of the fundamental client programs packaged with Argus is ra. The help options in the 3.0 ra client are extensive:

hacom:/usr/local/argus-clients-3.0.0.rc.69/bin# ./ra -h
Ra Version 3.0.0.rc.69
usage: ra
usage: ra [options] -S remoteServer [- filter-expression]
usage: ra [options] -r argusDataFile [- filter-expression]

options: -A                    print record summaries on termination.
         -b                    dump packet-matching code.
         -c              specify a delimiter  for output columns.
         -C <[host]:port>      specify Cisco Netflow source.
         -e            convert user data using  method.
                               Supported types are  and .
         -E              write records that are rejected by the filter
                               into 
         -F          read configuration from .
         -h                    print help.
         -M rmon               convert bi-directional flow data to RMON in/out stats
            poll               attach to remote server to get MAR and then disconnect
         -n                    don't convert numbers to names.
         -p            print fractional time with  precision.
         -q                    quiet mode. don't print record outputs.
         -r              read argus data . '-' denotes stdin.
         -R &ltdir>              recursively process files in directory
         -s [-][+[#]]field[:w] specify fields to print.
                   fields:     srcid, stime, ltime, sstime, dstime, sltime, dltime,
                               trans, seq, flgs, dur, avgdur, stddev, mindur, maxdur,
                               saddr, daddr, proto, sport, dport, stos, dtos, sdsb, ddsb
                               sco, dco, sttl, dttl, sipid, dipid, smpls, dmpls, svlan, dvlan
                               svid, dvid, svpri, dvpri, [s|d]pkts, [s|d]bytes,
                               [s||d]appbytes, [s|d]load, [s|d]loss, [s|d]ploss, [s|d]rate,
                               smac, dmac, dir, [s|d]intpkt, [s|d]jit, state, suser, duser,
                               swin, dwin, trans, srng, erng, stcpb, dtcpb, tcprtt, inode,
                               offset, smaxsz, dmaxsz, sminsz, dminsz
         -S 

The following is one way to invoke ra. I tell ra to show time in Unix format, with TCP flags, column headers, no resolution of IP addresses, while reading the specified file. The format of the remainder of the output is the same as demonstrated earlier. I conclude with a filter "- tcp" to only display TCP records, and I use the head command to limit the output to 10 lines.

hacom:/usr/local/argus-clients-3.0.0.rc.69/bin# ./ra -u -Z b -L0 -n -r /nsm/argus/argus3.test.arg -s stime proto saddr sport daddr dport spkts sappbytes dpkts dappbytes state - tcp | head
    StartTime  Proto     SrcAddr  Sport     DstAddr  Dport  SrcPkts    SAppBytes  DstPkts    DAppBytes State 
 1204513818.770060    tcp  69.255.105.234.2377  208.73.181.192.5223     1  0   1            0   A_A
 1204513828.429709    tcp  64.233.185.19.443   69.255.105.234.40237    1   0   1   0  FA_A
 1204513828.858757    tcp  69.255.105.234.2377  208.73.181.192.5223     1   0   1            0   A_A
 1204513830.041338    tcp  69.255.105.234.3192   209.40.96.212.995     15   421  17    3974 FSPA_

This sort of raw output is helpful if you want to inspect specific records. For a more top-down approach I turn to racluster. Note the first and third records describe the same session. This must be a persistent connection. We can use the racluster client to aggregate these, assuming Argus has seen most or all of the session from the beginning.

Using the Argus racluster client

The Argus client racluster is useful for getting a higher-level view of session data. The client merges records from the same flows. First I show the help options for racluster:

hacom:/usr/local/argus-clients-3.0.0.rc.69/bin# ./racluster -h
Racluster Version 3.0.0.rc.69
usage:  racluster [-f racluster.conf]
usage:  racluster [-f racluster.conf] [ra-options] [- filter-expression]

options:  -f       read aggregation rules from .
          -m flow key fields       specify fields to be used as flow keys.
          -M modes                 modify mode of operation.
             Available modes:      
                ind                aggregate multiple files independently
                norep              do not report aggregation statistics
                rmon               convert bi-directional data into rmon in/out data
                replace            replace input files with aggregation output
          -V                       verbose mode.

In the following example I tell racluster to collect all records involving host 64.22.185.19 and present the first four results:

hacom:/usr/local/argus-clients-3.0.0.rc.69/bin# ./racluster -L0 -M norep -u -n -r /nsm/argus/argus3.test.arg - host 64.233.185.19 and tcp | head -n 5
 StartTime   Flgs  Proto  SrcAddr  Sport   Dir  DstAddr  Dport  TotPkts   TotBytes State 
 1204513828.429709  e    tcp 64.233.185.19.443     69.255.105.234.40237  6   421   RST
 1204513862.958283  e    tcp 69.255.105.234.40238  ->   64.233.185.19.443  25  7741   RST
 1204513878.329589  e    tcp 69.255.105.234.40239  ->   64.233.185.19.443  23  5684   RST
 1204514058.412265  e    tcp 69.255.105.234.57330  ->   64.233.185.19.443  23  5684   RST

The following shows aggregation based on unique source and destination IP address pairs:

hacom:/usr/local/argus-clients-3.0.0.rc.69/bin# ./racluster -L0 -M norep -m saddr daddr -u -n -r /nsm/argus/argus3.test.arg - tcp | head -n 5
         StartTime    Flgs  Proto       SrcAddr  Sport   Dir       DstAddr  Dport  TotPkts   TotBytes State 
 1204516393.988996  e     ip     24.188.136.188     <->     69.255.105.234          4        276   CON
 1204513828.429709  e     ip      64.233.185.19     <->     69.255.105.234       1513     377809   CON
 1204513831.418396  e     ip      64.233.185.83     <->     69.255.105.234       1679     446483   CON
 1204516648.005118  e     ip       65.214.44.29     <->     69.255.105.234          8        504   CON

To change the order of the output, call rasort. Although I won't show the output here, you could use the following command to sort the racluster output by bytes.

hacom:/usr/local/argus-clients-3.0.0.rc.69/bin# ./racluster -M norep -m saddr daddr –u
-n -r /nsm/argus/argus3.test.arg -w - - tcp | ./rasort -m bytes -L0 | head -5

This article has only scratched the surface of Argus 3.0. If you're interested in using Argus for the collection and analysis of network session data, I recommend starting to record some data using the Argus server. Once you have data to inspect you can begin trying the client programs.

About the author


Richard Bejtlich is the founder of TaoSecurity, author of several books on network security monitoring, including
Extrusion Detection: Security Monitoring for Internal Intrusions, and operator of the TaoSecurity blog.

-ADS BY GOOGLE

SearchStorage.com.au

SearchSecurity.com.au

SearchCIO

SearchCIOMidmarket

SearchCompliance

Close