This is VERMONT - VERsatile MONitoring Tool.
Released under GPL2

Vermont (VERsatile MONitoring Toolkit) is an open-source software toolkit for the creation and processing of network flow data, based on monitored Internet packet data. The IETF standard IPFIX (IP Flow Information eXport) defines the formats and procedures for handling these flows. Furthermore, the Netflow.v9 and the PSAMP (Packet Sampling) standards are supported. Vermont runs on Linux and derivatives of BSD. It can receive and process raw packets via PCAP (up to 10 GBit/s) as well as IPFIX/Netflow.v9 flow data.

This is a fork of Vermont that focuses on the aggregation of HTTP related information into IPFIX in high-speed networks. The used HTTP related IE are standardized and registered with IANA. The new IpfixIDS (aka FIXIDS) module uses these IEs for intrusion detection.

For more information see also the wiki at https://github.com/felixe/ccsVermont/wiki.

For more into depth info, there are several publications relying on this version of Vermont:
"FIXIDS: A High-Speed Signature-based Flow Intrusion Detection System", http://www.ccs-labs.org/bib/erlacher2018fixids/
"High Performance Intrusion Detection Using HTTP-Based Payload Aggregation",  http://www.ccs-labs.org/bib/erlacher2017high/
"Improving Network Monitoring Through Aggregation of HTTP/1.1 Dialogs in IPFIX", http://www.ccs-labs.org/bib/erlacher2016improving/


------------
REQUIREMENTS
------------

The following tools and libraries are needed for Vermont (including header files). In parenthesis you will find versions that Vermont has been tested with (on Debian 8 and Ubuntu 16.04), but also other versions might work.

    gcc (4.9,5.4)
    cmake (and ev. 'cmake-curses-gui')
    libpcap-dev
    libpcre3-dev
    libboost-all-dev (1.55,1.58,1.62)
    libxml2-dev
    libsctp-dev (optional)
    postgresql (>= 8.3, optional)
    mysql (optional)
    libpq (optional)


-------------------------
BUILDING AND INSTALLATION
-------------------------

This project uses cmake for setting platform- and user-specific compile 
options. In order to generate the Makefile for actual compilation, you 
need to call in the root of the source directory:

$ cmake .

In order to change the default compile options, use:

$ cmake -D OPTION1=value1 -D OPTION2=value2 ...

To get a list of the most important options, call:

$ cmake -LH
   
As a user-friendly alternative, you can use the interactive user interface.
Please note that this requires the package cmake-curses-gui, if you are using
Ubuntu/Debian.

$ ccmake .

If some libraries are installed in custom directories, use:

$ cmake -D CMAKE_PREFIX_PATH=/custom/directory1:/custom/directory2

After successfully generating the Makefile with cmake, start the 
compilation with:

$ make

Although not strictly necessary, VERMONT binaries and data files can be 
copied to the usual install location by running:

$ make install 

------------------------------
BUILDING WITH PF_RING_ZC SUPPORT
------------------------------

PF_RING Zero Copy is a driver and library suite that allows for very fast packet capturing. For more info on PF_RING see:
https://www.ntop.org/products/packet-capture/pf_ring/
https://github.com/ntop/PF_RING

If the option "USE_PFRING_ZC" is enabled in cmake the building process requires corresponding libraries and an approriate NIC driver at runtime. The easiest and hassle-free way to provide the required components is to download the above mention github PF_RING repository and follow these steps:
cd into the repo, checkout the latest stable branch (at time of writing 7.0.0-stable), compile everything with #: make, then install the driver for you NIC with the following commands #: cd ../../drivers/intel/<YOURDRIVER>/src/; sudo ./load_driver.sh
In cmake enable the option USE_PFRING_ZC and provide the path to the PF_RING git repo in PFRING_ZC_PATH.
If the make process complains about not finding pfring.h, chances are high that you provided the wrong PFRING_ZC_PATH.
Now Vermont is ready to use PF_RING. If you want Vermont to use PF_RING Zero Copy (direct acces to the NIC memory from the userland application) prepend a zc: to the network device in the vermont configuration (e.g.: zc:eth0). At the moment there is a 5 min restriction if you want to use Zero Copy without a license. Ntop.org offers free academic licenses or commercial licenses at fair prices.

-----------------------------------
BUILDING WITH DTLS-OVER-UDP SUPPORT
-----------------------------------

VERMONT's DTLS support is based on OpenSSL version 1.0.0 (and maybe higher). 

Since the DTLS implementation in OpenSSL is fairly new and not as mature as 
the TLS/SSL implementation, you should use the latest version of OpenSSL which 
you can get from 

http://openssl.org/source/ 

At the time of writing of the DTLS part (July 2010), the latest version is 1.0.0a.

$ wget http://openssl.org/source/openssl-1.0.0a.tar.gz
$ tar xzf openssl-1.0.0a.tar.gz
$ cd openssl-1.0.0a/

If you want to profit from the most recent bugfixes, you can check out the 
sources from the OpenSSL CVS repository instead:

$ cvs -z9 -d anonymous@cvs.openssl.org:/openssl-cvs co openssl
$ cd openssl/

In order to avoid incompatibilities with other packages of your distribution,
you probably do not want the new version of OpenSSL to become the default 
OpenSSL library on your system. Therefore, it is recommended to install the 
new version in a local directory by using the --prefix option of the config
script.

To build OpenSSL and install it into a built/ subdirectory within the OpenSSL
source directory, call the following commands:

$ ./config -d no-dso no-shared --prefix=`pwd`/built
$ make 
$ make install

The configure option "no-dso" turns off the use of shared-library methods which 
avoids linking problems related to libdl on the Linux platform.
With the option "no-shared", only static libraries are built which makes it 
easier to link VERMONT to the correct version of OpenSSL.

In order to compile VERMONT with DTLS-over-UDP support, change into the root
of VERMONT's source directory and execute cmake with the OpenSSL include and 
library paths (replace "/path/to/openssl" by your OpenSSL source directory):

$ cmake -DSUPPORT_DTLS=YES -DCMAKE_INCLUDE_PATH=/path/to/openssl/built/include -DCMAKE_LIBRARY_PATH=/path/to/openssl/built/lib

On 64 bit platforms, the library path might be different (mind the "64" at the 
very end!):

$ cmake -DSUPPORT_DTLS=YES -DCMAKE_INCLUDE_PATH=/path/to/openssl/built/include -DCMAKE_LIBRARY_PATH=/path/to/openssl/built/lib64

If you have previously built VERMONT with OpenSSL located in another 
directory, you might need to manually remove the file CMakeCache.txt before 
calling cmake.

After cmake has finished, you should be able to build VERMONT with 
DTLS-over-UDP support by calling

$ make 

Please read the next section if you require support for DTLS over SCTP as well.


------------------------------------
BUILDING WITH DTLS-OVER-SCTP SUPPORT
------------------------------------

At the time of writing of this paragraph (July 2010), DTLS over SCTP can be used on FreeBSD only!
This is due to the fact that FreeBSD is currently the only OS which supports 
the SCTP-AUTH extension (see RFC 4895) which is required by DTLS.

The current version of OpenSSL (1.0.0a) has no native support for SCTP. You 
have to download additional patches from

http://sctp.fh-muenster.de/

and apply them to the OpenSSL sourcese before building OpenSSL. Make sure that
the patches fit to your local version of OpenSSL. Otherwise, you might need to
manually adapt the patch files.

Also, make sure to add the command line argument "sctp" when running OpenSSL's
./config to build SCTP support into OpenSSL.

In order to compile VERMONT with DTLS-over-SCTP support, you need to run cmake
with the following three options:

-DSUPPORT_SCTP
-DSUPPORT_DTLS
-DSUPPORT_DTLS_OVER_SCTP

In addition, you need to indicate the include and library paths to your patched
version of OpenSSL as explained for DTLS-over-UDP.


-----------------------
USAGE AND CONFIGURATION
-----------------------

In order to run VERMONT, a configuration file is needed which specifies the 
modules to be used and their parameters:

$ ./vermont -f <config-file>

Example configuration files can be found in configs/.
A documentation of the available modules and their configuration parameters
can be found in the wiki at https://github.com/felixe/ccsVermont/wiki.

Use Ctrl-C to stop VERMONT. If VERMONT does not exit properly, enter Ctrl-C
for a second time.


----------------------------------------------------
OPERATION AS COLLECTOR: TUNING SOCKET RECEIVE BUFFER
----------------------------------------------------

VERMONT can be used as an IPFIX/PSAMP and NetFlow.v9 collector. As the 
incoming IPFIX/PSAMP/NetFlow messages usually arrive in bursts, losses 
may occur due to insufficient buffer space.

As a solution, the socket receive buffer can be increased. To check the
current settings, use the following system calls on Linux systems with 
/proc file system:

$ cat /proc/sys/net/core/rmem_default
$ cat /proc/sys/net/core/rmem_max

In order to configure a new value X (in bytes), call:

$ sysctl -w net/core/rmem_default=X
$ sysctl -w net/core/rmem_max=X

-------------------------------------
EFFECTS OF RECEIVE OFFLOAD MECHANISMS
-------------------------------------

Several mechanisms have been implemented in modern network interface cards,
drivers, and kernels to offload common functions from the protocol stack and 
the application. One particular focus is on TCP segmentation and reassembly.

Receive offload mechanisms aim at reassembling subsequent TCP segments into
a single large packet before passing it to the IP/TCP protocol stack and 
finally to the application. In the Linux kernel, this is done by generic 
receive offload (GRO) if the network interface card and the driver support 
NAPI. Latest Intel 10GE controllers (e.g., 82599) support receive side 
coalescing (RSC) which performs TCP reassembly in hardware.

If any receive offload mechanism is enabled, VERMONT (like any other 
pcap-based application) does not observe the actually captured TCP packets 
but the reassembled ones. One consequence is that packet counts of flows will 
be smaller than the true number of packets.

In order to avoid such distortions, all receive offload mechanisms need to 
be disabled. In the case of GRO (and the older LRO), this can be done with 
ethtool. The following call returns a list of the current status of all 
offload mechanisms for interface <dev>:

$ ethtool -k <dev>

If GRO is not shown, you probably need to install a newer version of ethtool.
To disable GRO (and LRO), execute:

$ ethtool -K <dev> gro off
$ ethtool -K <dev> lro off

Hardware-based RSC can be deactivated at compile time of the driver.