lpicollector 1.0.0

--------------------------------------------------------------------------
Copyright (c) 2013, 2014 The University of Waikato, Hamilton, New Zealand.
All rights reserved.

This code has been developed by the University of Waikato WAND
research group. For further information please see http://www.wand.net.nz/.
---------------------------------------------------------------------------

lpicollector is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.

lpicollector is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with lpicollector. If not, see <http://www.gnu.org/licenses/>.


Report any bugs, questions or comments to contact@wand.net.nz

Authors:
  Meenakshee Mungro
  Shane Alcock


Introduction
============
This software aims to be a reliable ongoing network monitor that periodically
reports counters for bytes, packets, flows and users for each application
protocol supported by libprotoident.

Upon starting, lpicollector will collect statistics about application protocol
usage from a given input source (e.g. a network interface or trace file) and
listen on a TCP socket for clients to connect. The statistics will be
periodically exported to the clients using a custom binary protocol.

Example clients written in Python is included with the source code in the
exampleclients/ directory. lpi.py will parse and display the periodic statistic records produced by lpicollector and simple_flow_reporter.py will parse and
display ongoing and expired flow records.

Developers may use this code as a starting point for developing their own
clients. Further information about the export protocol can be found in the
appropriate source files for the collector, particularly src/lpicp.h.

Required Libraries
==================
libtrace
 * available from http://research.wand.net.nz/software/libtrace.php

libprotoident
 * available from http://research.wand.net.nz/software/libprotoident.php

libflowmanager
 * available from http://research.wand.net.nz/software/libflowmanager.php

libfifo
 * available from http://research.wand.net.nz/software/libfifo.php

libwandevent 3.0.0 or later
 * available from http://research.wand.net.nz/software/libwandevent.php


Installation
============
After having installed the required libraries, running the following series
of commands should install lpicollector

        ./bootstrap.sh    (only if you have cloned from git!)
        ./configure
        make
        make install

By default, lpicollector installs to /usr/local - this can be changed by
appending the --prefix=<new location> option to ./configure.

Also, if you have installed any of the required libraries to a non-standard
location, you will need to tell ./configure where the libraries and header
files can be found using LDFLAGS and CPPFLAGS.


Running lpicollector
====================
You can start lpicollector using the following syntax:
  lpicollector <options> <input>

The input must be a valid libtrace URI. A URI may be a trace file name,
an interface name (e.g eth0) or a device (e.g. /dev/dag0), depending on the
input source.
See https://github.com/wanduow/libtrace/wiki/Supported-Trace-Formats for more
information on libtrace URIs and supported input formats.

lpicollector supports a large number of command-line options. Some of the
options have not been fully tested -- these are marked as experimental in
the list below. If an option is experimental, use it at your own risk!

Basic options:
 -f <filter>:  Ignore packets that do not match the given BPF filter string.
 -p <port>:    Listen for incoming connections on this port -- default is 3678.
 -c <clients>: Limit the number of connected clients to this number.
 -s <bytes>:   Size of the internal message fifo in bytes. Default is 100MB.
 -R:           If set, all flows using RFC 1918 address space are ignored.
 -r <freq>:    Report statistics every <freq> seconds.
 -i <id>:      ID string to use for this monitor, defaults to $HOSTNAME.

Direction tagging  (choose one of these options):
 -P:           Use port number to determine direction. This is the least
               accurate method of assigning direction but will always work.

 -T:           Use existing direction tags to determine direction. This is
               the best method, but requires your packet capture to correctly
               assign direction to each packet. Don't use this with a pcap
               source (file or interface).

 -l <mac>:     Assign direction to each packet based on the given MAC address
               representing the 'inside' portion of the network. Therefore,
               packets with a source MAC address of <mac> will be deemed as
               outgoing and packets with a dest MAC address of <mac> will be
               deemed as incoming.

If no tagging method is provided, -T is assumed.

User tracking (experimental):
 -U:           Track and report LPI statistics separately for each observed
               'local' IP address. An IP address is determined to be local
               based on the direction tag, so only use this option if you
               have accurate direction tagging. Don't use this if you have
               set the -P option!

               All addresses that receive incoming packets or send outgoing
               packets are treated as local addresses for this purpose.

               Enabling this will greatly increase the amount of memory
               required by the lpicollector, especially if you are monitoring
               a large network with a lot of users.


Individual Flow monitoring (experimental):
 -E, -O:       Enable reporting expired and ongoing flows, respectively. This
               will cause lpicollector to act as a flow meter. I wouldn't
               recommend enabling this, unless you really know what you're
               doing.
 -e <freq>:    Report expired flows every <freq> seconds.
 -v <freq>:    Report ongoing flows every <freq> seconds.
 -o <seconds>: Number of seconds that a flow must run to be classified as
               ongoing.