Add option on Tile-Gx for logging for fast.log alerts over PCIe

When running on a TILEncore-Gx PCIe card, setting the filetype of fast.log
to pcie, will open a connection over PCIe to a host application caleld
tile-pcie-logd, that receives the alert strings and writes them to a file
on the host. The file name to open is also passed over the PCIe link.

This allows running Suricata on the TILEncore-Gx PCIe card, but have the
alerts logged to the host system's file system efficiently. The PCIe API that
is used is the Tilera Packet Queue (PQ) API which can access PCIe from User
Space, thus avoiding system calls.

Created util-logopenfile-tile.c and util-logopen-tile.h for the TILE
specific PCIe logging functionality.

Using Write() and Close() function pointers in LogFileCtx, which
default to standard write and close for files and sockets, but are
changed to PCIe write and close functions when a PCIe channel is
openned for logging.

Moved Logging contex out of tm-modules.h into util-logopenfile.h,
where it makes more sense. This required including util-logopenfile.h
into a couple of alert-*.c files, which previously were getting the
definitions from tm-modules.h.

The source and Makefile for tile-pcie-logd are added in contrib/tile-pcie-logd.

By default, the file name for fast.log specified in suricata.yaml is used as
the filename on the host. An optional argument to tile-pcie-logd, --prefix=,
can be added to prepend the supplied file path. For example, is the file
in suricata.yaml is specified as "/var/log/fast.log" and --prefix="/tmp",
then the file will be written to "/tmp/var/log/fast.log".

Check for TILERA_ROOT environment variable before building tile_pcie_logd

Building tile_pcie_logd on x86 requires the Tilera MDE for its PCIe libraries
and API header files. Configure now checs for TILERA_ROOT before enabling
builing tile_pcie_logd in contrib/tile_pcie_logd

configure.ac

@@ -399,6 +399,8 @@
         AC_DEFINE([HAVE_DETECT_DISABLED], [1], [Detection is disabled])]
     ])
 
+    AM_CONDITIONAL([BUILD_PCIE_LOGGING], [test ! -z "$TILERA_ROOT"])
+
 # libraries
 
     AC_MSG_CHECKING([for Mpipe])
@@ -407,7 +409,7 @@
         [
         AC_MSG_RESULT([yes])
         AC_DEFINE([HAVE_MPIPE],[1],[mPIPE support is available])
-        LDFLAGS="$LDFLAGS -lgxio -ltmc"
+        LDFLAGS="$LDFLAGS -lgxpci -lgxio -ltmc"
         ],
         [AC_MSG_RESULT([no])])
 
@@ -1628,7 +1630,7 @@ AC_SUBST(e_localstatedir)
 AC_DEFINE_UNQUOTED([CONFIG_DIR],["$e_sysconfdir"],[Our CONFIG_DIR])
 AC_SUBST(e_magic_file)
 
-AC_OUTPUT(Makefile src/Makefile qa/Makefile qa/coccinelle/Makefile rules/Makefile doc/Makefile contrib/Makefile contrib/file_processor/Makefile contrib/file_processor/Action/Makefile contrib/file_processor/Processor/Makefile suricata.yaml scripts/Makefile scripts/suricatasc/Makefile scripts/suricatasc/suricatasc)
+AC_OUTPUT(Makefile src/Makefile qa/Makefile qa/coccinelle/Makefile rules/Makefile doc/Makefile contrib/Makefile contrib/file_processor/Makefile contrib/file_processor/Action/Makefile contrib/file_processor/Processor/Makefile contrib/tile_pcie_logd/Makefile suricata.yaml scripts/Makefile scripts/suricatasc/Makefile scripts/suricatasc/suricatasc)
 
 SURICATA_BUILD_CONF="Suricata Configuration:
   AF_PACKET support:                       ${enable_af_packet}

contrib/Makefile.am

@@ -1,3 +1,3 @@
-SUBDIRS = file_processor
+SUBDIRS = file_processor tile_pcie_logd
 
 EXTRA_DIST = suri-graphite

contrib/tile_pcie_logd/Makefile.am

@@ -0,0 +1,14 @@
+
+EXTRA_DIST = LICENSE README
+
+if BUILD_PCIE_LOGGING
+bin_PROGRAMS = tile_pcie_logd
+
+tile_pcie_logd_SOURCE = tile_pcie_logd.c
+
+AM_CFLAGS = -std=gnu99 -Wall -Werror -g -O2 -I$(TILERA_ROOT)/include \
+            -DTILEPCI_HOST
+
+tile_pcie_logd_LDDADD = -pthread
+
+endif

contrib/tile_pcie_logd/README

@@ -0,0 +1,38 @@
+Introduction
+------------
+
+This application allows writing files to an x86 host from a TILEncore-Gx
+PCIe card.  The file name and data are sent over PCIe using the Tilera
+Packet Queue API from an aplication running on the Tilera processor.
+
+The original purpose is to write log files from Suricata (Intrusion
+Dectection System) on the x86 host's file system.
+
+Running The Logger
+------------------
+
+To run the application, set the TILERA_ROOT environment variable to
+point to a valide Tilera MDE, then do:
+
+   make run
+
+The application should be started before the application on the Tile
+side that will be generating the log data.
+
+By default, queue number 0 is used. The --queue_index=N command line
+argument can be used to change the queue number.
+
+If more than one TILEncore-Gx PCIe card is installed, the --card=M
+argument changes to listening to card M.
+
+Caveats
+-------
+
+Due to the fact that the host driver allocates 4MB physically
+contiguous memory for the packet queue ring buffer, it is possible
+that this allocation could fail on a host whose memory has been
+considerably fragmented. If the host program exits with the following
+error, reboot the host and run the test again.
+
+Host: Failed to open '/dev/tilegxpci%d/packet_queue/t2h/0': Cannot
+allocate memory