HTTPS clone URL
Subversion checkout URL
Correlate packets to the responsible processes in Linux systems. Diagnose connections by adding process information.
C Makefile Shell
Copyright (C) 2011 Battelle Memorial Institute Licensed under the GNU General Public License Version 2. See LICENSE for the full text of the license. See DISCLAIMER for additional disclaimers. Author: Brandon Carpenter Contributors: Alex Malozemoff and Glenn Fink H H OOO N N EEEEE H H O O NN N E HHHHH O O N N N EEEE H H O O N NN E H H OOO N N EEEEE Hone is a unique tool for correlating packets to processes to bridge the HOst-NEtwork divide. INTRODUCTION ============ When diagnosing anomalous behavior on a network, a system administrator has two separate areas to focus on: the packets traveling over the network (i.e., the network view), and the information contained on the individual hosts connected to the network (i.e., the host view). The network view provides a glimpse into the overall communication activity of the network, but it does not reveal what processes are causing this activity. On the other hand, the host view contains details on the processes producing the network traffic, but it does not contain information on which packets are associated with which process. This inability to correlate packets with their associated process is a fundamental (although intentional) shortcoming of the modern network stack. To bridge this gap we introduce the Hone (Host-network) correlator, an open-source tool that correlates packets to processes to diagnose problems seen on a network. While the idea of correlating packets to processes is a simple one, the implementation requires kernel modifications that alter the way the network stack works. Perhaps this complication is responsible for the fact that no other tool takes this approach. While there have been several tools that have come close to the packet-process correlation approach taken here, they differ from Hone in fundamental ways. Discussions of these tools and their differences from Hone can be found in: FINK, G., DUGGIRALA, V., CORREA, R., AND NORTH, C. Bridging the host-network divide: survey, taxonomy, and solution. In Proceedings of the 20th Large Installation System Administration Conference (2006), pp. 247–262. We plan to publish a follow-on paper on this work soon. HOW IT WORKS ============ The Linux Hone correlator uses the netfilter framework, which provides hooks for packet capture and modification at various places in the Linux network stack. Hone registers callbacks with the INPUT and OUTPUT chains of the filter table, providing notification of packets destined for and leaving the local system. Packets are correlated with a specific connection based on the socket associated with the given packet. Outgoing packets already contain a pointer to the originating socket, whereas incoming packets require protocol-specific lookup functions from the transport layer to determine the destination socket. Packet and socket structures do not contain any information linking them to a particular process. The owning process is determined by taking advantage of the fact that sockets are created in the context of the owning process. By hooking the socket creation routines, the creating process is reported along with a socket identifier for later lookup. Unfortunately, a problem arises when one performs a fork() or other socket-duplicating operation, as it is currently not possible to tell which process actually receives a given packet under these conditions. We are currently investigating a solution to this problem. The Linux correlator is comprised of multiple loadable kernel modules: process_notify, socket_notify, packet_notify, hone_notify and honeevent. The process_notify, socket_notify and packet_notify modules provide notification routines to notify registrants of process, socket and packet events. The hone_notify module registers for to receive all three event types and provides hone event notifications to the honeevent module. The honeevent module provides a character special device to provide hone events to userspace in plain text or PCAP-NG format. Process events are collected by hooking copy_process(), do_execve(), compat_do_execve() and do_exit() kernel routines using kprobes. This approach has drawbacks, but does allows the Hone driver to be used without modifying the kernel. PREREQUISITES ============= Building the Hone sensor kernel modules requires a few prerequisites: 1. Build tools, such as the gcc tool chain and GNU make 2. Kernel source header files for the target distribution 3. Kernel configuration (.config, typically included with the header files) 4. System map file from kernel build 5. Kernel with kprobes support (CONFIG_KPROBES) (would like to eliminate this requirement) If you can't build a Linux kernel, then you likely don't have the tools to build a kernel module. Be sure to install the recommended tools for your distribution. For instance, on Debian, install the build-essential package. By default, the build process will look for the kernel header files and the configuration file in /lib/modules/$(KVER)/build (default for KSRC), where KVER defaults to `uname -r`. If you wish to build for a different kernel version installed on the system, set KVER to the appropriate version number when running make. If the kernel headers live elsewhere, you can tell make about it via the KSRC variable. Some of the Hone modules use symbols that are not exported. Therefore, their address must be mapped statically using the system map file. Make will search for the appropriate map file in the following order: 1. $(PWD)/System.map 2. /boot/System.map-$(KERNELRELEASE) 3. /lib/modules/$(KERNELRELEASE)/build/System.map 4. /proc/kallsyms (only if the running and target versions are the same) 5. $(sys)/System.map You can override this search by setting the SYSMAP variable to the path of the appropriate system map when calling make. BUILDING ======== Assuming the above prerequisites are met, building the modules is a trivial process. Just change to the Hone source directory and execute make. Here are a few examples: Basic build: $ make OR $ make all If for some reason the prereq_check target fails, you can try skipping it: $ make modules Building for a different version than the running kernel: $ make KVER=2.6.38 Building for a kernel that is not installed $ make KSRC=/usr/src/linux-2.6.38 SYSMAP=/usr/src/linux-2.6.38/System.map INSTALLING ========== To install the modules, just change to the Hone source directory and execute `make install`. If you set KVER, KSRC, SYSMAP, ARCH, or any other make variables, you should set them for install as well. If the modules are built outside the kernel tree, it may be necessary to run depmod manually. If, after installing the modules using `make install`, you cannot use modprobe to insert the modules, try running depmod. Running depmod for the currently running kernel # /sbin/depmod -a Check out the depmod(8) man page if you wish to do something fancier, like depmod against a specific System.map file. BUILDING PACKAGES ================= There are three package types included in the Hone Makefile: tarball, deb and rpm. You can build all three by using the packages target or build them individually using the tarball, deb, or rpm targets. The deb and rpm packages depend on dkms. USAGE ===== After installing the Hone sensor, the kernel module can be loaded using modprobe: # modprobe honeevent If for some reason modprobe fails (i.e. depmod failed or was not run), the modules can be loaded manually using insmod: # insmod process_notify.ko # insmod socket_notify.ko # insmod packet_notify.ko # insmod hone_notify.ko # insmod honeevent.ko This will load a character special device driver. udev should automatically create two devices in /dev: /dev/hone and /dev/honet. /dev/hone provides PCAP-NG output while /dev/honet provides plain text output, one record per line. If udev is not being used, the devices can be created using mknod. The files can be created on any filesystem without the "nodev" mount option, however, they are normally made in /dev. Use the following command replacing $MAJOR and $MINOR with the device major and minor numbers (see the next paragraph). # mknod /dev/hone c $MAJOR $MINOR The major number is dynamically assigned when the module is loaded. It can be determined by reading /sys/module/honeevent/parameters/major or by locating the insertion message in syslog. For PCAP-NG output, use minor number 0. For plain text, use minor number 1. There are several module parameters that can be passed to the module at load time: devname - optional base name to give the device in sysfs; default is hone major - optional major number to assign the device; default is 0 meaning assigned automatically by the kernel hostid - optional GUID to include in the section header block used to identify the system on which the capture occurred. comment - optional comment to include in the section header block; spaces may be encoded as \040 or \x20 snaplen - set the maximum packet capture length; default is 0 meaning full packet capture pageorder - set the pageorder value used when allocating pages for the ring buffer where buffer size = PAGE_SIZE * pow(2, pageorder); default is 1 (2 pages) for 32-bit systems and 2 (4 pages) on 64-bit systems