diff --git a/docs/changes.md b/docs/changes.md new file mode 100644 index 0000000..4c2bbc3 --- /dev/null +++ b/docs/changes.md @@ -0,0 +1,14 @@ +# Change Log + +## 1.4.0: +* Refactor FakeNet-NG to unify Windows and Linux packet handling +* Remove Proxy Listener UDP stream abstraction to prevent issue where + subsequent clients do not receive response packets because the proxy listener + continues to send them to the old (expired) ephemeral port for the previous + client +* Stop flag command-line support for rudimentary IPC-based start/stop + automation +* Integration test script for MultiHost and SingleHost mode +* Fixed Errno 98 (`TIME_WAIT`) issue with `RawTcpListener` +* WinDivert `GetLastError` exception work-around for [WinDivert issue + #32](https://github.com/ffalcinelli/pydivert/issues/32) diff --git a/docs/contributors.md b/docs/contributors.md new file mode 100644 index 0000000..ed57688 --- /dev/null +++ b/docs/contributors.md @@ -0,0 +1,35 @@ +# Contributors + +This document credits those who conceptualized and/or implemented features for +FakeNet-NG. + +## Legacy + +FakeNet-NG is based on the original +[FakeNet](https://practicalmalwareanalysis.com/fakenet/) tool developed by +Andrew Honig and Michael Sikorski, which is still the tool of choice for +malware analysis on Windows XP. + +## Windows + +Peter Kacherginsky [implemented +FakeNet-NG](https://www.fireeye.com/blog/threat-research/2016/08/fakenet-ng_next_gen.html) +targeting modern versions of Windows. + +## Linux and Core + +Michael Bailey [implemented FakeNet-NG on +Linux](https://www.fireeye.com/blog/threat-research/2017/07/linux-support-for-fakenet-ng.html), +and later refactored FakeNet-NG to use this as the unified packet processing +logic for both Windows and Linux. + +## Content-Based Protocol Detection + +The original FakeNet-NG was able to automatically handle SSL; meanwhile, Joshua +Homan developed the original concept of using a protocol "taste" callback to +sample traffic and direct clients to the appropriate server ports. Matthew +Haigh, Michael Bailey, and Peter Kacherginsky conceptualized the Proxy Listener +and Hidden Listener mechanisms for introducing both of these content-based +protocol detection features to FakeNet-NG. Matthew Haigh then [implemented +Content-Based Protocol +Detection](https://www.fireeye.com/blog/threat-research/2017/10/fakenet-content-based-protocol-detection.html). diff --git a/docs/internals.md b/docs/internals.md index 746655d..c44a340 100644 --- a/docs/internals.md +++ b/docs/internals.md @@ -1,15 +1,9 @@ # FakeNet-NG Internals -## FakeNet-NG Linux Diverter Internals +This documentation was originally written for the Linux implementation, and +where specifics are called for, it currently references Linux. -The FakeNet-NG Diverter implementation for Linux uses -[netfilter](http://netfilter.org/) to examine, log, and redirect packets. -Netfilter is available at the kernel level through loadable kernel modules -(LKMs) and in userspace through the -[libnetfilter_queue](http://netfilter.org/projects/libnetfilter_queue/index.html) -library. Since FakeNet-NG is Python-based, it uses libnetfilter_queue via the -[python-netfilterqueue](https://github.com/kti/python-netfilterqueue/) wrapper -which is written in Cython. +## FakeNet-NG Diverter Internals For purposes of this documentation, some rigorously-defined terms will be repurposed or replaced: @@ -21,6 +15,20 @@ repurposed or replaced: we will use the more general term _conversation_ to represent the concept of a pair of endpoints (again, using that term loosely) that are communicating. +FakeNet-NG can also operate in two modes on Linux: +* `SingleHost` - simulate the Internet for the local machine +* `MultiHost` - simulate the Internet by acting as the gateway for another + machine + +Each implementation of FakeNet-NG ultimately relies on a driver or kernel +module that supports network hooking and a library that makes this accessible +from user space. The Windows Diverter uses +[PyDivert](https://github.com/ffalcinelli/pydivert) to control the +[WinDivert](https://reqrypt.org/windivert.html) driver. The Linux Diverter uses +[python-netfilterqueue](https://github.com/kti/python-netfilterqueue) to access +[libnetfilter_queue](https://netfilter.org/projects/libnetfilter_queue/) and in +turn [NetFilter](https://netfilter.org/). + ### Traffic Flow Condition Evaluation The simplest case for the Linux implementation of the FakeNet-NG Diverter is @@ -30,10 +38,9 @@ we use `iptables` to implement a `REDIRECT` rule in the `PREROUTING` chain. In this use case, FakeNet-NG implements only dynamic port forwarding (DPF) using python-netfilterqueue. -The most complicated case is `SingleHost` mode (experimental), in which both -DPF and NAT must be controlled by FakeNet-NG to permit process blacklisting -and other configuration settings. In this case, FakeNet-NG uses -python-netfilterqueue to evaluate four conditions: +The more complicated case is `SingleHost` mode, in which both DPF and NAT must +be controlled by FakeNet-NG to permit process blacklisting and other +configuration settings. In this case, FakeNet-NG evaluates four conditions: 1. When a packet is produced, is it destined for a foreign IP address? (if so, fix up its destination address to be a local address) 2. When a packet is about to be consumed, is it destined for an unbound port? @@ -45,8 +52,8 @@ python-netfilterqueue to evaluate four conditions: that has been NATted? (if so, fix up its source IP) Given two processes `P1` and `P2`, here is a diagram of communication and -condition evaluation using the `INPUT` and `OUTPUT` chains provided by -Netfilter: +condition evaluation specific to Linux, using the `INPUT` and `OUTPUT` chains +provided by Netfilter: ``` (1) (2) @@ -288,7 +295,6 @@ Or, in Python: (not sport_bound and not dport_bound)) ``` - ### Future #### NetworkMode Auto for Linux @@ -312,11 +318,12 @@ troubleshooting problems transferring the 24KB file `FakeNet.gif` over FTP. This is fine for `MultiHost` mode because external interfaces (e.g. `eth0`) frequently have a maximum transmittal unit (MTU) of 1500. However, for loopback -communications where the MTU is 65536, this causes errors. It is possible to -fix these errors by changing the buffer size to 65616, however this may be -overridden by future installations of python-netfilterqueue either via the -package management system specific to the Linux distribution, Pip, etc. +communications where the MTU may be something like 65536, this causes errors. +It is possible to fix these errors by changing the buffer size to 65616 +(accounting for 80 bytes of overhead), however this could be overridden by +future installations of python-netfilterqueue either via the package management +system specific to the Linux distribution, Pip, etc. A work-around for this issue is to send all NAT packets through an externally -facing IP address instead of 127.0.0.1 to avoid exposing ourselves to -`BufferSize < MTU` conditions such as in the transfer of large files. +facing IP address instead of 127.0.0.1 to avoid exposing traffic to `BufferSize +< MTU` conditions such as in the transfer of large files. diff --git a/fakenet/configs/default.ini b/fakenet/configs/default.ini index 606ab1f..33b1a11 100644 --- a/fakenet/configs/default.ini +++ b/fakenet/configs/default.ini @@ -26,9 +26,26 @@ DivertTraffic: Yes # NetworkMode: MultiHost NetworkMode: Auto -# DebugLevel (Linux only as of this writing): specify fine-grained debug print -# flags to enable. Enabling all logging when verbose mode is selected results -# in an unacceptable overhead cost, hence this setting. +# DebugLevel: specify fine-grained debug print flags to enable. Enabling all +# logging when verbose mode is selected results in overwhelming output, hence +# this setting. Valid values (comma-separated) are: +# +# GENPKT Generic packet information +# GENPKTV Packet analysis, displays IP, TCP, UDP fields, very wide output +# CB Diverter packet handler callback start/finish logging +# NONLOC Nonlocal packet verbose logging +# DPF Dynamic port forwarding decisions +# DPFV Dynamic port forwarding table activity +# IPNAT NAT decisions +# MANGLE Packet mangling (modification) activity +# PCAP PCAP writes of original and mangled packets +# IGN Cases where packets are forwarded as is +# FTP FTP-specific logic +# IGN-FTP Cases where packets are forwarded as is due to FTP Active Mode +# MISC Miscellaneous +# NFQUEUE NetfilterQueue activity (Linux only) +# PROCFS Procfs read/write activity (Linux only) +# IPTABLES iptables firewall rule activity (Linux only) DebugLevel: Off # MultiHost mode only: Specify what interfaces the Linux Diverter should create diff --git a/fakenet/diverters/debuglevels.py b/fakenet/diverters/debuglevels.py new file mode 100644 index 0000000..535dea9 --- /dev/null +++ b/fakenet/diverters/debuglevels.py @@ -0,0 +1,42 @@ +# Debug print levels for fine-grained debug trace output control +DNFQUEUE = (1 << 0) # netfilterqueue +DGENPKT = (1 << 1) # Generic packet handling +DGENPKTV = (1 << 2) # Generic packet handling with TCP analysis +DCB = (1 << 3) # Packet handlign callbacks +DPROCFS = (1 << 4) # procfs +DIPTBLS = (1 << 5) # iptables +DNONLOC = (1 << 6) # Nonlocal-destined datagrams +DDPF = (1 << 7) # DPF (Dynamic Port Forwarding) +DDPFV = (1 << 8) # DPF (Dynamic Port Forwarding) Verbose +DIPNAT = (1 << 9) # IP redirection for nonlocal-destined datagrams +DMANGLE = (1 << 10) # Packet mangling +DPCAP = (1 << 11) # Pcap write logic +DIGN = (1 << 12) # Packet redirect ignore conditions +DFTP = (1 << 13) # FTP checks +DMISC = (1 << 27) # Miscellaneous + +DCOMP = 0x0fffffff # Component mask +DFLAG = 0xf0000000 # Flag mask +DEVERY = 0x0fffffff # Log everything, low verbosity +DEVERY2 = 0x8fffffff # Log everything, complete verbosity + +DLABELS = { + DNFQUEUE: 'NFQUEUE', + DGENPKT: 'GENPKT', + DGENPKTV: 'GENPKTV', + DCB: 'CB', + DPROCFS: 'PROCFS', + DIPTBLS: 'IPTABLES', + DNONLOC: 'NONLOC', + DDPF: 'DPF', + DDPFV: 'DPFV', + DIPNAT: 'IPNAT', + DMANGLE: 'MANGLE', + DPCAP: 'PCAP', + DIGN: 'IGN', + DFTP: 'FTP', + DIGN | DFTP: 'IGN-FTP', + DMISC: 'MISC', +} + +DLABELS_INV = {v.upper(): k for k, v in DLABELS.iteritems()} diff --git a/fakenet/diverters/diverterbase.py b/fakenet/diverters/diverterbase.py index 73461c4..c2919ca 100644 --- a/fakenet/diverters/diverterbase.py +++ b/fakenet/diverters/diverterbase.py @@ -1,28 +1,501 @@ import os +import abc import sys import time import dpkt import signal import socket import logging -import fnconfig import threading import subprocess +from . import fnpacket +from . import fnconfig +from debuglevels import * from collections import namedtuple from collections import OrderedDict -class DiverterBase(fnconfig.Config): +class DivertParms(object): + """Class to abstract all criteria possible out of the Windows and Linux + diverters. + + These criteria largely derive from both the diverter state and the packet + contents. This class is sometimes passed around alongside the packet to + provide context wtihout loading down the fnpacket.PacketCtx with extraneous + concepts. + + Many of these critera are only applicable if the transport layer has + been parsed and validated. + """ + + def __init__(self, diverter, pkt): + self.diverter = diverter + self.pkt = pkt + + @property + def is_loopback0(self): + return (self.pkt.src_ip0 == self.pkt.dst_ip0 == + self.diverter.loopback_ip) + + @property + def is_loopback(self): + return self.pkt.src_ip == self.pkt.dst_ip == self.diverter.loopback_ip + + @property + def dport_hidden_listener(self): + """Does the destination port for the packet correspond to a hidden + listener (i.e. should the packet be redirected to the proxy)? + + Returns: + True if dport corresponds to hidden listener, else False + """ + return self.diverter.listener_ports.isHidden(self.pkt.proto, + self.pkt.dport) + + @property + def src_local(self): + """Is the source address one of the local IPs of this system? + + Returns: + True if local source IP, else False + """ + return self.pkt.src_ip in self.diverters.ip_addrs[self.pkt.ipver] + + @property + def sport_bound(self): + """Is the source port bound by a FakeNet-NG listener? + + Returns: + True if sport is bound by FakeNet-NG, else False + """ + return self.diverter.listener_ports.isListener(self.pkt.proto, + self.pkt.sport) + + @property + def dport_bound(self): + """Is the destination port bound by a FakeNet-NG listener? + + Returns: + True if dport is bound by FakeNet-NG, else False + """ + return self.diverter.listener_ports.isListener(self.pkt.proto, + self.pkt.dport) + + @property + def first_packet_new_session(self): + """Is this the first datagram from this conversation? + + Returns: + True if this pair of endpoints hasn't conversed before, else False + """ + return not (self.diverter.sessions.get(self.pkt.sport) == + (self.pkt.dst_ip, self.pkt.dport)) + + +class DiverterPerOSDelegate(object): + """Delegate class for OS-specific methods that FakeNet-NG implementors must + override. + + TODO: The following methods may need to be combined to ensure that there is + at least a single Ethernet interface with all valid settings (instead of, + say, several interfaces, each with only one of the components that are + needed to make the system work). + check_active_ethernet_adapters + check_ipaddresses + check_gateways (currently only a warning) + check_dns_servers (currently only a warning) + """ + __metaclass__ = abc.ABCMeta + + @abc.abstractmethod + def check_active_ethernet_adapters(self): + """Check that there is at least one Ethernet interface. + + Returns: + True if there is at least one interface, else False + """ + pass + + @abc.abstractmethod + def check_ipaddresses(self): + """Check that there is at least one non-null IP address associated with + at least one interface. + + Returns: + True if at least one IP address, else False + """ + pass + + @abc.abstractmethod + def check_gateways(self): + """Check that at least one interface has a non-NULL gateway set. + + Returns: + True if at least one gateway, else False + """ + pass + + @abc.abstractmethod + def fix_gateway(self): + """Check if there is a gateway configured on any of the Ethernet + interfaces. If not, then locate a configured IP address and set a gw + automatically. This is necessary for VMware's Host-Only DHCP server + which leaves the default gateway empty. + + Returns: + True if successful, else False + """ + pass + + @abc.abstractmethod + def check_dns_servers(self): + """Check that a DNS server is set. + + Returns: + True if a DNS server is set, else False + """ + pass + + @abc.abstractmethod + def fix_dns(self): + """Check if there is a DNS server on any of the Ethernet interfaces. If + not, then locate configured IP address and set a DNS server + automatically. + + Returns: + True if successful, else False + """ + pass + + @abc.abstractmethod + def get_pid_comm(self, pkt): + """Get the PID and process name by IP/port info. + + NOTE: the term "comm" is short for "command" and comes from the Linux + term for process name within task_struct and displayed in ps. + + Args: + pkt: A fnpacket.PacketCtx or derived object + + Returns: + Tuple of length 2, containing: + (pid, comm) + If the pid or comm cannot be discerned, the corresponding member of + the tuple will be None. + """ + pass + + @abc.abstractmethod + def getNewDestinationIp(self, src_ip): + """Get IP to redirect to after a redirection decision has been made. + + This is OS-specific due to varying empirical results on Windows and + Linux, and may be subject to change. + + On Windows, and possibly other operating systems, simply redirecting + external packets to the loopback address will cause the packets not to + be routable, so it is necessary to choose an external interface IP in + some cases. + Contrarywise, the Linux FTP tests will fail if all redirections are not + routed to 127.0.0.1. - def init_base(self, diverter_config, listeners_config, ip_addrs, - logging_level=logging.INFO): + Args: + src_ip: A str of the source IP address represented in ASCII + + Returns: + A str of the destination IP address represented in ASCII that the + packet should be redirected to. + """ + pass + + +class ListenerAlreadyBoundThere(Exception): + pass + + +class ListenerBlackWhiteList(Exception): + pass + + +class ListenerMeta(object): + """Info about each listener. + + Makes hidden listeners explicit. Organizes process and host black/white + lists and ExecuteCmd format strings. + + Mutators are here for building listener metadata before adding it to the + group. Accessors are in ListenerPorts for querying the collection for + listeners and their attributes. + """ + def __init__(self, proto, port, hidden=False): + self.proto = proto + self.port = port + self.hidden = hidden + self.proc_bl = None + self.proc_wl = None + self.host_bl = None + self.host_wl = None + self.cmd_template = None + + def _splitBlackWhiteList(self, configtext): + """Return list from comma-separated config line.""" + return [item.strip() for item in configtext.split(',')] + + def _validateBlackWhite(self): + """Validate that only a black or a white list of either type (host or + process) is configured. + + Side-effect: + Raises ListenerBlackWhiteList if invalid + """ + msg = None + fmt = 'Cannot specify both %s blacklist and whitelist for port %d' + if self.proc_wl and self.proc_bl: + msg = fmt % ('process', self.port) + self.proc_wl = self.proc_bl = None + elif self.host_wl and self.host_bl: + msg = fmt % ('host', self.port) + self.host_wl = self.host_bl = None + if msg: + raise ListenerBlackWhiteList(msg) + + def setProcessWhitelist(self, configtext): + self.proc_wl = self._splitBlackWhiteList(configtext) + self._validateBlackWhite() + + def setProcessBlacklist(self, configtext): + self.proc_bl = self._splitBlackWhiteList(configtext) + self._validateBlackWhite() + + def setHostWhitelist(self, configtext): + self.host_wl = self._splitBlackWhiteList(configtext) + self._validateBlackWhite() + + def setHostBlacklist(self, configtext): + self.host_bl = self._splitBlackWhiteList(configtext) + self._validateBlackWhite() + + def setExecuteCmd(self, configtext): + self.cmd_template = configtext + + +class ListenerPorts(object): + """Collection of listeners with convenience accessors. + + Previously, FakeNet-NG had several parallel dictionaries associated with + listener settings and lots of code like this: + 1.) Does this dictionary have a 'TCP' key? + 2.) Oh, yeah? Well, is this port in the dictionary under 'TCP'? + 3.) Ah, great! Now I can ask my question. Is there an ExecuteCmd for + this port? + + At a cost of having to add a bit of code and a few more comment lines, This + class takes care of the checks and turns queries like this into one-liners + like this one: + cmd = obj.getExecuteCmd('TCP', 80) # Returns None if not applicable + """ + + def __init__(self): + """Initialize dictionary of dictionaries: + protocol name => dict + portno => ListenerMeta + """ + self.protos = {} + + def addListener(self, listener): + """Add a ListenerMeta under the corresponding protocol and port.""" + proto = listener.proto + port = listener.port + + if not proto in self.protos: + self.protos[proto] = {} + + if port in self.protos[proto]: + raise ListenerAlreadyBoundThere( + 'Listener already bound to %s port %s' % (proto, port)) + + self.protos[proto][port] = listener + + def getListenerMeta(self, proto, port): + if proto in self.protos: + return self.protos[proto].get(port) + + def isListener(self, proto, port): + """Is this port associated with a listener?""" + return bool(self.getListenerMeta(proto, port)) + + def isHidden(self, proto, port): + """Is this port associated with a listener that is hidden?""" + listener = self.getListenerMeta(proto, port) + return listener.hidden if listener else False + + def getPortList(self, proto): + if proto in self.protos: + return self.protos[proto].keys() + return [] + + def intersectsWithPorts(self, proto, ports): + """Check if ports intersect with bound listener ports. + + Convenience method for checking whether source or destination port are + bound to a FakeNet-NG listener. + """ + return set(ports).intersection(self.getPortList(proto)) + + def getExecuteCmd(self, proto, port): + """Get the ExecuteCmd format string specified by the operator. + + Args: + proto: The protocol name + port: The port number + + Returns: + The format string if applicable + None, otherwise + """ + listener = self.getListenerMeta(proto, port) + if listener: + return listener.cmd_template + + def _isWhiteListMiss(self, thing, whitelist): + """Check if thing is NOT in whitelist. + + Args: + thing: thing to check whitelist for + whitelist: list of entries + + Returns: + True if thing is in whitelist + False otherwise, or if there is no whitelist + """ + if not whitelist: + return False + return not (thing in whitelist) + + def _isBlackListHit(self, thing, blacklist): + """Check if thing is in blacklist. + + Args: + thing: thing to check blacklist for + blacklist: list of entries + + Returns: + True if thing is in blacklist + False otherwise, or if there is no blacklist + """ + if not blacklist: + return False + return (thing in blacklist) + + def isProcessWhiteListMiss(self, proto, port, proc): + """Check if proc is OUTSIDE the process WHITElist for a port. + + Args: + proto: The protocol name + port: The port number + proc: The process name + + Returns: + False if no listener on this port + Return value of _isWhiteListMiss otherwise + """ + listener = self.getListenerMeta(proto, port) + if not listener: + return False + return self._isWhiteListMiss(proc, listener.proc_wl) + + def isProcessBlackListHit(self, proto, port, proc): + """Check if proc is IN the process BLACKlist for a port. + + Args: + proto: The protocol name + port: The port number + proc: The process name + + Returns: + False if no listener on this port + Return value of _isBlackListHit otherwise + """ + listener = self.getListenerMeta(proto, port) + if not listener: + return False + return self._isBlackListHit(proc, listener.proc_bl) + + def isHostWhiteListMiss(self, proto, port, host): + """Check if host is OUTSIDE the process WHITElist for a port. + + Args: + proto: The protocol name + port: The port number + host: The process name + + Returns: + False if no listener on this port + Return value of _isWhiteListMiss otherwise + """ + listener = self.getListenerMeta(proto, port) + if not listener: + return False + return self._isWhiteListMiss(host, listener.host_wl) + + def isHostBlackListHit(self, proto, port, host): + """Check if host is IN the process BLACKlist for a port. + + Args: + proto: The protocol name + port: The port number + host: The process name + + Returns: + False if no listener on this port + Return value of _isBlackListHit otherwise + """ + listener = self.getListenerMeta(proto, port) + if not listener: + return False + return self._isBlackListHit(host, listener.host_bl) + + +class DiverterBase(fnconfig.Config): + """The beating heart. + + You must implement the following methods to ride: + startCallback() + stopCallback() + """ + + def __init__(self, diverter_config, listeners_config, ip_addrs, + logging_level=logging.INFO): + """Initialize the DiverterBase. + + TODO: Replace the sys.exit() calls from this function with exceptions + or some other mechanism appropriate for allowing the user of this class + to programmatically detect and handle these cases in their own way. + This may entail moving configuration parsing to a method with a return + value, or modifying fakenet.py to handle Diverter exceptions. + + Args: + diverter_config: A dict of [Diverter] config section + listeners_config: A dict of listener configuration sections + ip_addrs: dictionary keyed by integers 4 and 6, with each element + being a list and each list member being a str that is an ASCII + representation of an IP address that is associated with a local + interface on this system. + logging_level: Optional integer logging level such as logging.DEBUG + + Returns: + None + """ # For fine-grained control of subclass debug output. Does not control # debug output from DiverterBase. To see DiverterBase debug output, # pass logging.DEBUG as the logging_level argument to init_base. self.pdebug_level = 0 self.pdebug_labels = dict() + # Override in Windows implementation + self.running_on_windows = False + self.pid = os.getpid() self.ip_addrs = ip_addrs @@ -49,38 +522,38 @@ def init_base(self, diverter_config, listeners_config, ip_addrs, # port tuples self.sessions = dict() - ####################################################################### - # Listener specific configuration - # NOTE: All of these definitions have protocol as the first key - # followed by a list or another nested dict with the actual - # definitions - - # Diverted ports - # TODO: a more meaningful name might be BOUND ports indicating ports - # that FakeNet-NG has bound to with a listener - self.diverted_ports = dict() - - # Listener Port Process filtering - # TODO: Allow PIDs - self.port_process_whitelist = dict() - self.port_process_blacklist = dict() - - # Listener Port Host filtering - # TODO: Allow domain name resolution - self.port_host_whitelist = dict() - self.port_host_blacklist = dict() - - # Execute command list - self.port_execute = dict() + # Manage logging of foreign-destined packets + self.nonlocal_ips_already_seen = [] + self.log_nonlocal_only_once = True + + # Port forwarding table, for looking up original unbound service ports + # when sending replies to foreign endpoints that have attempted to + # communicate with unbound ports. Allows fixing up source ports in + # response packets. Similar to the `sessions` member of the Windows + # Diverter implementation. + self.port_fwd_table = dict() + self.port_fwd_table_lock = threading.Lock() + + # Track conversations that will be ignored so that e.g. an RST response + # from a closed port does not erroneously trigger port forwarding and + # silence later replies to legitimate clients. + self.ignore_table = dict() + self.ignore_table_lock = threading.Lock() + + # IP forwarding table, for looking up original foreign destination IPs + # when sending replies to local endpoints that have attempted to + # communicate with other machines e.g. via hard-coded C2 IP addresses. + self.ip_fwd_table = dict() + self.ip_fwd_table_lock = threading.Lock() + + # Ports bound by FakeNet-NG listeners + self.listener_ports = ListenerPorts() # Parse listener configurations self.parse_listeners_config(listeners_config) ####################################################################### - # Diverter settings and filters - - # Intercept filter - self.filter = None + # Diverter settings # Default TCP/UDP listeners self.default_listener = dict() @@ -100,33 +573,124 @@ def init_base(self, diverter_config, listeners_config, ip_addrs, # Parse diverter config self.parse_diverter_config() + slists = ['DebugLevel', ] + self.reconfigure(portlists=[], stringlists=slists) + + dbg_lvl = 0 + if self.is_configured('DebugLevel'): + for label in self.getconfigval('DebugLevel'): + label = label.upper() + if label == 'OFF': + dbg_lvl = 0 + break + if not label in DLABELS_INV: + self.logger.warning('No such DebugLevel as %s' % (label)) + else: + dbg_lvl |= DLABELS_INV[label] + self.set_debug_level(dbg_lvl, DLABELS) + ####################################################################### # Network verification - Implemented in OS-specific mixin # Check active interfaces if not self.check_active_ethernet_adapters(): - self.logger.warning('WARNING: No active ethernet interfaces ' + + self.logger.warning('WARNING: No active ethernet interfaces ' 'detected!') self.logger.warning(' Please enable a network interface.') + sys.exit(1) + + # Check configured ip addresses + if not self.check_ipaddresses(): + self.logger.warning('ERROR: No interface had IP address ' + 'configured!') + self.logger.warning(' Please configure an IP address on a ' + 'network interface.') + sys.exit(1) # Check configured gateways - if not self.check_gateways(): + gw_ok = self.check_gateways() + if not gw_ok: self.logger.warning('WARNING: No gateways configured!') + if self.is_set('fixgateway'): + gw_ok = self.fix_gateway() + if not gw_ok: + self.logger.warning('Cannot fix gateway') + + if not gw_ok: self.logger.warning(' Please configure a default ' + 'gateway or route in order to intercept ' + 'external traffic.') + self.logger.warning(' Current interception abilities ' + + 'are limited to local traffic.') # Check configured DNS servers - if not self.check_dns_servers(): + dns_ok = self.check_dns_servers() + if not dns_ok: self.logger.warning('WARNING: No DNS servers configured!') - self.logger.warning(' Please configure a DNS server in ' + - 'order to allow network resolution.') + if self.is_set('fixdns'): + dns_ok = self.fix_dns() + if not dns_ok: + self.logger.warning('Cannot fix DNS') + + if not dns_ok: + self.logger.warning(' Please configure a DNS server ' + + 'in order to allow network resolution.') # OS-specific Diverters must initialize e.g. WinDivert, # libnetfilter_queue, pf/alf, etc. + def start(self): + """This method currently only serves the purpose of codifying what must + be implemented on a given OS to bring FakeNet-NG to that OS. + + Further refactoring should be done to unify network interface checks, + gateway and DNS configuration, etc. into this method while calling out + to the already-defined (and potentially some yet-to-be-defined) + abstract methods that handle the real OS-specific stuff. + """ + self.logger.info('Starting...') + return self.startCallback() + + def stop(self): + self.logger.info('Stopping...') + return self.stopCallback() + + @abc.abstractmethod + def startCallback(self): + """Initiate packet processing and return immediately. + + Generally, install hooks/filters and start one or more threads to + siphon packet events. + + Returns: + True if successful, else False + """ + pass + + @abc.abstractmethod + def stopCallback(self): + """Terminate packet processing. + + Generally set a flag to tell the thread to stop, join with the thread, + uninstall hooks, and change network settings back to normal. + + Returns: + True if successful, else False + """ + pass + def set_debug_level(self, lvl, labels={}): - """Enable debug output if necessary and set the debug output level.""" + """Enable debug output if necessary, set the debug output level, and + maintain a reference to the dictionary of labels to print when a given + logging level is encountered. + + Args: + lvl: An int mask of all debug logging levels + labels: A dict of int => str assigning names to each debug level + + Returns: + None + """ if lvl: self.logger.setLevel(logging.DEBUG) @@ -135,13 +699,27 @@ def set_debug_level(self, lvl, labels={}): self.pdebug_labels = labels def pdebug(self, lvl, s): - """Log only the debug trace messages that have been enabled.""" + """Log only the debug trace messages that have been enabled via + set_debug_level. + + Args: + lvl: An int indicating the debug level of this message + s: The mssage + + Returns: + None + """ if self.pdebug_level & lvl: label = self.pdebug_labels.get(lvl) prefix = '[' + label + '] ' if label else '[some component] ' self.logger.debug(prefix + str(s)) def check_privileged(self): + """UNIXy and Windows-oriented check for superuser privileges. + + Returns: + True if superuser, else False + """ try: privileged = (os.getuid() == 0) except AttributeError: @@ -150,6 +728,19 @@ def check_privileged(self): return privileged def parse_listeners_config(self, listeners_config): + """Parse listener config sections. + + TODO: Replace the sys.exit() calls from this function with exceptions + or some other mechanism appropriate for allowing the user of this class + to programmatically detect and handle these cases in their own way. + This may entail modifying fakenet.py. + + Args: + listeners_config: A dict of listener configuration sections + + Returns: + None + """ ####################################################################### # Populate diverter ports and process filters from the configuration @@ -159,7 +750,8 @@ def parse_listeners_config(self, listeners_config): port = int(listener_config['port']) - hidden = listener_config.get('hidden', 'false') == 'True' + hidden = (listener_config.get('hidden', 'false').lower() == + 'true') if not 'protocol' in listener_config: self.logger.error('ERROR: Protocol not defined for ' + @@ -173,18 +765,12 @@ def parse_listeners_config(self, listeners_config): 'listener %s', protocol, listener_name) sys.exit(1) - # diverted_ports[protocol][port] is True if the listener is - # configured as 'Hidden', which means it will not receive - # packets unless the ProxyListener determines that the protocol - # matches the listener - if not protocol in self.diverted_ports: - self.diverted_ports[protocol] = dict() - - self.diverted_ports[protocol][port] = hidden + listener = ListenerMeta(protocol, port, hidden) ############################################################### # Process filtering configuration - if 'processwhitelist' in listener_config and 'processblacklist' in listener_config: + if ('processwhitelist' in listener_config and + 'processblacklist' in listener_config): self.logger.error('ERROR: Listener can\'t have both ' + 'process whitelist and blacklist.') sys.exit(1) @@ -193,70 +779,55 @@ def parse_listeners_config(self, listeners_config): self.logger.debug('Process whitelist:') - if not protocol in self.port_process_whitelist: - self.port_process_whitelist[protocol] = dict() - - self.port_process_whitelist[protocol][port] = [ - process.strip() for process in - listener_config['processwhitelist'].split(',')] + whitelist = listener_config['processwhitelist'] + listener.setProcessWhitelist(whitelist) - for port in self.port_process_whitelist[protocol]: - self.logger.debug(' Port: %d (%s) Processes: %s', - port, protocol, ', '.join( - self.port_process_whitelist[protocol][port])) + # for port in self.port_process_whitelist[protocol]: + # self.logger.debug(' Port: %d (%s) Processes: %s', + # port, protocol, ', '.join( + # self.port_process_whitelist[protocol][port])) elif 'processblacklist' in listener_config: self.logger.debug('Process blacklist:') - if not protocol in self.port_process_blacklist: - self.port_process_blacklist[protocol] = dict() + blacklist = listener_config['processblacklist'] + listener.setProcessBlacklist(blacklist) - self.port_process_blacklist[protocol][port] = [ - process.strip() for process in - listener_config['processblacklist'].split(',')] - - for port in self.port_process_blacklist[protocol]: - self.logger.debug(' Port: %d (%s) Processes: %s', - port, protocol, ', '.join( - self.port_process_blacklist[protocol][port])) + # for port in self.port_process_blacklist[protocol]: + # self.logger.debug(' Port: %d (%s) Processes: %s', + # port, protocol, ', '.join( + # self.port_process_blacklist[protocol][port])) ############################################################### # Host filtering configuration - if 'hostwhitelist' in listener_config and 'hostblacklist' in listener_config: + if ('hostwhitelist' in listener_config and + 'hostblacklist' in listener_config): self.logger.error('ERROR: Listener can\'t have both ' + 'host whitelist and blacklist.') sys.exit(1) elif 'hostwhitelist' in listener_config: - self.logger.debug('Host whitelist:') + host_whitelist = listener_config['hostwhitelist'] + listener.setHostWhitelist(host_whitelist) - if not protocol in self.port_host_whitelist: - self.port_host_whitelist[protocol] = dict() - - self.port_host_whitelist[protocol][port] = [host.strip() - for host in - listener_config['hostwhitelist'].split(',')] - - for port in self.port_host_whitelist[protocol]: - self.logger.debug(' Port: %d (%s) Hosts: %s', port, - protocol, ', '.join( - self.port_host_whitelist[protocol][port])) + # for port in self.port_host_whitelist[protocol]: + # self.logger.debug(' Port: %d (%s) Hosts: %s', port, + # protocol, ', '.join( + # self.port_host_whitelist[protocol][port])) elif 'hostblacklist' in listener_config: self.logger.debug('Host blacklist:') + host_blacklist = listener_config['hostblacklist'] + listener.setHostBlacklist(host_blacklist) - if not protocol in self.port_host_blacklist: - self.port_host_blacklist[protocol] = dict() + # for port in self.port_host_blacklist[protocol]: + # self.logger.debug(' Port: %d (%s) Hosts: %s', port, + # protocol, ', '.join( + # self.port_host_blacklist[protocol][port])) - self.port_host_blacklist[protocol][port] = [host.strip() - for host in - listener_config['hostblacklist'].split(',')] - - for port in self.port_host_blacklist[protocol]: - self.logger.debug(' Port: %d (%s) Hosts: %s', port, - protocol, ', '.join( - self.port_host_blacklist[protocol][port])) + # Listener metadata is now configured, add it to the dictionary + self.listener_ports.addListener(listener) ############################################################### # Execute command configuration @@ -265,7 +836,8 @@ def parse_listeners_config(self, listeners_config): # Would prefer not to get into the middle of a debug # session and learn that a typo has ruined the day, so we - # test beforehand by + # test beforehand to make sure all the user-specified + # insertion strings are valid. test = self._build_cmd(template, 0, 'test', '1.2.3.4', 12345, '4.3.2.1', port) if not test: @@ -274,26 +846,64 @@ def parse_listeners_config(self, listeners_config): 'listener %s') % (listener_name)) sys.exit(1) - if not protocol in self.port_execute: - self.port_execute[protocol] = dict() + listener.setExecuteCmd(template) - self.port_execute[protocol][port] = \ - listener_config['executecmd'].strip() self.logger.debug('Port %d (%s) ExecuteCmd: %s', port, protocol, - self.port_execute[protocol][port]) + template) + + def build_cmd(self, pkt, pid, comm): + """Retrieve the ExecuteCmd directive if applicable and build the + command to execute. + + Args: + pkt: An fnpacket.PacketCtx or derived object + pid: Process ID associated with the packet + comm: Process name (command) that sent the packet + + Returns: + A str that is the resultant command to execute + """ + cmd = None + + template = self.listener_ports.getExecuteCmd(pkt.proto, pkt.dport) + if template: + cmd = self._build_cmd(template, pid, comm, pkt.src_ip, pkt.sport, + pkt.dst_ip, pkt.dport) + + return cmd def _build_cmd(self, tmpl, pid, comm, src_ip, sport, dst_ip, dport): + """Build a command based on the template specified in an ExecuteCmd + config directive, applying the parameters as needed. + + Accepts individual arguments instead of an fnpacket.PacketCtx so that + the Diverter can test any ExecuteCmd directives at configuration time + without having to synthesize a fnpacket.PacketCtx or construct a + NamedTuple to satisfy the requirement for such an argument. + + Args: + tmpl: A str containing the body of the ExecuteCmd config directive + pid: Process ID associated with the packet + comm: Process name (command) that sent the packet + src_ip: The source IP address that originated the packet + sport: The source port that originated the packet + dst_ip: The destination IP that the packet was directed at + dport: The destination port that the packet was directed at + + Returns: + A str that is the resultant command to execute + """ cmd = None try: cmd = tmpl.format( - pid = str(pid), - procname = str(comm), - src_addr = str(src_ip), - src_port = str(sport), - dst_addr = str(dst_ip), - dst_port = str(dport)) + pid=str(pid), + procname=str(comm), + src_addr=str(src_ip), + src_port=str(sport), + dst_addr=str(dst_ip), + dst_port=str(dport)) except KeyError as e: self.logger.error(('Failed to build ExecuteCmd for port %d due ' + 'to erroneous format key: %s') % @@ -301,55 +911,79 @@ def _build_cmd(self, tmpl, pid, comm, src_ip, sport, dst_ip, dport): return cmd - ########################################################################### - # Execute process and detach - def execute_detached(self, execute_cmd, winders=False): - """Supposedly OS-agnostic asynchronous subprocess creation. + def execute_detached(self, execute_cmd): + """OS-agnostic asynchronous subprocess creation. + + Executes the process with the appropriate subprocess.Popen parameters + for UNIXy or Windows platforms to isolate the process from FakeNet-NG + to prevent it from being interrupted by termination of FakeNet-NG, + Ctrl-C, etc. - Written in anticipation of re-factoring diverters into a common class - parentage. + Args: + execute_cmd: A str that is the command to execute - Not tested on Windows. Override or fix this if it does not work, for - instance to use the Popen creationflags argument or omit the close_fds - argument on Windows. + Side-effects: + Creates the specified process. + + Returns: + Success => an int that is the pid of the new process + Failure => None """ DETACHED_PROCESS = 0x00000008 - cflags = DETACHED_PROCESS if winders else 0 - cfds = False if winders else True - shl = False if winders else True + cflags = DETACHED_PROCESS if self.running_on_windows else 0 + cfds = False if self.running_on_windows else True + shl = False if self.running_on_windows else True def ign_sigint(): # Prevent KeyboardInterrupt in FakeNet-NG's console from # terminating child processes signal.signal(signal.SIGINT, signal.SIG_IGN) - # import pdb - # pdb.set_trace() + preexec = None if self.running_on_windows else ign_sigint + try: pid = subprocess.Popen(execute_cmd, creationflags=cflags, shell=shl, - close_fds = cfds, - preexec_fn = ign_sigint).pid - except Exception, e: - self.logger.error('Error: Failed to execute command: %s', execute_cmd) + close_fds=cfds, + preexec_fn=preexec).pid + except Exception as e: + self.logger.error('Exception of type %s' % (str(type(e)))) + self.logger.error('Error: Failed to execute command: %s', + execute_cmd) self.logger.error(' %s', e) else: return pid - def build_cmd(self, proto_name, pid, comm, src_ip, sport, dst_ip, dport): - cmd = None + def parse_diverter_config(self): + """Parse [Diverter] config section. - if ((proto_name in self.port_execute) and - (dport in self.port_execute[proto_name]) - ): - template = self.port_execute[proto_name][dport] - cmd = self._build_cmd(template, pid, comm, src_ip, sport, dst_ip, - dport) + Args: N/A - return cmd + Side-effects: + Diverter members (whitelists, pcap, etc.) initialized. - def parse_diverter_config(self): - if self.getconfigval('processwhitelist') and self.getconfigval('processblacklist'): + Returns: + None + """ + # SingleHost vs MultiHost mode + self.network_mode = 'SingleHost' # Default + self.single_host_mode = True + if self.is_configured('networkmode'): + self.network_mode = self.getconfigval('networkmode') + available_modes = ['singlehost', 'multihost'] + + # Constrain argument values + if self.network_mode.lower() not in available_modes: + self.logger.error('NetworkMode must be one of %s' % + (available_modes)) + sys.exit(1) + + # Adjust previously assumed mode if operator specifies MultiHost + if self.network_mode.lower() == 'multihost': + self.single_host_mode = False + + if (self.getconfigval('processwhitelist') and + self.getconfigval('processblacklist')): self.logger.error('ERROR: Diverter can\'t have both process ' + 'whitelist and blacklist.') sys.exit(1) @@ -360,7 +994,7 @@ def parse_diverter_config(self): time.strftime('%Y%m%d_%H%M%S')) self.logger.info('Capturing traffic to %s', self.pcap_filename) self.pcap = dpkt.pcap.Writer(open(self.pcap_filename, 'wb'), - linktype=dpkt.pcap.DLT_RAW) + linktype=dpkt.pcap.DLT_RAW) self.pcap_lock = threading.Lock() # Do not redirect blacklisted processes @@ -379,11 +1013,12 @@ def parse_diverter_config(self): # Do not redirect blacklisted hosts if self.is_configured('hostblacklist'): + self.blacklist_hosts = self.getconfigval('hostblacklist') self.logger.debug('Blacklisted hosts: %s', ', '.join( [str(p) for p in self.getconfigval('hostblacklist')])) # Redirect all traffic - self.default_listener = dict() + self.default_listener = {'TCP': None, 'UDP': None} if self.is_set('redirectalltraffic'): if self.is_unconfigured('defaulttcplistener'): self.logger.error('ERROR: No default TCP listener specified ' + @@ -395,28 +1030,34 @@ def parse_diverter_config(self): 'in the configuration.') sys.exit(1) - elif not self.getconfigval('defaulttcplistener').lower() in self.listeners_config: + elif not (self.getconfigval('defaulttcplistener').lower() in + self.listeners_config): self.logger.error('ERROR: No configuration exists for ' + - 'default TCP listener %s', self.getconfigval( - 'defaulttcplistener')) + 'default TCP listener %s', + self.getconfigval('defaulttcplistener')) sys.exit(1) - elif not self.getconfigval('defaultudplistener').lower() in self.listeners_config: + elif not (self.getconfigval('defaultudplistener').lower() in + self.listeners_config): self.logger.error('ERROR: No configuration exists for ' + - 'default UDP listener %s', self.getconfigval( - 'defaultudplistener')) + 'default UDP listener %s', + self.getconfigval('defaultudplistener')) sys.exit(1) else: - self.default_listener['TCP'] = int( - self.listeners_config[self.getconfigval('defaulttcplistener').lower()]['port']) - self.logger.error('Using default listener %s on port %d', self.getconfigval( - 'defaulttcplistener').lower(), self.default_listener['TCP']) - - self.default_listener['UDP'] = int( - self.listeners_config[self.getconfigval('defaultudplistener').lower()]['port']) - self.logger.error('Using default listener %s on port %d', self.getconfigval( - 'defaultudplistener').lower(), self.default_listener['UDP']) + default_listener = self.getconfigval('defaulttcplistener').lower() + default_port = self.listeners_config[default_listener]['port'] + self.default_listener['TCP'] = int(default_port) + self.logger.error('Using default listener %s on port %d', + self.getconfigval('defaulttcplistener').lower(), + self.default_listener['TCP']) + + default_listener = self.getconfigval('defaultudplistener').lower() + default_port = self.listeners_config[default_listener]['port'] + self.default_listener['UDP'] = int(default_port) + self.logger.error('Using default listener %s on port %d', + self.getconfigval('defaultudplistener').lower(), + self.default_listener['UDP']) # Re-marshall these into a readily usable form... @@ -434,113 +1075,671 @@ def parse_diverter_config(self): self.logger.debug('Blacklisted UDP ports: %s', ', '.join( [str(p) for p in self.getconfigval('BlackListPortsUDP')])) - def write_pcap(self, data): + def write_pcap(self, pkt): + """Writes a packet to the pcap. + + Args: + pkt: A fnpacket.PacketCtx or derived object + + Returns: + None + + Side-effects: + Calls dpkt.pcap.Writer.writekpt to persist the octets + """ if self.pcap and self.pcap_lock: - self.pcap_lock.acquire() - try: - self.pcap.writepkt(data) - finally: - self.pcap_lock.release() - - -def test_redir_logic(diverter_factory): - diverter_config = dict() - diverter_config['dumppackets'] = 'Yes' - diverter_config['dumppacketsfileprefix'] = 'packets' - diverter_config['modifylocaldns'] = 'No' - diverter_config['stopdnsservice'] = 'Yes' - diverter_config['redirectalltraffic'] = 'Yes' - diverter_config['defaulttcplistener'] = 'RawTCPListener' - diverter_config['defaultudplistener'] = 'RawUDPListener' - diverter_config['blacklistportstcp'] = '139' - diverter_config['blacklistportsudp'] = '67, 68, 137, 138, 1900, 5355' - - listeners_config = OrderedDict() - - listeners_config['dummytcp'] = dict() - listeners_config['dummytcp']['enabled'] = 'True' - listeners_config['dummytcp']['port'] = '65535' - listeners_config['dummytcp']['protocol'] = 'TCP' - listeners_config['dummytcp']['listener'] = 'RawListener' - listeners_config['dummytcp']['usessl'] = 'No' - listeners_config['dummytcp']['timeout'] = '10' - - listeners_config['rawtcplistener'] = dict() - listeners_config['rawtcplistener']['enabled'] = 'True' - listeners_config['rawtcplistener']['port'] = '1337' - listeners_config['rawtcplistener']['protocol'] = 'TCP' - listeners_config['rawtcplistener']['listener'] = 'RawListener' - listeners_config['rawtcplistener']['usessl'] = 'No' - listeners_config['rawtcplistener']['timeout'] = '10' - - listeners_config['dummyudp'] = dict() - listeners_config['dummyudp']['enabled'] = 'True' - listeners_config['dummyudp']['port'] = '65535' - listeners_config['dummyudp']['protocol'] = 'UDP' - listeners_config['dummyudp']['listener'] = 'RawListener' - listeners_config['dummyudp']['usessl'] = 'No' - listeners_config['dummyudp']['timeout'] = '10' - - listeners_config['rawudplistener'] = dict() - listeners_config['rawudplistener']['enabled'] = 'True' - listeners_config['rawudplistener']['port'] = '1337' - listeners_config['rawudplistener']['protocol'] = 'UDP' - listeners_config['rawudplistener']['listener'] = 'RawListener' - listeners_config['rawudplistener']['usessl'] = 'No' - listeners_config['rawudplistener']['timeout'] = '10' - - listeners_config['httplistener80'] = dict() - listeners_config['httplistener80']['enabled'] = 'True' - listeners_config['httplistener80']['port'] = '80' - listeners_config['httplistener80']['protocol'] = 'TCP' - listeners_config['httplistener80']['listener'] = 'HTTPListener' - listeners_config['httplistener80']['usessl'] = 'No' - listeners_config['httplistener80']['webroot'] = 'defaultFiles/' - listeners_config['httplistener80']['timeout'] = '10' - listeners_config['httplistener80']['dumphttpposts'] = 'Yes' - listeners_config['httplistener80']['dumphttppostsfileprefix'] = 'http' - - ip_addrs = dict() - ip_addrs[4] = ['192.168.19.222', '127.0.0.1'] - ip_addrs[6] = [] - - div = diverter_factory(diverter_config, listeners_config, ip_addrs) - testcase = namedtuple( - 'testcase', ['src', 'sport', 'dst', 'dport', 'expect']) - - foreign = '192.168.19.132' - LOCAL = '192.168.19.222' - LOOPBACK = '127.0.0.1' - unbound = 33333 - BOUND = 80 - - bound_ports = [] - for k, v in listeners_config.iteritems(): - bound_ports.append(int(v['port'], 10)) - - testcases = [ - testcase(foreign, unbound, LOCAL, unbound, True), - testcase(foreign, unbound, LOCAL, BOUND, False), - testcase(foreign, BOUND, LOCAL, unbound, True), - testcase(foreign, BOUND, LOCAL, BOUND, False), - - testcase(LOCAL, unbound, foreign, unbound, True), - testcase(LOCAL, unbound, foreign, BOUND, False), - testcase(LOCAL, BOUND, foreign, unbound, False), - testcase(LOCAL, BOUND, foreign, BOUND, False), - - testcase(LOOPBACK, unbound, LOOPBACK, unbound, True), - testcase(LOOPBACK, unbound, LOOPBACK, BOUND, False), - testcase(LOOPBACK, BOUND, LOOPBACK, unbound, False), - testcase(LOOPBACK, BOUND, LOOPBACK, BOUND, False), - ] - - for tc in testcases: - r = div.decide_redir_port(4, 'TCP', 1337, bound_ports, tc.src, - tc.sport, tc.dst, tc.dport) - if r != tc.expect: - print('TEST CASE FAILED: %s:%d -> %s:%d expected %d got %d' % - (tc.src, tc.sport, tc.dst, tc.dport, tc.expect, r)) + with self.pcap_lock: + mangled = 'mangled' if pkt.mangled else 'initial' + self.pdebug(DPCAP, 'Writing %s packet %s' % + (mangled, pkt.hdrToStr2())) + self.pcap.writepkt(pkt.octets) + + def handle_pkt(self, pkt, callbacks3, callbacks4): + """Generic packet hook. + + Applies FakeNet-NG decision-making to packet, deferring as necessary to + callbacks. + + Args: + pkt: A fnpacket.PacketCtx child class + callbacks3: Layer 3 (network) callbacks + callbacks4: Layer 4 (network) callbacks + + Side-effects: + 1.) Unconditionally Write unmangled packet to pcap + 2.) Call layer 3 (network) callbacks... + 3.) Call layer 4 (transport) callbacks... + 4.) If the packet headers have been modified, double-write the + mangled packet to the pcap for SSL decoding purposes + + The caller is responsible for checking if the packet was mangled, + updating the contents of the datagram with the network hooking specific + to their OS, and accepting/transmitting the final packet. + + Params + ------ + pkt: fnpacket.PacketCtx object + callbacks3: Array of L3 (network) callbacks + callbacks4: Array of L4 (transport) callbacks + + Side-effects: + Mangles pkt as necessary + + Returns: + None + """ + + # 1: Unconditionally write unmangled packet to pcap + self.write_pcap(pkt) + + no_further_processing = False + + if pkt.ipver is None: + self.logger.warning('%s: Failed to parse IP packet' % (pkt.label)) else: - print('Test case passed: %s:%d -> %s:%d expected %d got %d' % - (tc.src, tc.sport, tc.dst, tc.dport, tc.expect, r)) + self.pdebug(DGENPKT, '%s %s' % (pkt.label, pkt.hdrToStr())) + + crit = DivertParms(self, pkt) + + # fnpacket has parsed all that can be parsed, so + pid, comm = self.get_pid_comm(pkt) + if self.pdebug_level & DGENPKTV: + logline = self.formatPkt(pkt, pid, comm) + self.pdebug(DGENPKTV, logline) + elif pid and (pid != self.pid) and crit.first_packet_new_session: + self.logger.info(' pid: %d name: %s' % + (pid, comm if comm else 'Unknown')) + + # 2: Call layer 3 (network) callbacks + for cb in callbacks3: + # These debug outputs are useful for figuring out which + # callback is responsible for an exception that was masked by + # python-netfilterqueue's global callback. + self.pdebug(DCB, 'Calling %s' % (cb)) + + cb(crit, pkt) + + self.pdebug(DCB, '%s finished' % (cb)) + + if pkt.proto: + + if len(callbacks4): + # Windows Diverter has always allowed loopback packets to + # fall where they may. This behavior now applies to all + # Diverters. + if crit.is_loopback: + self.logger.debug('Ignoring loopback packet') + self.logger.debug(' %s:%d -> %s:%d', pkt.src_ip, + pkt.sport, pkt.dst_ip, pkt.dport) + no_further_processing = True + + # 3: Layer 4 (Transport layer) callbacks + if not no_further_processing: + for cb in callbacks4: + # These debug outputs are useful for figuring out + # which callback is responsible for an exception + # that was masked by python-netfilterqueue's global + # callback. + self.pdebug(DCB, 'Calling %s' % (cb)) + + cb(crit, pkt, pid, comm) + + self.pdebug(DCB, '%s finished' % (cb)) + + else: + self.pdebug(DGENPKT, '%s: Not handling protocol %s' % + (pkt.label, pkt.proto)) + + # 4: Double write mangled packets to represent changes made by + # FakeNet-NG while still allowing SSL decoding with the old packets + if pkt.mangled: + self.write_pcap(pkt) + + def formatPkt(self, pkt, pid, comm): + """Format a packet analysis log line for DGENPKTV. + + Args: + pkt: A fnpacket.PacketCtx or derived object + pid: Process ID associated with the packet + comm: Process executable name + + Returns: + A str containing the log line + """ + logline = '' + + if pkt.proto == 'UDP': + fmt = '| {label} {proto} | {pid:>6} | {comm:<8} | {src:>15}:{sport:<5} | {dst:>15}:{dport:<5} | {length:>5} | {flags:<11} | {seqack:<35} |' + logline = fmt.format( + label=pkt.label, + proto=pkt.proto, + pid=pid, + comm=comm, + src=pkt.src_ip, + sport=pkt.sport, + dst=pkt.dst_ip, + dport=pkt.dport, + length=len(pkt), + flags='', + seqack='', + ) + + elif pkt.proto == 'TCP': + tcp = pkt._hdr.data + + sa = 'Seq=%d, Ack=%d' % (tcp.seq, tcp.ack) + + f = [] + if (tcp.flags & dpkt.tcp.TH_RST) != 0: + f.append('RST') + if (tcp.flags & dpkt.tcp.TH_SYN) != 0: + f.append('SYN') + if (tcp.flags & dpkt.tcp.TH_ACK) != 0: + f.append('ACK') + if (tcp.flags & dpkt.tcp.TH_FIN) != 0: + f.append('FIN') + if (tcp.flags & dpkt.tcp.TH_PUSH) != 0: + f.append('PSH') + + fmt = '| {label} {proto} | {pid:>6} | {comm:<8} | {src:>15}:{sport:<5} | {dst:>15}:{dport:<5} | {length:>5} | {flags:<11} | {seqack:<35} |' + logline = fmt.format( + label=pkt.label, + proto=pkt.proto, + pid=pid, + comm=comm, + src=pkt.src_ip, + sport=pkt.sport, + dst=pkt.dst_ip, + dport=pkt.dport, + length=len(pkt), + flags=','.join(f), + seqack=sa, + ) + else: + fmt = '| {label} {proto} | {pid:>6} | {comm:<8} | {src:>15}:{sport:<5} | {dst:>15}:{dport:<5} | {length:>5} | {flags:<11} | {seqack:<35} |' + logline = fmt.format( + label=pkt.label, + proto='UNK', + pid=pid, + comm=comm, + src=str(pkt.src_ip), + sport=str(pkt.sport), + dst=str(pkt.dst_ip), + dport=str(pkt.dport), + length=len(pkt), + flags='', + seqack='', + ) + return logline + + def check_should_ignore(self, pkt, pid, comm): + """Indicate whether a packet should be passed without mangling. + + Checks whether the packet matches black and whitelists, or whether it + signifies an FTP Active Mode connection. + + Args: + pkt: A fnpacket.PacketCtx or derived object + pid: Process ID associated with the packet + comm: Process executable name + + Returns: + True if the packet should be left alone, else False. + """ + + src_ip = pkt.src_ip0 + sport = pkt.sport0 + dst_ip = pkt.dst_ip0 + dport = pkt.dport0 + + if not self.is_set('redirectalltraffic'): + self.pdebug(DIGN, 'Ignoring %s packet %s' % + (pkt.proto, pkt.hdrToStr())) + return True + + # SingleHost mode checks + if self.single_host_mode: + if comm: + # Check process blacklist + if comm in self.blacklist_processes: + self.pdebug(DIGN, ('Ignoring %s packet from process %s ' + + 'in the process blacklist.') % (pkt.proto, + comm)) + self.pdebug(DIGN, ' %s' % + (pkt.hdrToStr())) + return True + + # Check process whitelist + elif (len(self.whitelist_processes) and (comm not in + self.whitelist_processes)): + self.pdebug(DIGN, ('Ignoring %s packet from process %s ' + + 'not in the process whitelist.') % (pkt.proto, + comm)) + self.pdebug(DIGN, ' %s' % + (pkt.hdrToStr())) + return True + + # Check per-listener blacklisted process list + elif self.listener_ports.isProcessBlackListHit( + pkt.proto, dport, comm): + self.pdebug(DIGN, ('Ignoring %s request packet from ' + + 'process %s in the listener process ' + + 'blacklist.') % (pkt.proto, comm)) + self.pdebug(DIGN, ' %s' % + (pkt.hdrToStr())) + return True + + # Check per-listener whitelisted process list + elif self.listener_ports.isProcessWhiteListMiss( + pkt.proto, dport, comm): + self.pdebug(DIGN, ('Ignoring %s request packet from ' + + 'process %s not in the listener process ' + + 'whitelist.') % (pkt.proto, comm)) + self.pdebug(DIGN, ' %s' % + (pkt.hdrToStr())) + return True + + # MultiHost mode checks + else: + pass # None as of yet + + # Checks independent of mode + + # Forwarding blacklisted port + if pkt.proto: + if set(self.blacklist_ports[pkt.proto]).intersection([sport, dport]): + self.pdebug(DIGN, 'Forwarding blacklisted port %s packet:' % + (pkt.proto)) + self.pdebug(DIGN, ' %s' % (pkt.hdrToStr())) + return True + + # Check host blacklist + global_host_blacklist = self.getconfigval('hostblacklist') + if global_host_blacklist and dst_ip in global_host_blacklist: + self.pdebug(DIGN, ('Ignoring %s packet to %s in the host ' + + 'blacklist.') % (str(pkt.proto), dst_ip)) + self.pdebug(DIGN, ' %s' % (pkt.hdrToStr())) + self.logger.error('IGN: host blacklist match') + return True + + # Check the port host whitelist + if self.listener_ports.isHostWhiteListMiss(pkt.proto, dport, dst_ip): + self.pdebug(DIGN, ('Ignoring %s request packet to %s not in ' + + 'the listener host whitelist.') % (pkt.proto, + dst_ip)) + self.pdebug(DIGN, ' %s' % (pkt.hdrToStr())) + return True + + # Check the port host blacklist + if self.listener_ports.isHostBlackListHit(pkt.proto, dport, dst_ip): + self.pdebug(DIGN, ('Ignoring %s request packet to %s in the ' + + 'listener host blacklist.') % (pkt.proto, dst_ip)) + self.pdebug(DIGN, ' %s' % (pkt.hdrToStr())) + return True + + # Duplicated from diverters/windows.py: + # HACK: FTP Passive Mode Handling + # Check if a listener is initiating a new connection from a + # non-diverted port and add it to blacklist. This is done to handle a + # special use-case of FTP ACTIVE mode where FTP server is initiating a + # new connection for which the response may be redirected to a default + # listener. NOTE: Additional testing can be performed to check if this + # is actually a SYN packet + if pid == self.pid: + if ( + ((dst_ip in self.ip_addrs[pkt.ipver]) and + (not dst_ip.startswith('127.'))) and + ((src_ip in self.ip_addrs[pkt.ipver]) and + (not dst_ip.startswith('127.'))) and + (not self.listener_ports.intersectsWithPorts(pkt.proto, [sport, dport])) + ): + + self.pdebug(DIGN | DFTP, 'Listener initiated %s connection' % + (pkt.proto)) + self.pdebug(DIGN | DFTP, ' %s' % (pkt.hdrToStr())) + self.pdebug(DIGN | DFTP, ' Blacklisting port %d' % (sport)) + self.blacklist_ports[pkt.proto].append(sport) + + return True + + return False + + def check_log_icmp(self, crit, pkt): + """Log an ICMP packet if the header was parsed as ICMP. + + Args: + crit: A DivertParms object + pkt: An fnpacket.PacketCtx or derived object + + Returns: + None + """ + if pkt.is_icmp: + self.logger.info('ICMP type %d code %d %s' % ( + pkt.icmp_type, pkt.icmp_code, pkt.hdrToStr())) + + def getOriginalDestPort(self, orig_src_ip, orig_src_port, proto): + """Return original destination port, or None if it was not redirected. + + The proxy listener uses this method to obtain and provide port + information to listeners in the taste() callback as an extra hint as to + whether the traffic may be appropriate for parsing by that listener. + + Args: + orig_src_ip: A str that is the ASCII representation of the peer IP + orig_src_port: An int that is the source port of the peer + + Returns: + The original destination port if the packet was redirected + None, otherwise + """ + + orig_src_key = fnpacket.PacketCtx.gen_endpoint_key(proto, orig_src_ip, + orig_src_port) + with self.port_fwd_table_lock: + return self.port_fwd_table.get(orig_src_key) + + def maybe_redir_ip(self, crit, pkt, pid, comm): + """Conditionally redirect foreign destination IPs to localhost. + + On Linux, this is used only under SingleHost mode. + + Args: + crit: DivertParms object + pkt: fnpacket.PacketCtx or derived object + pid: int process ID associated with the packet + comm: Process name (command) that sent the packet + + Side-effects: + May mangle the packet by modifying the destination IP to point to a + loopback or external interface IP local to the system where + FakeNet-NG is running. + + Returns: + None + """ + if self.check_should_ignore(pkt, pid, comm): + return + + self.pdebug(DIPNAT, 'Condition 1 test') + # Condition 1: If the remote IP address is foreign to this system, + # then redirect it to a local IP address. + if self.single_host_mode and (pkt.dst_ip not in self.ip_addrs[pkt.ipver]): + self.pdebug(DIPNAT, 'Condition 1 satisfied') + with self.ip_fwd_table_lock: + self.ip_fwd_table[pkt.skey] = pkt.dst_ip + + newdst = self.getNewDestinationIp(pkt.src_ip) + + self.pdebug(DIPNAT, 'REDIRECTING %s to IP %s' % + (pkt.hdrToStr(), newdst)) + pkt.dst_ip = newdst + + else: + # Delete any stale entries in the IP forwarding table: If the + # local endpoint appears to be reusing a client port that was + # formerly used to connect to a foreign host (but not anymore), + # then remove the entry. This prevents a packet hook from + # faithfully overwriting the source IP on a later packet to + # conform to the foreign endpoint's stale connection IP when + # the host is reusing the port number to connect to an IP + # address that is local to the FakeNet system. + + with self.ip_fwd_table_lock: + if pkt.skey in self.ip_fwd_table: + self.pdebug(DIPNAT, ' - DELETING ipfwd key entry: %s' % + (pkt.skey)) + del self.ip_fwd_table[pkt.skey] + + def maybe_fixup_srcip(self, crit, pkt, pid, comm): + """Conditionally fix up the source IP address if the remote endpoint + had their connection IP-forwarded. + + Check is based on whether the remote endpoint corresponds to a key in + the IP forwarding table. + + Args: + crit: DivertParms object + pkt: fnpacket.PacketCtx or derived object + pid: int process ID associated with the packet + comm: Process name (command) that sent the packet + + Side-effects: + May mangle the packet by modifying the source IP to reflect the + original destination IP that was overwritten by maybe_redir_ip. + + Returns: + None + """ + # Condition 4: If the local endpoint (IP/port/proto) combo + # corresponds to an endpoint that initiated a conversation with a + # foreign endpoint in the past, then fix up the source IP for this + # incoming packet with the last destination IP that was requested + # by the endpoint. + self.pdebug(DIPNAT, "Condition 4 test: was remote endpoint IP fwd'd?") + with self.ip_fwd_table_lock: + if self.single_host_mode and (pkt.dkey in self.ip_fwd_table): + self.pdebug(DIPNAT, 'Condition 4 satisfied') + self.pdebug(DIPNAT, ' = FOUND ipfwd key entry: ' + pkt.dkey) + new_srcip = self.ip_fwd_table[pkt.dkey] + self.pdebug(DIPNAT, 'MASQUERADING %s from IP %s' % + (pkt.hdrToStr(), new_srcip)) + pkt.src_ip = new_srcip + else: + self.pdebug(DIPNAT, ' ! NO SUCH ipfwd key entry: ' + pkt.dkey) + + def maybe_redir_port(self, crit, pkt, pid, comm): + """Conditionally send packets to the default listener for this proto. + + Args: + crit: DivertParms object + pkt: fnpacket.PacketCtx or derived object + pid: int process ID associated with the packet + comm: Process name (command) that sent the packet + + Side-effects: + May mangle the packet by modifying the destination port to point to + the default listener. + + Returns: + None + """ + # Pre-condition 1: there must be a default listener for this protocol + default = self.default_listener.get(pkt.proto) + if not default: + return + + # Pre-condition 2: destination must not be present in port forwarding + # table (prevents masqueraded ports responding to unbound ports from + # being mistaken as starting a conversation with an unbound port). + with self.port_fwd_table_lock: + # Uses dkey to cross-reference + if pkt.dkey in self.port_fwd_table: + return + + # Proxy-related check: is the dport bound by a listener that is hidden? + dport_hidden_listener = crit.dport_hidden_listener + + # Condition 2: If the packet is destined for an unbound port, then + # redirect it to a bound port and save the old destination IP in + # the port forwarding table keyed by the source endpoint identity. + + bound_ports = self.listener_ports.getPortList(pkt.proto) + if dport_hidden_listener or self.decide_redir_port(pkt, bound_ports): + self.pdebug(DDPFV, 'Condition 2 satisfied: Packet destined for ' + 'unbound port or hidden listener') + + # Post-condition 1: General ignore conditions are not met, or this + # is part of a conversation that is already being ignored. + # + # Placed after the decision to redirect for three reasons: + # 1.) We want to ensure that the else condition below has a chance + # to check whether to delete a stale port forwarding table + # entry. + # 2.) Checking these conditions is, on average, more expensive than + # checking if the packet would be redirected in the first + # place. + # 3.) Reporting of packets that are being ignored (i.e. not + # redirected), which is integrated into this check, should only + # appear when packets would otherwise have been redirected. + + # Is this conversation already being ignored for DPF purposes? + with self.ignore_table_lock: + if ((pkt.dkey in self.ignore_table) and + (self.ignore_table[pkt.dkey] == pkt.sport)): + # This is a reply (e.g. a TCP RST) from the + # non-port-forwarded server that the non-port-forwarded + # client was trying to talk to. Leave it alone. + return + + if self.check_should_ignore(pkt, pid, comm): + with self.ignore_table_lock: + self.ignore_table[pkt.skey] = pkt.dport + return + + # Record the foreign endpoint and old destination port in the port + # forwarding table + self.pdebug(DDPFV, ' + ADDING portfwd key entry: ' + pkt.skey) + with self.port_fwd_table_lock: + self.port_fwd_table[pkt.skey] = pkt.dport + + self.pdebug(DDPF, 'Redirecting %s to go to port %d' % + (pkt.hdrToStr(), default)) + pkt.dport = default + + else: + # Delete any stale entries in the port forwarding table: If the + # foreign endpoint appears to be reusing a client port that was + # formerly used to connect to an unbound port on this server, + # remove the entry. This prevents the OUTPUT or other packet + # hook from faithfully overwriting the source port to conform + # to the foreign endpoint's stale connection port when the + # foreign host is reusing the port number to connect to an + # already-bound port on the FakeNet system. + + self.delete_stale_port_fwd_key(pkt.skey) + + if crit.first_packet_new_session: + self.addSession(pkt) + + # Execute command if applicable + self.maybeExecuteCmd(pkt, pid, comm) + + def maybe_fixup_sport(self, crit, pkt, pid, comm): + """Conditionally fix up source port if the remote endpoint had their + connection port-forwarded to the default listener. + + Check is based on whether the remote endpoint corresponds to a key in + the port forwarding table. + + Side-effects: + May mangle the packet by modifying the source port to masquerade + traffic coming from the default listener to look as if it is coming + from the port that the client originally requested. + + Returns: + None + """ + hdr_modified = None + + # Condition 3: If the remote endpoint (IP/port/proto) combo + # corresponds to an endpoint that initiated a conversation with an + # unbound port in the past, then fix up the source port for this + # outgoing packet with the last destination port that was requested + # by that endpoint. The term "endpoint" is (ab)used loosely here to + # apply to UDP host/port/proto combos and any other protocol that + # may be supported in the future. + new_sport = None + self.pdebug(DDPFV, "Condition 3 test: was remote endpoint port fwd'd?") + + with self.port_fwd_table_lock: + new_sport = self.port_fwd_table.get(pkt.dkey) + + if new_sport: + self.pdebug(DDPFV, 'Condition 3 satisfied: must fix up ' + + 'source port') + self.pdebug(DDPFV, ' = FOUND portfwd key entry: ' + pkt.dkey) + self.pdebug(DDPF, 'MASQUERADING %s to come from port %d' % + (pkt.hdrToStr(), new_sport)) + pkt.sport = new_sport + else: + self.pdebug(DDPFV, ' ! NO SUCH portfwd key entry: ' + pkt.dkey) + + return pkt.hdr if pkt.mangled else None + + def delete_stale_port_fwd_key(self, skey): + with self.port_fwd_table_lock: + if skey in self.port_fwd_table: + self.pdebug(DDPFV, ' - DELETING portfwd key entry: ' + skey) + del self.port_fwd_table[skey] + + def decide_redir_port(self, pkt, bound_ports): + """Decide whether to redirect a port. + + Optimized logic derived by truth table + k-map. See docs/internals.md + for details. + + Args: + pkt: fnpacket.PacketCtx or derived object + bound_ports: Set of ports that are bound for this protocol + + Returns: + True if the packet must be redirected to the default listener + False otherwise + """ + # A, B, C, D for easy manipulation; full names for readability only. + a = src_local = (pkt.src_ip in self.ip_addrs[pkt.ipver]) + c = sport_bound = pkt.sport in (bound_ports) + d = dport_bound = pkt.dport in (bound_ports) + + if self.pdebug_level & DDPFV: + # Unused logic term not calculated except for debug output + b = dst_local = (pkt.dst_ip in self.ip_addrs[pkt.ipver]) + + self.pdebug(DDPFV, 'src %s (%s)' % + (str(pkt.src_ip), ['foreign', 'local'][a])) + self.pdebug(DDPFV, 'dst %s (%s)' % + (str(pkt.dst_ip), ['foreign', 'local'][b])) + self.pdebug(DDPFV, 'sport %s (%sbound)' % + (str(pkt.sport), ['un', ''][c])) + self.pdebug(DDPFV, 'dport %s (%sbound)' % + (str(pkt.dport), ['un', ''][d])) + + # Convenience function: binary representation of a bool + def bn(x): + return '1' if x else '0' # Bool -> binary + + self.pdebug(DDPFV, 'abcd = ' + bn(a) + bn(b) + bn(c) + bn(d)) + + return (not a and not d) or (not c and not d) + + def addSession(self, pkt): + """Add a connection to the sessions hash table. + + Args: + pkt: fnpacket.PacketCtx or derived object + + Returns: + None + """ + self.sessions[pkt.sport] = (pkt.dst_ip, pkt.dport) + + def maybeExecuteCmd(self, pkt, pid, comm): + """Execute any ExecuteCmd associated with this port/listener. + + Args: + pkt: fnpacket.PacketCtx or derived object + pid: int process ID associated with the packet + comm: Process name (command) that sent the packet + + Returns: + None + """ + if not pid: + return + + execCmd = self.build_cmd(pkt, pid, comm) + if execCmd: + self.logger.info('Executing command: %s' % (execCmd)) + self.execute_detached(execCmd) + diff --git a/fakenet/diverters/fnconfig.py b/fakenet/diverters/fnconfig.py index e2d7332..5a97061 100644 --- a/fakenet/diverters/fnconfig.py +++ b/fakenet/diverters/fnconfig.py @@ -1,4 +1,4 @@ -class Config: +class Config(object): """Configuration primitives. Inherit from or instantiate this class and call configure() when you've got diff --git a/fakenet/diverters/fnpacket.py b/fakenet/diverters/fnpacket.py new file mode 100644 index 0000000..91355c1 --- /dev/null +++ b/fakenet/diverters/fnpacket.py @@ -0,0 +1,305 @@ +import dpkt +import socket +import logging +import debuglevels + + +class PacketCtx(object): + """Library-agnostic representation of packet and metadata. + + Attempt to abstract the following out of FakeNet-NG code: + * OS-specific metadata + * Packet display + * Packet mangling + * Use of underlying packet libraries e.g. dpkt + """ + + @staticmethod + def gen_endpoint_key(proto, ip, port): + """e.g. 192.168.19.132:tcp/3030 + + Need static method because getOriginalDestPort (called by proxy + listener) uses this. + """ + return str(ip) + ':' + str(proto) + '/' + str(port) + + def __init__(self, label, raw): + self.logger = logging.getLogger('Diverter') + + # Universal parameters + self.label = label + self._raw = raw + self._mangled = False # Determines whether to recalculate csums + + self.handled_protocols = { + dpkt.ip.IP_PROTO_TCP: 'TCP', + dpkt.ip.IP_PROTO_UDP: 'UDP', + } + + self._is_ip = False + self._is_icmp = False + + # Some packet attributes are cached in duplicate members below for code + # simplicity and uniformity rather than having to query which packet + # headers were or were not parsed. + + # L3 (IP) parameters + self.ipver = None + self._ipcsum0 = None # Initial checksum + self._hdr = None + self.proto_num = None + self.proto = None # Abused as flag: is L4 protocol handled? + self._src_ip0 = None # Initial source IP address + self._src_ip = None # Cached in ASCII form + self._dst_ip0 = None # Initial destination IP address + self._dst_ip = None # Again cached in ASCII + + # L4 (TCP or UDP) parameters + self._tcpudpcsum0 = None # Initial checksum + self._sport0 = None # Initial source port + self._sport = None # Cached for uniformity/ease + self.skey = None + self._dport0 = None # Initial destination port + self._dport = None # Cached for uniformity/ease + self.dkey = None + + # Parse as much as possible + self.ipver = ((ord(self._raw[0]) & 0xf0) >> 4) + if self.ipver == 4: + self._parseIpv4() + elif self.ipver == 6: + self._parseIpv6() + self._parseIp() # If _parseIpv4 or _parseIpv6 worked... + self._parseIcmp() # Or handle ICMP packets + + def __len__(self): + if self._mangled: + self._updateRaw() + + return len(self._raw) + + # Data + + @property + def mangled(self): + return self._mangled + + @property + def hdr(self): + if self._mangled: + self._calcCsums() + + return self._hdr + + @property + def octets(self): + if self._mangled: + self._updateRaw() + + return self._raw + + # csums (NOTE: IPv6 has no csum, will return None) + + @property + def l3csum0(self): + return self._ipcsum0 + + @property + def l3csum(self): + if self.ipver == 4: + return self._hdr.sum + return None + + @property + def l4csum0(self): + return self._tcpudpcsum0 + + @property + def l4csum(self): + if self.proto: + return self._hdr.data.sum + return None + + # src_ip + + @property + def src_ip0(self): + return self._src_ip0 + + @property + def src_ip(self): + return self._src_ip + + @src_ip.setter + def src_ip(self, new_srcip): + if self._is_ip: + self._src_ip = new_srcip + self._hdr.src = socket.inet_aton(new_srcip) + self._mangled = True + + # dst_ip + + @property + def dst_ip0(self): + return self._dst_ip0 + + @property + def dst_ip(self): + return self._dst_ip + + @dst_ip.setter + def dst_ip(self, new_dstip): + if self._is_ip: + self._dst_ip = new_dstip + self._hdr.dst = socket.inet_aton(new_dstip) + self._mangled = True + + # sport + + @property + def sport0(self): + return self._sport0 + + @property + def sport(self): + return self._sport + + @sport.setter + def sport(self, new_sport): + if self._is_ip: + self._sport = new_sport + self._hdr.data.sport = new_sport + self._mangled = True + + # dport + + @property + def dport0(self): + return self._dport0 + + @property + def dport(self): + return self._dport + + @dport.setter + def dport(self, new_dport): + if self._is_ip: + self._dport = new_dport + self._hdr.data.dport = new_dport + self._mangled = True + + # ICMP + + @property + def is_icmp(self): + return self._is_icmp + + @property + def icmp_type(self): + if self._is_icmp: + return self._hdr.data.type + return None + + @property + def icmp_code(self): + if self._is_icmp: + return self._hdr.data.code + return None + + def fmtL3Csums(self): + s = 'IP csum N/A' + if self._is_ip: + if self.ipver == 4: + csum0 = hex(self._ipcsum0).rstrip('L') + if self._mangled: + self._calcCsums() + csum = hex(self._hdr.sum).rstrip('L') + s = 'IPv4 csum %s->%s' % (csum0, csum) + else: + s = 'IPv4 csum %s' % (csum0) + elif self.ipver == 6: + s = 'IPv6 csum N/A' + return s + + def fmtL4Csums(self): + s = 'L4 csum N/A' + if self.proto: + csum0 = hex(self._tcpudpcsum0).rstrip('L') + if self._mangled: + self._calcCsums() + csum = hex(self._hdr.data.sum).rstrip('L') + s = '%s csum %s->%s' % (self.proto, csum0, csum) + else: + s = '%s csum %s' % (self.proto, csum0) + return s + + def fmtCsumData(self, sep='/'): + if self._is_ip: + return '%s %s %s ' % (self.fmtL3Csums(), sep, self.fmtL4Csums()) + else: + return 'No identifying info' + + def hdrToStr2(self, sep='/'): + return '%s %s %s' % (self.hdrToStr(), sep, self.fmtCsumData(sep)) + + def hdrToStr(self): + s = 'No valid IP headers parsed' + if self._is_ip: + if self.proto: + s = '%s %s:%d->%s:%d' % (self.proto, self._src_ip, + self._hdr.data.sport, self._dst_ip, + self._hdr.data.dport) + else: + s = '%s->%s' % (self._src_ip, self._dst_ip) + + return s + + def _parseIp(self): + """Parse IP src/dst fields and next-layer fields if recognized.""" + if self._is_ip: + self._src_ip0 = self._src_ip = socket.inet_ntoa(self._hdr.src) + self._dst_ip0 = self._dst_ip = socket.inet_ntoa(self._hdr.dst) + self.proto = self.handled_protocols.get(self.proto_num) + + # If this is a transport protocol we handle... + if self.proto: + self._tcpudpcsum0 = self._hdr.data.sum + self._sport0 = self._sport = self._hdr.data.sport + self._dport0 = self._dport = self._hdr.data.dport + self.skey = self._genEndpointKey(self._src_ip, self._sport) + self.dkey = self._genEndpointKey(self._dst_ip, self._dport) + + def _genEndpointKey(self, ip, port): + return PacketCtx.gen_endpoint_key(self.proto, ip, port) + + def _parseIcmp(self): + self._is_icmp = (self.proto_num == dpkt.ip.IP_PROTO_ICMP) + + def _parseIpv4(self): + hdr = dpkt.ip.IP(self._raw) + # An IP header length less than 5 is invalid + if hdr.hl >= 5: + self._is_ip = True + self._hdr = hdr + self.proto_num = hdr.p + self._ipcsum0 = hdr.sum + return bool(self._hdr) + + def _parseIpv6(self): + self._is_ip = True + self._hdr = dpkt.ip6.IP6(self._raw) + self.proto_num = self._hdr.nxt + return True + + def _calcCsums(self): + """The roundabout dance of inducing dpkt to recalculate checksums...""" + self._hdr.sum = 0 + self._hdr.data.sum = 0 + # This has the side-effect of invoking dpkt.in_cksum() et al: + str(self._hdr) + + def _updateRaw(self): + self._calcCsums() + self._raw = self._hdr.pack() + + diff --git a/fakenet/diverters/linutil.py b/fakenet/diverters/linutil.py index 72cf3af..145f0a3 100644 --- a/fakenet/diverters/linutil.py +++ b/fakenet/diverters/linutil.py @@ -8,46 +8,12 @@ import threading import subprocess import netfilterqueue +from debuglevels import * from collections import defaultdict +from . import diverterbase -# Debug print levels for fine-grained debug trace output control -DNFQUEUE = (1 << 0) # netfilterqueue -DGENPKT = (1 << 1) # Generic packet handling -DGENPKTV = (1 << 2) # Generic packet handling with TCP analysis -DCB = (1 << 3) # Packet handlign callbacks -DPROCFS = (1 << 4) # procfs -DIPTBLS = (1 << 5) # iptables -DNONLOC = (1 << 6) # Nonlocal-destined datagrams -DDPF = (1 << 7) # DPF (Dynamic Port Forwarding) -DDPFV = (1 << 8) # DPF (Dynamic Port Forwarding) Verbose -DIPNAT = (1 << 9) # IP redirection for nonlocal-destined datagrams -DIGN = (1 << 10) # Packet redirect ignore conditions -DFTP = (1 << 11) # FTP checks -DMISC = (1 << 27) # Miscellaneous - -DCOMP = 0x0fffffff # Component mask -DFLAG = 0xf0000000 # Flag mask -DEVERY = 0x0fffffff # Log everything, low verbosity -DEVERY2 = 0x8fffffff # Log everything, complete verbosity - -DLABELS = { - DNFQUEUE: 'NFQUEUE', - DGENPKT: 'GENPKT', - DGENPKTV: 'GENPKTV', - DPROCFS: 'PROCFS', - DIPTBLS: 'IPTABLES', - DNONLOC: 'NONLOC', - DDPF: 'DPF', - DDPFV: 'DPFV', - DIPNAT: 'IPNAT', - DIGN: 'IGN', - DIGN | DFTP: 'IGN-FTP', - DMISC: 'MISC', -} - -DLABELS_INV = {v.upper(): k for k, v in DLABELS.iteritems()} - -class IptCmdTemplate: + +class IptCmdTemplate(object): """For managing insertion and removal of iptables rules. Construct and execute iptables command lines to add (-I or -A) and remove @@ -61,16 +27,20 @@ def __init__(self, fmt, args=[], add='-I', rem='-D', add_idx=0, rem_idx=0): self._addcmd = fmt % tuple(args[0:add_idx] + [add] + args[add_idx:]) self._remcmd = fmt % tuple(args[0:add_idx] + [rem] + args[rem_idx:]) - def gen_add_cmd(self): return self._addcmd + def gen_add_cmd(self): + return self._addcmd - def gen_remove_cmd(self): return self._remcmd + def gen_remove_cmd(self): + return self._remcmd - def add(self): return subprocess.call(self._addcmd.split()) + def add(self): + return subprocess.call(self._addcmd.split()) - def remove(self): return subprocess.call(self._remcmd.split()) + def remove(self): + return subprocess.call(self._remcmd.split()) -class LinuxDiverterNfqueue: +class LinuxDiverterNfqueue(object): """NetfilterQueue object wrapper. Handles iptables rule addition/removal, NetfilterQueue management, @@ -188,7 +158,7 @@ def stop(self): self._rule.remove() # Shell out to iptables to remove the rule -class ProcfsReader: +class ProcfsReader(object): """Standard row/field reading for proc files.""" def __init__(self, path, skip, cb): self.path = path @@ -218,7 +188,8 @@ def parse(self, multi=False, max_col=None): # Insufficient columns => ValueError if max_col and (len(line) < max_col): - raise ValueError('Line %d in %s has less than %d columns' % + raise ValueError(('Line %d in %s has less than %d ' + 'columns') % (n, self.path, max_col)) # Skip header lines if self.skip: @@ -241,7 +212,7 @@ def parse(self, multi=False, max_col=None): return retval -class LinUtilMixin(): +class LinUtilMixin(diverterbase.DiverterPerOSDelegate): """Automate addition/removal of iptables rules, checking interface names, checking available netfilter queue numbers, etc. """ @@ -250,6 +221,12 @@ def init_linux_mixin(self): self.old_dns = None self.iptables_captured = '' + def getNewDestinationIp(self, ip): + """On Linux, FTP tests fail if IP redirection uses the external IP, so + always return localhost. + """ + return '127.0.0.1' + def check_active_ethernet_adapters(self): return (len(self._linux_get_ifaces()) > 0) @@ -260,6 +237,18 @@ def check_dns_servers(self): # TODO: Implement return True + def check_ipaddresses(self): + # TODO: Implement + return True + + def fix_gateway(self): + # TODO: Implement + return False + + def fix_dns(self): + # TODO: Implement + return False + def linux_capture_iptables(self): self.iptables_captured = '' ret = None @@ -348,10 +337,10 @@ def linux_get_current_nfnlq_bindings(self): str(queue_nr) + ' per ') + procfs_path) qnos.append(queue_nr) except IOError as e: - self.logger.debug(('Failed to open %s to enumerate netfilter ' + - 'netlink queues, caller may proceed as if ' + - 'none are in use: %s') % - (procfs_path, e.message)) + self.logger.debug(('Failed to open %s to enumerate netfilter ' + 'netlink queues, caller may proceed as if ' + 'none are in use: %s') % + (procfs_path, e.message)) return qnos @@ -486,8 +475,8 @@ def linux_restore_local_dns(self): e.message)) def linux_find_processes(self, names): - """Yeah great, but what if a blacklisted process spawns after we call - this? We'd have to call this every time we do anything - expensive! Then again, + """But what if a blacklisted process spawns after we call + this? We'd have to call this every time we do anything. """ pids = [] @@ -523,6 +512,16 @@ def _ip_port_for_proc_net_tcp(self, ipver, ip_dotdecimal, port): def linux_find_sock_by_endpoint(self, ipver, proto_name, ip, port, local=True): + """Check args and call _linux_find_sock_by_endpoint_unsafe.""" + + if proto_name and ip and port: + return self._linux_find_sock_by_endpoint_unsafe(ipver, proto_name, + ip, port, local) + else: + return None + + def _linux_find_sock_by_endpoint_unsafe(self, ipver, proto_name, ip, port, + local=True): """Search /proc/net/tcp for a socket whose local (field 1, zero-based) or remote (field 2) address matches ip:port and return the corresponding inode (field 9). @@ -532,7 +531,7 @@ def linux_find_sock_by_endpoint(self, ipver, proto_name, ip, port, Example contents of /proc/net/tcp (wrapped and double-spaced) sl local_address rem_address st tx_queue rx_queue tr tm->when - retrnsmt uid timeout inode + retrnsmt uid timeout inode 0: 0100007F:0277 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 53320 1 0000000000000000 100 0 0 10 0 @@ -718,6 +717,10 @@ def linux_get_pid_comm_by_endpoint(self, ipver, proto_name, ip, port): return pid, comm + def get_pid_comm(self, pkt): + return self.linux_get_pid_comm_by_endpoint(pkt.ipver, pkt.proto, + pkt.src_ip, pkt.sport) + def linux_endpoint_owned_by_processes(self, ipver, proto_name, ip, port, names): inode = self.linux_find_sock_by_endpoint(ipver, proto_name, ip, port) @@ -736,3 +739,4 @@ def linux_endpoint_owned_by_processes(self, ipver, proto_name, ip, port, (ip, port, t)) return False + diff --git a/fakenet/diverters/linux.py b/fakenet/diverters/linux.py index 4d88e54..b3695c6 100644 --- a/fakenet/diverters/linux.py +++ b/fakenet/diverters/linux.py @@ -3,206 +3,32 @@ import time import socket import logging +import traceback import threading import subprocess -import diverterbase import netfilterqueue from linutil import * +from . import fnpacket +from debuglevels import * from diverterbase import * from collections import namedtuple from netfilterqueue import NetfilterQueue -class PacketHandler: - """Used to encapsulate common patterns in packet hooks.""" +class LinuxPacketCtx(fnpacket.PacketCtx): + def __init__(self, lbl, nfqpkt): + self.nfqpkt = nfqpkt + raw = nfqpkt.get_payload() - def __init__(self, pkt, diverter, label, callbacks3, callbacks4): - self.logger = logging.getLogger('Diverter') - - self.pkt = pkt - self.diverter = diverter # Relies on Diverter for certain operations - self.label = label - self.callbacks3 = callbacks3 - self.callbacks4 = callbacks4 - - self.raw = self.pkt.get_payload() - self.ipver = ((ord(self.raw[0]) & 0xf0) >> 4) - self.hdr, self.proto = self.diverter.parse_pkt[self.ipver](self.ipver, - self.raw) - - def handle_pkt(self): - """Generic packet hook. - - 1.) Common prologue: - A.) Unconditionally Write unmangled packet to pcap - B.) Parse IP packet - - 2.) Call layer 3 (network) callbacks... - - 3.) Parse higher-layer protocol (TCP, UDP) for port numbers - - 4.) Call layer 4 (transport) callbacks... - - 5.) Common epilogue: - A.) If the packet headers have been modified: - i.) Double-write the mangled packet to the pcap for SSL - decoding purposes - ii.) Update the packet payload with NetfilterQueue - B.) Accept the packet with NetfilterQueue - """ - - # 1A: Unconditionally write unmangled packet to pcap - self.diverter.write_pcap(self.hdr.pack()) - - if (self.hdr, self.proto) == (None, None): - self.logger.warning('%s: Failed to parse IP packet' % (self.label)) - else: - proto_name = self.diverter.handled_protocols.get(self.proto) - - self.diverter.pdebug(DGENPKT, '%s %s' % (self.label, - self.diverter.hdr_to_str(proto_name, - self.hdr))) - - # 1B: Parse IP packet (actually done in ctor) - self.src_ip = socket.inet_ntoa(self.hdr.src) - self.dst_ip = socket.inet_ntoa(self.hdr.dst) - - # 2: Call layer 3 (network) callbacks - for cb in self.callbacks3: - # These debug outputs are useful for figuring out which - # callback is responsible for an exception that was masked by - # python-netfilterqueue's global callback. - self.diverter.pdebug(DCB, 'Calling %s' % (cb)) - - cb(self.label, self.hdr, self.ipver, self.proto, proto_name, - self.src_ip, self.dst_ip) - - self.diverter.pdebug(DCB, '%s finished' % (cb)) - - if proto_name: - - if len(self.callbacks4): - # 3: Parse higher-layer protocol - self.sport = self.hdr.data.sport - self.dport = self.hdr.data.dport - self.skey = self.diverter.gen_endpoint_key(proto_name, - self.src_ip, - self.sport) - self.dkey = self.diverter.gen_endpoint_key(proto_name, - self.dst_ip, - self.dport) - - pid, comm = self.diverter.linux_get_pid_comm_by_endpoint( - self.ipver, proto_name, self.src_ip, self.sport) - - if proto_name == 'UDP': - fmt = '| {label} {proto} | {pid:>6} | {comm:<8} | {src:>15}:{sport:<5} | {dst:>15}:{dport:<5} | {length:>5} | {flags:<11} | {seqack:<35} |' - logline = fmt.format( - label=self.label, - proto=proto_name, - pid=pid, - comm=comm, - src=self.src_ip, - sport=self.sport, - dst=self.dst_ip, - dport=self.dport, - length=len(self.raw), - flags='', - seqack='', - ) - self.diverter.pdebug(DGENPKTV, logline) - - elif proto_name == 'TCP': - tcp = self.hdr.data - # Interested in: - # SYN - # SYN,ACK - # ACK - # PSH - # FIN - syn = (tcp.flags & dpkt.tcp.TH_SYN) != 0 - ack = (tcp.flags & dpkt.tcp.TH_ACK) != 0 - fin = (tcp.flags & dpkt.tcp.TH_FIN) != 0 - psh = (tcp.flags & dpkt.tcp.TH_PUSH) != 0 - rst = (tcp.flags & dpkt.tcp.TH_RST) != 0 - - sa = 'Seq=%d, Ack=%d' % (tcp.seq, tcp.ack) - f = [] - if rst: - f.append('RST') - if syn: - f.append('SYN') - if ack: - f.append('ACK') - if fin: - f.append('FIN') - if psh: - f.append('PSH') - - fmt = '| {label} {proto} | {pid:>6} | {comm:<8} | {src:>15}:{sport:<5} | {dst:>15}:{dport:<5} | {length:>5} | {flags:<11} | {seqack:<35} |' - logline = fmt.format( - label=self.label, - proto=proto_name, - pid=pid, - comm=comm, - src=self.src_ip, - sport=self.sport, - dst=self.dst_ip, - dport=self.dport, - length=len(self.raw), - flags=','.join(f), - seqack=sa, - ) - self.diverter.pdebug(DGENPKTV, logline) - - if ((not (self.diverter.pdebug_level & DGENPKTV)) and - pid and (pid != self.diverter.pid)): - self.logger.info(' pid: %d name: %s' % - (pid, comm if comm else 'Unknown')) - - hdr_latest = self.hdr - modified = False - - # 4: Layer 4 (Transport layer) callbacks - for cb in self.callbacks4: - # These debug outputs are useful for figuring out which - # callback is responsible for an exception that was - # masked by python-netfilterqueue's global callback. - self.diverter.pdebug(DCB, 'Calling %s' % (cb)) - - hdr_mod = cb(self.label, pid, comm, self.ipver, - hdr_latest, proto_name, - self.src_ip, self.sport, self.skey, - self.dst_ip, self.dport, self.dkey) - - if hdr_mod: - hdr_latest = hdr_mod - modified = True - - self.diverter.pdebug(DCB, '%s finished' % (cb)) - - if modified: - # 5Ai: Double write mangled packets to represent changes - # made by FakeNet-NG while still allowing SSL decoding - self.diverter.write_pcap(hdr_latest.pack()) - - # 5Aii: Finalize changes with nfq - self.pkt.set_payload(hdr_latest.pack()) - else: - self.diverter.pdebug(DGENPKT, '%s: Not handling protocol %s' % - (self.label, self.proto)) - - # 5B: NF_ACCEPT - self.pkt.accept() + super(LinuxPacketCtx, self).__init__(lbl, raw) class Diverter(DiverterBase, LinUtilMixin): - def __init__(self, diverter_config, listeners_config, ip_addrs, logging_level=logging.INFO): - self.init_base(diverter_config, listeners_config, ip_addrs, - logging_level) + super(Diverter, self).__init__(diverter_config, listeners_config, + ip_addrs, logging_level) self.init_linux_mixin() self.init_diverter_linux() @@ -212,103 +38,24 @@ def init_diverter_linux(self): # String list configuration item that is specific to the Linux # Diverter, will not be parsed by DiverterBase, and needs to be # accessed as an array in the future. - slists = ['linuxredirectnonlocal', 'DebugLevel'] + slists = ['linuxredirectnonlocal', ] self.reconfigure(portlists=[], stringlists=slists) - dbg_lvl = 0 - if self.is_configured('DebugLevel'): - for label in self.getconfigval('DebugLevel'): - label = label.upper() - if label == 'OFF': - dbg_lvl = 0 - break - if not label in DLABELS_INV: - self.logger.warning('No such DebugLevel as %s' % (label)) - else: - dbg_lvl |= DLABELS_INV[label] - self.set_debug_level(dbg_lvl, DLABELS) - - # SingleHost vs MultiHost mode - mode = 'SingleHost' # Default - self.single_host_mode = True - if self.is_configured('networkmode'): - mode = self.getconfigval('networkmode') - available_modes = ['singlehost', 'multihost'] - - # Constrain argument values - if mode.lower() not in available_modes: - self.logger.error('NetworkMode must be one of %s' % - (available_modes)) - sys.exit(1) - - # Adjust previously assumed mode if user specifies MultiHost - if mode.lower() == 'multihost': - self.single_host_mode = False - - if self.single_host_mode: - while True: - prompt = ('You acknowledge that SingleHost mode on Linux is ' + - 'experimental and not functionally complete? ' + - '[Y/N] ') - acknowledgement = raw_input(prompt) - okay = ['y', 'yes', 'yeah', 'sure', 'okay', 'whatever'] - nope = ['n', 'no', 'nah', 'nope'] - if acknowledgement.lower() in okay: - self.logger.info('Okay, we\'ll take it for a spin!') - break - elif acknowledgement.lower() in nope: - self.logger.error('User opted out of crowd-sourced ' + - 'alpha testing program ;-)') - sys.exit(1) - - self.logger.info('Running in %s mode' % (mode)) - - self.parse_pkt = dict() - self.parse_pkt[4] = self.parse_ipv4 - self.parse_pkt[6] = self.parse_ipv6 + self.logger.info('Running in %s mode' % (self.network_mode)) self.nfqueues = list() - self.handled_protocols = { - dpkt.ip.IP_PROTO_TCP: 'TCP', - dpkt.ip.IP_PROTO_UDP: 'UDP', - } - # Track iptables rules not associated with any nfqueue object self.rules_added = [] - # Manage logging of foreign-destined packets - self.nonlocal_ips_already_seen = [] - self.log_nonlocal_only_once = True - - # Port forwarding table, for looking up original unbound service ports - # when sending replies to foreign endpoints that have attempted to - # communicate with unbound ports. Allows fixing up source ports in - # response packets. Similar to the `sessions` member of the Windows - # Diverter implementation. - self.port_fwd_table = dict() - self.port_fwd_table_lock = threading.Lock() - - # Track conversations that will be ignored so that e.g. an RST response - # from a closed port does not erroneously trigger port forwarding and - # silence later replies to legitimate clients. - self.ignore_table = dict() - self.ignore_table_lock = threading.Lock() - - # IP forwarding table, for looking up original foreign destination IPs - # when sending replies to local endpoints that have attempted to - # communicate with other machines e.g. via hard-coded C2 IP addresses. - self.ip_fwd_table = dict() - self.ip_fwd_table_lock = threading.Lock() - # NOTE: Constraining cache size via LRU or similar is a non-requirement # due to the short anticipated runtime of FakeNet-NG. If you see your # FakeNet-NG consuming large amounts of memory, contact your doctor to # find out if Ctrl+C is right for you. - # The below callbacks are configured to be efficiently executed by a - # PacketHandler object within the nonlocal, incoming, and outgoing - # packet hooks installed by the start method. + # The below callbacks are configured to be efficiently executed by the + # handle_pkt method, incoming, and outgoing packet hooks installed by + # the start method. # Network layer callbacks for nonlocal-destined packets # @@ -347,9 +94,7 @@ def init_diverter_linux(self): if self.single_host_mode: self.outgoing_trans_cbs.append(self.maybe_redir_ip) - def start(self): - self.logger.info('Starting Linux Diverter...') - + def startCallback(self): if not self.check_privileged(): self.logger.error('The Linux Diverter requires administrative ' + 'privileges') @@ -415,12 +160,13 @@ def start(self): if self.single_host_mode and self.is_set('modifylocaldns'): self.linux_modifylocaldns_ephemeral() - if self.is_configured('linuxflushdnscommand') and self.single_host_mode: + if (self.is_configured('linuxflushdnscommand') and + self.single_host_mode): cmd = self.getconfigval('linuxflushdnscommand') ret = subprocess.call(cmd.split()) if ret != 0: - self.logger.error( - 'Failed to flush DNS cache. Local machine may use cached DNS results.') + self.logger.error('Failed to flush DNS cache. Local machine ' + 'may use cached DNS results.') if self.is_configured('linuxredirectnonlocal'): self.pdebug(DMISC, 'Processing LinuxRedirectNonlocal') @@ -447,14 +193,14 @@ def start(self): self.rules_added.append(rule) - def stop(self): - self.logger.info('Stopping Linux Diverter...') + return True + def stopCallback(self): self.pdebug(DNFQUEUE, 'Notifying NFQUEUE objects of imminent stop') for q in self.nfqueues: q.stop_nonblocking() - self.pdebug(DIPTBLS, 'Removing iptables rules not associated with any ' + + self.pdebug(DIPTBLS, 'Removing iptables rules not associated with any ' 'NFQUEUE object') self.linux_remove_iptables_rules(self.rules_added) @@ -463,7 +209,7 @@ def stop(self): q.stop() if self.pcap: - self.pdebug(DMISC, 'Closing pcap file %s' % (self.pcap_filename)) + self.pdebug(DPCAP, 'Closing pcap file %s' % (self.pcap_filename)) self.pcap.close() # Only after all queues are stopped self.logger.info('Stopped Linux Diverter') @@ -473,32 +219,32 @@ def stop(self): self.linux_restore_iptables() - def getOriginalDestPort(self, orig_src_ip, orig_src_port, proto): - """Return original destination port, or None if it was not redirected - """ - - orig_src_key = self.gen_endpoint_key(proto, orig_src_ip, orig_src_port) - self.port_fwd_table_lock.acquire() - - try: - if orig_src_key in self.port_fwd_table: - return self.port_fwd_table[orig_src_key] - - return None - finally: - self.port_fwd_table_lock.release() + return True - def handle_nonlocal(self, pkt): + def handle_nonlocal(self, nfqpkt): """Handle comms sent to IP addresses that are not bound to any adapter. This allows analysts to observe when malware is communicating with hard-coded IP addresses in MultiHost mode. """ - h = PacketHandler(pkt, self, 'handle_nonlocal', self.nonlocal_net_cbs, - []) - h.handle_pkt() - - def handle_incoming(self, pkt): + try: + pkt = LinuxPacketCtx('handle_nonlocal', nfqpkt) + self.handle_pkt(pkt, self.nonlocal_net_cbs, []) + if pkt.mangled: + nfqpkt.set_payload(pkt.octets) + # Catch-all exceptions are usually bad practice, agreed, but + # python-netfilterqueue has a catch-all that will not print enough + # information to troubleshoot with, so if there is going to be a + # catch-all exception handler anyway, it might as well be mine so that + # I can print out the stack trace before I lose access to this valuable + # debugging information. + except Exception: + self.logger.error('Exception: %s' % (traceback.format_exc())) + raise + + nfqpkt.accept() + + def handle_incoming(self, nfqpkt): """Incoming packet hook. Specific to incoming packets: @@ -510,11 +256,19 @@ def handle_incoming(self, pkt): No return value. """ - h = PacketHandler(pkt, self, 'handle_incoming', self.incoming_net_cbs, - self.incoming_trans_cbs) - h.handle_pkt() - - def handle_outgoing(self, pkt): + try: + pkt = LinuxPacketCtx('handle_incoming', nfqpkt) + self.handle_pkt(pkt, self.incoming_net_cbs, + self.incoming_trans_cbs) + if pkt.mangled: + nfqpkt.set_payload(pkt.octets) + except Exception: + self.logger.error('Exception: %s' % (traceback.format_exc())) + raise + + nfqpkt.accept() + + def handle_outgoing(self, nfqpkt): """Outgoing packet hook. Specific to outgoing packets: @@ -529,526 +283,40 @@ def handle_outgoing(self, pkt): No return value. """ - h = PacketHandler(pkt, self, 'handle_outgoing', self.outgoing_net_cbs, - self.outgoing_trans_cbs) - h.handle_pkt() - - def parse_ipv4(self, ipver, raw): - hdr = dpkt.ip.IP(raw) - if hdr.hl < 5: - return (None, None) # An IP header length less than 5 is invalid - return hdr, hdr.p - - def parse_ipv6(self, ipver, raw): - hdr = dpkt.ip6.IP6(raw) - return hdr, hdr.nxt - - def gen_endpoint_key(self, proto_name, ip, port): - """e.g. 192.168.19.132:tcp/3030""" - return str(ip) + ':' + str(proto_name) + '/' + str(port) - - def check_log_icmp(self, label, hdr, ipver, proto, proto_name, src_ip, - dst_ip): - if proto == dpkt.ip.IP_PROTO_ICMP: - self.logger.info('ICMP type %d code %d %s' % ( - hdr.data.type, hdr.data.code, self.hdr_to_str(None, hdr))) - - return None - - def check_log_nonlocal(self, label, hdr, ipver, proto, proto_name, src_ip, - dst_ip): - if dst_ip not in self.ip_addrs[ipver]: - self._maybe_log_nonlocal(hdr, ipver, proto, dst_ip) - - return None - - def _maybe_log_nonlocal(self, hdr, ipver, proto, dst_ip): + try: + pkt = LinuxPacketCtx('handle_outgoing', nfqpkt) + self.handle_pkt(pkt, self.outgoing_net_cbs, + self.outgoing_trans_cbs) + if pkt.mangled: + nfqpkt.set_payload(pkt.octets) + except Exception: + self.logger.error('Exception: %s' % (traceback.format_exc())) + raise + + nfqpkt.accept() + + def check_log_nonlocal(self, crit, pkt): """Conditionally log packets having a foreign destination. Each foreign destination will be logged only once if the Linux Diverter's internal log_nonlocal_only_once flag is set. Otherwise, any foreign destination IP address will be logged each time it is observed. """ - proto_name = self.handled_protocols.get(proto) - - self.pdebug(DNONLOC, 'Nonlocal %s' % - (self.hdr_to_str(proto_name, hdr))) - first_sighting = (dst_ip not in self.nonlocal_ips_already_seen) + if pkt.dst_ip not in self.ip_addrs[pkt.ipver]: + self.pdebug(DNONLOC, 'Nonlocal %s' % pkt.hdrToStr()) + first_sighting = (pkt.dst_ip not in self.nonlocal_ips_already_seen) + if first_sighting: + self.nonlocal_ips_already_seen.append(pkt.dst_ip) + # Log when a new IP is observed OR if we are not restricted to + # logging only the first occurrence of a given nonlocal IP. + if first_sighting or (not self.log_nonlocal_only_once): + self.logger.info( + 'Received nonlocal IPv%d datagram destined for %s' % + (pkt.ipver, pkt.dst_ip)) - if first_sighting: - self.nonlocal_ips_already_seen.append(dst_ip) - - # Log when a new IP is observed OR if we are not restricted to - # logging only the first occurrence of a given nonlocal IP. - if first_sighting or (not self.log_nonlocal_only_once): - self.logger.info( - 'Received nonlocal IPv%d datagram destined for %s' % - (ipver, dst_ip)) - - def check_should_ignore(self, pid, comm, ipver, hdr, proto_name, src_ip, - sport, dst_ip, dport): - # SingleHost mode checks - if self.single_host_mode: - if comm: - if comm in self.blacklist_processes: - self.pdebug(DIGN, ('Ignoring %s packet from process %s ' + - 'in the process blacklist.') % (proto_name, - comm)) - self.pdebug(DIGN, ' %s' % - (self.hdr_to_str(proto_name, hdr))) - return True - - elif (len(self.whitelist_processes) and (comm not in - self.whitelist_processes)): - self.pdebug(DIGN, ('Ignoring %s packet from process %s ' + - 'not in the process whitelist.') % (proto_name, - comm)) - self.pdebug(DIGN, ' %s' % - (self.hdr_to_str(proto_name, hdr))) - return True - - # Check per-listener blacklisted process list - elif ((proto_name in self.port_process_blacklist) and - (dport in self.port_process_blacklist[proto_name])): - # If program DOES match blacklist - if comm in self.port_process_blacklist[proto_name][dport]: - self.pdebug(DIGN, ('Ignoring %s request packet from ' + - 'process %s in the listener process ' + - 'blacklist.') % (proto_name, comm)) - self.pdebug(DIGN, ' %s' % - (self.hdr_to_str(proto_name, hdr))) - - return True - - # Check per-listener whitelisted process list - elif ((proto_name in self.port_process_whitelist) and - (dport in self.port_process_whitelist[proto_name])): - # If program does NOT match whitelist - if not comm in self.port_process_whitelist[proto_name][dport]: - self.pdebug(DIGN, ('Ignoring %s request packet from ' + - 'process %s not in the listener process ' + - 'whitelist.') % (proto_name, comm)) - self.pdebug(DIGN, ' %s' % - (self.hdr_to_str(proto_name, hdr))) - return True - - # MultiHost mode checks - else: - pass # None as of yet - - # Checks independent of mode - - if set(self.blacklist_ports[proto_name]).intersection([sport, dport]): - self.pdebug(DIGN, 'Forwarding blacklisted port %s packet:' % - (proto_name)) - self.pdebug(DIGN, ' %s' % (self.hdr_to_str(proto_name, hdr))) - return True - - global_host_blacklist = self.getconfigval('hostblacklist') - if global_host_blacklist and dst_ip in global_host_blacklist: - self.pdebug(DIGN, ('Ignoring %s packet to %s in the host ' + - 'blacklist.') % (proto_name, dst_ip)) - self.pdebug(DIGN, ' %s' % (self.hdr_to_str(proto_name, hdr))) - return True - - if ((proto_name in self.port_host_whitelist) and - (dport in self.port_host_whitelist[proto_name])): - # If host does NOT match whitelist - if not dst_ip in self.port_host_whitelist[proto_name][dport]: - self.pdebug(DIGN, ('Ignoring %s request packet to %s not in ' + - 'the listener host whitelist.') % (proto_name, - dst_ip)) - self.pdebug(DIGN, ' %s' % (self.hdr_to_str(proto_name, hdr))) - return True - - if ((proto_name in self.port_host_blacklist) and - (dport in self.port_host_blacklist[proto_name])): - # If host DOES match blacklist - if dst_ip in self.port_host_blacklist[proto_name][dport]: - self.pdebug(DIGN, ('Ignoring %s request packet to %s in the ' + - 'listener host blacklist.') % (proto_name, dst_ip)) - self.pdebug(DIGN, ' %s' % (self.hdr_to_str(proto_name, hdr))) - return True - - # Duplicated from diverters/windows.py: - # HACK: FTP Passive Mode Handling - # Check if a listener is initiating a new connection from a - # non-diverted port and add it to blacklist. This is done to handle a - # special use-case of FTP ACTIVE mode where FTP server is initiating a - # new connection for which the response may be redirected to a default - # listener. NOTE: Additional testing can be performed to check if this - # is actually a SYN packet - if pid == self.pid: - if ( - ((dst_ip in self.ip_addrs[ipver]) and - (not dst_ip.startswith('127.'))) and - ((src_ip in self.ip_addrs[ipver]) and - (not dst_ip.startswith('127.'))) and - (not set([sport, dport]).intersection(self.diverted_ports[proto_name])) - ): - - self.pdebug(DIGN | DFTP, 'Listener initiated %s connection' % - (proto_name)) - self.pdebug(DIGN | DFTP, ' %s' % (self.hdr_to_str(proto_name, hdr))) - self.pdebug(DIGN | DFTP, ' Blacklisting port %d' % (sport)) - self.blacklist_ports[proto_name].append(sport) - - return True - - return False - - def maybe_redir_ip(self, label, pid, comm, ipver, hdr, proto_name, src_ip, - sport, skey, dst_ip, dport, dkey): - """Conditionally redirect foreign destination IPs to localhost. - - Used only under SingleHost mode. - - Returns: - None - if unmodified - dpkt.ip.hdr - if modified - """ - hdr_modified = None - - if self.check_should_ignore(pid, comm, ipver, hdr, proto_name, src_ip, - sport, dst_ip, dport): - return hdr_modified # None - - self.pdebug(DIPNAT, 'Condition 1 test') - # Condition 1: If the remote IP address is foreign to this system, - # then redirect it to a local IP address. - if self.single_host_mode and (dst_ip not in self.ip_addrs[ipver]): - self.pdebug(DIPNAT, 'Condition 1 satisfied') - self.ip_fwd_table_lock.acquire() - try: - self.ip_fwd_table[skey] = dst_ip - - finally: - self.ip_fwd_table_lock.release() - - newdst = '127.0.0.1' - hdr_modified = self.mangle_dstip(hdr, proto_name, dst_ip, newdst) - - else: - # Delete any stale entries in the IP forwarding table: If the - # local endpoint appears to be reusing a client port that was - # formerly used to connect to a foreign host (but not anymore), - # then remove the entry. This prevents a packet hook from - # faithfully overwriting the source IP on a later packet to - # conform to the foreign endpoint's stale connection IP when - # the host is reusing the port number to connect to an IP - # address that is local to the FakeNet system. - - self.ip_fwd_table_lock.acquire() - try: - if skey in self.ip_fwd_table: - self.pdebug(DIPNAT, ' - DELETING ipfwd key entry: ' + skey) - del self.ip_fwd_table[skey] - finally: - self.ip_fwd_table_lock.release() - - return hdr_modified - - def maybe_fixup_srcip(self, label, pid, comm, ipver, hdr, proto_name, - src_ip, sport, skey, dst_ip, dport, dkey): - """Conditionally fix up the source IP address if the remote endpoint - had their connection IP-forwarded. - - Check is based on whether the remote endpoint corresponds to a key in - the IP forwarding table. - - Returns: - None - if unmodified - dpkt.ip.hdr - if modified - """ - hdr_modified = None - - # Condition 4: If the local endpoint (IP/port/proto) combo - # corresponds to an endpoint that initiated a conversation with a - # foreign endpoint in the past, then fix up the source IP for this - # incoming packet with the last destination IP that was requested - # by the endpoint. - self.pdebug(DIPNAT, "Condition 4 test: was remote endpoint IP fwd'd?") - self.ip_fwd_table_lock.acquire() - try: - if self.single_host_mode and (dkey in self.ip_fwd_table): - self.pdebug(DIPNAT, 'Condition 4 satisfied') - self.pdebug(DIPNAT, ' = FOUND ipfwd key entry: ' + dkey) - new_srcip = self.ip_fwd_table[dkey] - hdr_modified = self.mangle_srcip( - hdr, proto_name, hdr.src, new_srcip) - else: - self.pdebug(DIPNAT, ' ! NO SUCH ipfwd key entry: ' + dkey) - finally: - self.ip_fwd_table_lock.release() - - return hdr_modified - - def maybe_redir_port(self, label, pid, comm, ipver, hdr, proto_name, - src_ip, sport, skey, dst_ip, dport, dkey): - hdr_modified = None - - # Get default listener port for this proto, or bail if none - default = None - if not proto_name in self.default_listener: - return hdr_modified # None - default = self.default_listener[proto_name] - - # Pre-condition 1: RedirectAllTraffic: Yes - # NOTE: This only applies to port redirection in the Windows Diverter; - # IP addresses will be modified by the Windows Diverter when - # RedirectAllTraffic is disabled. So, the Linux Diverter implementation - # will follow suit. - if not self.is_set('redirectalltraffic'): - self.pdebug(DIGN, 'Ignoring %s packet %s' % - (proto_name, self.hdr_to_str(proto_name, hdr))) - return hdr_modified # None - - # Pre-condition 1: destination must not be present in port forwarding - # table (prevents masqueraded ports responding to unbound ports from - # being mistaken as starting a conversation with an unbound port). - found = False - self.port_fwd_table_lock.acquire() - try: - # Uses dkey to cross-reference - found = dkey in self.port_fwd_table - finally: - self.port_fwd_table_lock.release() - - if found: - return hdr_modified # None - - bound_ports = self.diverted_ports.get(proto_name, []) - - # First, check if this packet is sent from a listener/diverter - # If so, don't redir for 'Hidden' status because it is already - # being forwarded from proxy listener to bound/hidden listener - # Next, check if listener for this port is 'Hidden'. If so, we need to - # divert it to the proxy as per the Hidden config - if (dport in bound_ports and pid != self.pid and - bound_ports[dport] is True): - - #divert to proxy - hdr_modified = self.mangle_dstport(hdr, proto_name, dport, default) - - # Record the foreign endpoint and old destination port in the port - # forwarding table - self.pdebug(DDPFV, ' + ADDING portfwd key entry: ' + skey) - self.port_fwd_table_lock.acquire() - try: - self.port_fwd_table[skey] = dport - finally: - self.port_fwd_table_lock.release() - - # Record the altered port for making the ExecuteCmd decision - dport = default - - # Condition 2: If the packet is destined for an unbound port, then - # redirect it to a bound port and save the old destination IP in - # the port forwarding table keyed by the source endpoint identity. - - elif self.decide_redir_port(ipver, proto_name, default, bound_ports, - src_ip, sport, dst_ip, dport): - self.pdebug(DDPFV, 'Condition 2 satisfied') - - # Post-condition 1: General ignore conditions are not met, or this - # is part of a conversation that is already being ignored. - # - # Placed after the decision to redirect for three reasons: - # 1.) We want to ensure that the else condition below has a chance - # to check whether to delete a stale port forwarding table - # entry. - # 2.) Checking these conditions is, on average, more expensive than - # checking if the packet would be redirected in the first - # place. - # 3.) Reporting of packets that are being ignored (i.e. not - # redirected), which is integrated into this check, should only - # appear when packets would otherwise have been redirected. - - # Is this conversation already being ignored for DPF purposes? - self.ignore_table_lock.acquire() - try: - if dkey in self.ignore_table and self.ignore_table[dkey] == sport: - # This is a reply (e.g. a TCP RST) from the - # non-port-forwarded server that the non-port-forwarded - # client was trying to talk to. Leave it alone. - return hdr_modified # None - finally: - self.ignore_table_lock.release() - - if self.check_should_ignore(pid, comm, ipver, hdr, proto_name, - src_ip, sport, dst_ip, dport): - self.ignore_table_lock.acquire() - try: - self.ignore_table[skey] = dport - finally: - self.ignore_table_lock.release() - return hdr_modified # None - - # Record the foreign endpoint and old destination port in the port - # forwarding table - self.pdebug(DDPFV, ' + ADDING portfwd key entry: ' + skey) - self.port_fwd_table_lock.acquire() - try: - self.port_fwd_table[skey] = dport - finally: - self.port_fwd_table_lock.release() - - hdr_modified = self.mangle_dstport(hdr, proto_name, dport, default) - - # Record the altered port for making the ExecuteCmd decision - dport = default - - else: - # Delete any stale entries in the port forwarding table: If the - # foreign endpoint appears to be reusing a client port that was - # formerly used to connect to an unbound port on this server, - # remove the entry. This prevents the OUTPUT or other packet - # hook from faithfully overwriting the source port to conform - # to the foreign endpoint's stale connection port when the - # foreign host is reusing the port number to connect to an - # already-bound port on the FakeNet system. - - self.delete_stale_port_fwd_key(skey) - - if not (sport in self.sessions and self.sessions[sport] == (dst_ip, - dport)): - self.sessions[sport] = (dst_ip, dport) - - if pid and (dst_ip in self.ip_addrs[ipver]): - cmd = self.build_cmd(proto_name, pid, comm, src_ip, - sport, dst_ip, dport) - if cmd: - self.logger.info('Executing command: %s', cmd) - self.execute_detached(cmd) - - return hdr_modified - - def delete_stale_port_fwd_key(self, skey): - self.port_fwd_table_lock.acquire() - try: - if skey in self.port_fwd_table: - self.pdebug(DDPFV, ' - DELETING portfwd key entry: ' + skey) - del self.port_fwd_table[skey] - finally: - self.port_fwd_table_lock.release() - - def maybe_fixup_sport(self, label, pid, comm, ipver, hdr, proto_name, - src_ip, sport, skey, dst_ip, dport, dkey): - """Conditionally fix up source port if the remote endpoint had their - connection port-forwarded. - - Check is based on whether the remote endpoint corresponds to a key in - the port forwarding table. - - Returns: - None - if unmodified - dpkt.ip.hdr - if modified - """ - hdr_modified = None - - # Condition 3: If the remote endpoint (IP/port/proto) combo - # corresponds to an endpoint that initiated a conversation with an - # unbound port in the past, then fix up the source port for this - # outgoing packet with the last destination port that was requested - # by that endpoint. The term "endpoint" is (ab)used loosely here to - # apply to UDP host/port/proto combos and any other protocol that - # may be supported in the future. - self.pdebug(DDPFV, "Condition 3 test: was remote endpoint port fwd'd?") - self.port_fwd_table_lock.acquire() - try: - if dkey in self.port_fwd_table: - self.pdebug(DDPFV, 'Condition 3 satisfied: must fix up ' + - 'source port') - self.pdebug(DDPFV, ' = FOUND portfwd key entry: ' + dkey) - new_sport = self.port_fwd_table[dkey] - hdr_modified = self.mangle_srcport( - hdr, proto_name, hdr.data.sport, new_sport) - else: - self.pdebug(DDPFV, ' ! NO SUCH portfwd key entry: ' + dkey) - finally: - self.port_fwd_table_lock.release() - - return hdr_modified - - def decide_redir_port(self, ipver, proto_name, default_port, bound_ports, - src_ip, sport, dst_ip, dport): - """Decide whether to redirect a port. - - Optimized logic derived by truth table + k-map. See docs/internals.md - for details. - """ - # A, B, C, D for easy manipulation; full names for readability only. - a = src_local = (src_ip in self.ip_addrs[ipver]) - c = sport_bound = sport in (bound_ports) - d = dport_bound = dport in (bound_ports) - - if self.pdebug_level & DDPFV: - # Unused logic term not calculated except for debug output - b = dst_local = (dst_ip in self.ip_addrs[ipver]) - - self.pdebug(DDPFV, 'src %s (%s)' % - (str(src_ip), ['foreign', 'local'][a])) - self.pdebug(DDPFV, 'dst %s (%s)' % - (str(dst_ip), ['foreign', 'local'][b])) - self.pdebug(DDPFV, 'sport %s (%sbound)' % - (str(sport), ['un', ''][c])) - self.pdebug(DDPFV, 'dport %s (%sbound)' % - (str(dport), ['un', ''][d])) - - def bn(x): return '1' if x else '0' # Bool -> binary - self.pdebug(DDPFV, 'abcd = ' + bn(a) + bn(b) + bn(c) + bn(d)) - - return (not a and not d) or (not c and not d) - - def mangle_dstip(self, hdr, proto_name, dstip, newdstip): - """Mangle destination IP for selected outgoing packets.""" - self.pdebug(DIPNAT, 'REDIRECTING %s to IP %s' % - (self.hdr_to_str(proto_name, hdr), newdstip)) - hdr.dst = socket.inet_aton(newdstip) - self._calc_csums(hdr) - return hdr - - def mangle_srcip(self, hdr, proto_name, src_ip, new_srcip): - """Mangle source IP for selected incoming packets.""" - self.pdebug(DIPNAT, 'MASQUERADING %s from IP %s' % - (self.hdr_to_str(proto_name, hdr), new_srcip)) - hdr.src = socket.inet_aton(new_srcip) - self._calc_csums(hdr) - return hdr - - def mangle_dstport(self, hdr, proto_name, dstport, newdstport): - """Mangle destination port for selected incoming packets.""" - self.pdebug(DDPF, 'REDIRECTING %s to port %d' % - (self.hdr_to_str(proto_name, hdr), newdstport)) - hdr.data.dport = newdstport - self._calc_csums(hdr) - return hdr - - def mangle_srcport(self, hdr, proto_name, srcport, newsrcport): - """Mangle source port for selected outgoing packets.""" - self.pdebug(DDPF, 'MASQUERADING %s from port %d' % - (self.hdr_to_str(proto_name, hdr), newsrcport)) - hdr.data.sport = newsrcport - self._calc_csums(hdr) - return hdr - - def hdr_to_str(self, proto_name, hdr): - src_ip = socket.inet_ntoa(hdr.src) - dst_ip = socket.inet_ntoa(hdr.dst) - if proto_name: - return '%s %s:%d->%s:%d' % (proto_name, src_ip, hdr.data.sport, - dst_ip, hdr.data.dport) - else: - return '%s->%s' % (src_ip, dst_ip) - - def _calc_csums(self, hdr): - """The roundabout dance of inducing dpkt to recalculate checksums.""" - hdr.sum = 0 - hdr.data.sum = 0 - str(hdr) # This has the side-effect of invoking dpkt.in_cksum() et al + return None if __name__ == '__main__': - logging.basicConfig(format='%(message)s') - diverterbase.test_redir_logic(Diverter) + raise NotImplementedError diff --git a/fakenet/diverters/windows.py b/fakenet/diverters/windows.py old mode 100755 new mode 100644 index 4f5d18c..e98b346 --- a/fakenet/diverters/windows.py +++ b/fakenet/diverters/windows.py @@ -8,432 +8,157 @@ import os import dpkt +from . import fnpacket import time import threading import platform from winutil import * +from diverterbase import * import subprocess -class Diverter(WinUtilMixin): - def __init__(self, diverter_config, listeners_config, logging_level = logging.INFO): +class WindowsPacketCtx(fnpacket.PacketCtx): + def __init__(self, lbl, wdpkt): + self.wdpkt = wdpkt + raw = wdpkt.raw.tobytes() - self.logger = logging.getLogger('Diverter') - self.logger.setLevel(logging_level) + super(WindowsPacketCtx, self).__init__(lbl, raw) - self.diverter_config = diverter_config - self.listeners_config = listeners_config + # Packet mangling properties are extended here to also write the data to + # the pydivert.Packet object. This is because there appears to be no way to + # populate the pydivert.Packet object with plain octets unless you can also + # provide @interface and @direction arguments which do not appear at a + # glance to be directly available as attributes of pydivert.Packet, + # according to https://ffalcinelli.github.io/pydivert/ + # + # Perhaps we can get these from wd_addr? - # Local IP addresses - self.external_ip = None - self.loopback_ip = None + # src_ip overrides - # Used for caching of DNS server names prior to changing - self.adapters_dns_server_backup = dict() + @property + def src_ip(self): + return self._src_ip - # Used to restore modified Interfaces back to DHCP - self.adapters_dhcp_restore = list() - self.adapters_dns_restore = list() + @src_ip.setter + def src_ip(self, new_srcip): + super(self.__class__, self.__class__).src_ip.fset(self, new_srcip) + self.wdpkt.src_addr = new_srcip - # Restore Npcap loopback adapter - self.restore_npcap_loopback = False + # dst_ip overrides - # Sessions cache - # NOTE: A dictionary of source ports mapped to destination address, port tuples - self.sessions = dict() + @property + def dst_ip(self): + return self._dst_ip - ####################################################################### - # Listener specific configuration - # NOTE: All of these definitions have protocol as the first key - # followed by a list or another nested dict with the actual definitions + @dst_ip.setter + def dst_ip(self, new_dstip): + super(self.__class__, self.__class__).dst_ip.fset(self, new_dstip) + self.wdpkt.dst_addr = new_dstip - # Diverted ports - self.diverted_ports = dict() + # sport overrides - # Listener Port Process filtering - # TODO: Allow PIDs - self.port_process_whitelist = dict() - self.port_process_blacklist = dict() + @property + def sport(self): + return self._sport - # Listener Port Host filtering - # TODO: Allow domain name resolution - self.port_host_whitelist = dict() - self.port_host_blacklist = dict() + @sport.setter + def sport(self, new_sport): + super(self.__class__, self.__class__).sport.fset(self, new_sport) + if self.proto: + self.wdpkt.src_port = new_sport - # Execute command list - self.port_execute = dict() + # dport overrides - # Parse listener configurations - self.parse_listeners_config(listeners_config) + @property + def dport(self): + return self._dport - ####################################################################### - # Diverter settings and filters + @dport.setter + def dport(self, new_dport): + super(self.__class__, self.__class__).dport.fset(self, new_dport) + if self.proto: + self.wdpkt.dst_port = new_dport - # Intercept filter - # NOTE: All relevant connections are recorded as outbound by WinDivert - # so additional filtering based on destination port will need to be - # performed in order to determine the correct traffic direction. - self.filter = None - # Default TCP/UDP listeners - self.default_listener_tcp_port = None - self.default_listener_udp_port = None +class Diverter(DiverterBase, WinUtilMixin): - # Global TCP/UDP port blacklist - self.blacklist_ports_tcp = [] - self.blacklist_ports_udp = [] + def __init__(self, diverter_config, listeners_config, ip_addrs, + logging_level=logging.INFO): - # Global process blacklist - # TODO: Allow PIDs - self.blacklist_processes = [] - - # Global host blacklist - # TODO: Allow domain resolution - self.blacklist_hosts = [] + # Populated by winutil and used to restore modified Interfaces back to + # DHCP + self.adapters_dhcp_restore = list() + self.adapters_dns_restore = list() - # Parse diverter config - self.parse_diverter_config() + super(Diverter, self).__init__(diverter_config, listeners_config, + ip_addrs, logging_level) - ####################################################################### - # Network verification + self.running_on_windows = True - # Check active interfaces - if not self.check_active_ethernet_adapters(): - self.logger.warning('ERROR: No active ethernet interfaces detected!') - self.logger.warning(' Please enable a network interface.') + if not self.single_host_mode: + self.logger.error('Windows diverter currently only supports ' + 'SingleHost mode') sys.exit(1) - # Check configured ip addresses - if not self.check_ipaddresses(): - self.logger.warning('ERROR: No interface had IP address configured!') - self.logger.warning(' Please configure an IP address on a network interfnace.') - sys.exit(1) - - # Check configured gateways - if not self.check_gateways(): - self.logger.warning('WARNING: No gateways configured!') - - # Check if there is a gateway configured on any of the Ethernet interfaces. If that's not the case, - # then locate configured IP address and set a gateway automatically. This is necessary for VMWare Host-Only - # DHCP server which leaves default gateway empty. - if self.diverter_config.get('fixgateway') and self.diverter_config['fixgateway'].lower() == 'yes': - - for adapter in self.get_adapters_info(): - - # Look for a DHCP interface with a set IP address but no gateway (Host-Only) - if self.check_ipaddresses_interface(adapter) and adapter.DhcpEnabled: - - (ip_address, netmask) = next(self.get_ipaddresses_netmask(adapter)) - gw_address = ip_address[:ip_address.rfind('.')]+'.254' - - interface_name = self.get_adapter_friendlyname(adapter.Index) - - # Don't set gateway on loopback interfaces (e.g. Npcap Loopback Adapter) - if not "loopback" in interface_name.lower(): - - self.adapters_dhcp_restore.append(interface_name) - - cmd_set_gw = "netsh interface ip set address name=\"%s\" static %s %s %s" % (interface_name, ip_address, netmask, gw_address) - - # Configure gateway - try: - subprocess.check_call(cmd_set_gw, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) - except subprocess.CalledProcessError, e: - self.logger.error(" Failed to set gateway %s on interface %s." % (gw_address, interface_name)) - else: - self.logger.info(" Setting gateway %s on interface %s" % (gw_address, interface_name)) - - - else: - self.logger.warning('WARNING: Please configure a default gateway or route in order to intercept external traffic.') - self.logger.warning(' Current interception abilities are limited to local traffic.') + # Used (by winutil) for caching of DNS server names prior to changing + self.adapters_dns_server_backup = dict() # Configure external and loopback IP addresses - self.external_ip = self.get_best_ipaddress() or self.get_ip_with_gateway() or socket.gethostbyname(socket.gethostname()) - self.loopback_ip = socket.gethostbyname('localhost') - - self.logger.info("External IP: %s Loopback IP: %s" % (self.external_ip, self.loopback_ip)) - - # Check configured DNS servers - if not self.check_dns_servers(): - self.logger.warning('WARNING: No DNS servers configured!') + self.external_ip = self.get_best_ipaddress() + if not self.external_ip: + self.external_ip = self.get_ip_with_gateway() + if not self.external_ip: + self.external_ip = socket.gethostbyname(socket.gethostname()) - # Check if there is a DNS server on any of the Ethernet interfaces. If that's not the case, - # then locate configured IP address and set a DNS server automatically. - if self.diverter_config.get('fixdns') and self.diverter_config['fixdns'].lower() == 'yes': + self.logger.info('External IP: %s Loopback IP: %s' % + (self.external_ip, self.loopback_ip)) - for adapter in self.get_adapters_info(): - - if self.check_ipaddresses_interface(adapter): - - ip_address = next(self.get_ipaddresses(adapter)) - dns_address = ip_address - - interface_name = self.get_adapter_friendlyname(adapter.Index) - - # Don't set DNS on loopback interfaces (e.g. Npcap Loopback Adapter) - if not "loopback" in interface_name.lower(): - - self.adapters_dns_restore.append(interface_name) - - cmd_set_dns = "netsh interface ip set dns name=\"%s\" static %s" % (interface_name, dns_address) - - # Configure DNS server - try: - subprocess.check_call(cmd_set_dns, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) - except subprocess.CalledProcessError, e: - self.logger.error(" Failed to set DNS %s on interface %s." % (dns_address, interface_name)) - else: - self.logger.info(" Setting DNS %s on interface %s" % (dns_address, interface_name)) + ####################################################################### + # Initialize filter and WinDivert driver - else: - self.logger.warning('WARNING: Please configure a DNS server in order to intercept domain name resolutions.') - self.logger.warning(' Current interception abilities are limited to IP only traffic.') + # Interpose on all IP datagrams so they appear in the pcap, let + # DiverterBase decide whether they're actually forwarded etc. + self.filter = 'outbound and ip' - ####################################################################### # Initialize WinDivert - try: self.handle = WinDivert(filter=self.filter) self.handle.open() except WindowsError, e: if e.winerror == 5: - self.logger.error('ERROR: Insufficient privileges to run windows diverter.') - self.logger.error(' Please restart with Administrator privileges.') + self.logger.error('ERROR: Insufficient privileges to run ' + 'windows diverter.') + self.logger.error(' Please restart with Administrator ' + 'privileges.') sys.exit(1) elif e.winerror == 3: - self.logger.error('ERROR: Could not locate WinDivert DLL or one of its components.') - self.logger.error(' Please make sure you have copied FakeNet-NG to the C: drive.') + self.logger.error('ERROR: Could not locate WinDivert DLL or ' + 'one of its components.') + self.logger.error(' Please make sure you have copied ' + 'FakeNet-NG to the C: drive.') sys.exit(1) else: - self.logger.error('ERROR: Failed to open a handle to the WinDivert driver: %s', e) + self.logger.error('ERROR: Failed to open a handle to the ' + 'WinDivert driver: %s', e) sys.exit(1) - # Capture packets configuration - self.capture_flag = False - self.dump_packets_file_prefix = "packets" - self.pcap = None - - if self.diverter_config.get('dumppackets') and self.diverter_config['dumppackets'].lower() == 'yes': - self.capture_flag = True - pcap_filename = "%s_%s.pcap" % (diverter_config.get('dumppacketsfileprefix', 'packets'), time.strftime("%Y%m%d_%H%M%S")) - - self.logger.info('Capturing traffic to %s', pcap_filename) - self.pcap = dpkt.pcap.Writer(open(pcap_filename, 'wb'), linktype=dpkt.pcap.DLT_RAW) - - def getOriginalDestPort(self, orig_src_ip, orig_src_port, proto): - """Return original destination port, or None if it was not redirected - """ - - if orig_src_port in self.sessions: - return self.sessions[orig_src_port] - return None - - ########################################################################### - # Parse listener specific settings and filters - - def parse_listeners_config(self, listeners_config): - - ####################################################################### - # Populate diverter ports and process filters from the configuration - for listener_name, listener_config in listeners_config.iteritems(): - - if 'port' in listener_config: - - port = int(listener_config['port']) - hidden = True if ('hidden' in listener_config and - listener_config['hidden'] == 'True') else False - - if not 'protocol' in listener_config: - self.logger.error('ERROR: Protocol not defined for listener %s', listener_name) - sys.exit(1) - - protocol = listener_config['protocol'].upper() - - if not protocol in ['TCP', 'UDP']: - self.logger.error('ERROR: Invalid protocol %s for listener %s', protocol, listener_name) - sys.exit(1) - - if not protocol in self.diverted_ports: - self.diverted_ports[protocol] = dict() - - self.diverted_ports[protocol][port] = hidden - - ############################################################### - # Process filtering configuration - if 'processwhitelist' in listener_config and 'processblacklist' in listener_config: - self.logger.error('ERROR: Listener can\'t have both process whitelist and blacklist.') - sys.exit(1) - - elif 'processwhitelist' in listener_config: - - self.logger.debug('Process whitelist:') - - if not protocol in self.port_process_whitelist: - self.port_process_whitelist[protocol] = dict() - - self.port_process_whitelist[protocol][port] = [process.strip() for process in listener_config['processwhitelist'].split(',')] - - for port in self.port_process_whitelist[protocol]: - self.logger.debug(' Port: %d (%s) Processes: %s', port, protocol, ', '.join(self.port_process_whitelist[protocol][port])) - - elif 'processblacklist' in listener_config: - self.logger.debug('Process blacklist:') - - if not protocol in self.port_process_blacklist: - self.port_process_blacklist[protocol] = dict() - - self.port_process_blacklist[protocol][port] = [process.strip() for process in listener_config['processblacklist'].split(',')] - - for port in self.port_process_blacklist[protocol]: - self.logger.debug(' Port: %d (%s) Processes: %s', port, protocol, ', '.join(self.port_process_blacklist[protocol][port])) - - ############################################################### - # Host filtering configuration - if 'hostwhitelist' in listener_config and 'hostblacklist' in listener_config: - self.logger.error('ERROR: Listener can\'t have both host whitelist and blacklist.') - sys.exit(1) - - elif 'hostwhitelist' in listener_config: - - self.logger.debug('Host whitelist:') - - if not protocol in self.port_host_whitelist: - self.port_host_whitelist[protocol] = dict() - - self.port_host_whitelist[protocol][port] = [host.strip() for host in listener_config['hostwhitelist'].split(',')] - - for port in self.port_host_whitelist[protocol]: - self.logger.debug(' Port: %d (%s) Hosts: %s', port, protocol, ', '.join(self.port_host_whitelist[protocol][port])) - - elif 'hostblacklist' in listener_config: - self.logger.debug('Host blacklist:') - - if not protocol in self.port_host_blacklist: - self.port_host_blacklist[protocol] = dict() - - self.port_host_blacklist[protocol][port] = [host.strip() for host in listener_config['hostblacklist'].split(',')] - - for port in self.port_host_blacklist[protocol]: - self.logger.debug(' Port: %d (%s) Hosts: %s', port, protocol, ', '.join(self.port_host_blacklist[protocol][port])) - - ############################################################### - # Execute command configuration - if 'executecmd' in listener_config: - - if not protocol in self.port_execute: - self.port_execute[protocol] = dict() - - self.port_execute[protocol][port] = listener_config['executecmd'].strip() - self.logger.debug('Port %d (%s) ExecuteCmd: %s', port, protocol, self.port_execute[protocol][port] ) - - ########################################################################### - # Parse diverter settings and filters - - def expand_ports(self, ports_list): - ports = [] - for i in ports_list.split(','): - if '-' not in i: - ports.append(int(i)) - else: - l,h = map(int, i.split('-')) - ports+= range(l,h+1) - return ports - - def parse_diverter_config(self): - if self.diverter_config.get('networkmode').lower() != 'singlehost': - self.logger.error('Windows diverter currently only supports SingleHost mode') - sys.exit(1) - - # Do not redirect blacklisted processes - if self.diverter_config.get('processblacklist') != None: - self.blacklist_processes = [process.strip() for process in self.diverter_config.get('processblacklist').split(',')] - self.logger.debug('Blacklisted processes: %s', ', '.join([str(p) for p in self.blacklist_processes])) - - # Do not redirect blacklisted hosts - if self.diverter_config.get('hostblacklist') != None: - self.blacklist_hosts = [host.strip() for host in self.diverter_config.get('hostblacklist').split(',')] - self.logger.debug('Blacklisted hosts: %s', ', '.join([str(p) for p in self.blacklist_hosts])) - - # Redirect all traffic - if self.diverter_config.get('redirectalltraffic') and self.diverter_config['redirectalltraffic'].lower() == 'yes': - self.filter = "outbound and ip and (icmp or tcp or udp)" - - if self.diverter_config.get('defaulttcplistener') == None: - self.logger.error('ERROR: No default TCP listener specified in the configuration.') - sys.exit(1) - - elif self.diverter_config.get('defaultudplistener') == None: - self.logger.error('ERROR: No default UDP listener specified in the configuration.') - sys.exit(1) - - elif not self.diverter_config.get('defaulttcplistener') in self.listeners_config: - self.logger.error('ERROR: No configuration exists for default TCP listener %s', self.diverter_config.get('defaulttcplistener')) - sys.exit(1) - - elif not self.diverter_config.get('defaultudplistener') in self.listeners_config: - self.logger.error('ERROR: No configuration exists for default UDP listener %s', self.diverter_config.get('defaultudplistener')) - sys.exit(1) - - else: - self.default_listener_tcp_port = int( self.listeners_config[ self.diverter_config['defaulttcplistener'] ]['port'] ) - self.logger.error('Using default listener %s on port %d', self.diverter_config['defaulttcplistener'], self.default_listener_tcp_port) - - self.default_listener_udp_port = int( self.listeners_config[ self.diverter_config['defaultudplistener'] ]['port'] ) - self.logger.error('Using default listener %s on port %d', self.diverter_config['defaultudplistener'], self.default_listener_udp_port) - - # Do not redirect blacklisted TCP ports - if self.diverter_config.get('blacklistportstcp') != None: - self.blacklist_ports_tcp = self.expand_ports(self.diverter_config.get('blacklistportstcp')) - self.logger.debug('Blacklisted TCP ports: %s', ', '.join([str(p) for p in self.blacklist_ports_tcp])) - - # Do not redirect blacklisted UDP ports - if self.diverter_config.get('blacklistportsudp') != None: - self.blacklist_ports_udp = self.expand_ports(self.diverter_config.get('blacklistportsudp')) - self.logger.debug('Blacklisted UDP ports: %s', ', '.join([str(p) for p in self.blacklist_ports_udp])) - - # Redirect only specific traffic, build the filter dynamically - else: - - filter_diverted_ports = list() - - if self.diverted_ports.get('TCP') != None: - for tcp_port in self.diverted_ports.get('TCP'): - filter_diverted_ports.append("tcp.DstPort == %s" % tcp_port) - filter_diverted_ports.append("tcp.SrcPort == %s" % tcp_port) - - if self.diverted_ports.get('UDP') != None: - for udp_port in self.diverted_ports.get('UDP'): - filter_diverted_ports.append("udp.DstPort == %s" % udp_port) - filter_diverted_ports.append("udp.SrcPort == %s" % udp_port) - - if len(filter_diverted_ports) > 0: - self.filter = "outbound and ip and (icmp or %s)" % " or ".join(filter_diverted_ports) - else: - self.filter = "outbound and ip" - ########################################################################### # Diverter controller functions - def start(self): - - self.logger.info('Starting...') - + def startCallback(self): # Set local DNS server IP address - if self.diverter_config.get('modifylocaldns') and self.diverter_config['modifylocaldns'].lower() == 'yes': + if self.is_set('modifylocaldns'): self.set_dns_server(self.external_ip) # Stop DNS service - if self.diverter_config.get('stopdnsservice') and self.diverter_config['stopdnsservice'].lower() == 'yes': - self.stop_service_helper('Dnscache') + if self.is_set('stopdnsservice'): + self.stop_service_helper('Dnscache') self.logger.info('Diverting ports: ') - if self.diverted_ports.get('TCP'): self.logger.info('TCP: %s', ', '.join("%d" % port for port in self.diverted_ports['TCP'])) - if self.diverted_ports.get('UDP'): self.logger.info('UDP: %s', ', '.join("%d" % port for port in self.diverted_ports['UDP'])) - + self.flush_dns() self.diverter_thread = threading.Thread(target=self.divert_thread) @@ -441,22 +166,58 @@ def start(self): self.diverter_thread.start() - def divert_thread(self): + return True + def divert_thread(self): try: while True: - packet = self.handle.recv() - self.handle_packet(packet) + wdpkt = self.handle.recv() + + if wdpkt is None: + self.logger.error('ERROR: Can\'t handle packet.') + continue + + pkt = WindowsPacketCtx('divert_thread', wdpkt) + + cb3 = [ + self.check_log_icmp, + self.redirIcmpIpUnconditionally + ] + cb4 = [ + self.maybe_redir_port, + self.maybe_fixup_sport, + self.maybe_redir_ip, + self.maybe_fixup_srcip, + ] + + self.handle_pkt(pkt, cb3, cb4) + + # Attempt to send the processed packet + self.setLastErrorNull() # WinDivert/LastError workaround + try: + self.handle.send(pkt.wdpkt) + except Exception, e: + + protocol = 'Unknown' + + if pkt.proto: + protocol = pkt.proto + elif pkt.is_icmp: + proto = 'ICMP' + + self.logger.error('ERROR: Failed to send %s %s %s packet', + self.pktDirectionStr(pkt), + self.pktInterfaceStr(pkt), proto) + self.logger.error(' %s' % (pkt.hdrToStr())) + self.logger.error(' %s', e) - # Handle errors related to shutdown process. except WindowsError as e: - if e.winerror in [4,6,995]: + if e.winerror in [4, 6, 995]: return else: raise - def stop(self): - self.logger.info('Stopping...') + def stopCallback(self): if self.pcap: self.pcap.close() @@ -465,328 +226,92 @@ def stop(self): # Restore DHCP adapter settings for interface_name in self.adapters_dhcp_restore: - cmd_set_dhcp = "netsh interface ip set address name=\"%s\" dhcp" % interface_name + cmd_set_dhcp = ('netsh interface ip set address name="%s" dhcp' % + interface_name) # Restore DHCP on interface try: - subprocess.check_call(cmd_set_dhcp, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + subprocess.check_call(cmd_set_dhcp, shell=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) except subprocess.CalledProcessError, e: - self.logger.error("Failed to restore DHCP on interface %s." % interface_name) + self.logger.error('Failed to restore DHCP on interface %s.' % + interface_name) else: - self.logger.info("Restored DHCP on interface %s" % interface_name) + self.logger.info('Restored DHCP on interface %s' % + interface_name) # Restore DHCP adapter settings for interface_name in self.adapters_dns_restore: - cmd_del_dns = "netsh interface ip delete dns name=\"%s\" all" % interface_name - cmd_set_dns_dhcp = "netsh interface ip set dns \"%s\" dhcp" % interface_name + cmd_del_dns = ('netsh interface ip delete dns name="%s" all' % + interface_name) + cmd_set_dns_dhcp = ('netsh interface ip set dns "%s" dhcp' % + interface_name) # Restore DNS on interface try: - subprocess.check_call(cmd_del_dns, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) - subprocess.check_call(cmd_set_dns_dhcp, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + subprocess.check_call(cmd_del_dns, shell=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + subprocess.check_call(cmd_set_dns_dhcp, shell=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) except subprocess.CalledProcessError, e: - self.logger.error("Failed to restore DNS on interface %s." % interface_name) + self.logger.error("Failed to restore DNS on interface %s." % + interface_name) else: - self.logger.info("Restored DNS on interface %s" % interface_name) + self.logger.info("Restored DNS on interface %s" % + interface_name) # Restore DNS server - if self.diverter_config.get('modifylocaldns') and self.diverter_config['modifylocaldns'].lower() == 'yes': + if self.is_set('modifylocaldns'): self.restore_dns_server() # Restart DNS service - if self.diverter_config.get('stopdnsservice') and self.diverter_config['stopdnsservice'].lower() == 'yes': - self.start_service_helper('Dnscache') + if self.is_set('stopdnsservice'): + self.start_service_helper('Dnscache') self.flush_dns() - - - def handle_icmp_packet(self, packet): - # Modify outgoing ICMP packet to target local Windows host which will reply to the ICMP messages. - # HACK: Can't intercept inbound ICMP server, but still works for now. - - if not ((packet.is_loopback and packet.src_addr == self.loopback_ip and packet.dst_addr == self.loopback_ip) or \ - (packet.src_addr == self.external_ip and packet.dst_addr == self.external_ip)): - - self.logger.info('Modifying %s ICMP packet:', 'loopback' if packet.is_loopback else 'external') - self.logger.info(' from: %s -> %s', packet.src_addr, packet.dst_addr) - - # Direct packet to the right interface IP address to avoid routing issues - packet.dst_addr = self.loopback_ip if packet.is_loopback else self.external_ip - - self.logger.info(' to: %s -> %s', packet.src_addr, packet.dst_addr) - - return packet - - def handle_tcp_udp_packet(self, packet, protocol, default_listener_port, blacklist_ports): - - # Meta strings - interface_string = 'loopback' if packet.is_loopback else 'external' - direction_string = 'inbound' if packet.is_inbound else 'outbound' - - # Protocol specific filters - diverted_ports = self.diverted_ports.get(protocol) - port_process_whitelist = self.port_process_whitelist.get(protocol) - port_process_blacklist = self.port_process_blacklist.get(protocol) - port_host_whitelist = self.port_host_whitelist.get(protocol) - port_host_blacklist = self.port_host_blacklist.get(protocol) - port_execute = self.port_execute.get(protocol) - - if (packet.is_loopback and packet.src_addr == self.loopback_ip and packet.dst_addr == self.loopback_ip): - self.logger.debug('Ignoring loopback packet') - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - elif packet.src_port in blacklist_ports or packet.dst_port in blacklist_ports: - self.logger.debug('Forwarding blacklisted port %s %s %s packet:', direction_string, interface_string, protocol) - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - # Check if a packet must be diverted to a local listener - # Rules: - # 1) Divert outbound packets only - # 2) Make sure we are not diverting response packet based on the source port - # 3) Make sure the destination port is a known diverted port or we have a default listener port defined - elif diverted_ports and (packet.dst_port in diverted_ports or default_listener_port != None) and not packet.src_port in diverted_ports: - - # Find which process ID is sending the request - conn_pid = self.get_pid_port_tcp(packet.src_port) if packet.tcp else self.get_pid_port_udp(packet.src_port) - process_name = self.get_process_image_filename(conn_pid) if conn_pid else None - - # Check process blacklist - if process_name and process_name in self.blacklist_processes: - self.logger.debug('Ignoring %s %s %s request packet from process %s in the process blacklist.', direction_string, interface_string, protocol, process_name) - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - # Check host blacklist - elif packet.dst_addr in self.blacklist_hosts: - self.logger.debug('Ignoring %s %s %s request packet to %s in the host blacklist.', direction_string, interface_string, protocol, packet.dst_addr) - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - # Check the port process whitelist - elif process_name and port_process_whitelist and \ - ((packet.dst_port in port_process_whitelist and not process_name in port_process_whitelist[packet.dst_port]) or\ - (default_listener_port and default_listener_port in port_process_whitelist and not process_name in port_process_whitelist[default_listener_port])) : - self.logger.debug('Ignoring %s %s %s request packet from process %s not in the listener process whitelist.', direction_string, interface_string, protocol, process_name) - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - # Check the port process blacklist - elif process_name and port_process_blacklist and \ - ((packet.dst_port in port_process_blacklist and process_name in port_process_blacklist[packet.dst_port]) or\ - (default_listener_port and default_listener_port in port_process_blacklist and process_name in port_process_blacklist[default_listener_port])) : - self.logger.debug('Ignoring %s %s %s request packet from process %s in the listener process blacklist.', direction_string, interface_string, protocol, process_name) - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - # Check the port host whitelist - elif packet.dst_addr and port_host_whitelist and \ - ((packet.dst_port in port_host_whitelist and not packet.dst_addr in port_host_whitelist[packet.dst_port]) or\ - (default_listener_port and default_listener_port in port_host_whitelist and not packet.dst_addr in port_host_whitelist[default_listener_port])) : - self.logger.debug('Ignoring %s %s %s request packet to %s not in the listener host whitelist.', direction_string, interface_string, protocol, packet.dst_addr) - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - # Check the port host blacklist - elif packet.dst_addr and port_host_blacklist and \ - ((packet.dst_port in port_host_blacklist and packet.dst_addr in port_host_blacklist[packet.dst_port]) or\ - (default_listener_port and default_listener_port in port_host_blacklist and packet.dst_addr in port_host_blacklist[default_listener_port])) : - self.logger.debug('Ignoring %s %s %s request packet to %s in the listener host blacklist.', direction_string, interface_string, protocol, packet.dst_addr) - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - # Make sure you are not intercepting packets from one of the FakeNet listeners - elif conn_pid and os.getpid() == conn_pid: - - # HACK: FTP Passive Mode Handling - # Check if a listener is initiating a new connection from a non-diverted port and add it to blacklist. This is done to handle a special use-case - # of FTP ACTIVE mode where FTP server is initiating a new connection for which the response may be redirected to a default listener. - # NOTE: Additional testing can be performed to check if this is actually a SYN packet - if packet.dst_addr == self.external_ip and packet.src_addr == self.external_ip and not packet.src_port in diverted_ports and not packet.dst_port in diverted_ports: - - self.logger.debug('Listener initiated connection %s %s %s:', direction_string, interface_string, protocol) - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - self.logger.debug(' Blacklisted port %d', packet.src_port) - - blacklist_ports.append(packet.src_port) - - else: - self.logger.debug('Skipping %s %s %s listener packet:', direction_string, interface_string, protocol) - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - # Modify the packet - else: - - # Adjustable log level output. Used to display info level logs for first packets of the session and - # debug level for the rest of the communication in order to reduce log output. - logger_level = self.logger.debug - - # First packet in a new session - if not (packet.src_port in self.sessions and self.sessions[packet.src_port] == (packet.dst_addr, packet.dst_port)): - - # Cache original target IP address based on source port - self.sessions[packet.src_port] = (packet.dst_addr, packet.dst_port) - - # Override log level to display all information on info level - logger_level = self.logger.info - - # Execute command - if conn_pid and port_execute and (packet.dst_port in port_execute or (default_listener_port and default_listener_port in port_execute)): - - - execute_cmd = port_execute[packet.dst_port if packet.dst_port in diverted_ports else default_listener_port].format(pid = conn_pid, - procname = process_name, - src_addr = packet.src_addr, - src_port = packet.src_port, - dst_addr = packet.dst_addr, - dst_port = packet.dst_port) - logger_level('Executing command: %s', execute_cmd) + return True - self.execute_detached(execute_cmd) + def pktInterfaceStr(self, pkt): + """WinDivert provides is_loopback which Windows Diverter uses to + display information about the disposition of packets it is + processing during error and other cases. + """ + return 'loopback' if pkt.wdpkt.is_loopback else 'external' + def pktDirectionStr(self, pkt): + """WinDivert provides is_inbound which Windows Diverter uses to + display information about the disposition of packets it is + processing during error and other cases. + """ + return 'inbound' if pkt.wdpkt.is_inbound else 'outbound' - logger_level('Modifying %s %s %s request packet:', direction_string, interface_string, protocol) - logger_level(' from: %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) + def redirIcmpIpUnconditionally(self, crit, pkt): + """Redirect ICMP to loopback or external IP if necessary. - # Direct packet to the right interface IP address to avoid routing issues - packet.dst_addr = self.loopback_ip if packet.is_loopback else self.external_ip + On Windows, we can't conveniently use an iptables REDIRECT rule to get + ICMP packets sent back home for free, so here is some code. + """ + if (pkt.is_icmp and + pkt.dst_ip not in [self.loopback_ip, self.external_ip]): + self.logger.info('Modifying ICMP packet (type %d, code %d):' % + (pkt.icmp_type, pkt.icmp_code)) + self.logger.info(' from: %s' % (pkt.hdrToStr())) + pkt.dst_ip = self.getNewDestinationIp(pkt.src_ip) + self.logger.info(' to: %s' % (pkt.hdrToStr())) - # Direct packet to an existing or a default listener - # check if 'hidden' config is set. If so, the packet is - # directed to the default listener which is the proxy - packet.dst_port = (packet.dst_port if ( - packet.dst_port in diverted_ports and - diverted_ports[packet.dst_port] is False) - else default_listener_port) - - logger_level(' to: %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - if conn_pid: - logger_level(' pid: %d name: %s', conn_pid, process_name if process_name else 'Unknown') - - - # Restore diverted response from a local listener - # NOTE: The response can come from a legitimate request - elif diverted_ports and packet.src_port in diverted_ports: - # The packet is a response from a listener. It needs to be - # redirected to the original source - - # Find which process ID is sending the request - conn_pid = self.get_pid_port_tcp(packet.dst_port) if packet.tcp else self.get_pid_port_udp(packet.dst_port) - process_name = self.get_process_image_filename(conn_pid) - - if not packet.dst_port in self.sessions: - self.logger.debug('Unknown %s %s %s response packet:', direction_string, interface_string, protocol) - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - # Restore original target IP address from the cache - else: - self.logger.debug('Modifying %s %s %s response packet:', direction_string, interface_string, protocol) - self.logger.debug(' from: %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - # Restore original target IP address based on destination port - packet.src_addr, packet.src_port = self.sessions[packet.dst_port] - - self.logger.debug(' to: %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - if conn_pid: - self.logger.debug(' pid: %d name: %s', conn_pid, process_name if process_name else 'Unknown') - - else: - # At this point whe know the packet is either a response packet - # from a listener(sport is bound) or is bound for a port with no - # listener (dport not bound) - - # Cache original target IP address based on source port - self.sessions[packet.src_port] = (packet.dst_addr, packet.dst_port) - - # forward to proxy - packet.dst_port = default_listener_port - - self.logger.debug('Redirected %s %s %s packet to proxy:', direction_string, interface_string, protocol) - self.logger.debug(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - - return packet - - def handle_packet(self, packet): - - if packet == None: - self.logger.error('ERROR: Can\'t handle packet.') - return - - # Preserve destination address to detect packet being diverted - dst_addr = packet.dst_addr - - ####################################################################### - # Capture packet and store raw packet in the PCAP - if self.capture_flag: - self.pcap.writepkt(packet.raw.tobytes()) - - ########################################################################### - # Verify the IP packet has an additional header - - if packet.ip: - - ####################################################################### - # Handle ICMP Packets - - if packet.icmp: - packet = self.handle_icmp_packet(packet) - - ####################################################################### - # Handle TCP/UDP Packets - - elif packet.tcp: - protocol = 'TCP' - packet = self.handle_tcp_udp_packet(packet, - protocol, - self.default_listener_tcp_port, - self.blacklist_ports_tcp) - - elif packet.udp: - protocol = 'UDP' - packet = self.handle_tcp_udp_packet(packet, - protocol, - self.default_listener_udp_port, - self.blacklist_ports_udp) - - else: - self.logger.error('ERROR: Unknown packet header type.') - - ####################################################################### - # Capture modified packet and store raw packet in the PCAP - # NOTE: While this results in potentially duplicate traffic capture, this is necessary - # to properly restore TLS/SSL sessions. - # TODO: Develop logic to record traffic before modification for both requests and - # responses to reduce duplicate captures. - if self.capture_flag and (dst_addr != packet.dst_addr): - self.pcap.writepkt(packet.raw.tobytes()) - - ####################################################################### - # Attempt to send the processed packet - try: - self.handle.send(packet) - except Exception, e: - - protocol = 'Unknown' - - if packet.tcp: - protocol = 'TCP' - elif packet.udp: - protocol = 'UDP' - elif packet.icmp: - protocol = 'ICMP' - - interface_string = 'loopback' if packet.is_loopback else 'external' - direction_string = 'inbound' if packet.is_inbound else 'outbound' - - self.logger.error('ERROR: Failed to send %s %s %s packet', direction_string, interface_string, protocol) - - if packet.src_port and packet.dst_port: - self.logger.error(' %s:%d -> %s:%d', packet.src_addr, packet.src_port, packet.dst_addr, packet.dst_port) - else: - self.logger.error(' %s -> %s', packet.src_addr, packet.dst_addr) + return pkt - self.logger.error(' %s', e) def main(): - diverter_config = {'redirectalltraffic': 'no', 'defaultlistener': 'DefaultListener', 'dumppackets': 'no'} + diverter_config = {'redirectalltraffic': 'no', + 'defaultlistener': 'DefaultListener', + 'dumppackets': 'no'} listeners_config = {'DefaultListener': {'port': '1337', 'protocol': 'TCP'}} diverter = Diverter(diverter_config, listeners_config) diff --git a/fakenet/diverters/winutil.py b/fakenet/diverters/winutil.py old mode 100755 new mode 100644 index 6440909..c2bef3f --- a/fakenet/diverters/winutil.py +++ b/fakenet/diverters/winutil.py @@ -1,7 +1,9 @@ #!/usr/bin/env python import logging -logging.basicConfig(format='%(asctime)s [%(name)18s] %(message)s', datefmt='%m/%d/%y %I:%M:%S %p', level=logging.DEBUG) +logging.basicConfig(format='%(asctime)s [%(name)18s] %(message)s', + datefmt='%m/%d/%y %I:%M:%S %p', level=logging.DEBUG) +import ctypes from ctypes import * from ctypes.wintypes import * @@ -9,6 +11,7 @@ import sys import socket import struct +from . import diverterbase import time @@ -16,10 +19,10 @@ import subprocess -NO_ERROR = 0 +NO_ERROR = 0 -AF_INET = 2 -AF_INET6 = 23 +AF_INET = 2 +AF_INET6 = 23 ULONG64 = c_uint64 @@ -28,34 +31,35 @@ # Services related functions ############################################################################## -SC_MANAGER_ALL_ACCESS = 0xF003F +SC_MANAGER_ALL_ACCESS = 0xF003F -SERVICE_ALL_ACCESS = 0xF01FF -SERVICE_STOP = 0x0020 -SERVICE_QUERY_STATUS = 0x0004 +SERVICE_ALL_ACCESS = 0xF01FF +SERVICE_STOP = 0x0020 +SERVICE_QUERY_STATUS = 0x0004 SERVICE_ENUMERATE_DEPENDENTS = 0x0008 -SC_STATUS_PROCESS_INFO = 0x0 +SC_STATUS_PROCESS_INFO = 0x0 -SERVICE_STOPPED = 0x1 -SERVICE_START_PENDING = 0x2 -SERVICE_STOP_PENDING = 0x3 -SERVICE_RUNNING = 0x4 -SERVICE_CONTINUE_PENDING = 0x5 -SERVICE_PAUSE_PENDING = 0x6 -SERVICE_PAUSED = 0x7 +SERVICE_STOPPED = 0x1 +SERVICE_START_PENDING = 0x2 +SERVICE_STOP_PENDING = 0x3 +SERVICE_RUNNING = 0x4 +SERVICE_CONTINUE_PENDING = 0x5 +SERVICE_PAUSE_PENDING = 0x6 +SERVICE_PAUSED = 0x7 -SERVICE_CONTROL_STOP = 0x1 -SERVICE_CONTROL_PAUSE = 0x2 -SERVICE_CONTROL_CONTINUE = 0x3 +SERVICE_CONTROL_STOP = 0x1 +SERVICE_CONTROL_PAUSE = 0x2 +SERVICE_CONTROL_CONTINUE = 0x3 -SERVICE_NO_CHANGE = 0xffffffff +SERVICE_NO_CHANGE = 0xffffffff + +SERVICE_AUTO_START = 0x2 +SERVICE_BOOT_START = 0x0 +SERVICE_DEMAND_START = 0x3 +SERVICE_DISABLED = 0x4 +SERVICE_SYSTEM_START = 0x1 -SERVICE_AUTO_START = 0x2 -SERVICE_BOOT_START = 0x0 -SERVICE_DEMAND_START = 0x3 -SERVICE_DISABLED = 0x4 -SERVICE_SYSTEM_START = 0x1 class SERVICE_STATUS_PROCESS(Structure): _fields_ = [ @@ -67,7 +71,7 @@ class SERVICE_STATUS_PROCESS(Structure): ("dwCheckPoint", DWORD), ("dwWaitHint", DWORD), ("dwProcessId", DWORD), - ("dwServiceFlags", DWORD), + ("dwServiceFlags", DWORD), ] ############################################################################## @@ -77,7 +81,9 @@ class SERVICE_STATUS_PROCESS(Structure): ############################################################################## # GetExtendedTcpTable constants and structures -TCP_TABLE_OWNER_PID_ALL = 5 + +TCP_TABLE_OWNER_PID_ALL = 5 + class MIB_TCPROW_OWNER_PID(Structure): _fields_ = [ @@ -89,6 +95,7 @@ class MIB_TCPROW_OWNER_PID(Structure): ("dwOwningPid", DWORD) ] + class MIB_TCPTABLE_OWNER_PID(Structure): _fields_ = [ ("dwNumEntries", DWORD), @@ -98,7 +105,9 @@ class MIB_TCPTABLE_OWNER_PID(Structure): ############################################################################## # GetExtendedUdpTable constants and structures -UDP_TABLE_OWNER_PID = 1 + +UDP_TABLE_OWNER_PID = 1 + class MIB_UDPROW_OWNER_PID(Structure): _fields_ = [ @@ -107,6 +116,7 @@ class MIB_UDPROW_OWNER_PID(Structure): ("dwOwningPid", DWORD) ] + class MIB_UDPTABLE_OWNER_PID(Structure): _fields_ = [ ("dwNumEntries", DWORD), @@ -116,6 +126,7 @@ class MIB_UDPTABLE_OWNER_PID(Structure): ############################################################################### # GetProcessImageFileName constants and structures + MAX_PATH = 260 PROCESS_QUERY_LIMITED_INFORMATION = 0x1000 @@ -126,15 +137,16 @@ class MIB_UDPTABLE_OWNER_PID(Structure): MIB_IF_TYPE_ETHERNET = 6 MIB_IF_TYPE_LOOPBACK = 28 -IF_TYPE_IEEE80211 = 71 +IF_TYPE_IEEE80211 = 71 ############################################################################### # GetAdaptersAddresses constants and structures MAX_ADAPTER_ADDRESS_LENGTH = 8 -MAX_DHCPV6_DUID_LENGTH = 130 +MAX_DHCPV6_DUID_LENGTH = 130 + +IFOPERSTATUSUP = 1 -IFOPERSTATUSUP = 1 class SOCKADDR(Structure): _fields_ = [ @@ -142,33 +154,40 @@ class SOCKADDR(Structure): ("sa_data", c_char * 14), ] + class SOCKET_ADDRESS(Structure): _fields_ = [ ("Sockaddr", POINTER(SOCKADDR)), ("SockaddrLength", INT), ] + class IP_ADAPTER_PREFIX(Structure): pass + + IP_ADAPTER_PREFIX._fields_ = [ - ("Length", ULONG), - ("Flags", DWORD), - ("Next", POINTER(IP_ADAPTER_PREFIX)), - ("Address", SOCKET_ADDRESS), - ("PrefixLength", ULONG), - ] + ("Length", ULONG), + ("Flags", DWORD), + ("Next", POINTER(IP_ADAPTER_PREFIX)), + ("Address", SOCKET_ADDRESS), + ("PrefixLength", ULONG), +] + class IP_ADAPTER_ADDRESSES(Structure): pass + + IP_ADAPTER_ADDRESSES._fields_ = [ ("Length", ULONG), ("IfIndex", DWORD), ("Next", POINTER(IP_ADAPTER_ADDRESSES)), ("AdapterName", LPSTR), - ("FirstUnicastAddress", c_void_p), # Not used - ("FirstAnycastAddress", c_void_p), # Not used - ("FirstMulticastAddress", c_void_p), # Not used - ("FirstDnsServerAddress", c_void_p), # Not used + ("FirstUnicastAddress", c_void_p), # Not used + ("FirstAnycastAddress", c_void_p), # Not used + ("FirstMulticastAddress", c_void_p), # Not used + ("FirstDnsServerAddress", c_void_p), # Not used ("DnsSuffix", LPWSTR), ("Description", LPWSTR), ("FriendlyName", LPWSTR), @@ -183,8 +202,8 @@ class IP_ADAPTER_ADDRESSES(Structure): ("FirstPrefix", POINTER(IP_ADAPTER_PREFIX)), ("TransmitLinkSpeed", ULONG64), ("ReceiveLinkSpeed", ULONG64), - ("FirstWinsServerAddress", c_void_p), # Not used - ("FirstGatewayAddress", c_void_p), # Not used + ("FirstWinsServerAddress", c_void_p), # Not used + ("FirstGatewayAddress", c_void_p), # Not used ("Ipv4Metric", ULONG), ("Ipv6Metric", ULONG), ("Luid", ULONG64), @@ -197,7 +216,7 @@ class IP_ADAPTER_ADDRESSES(Structure): ("Dhcpv6ClientDuid", BYTE * MAX_DHCPV6_DUID_LENGTH), ("Dhcpv6ClientDuidLength", ULONG), ("Dhcpv6Iaid", ULONG), - ("FirstDnsSuffix", c_void_p), # Not used + ("FirstDnsSuffix", c_void_p), # Not used ] ############################################################################### @@ -209,50 +228,58 @@ class IP_ADAPTER_ADDRESSES(Structure): MIB_IF_TYPE_ETHERNET = 6 MIB_IF_TYPE_LOOPBACK = 28 -IF_TYPE_IEEE80211 = 71 +IF_TYPE_IEEE80211 = 71 + class IP_ADDRESS_STRING(Structure): _fields_ = [ ("String", c_char * 16), ] + class IP_MASK_STRING(Structure): _fields_ = [ ("String", c_char * 16), ] + class IP_ADDR_STRING(Structure): pass + + IP_ADDR_STRING._fields_ = [ - ("Next", POINTER(IP_ADDR_STRING)), - ("IpAddress", IP_ADDRESS_STRING), - ("IpMask", IP_MASK_STRING), - ("Context", DWORD), - ] + ("Next", POINTER(IP_ADDR_STRING)), + ("IpAddress", IP_ADDRESS_STRING), + ("IpMask", IP_MASK_STRING), + ("Context", DWORD), +] + class IP_ADAPTER_INFO(Structure): pass + + IP_ADAPTER_INFO._fields_ = [ - ("Next", POINTER(IP_ADAPTER_INFO)), - ("ComboIndex", DWORD), - ("AdapterName", c_char * (MAX_ADAPTER_NAME_LENGTH + 4)), - ("Description", c_char * (MAX_ADAPTER_DESCRIPTION_LENGTH + 4)), - ("AddressLength", UINT), - ("Address", BYTE * MAX_ADAPTER_LENGTH), - ("Index", DWORD), - ("Type", UINT), - ("DhcpEnabled", UINT), - ("CurrentIpAddress", c_void_p), # Not used - ("IpAddressList", IP_ADDR_STRING), - ("GatewayList", IP_ADDR_STRING), - ("DhcpServer", IP_ADDR_STRING), - ("HaveWins", BOOL), - ("PrimaryWinsServer", IP_ADDR_STRING), - ("SecondaryWinsServer", IP_ADDR_STRING), - ("LeaseObtained", c_ulong), - ("LeaseExpires", c_ulong), + ("Next", POINTER(IP_ADAPTER_INFO)), + ("ComboIndex", DWORD), + ("AdapterName", c_char * (MAX_ADAPTER_NAME_LENGTH + 4)), + ("Description", c_char * (MAX_ADAPTER_DESCRIPTION_LENGTH + 4)), + ("AddressLength", UINT), + ("Address", BYTE * MAX_ADAPTER_LENGTH), + ("Index", DWORD), + ("Type", UINT), + ("DhcpEnabled", UINT), + ("CurrentIpAddress", c_void_p), # Not used + ("IpAddressList", IP_ADDR_STRING), + ("GatewayList", IP_ADDR_STRING), + ("DhcpServer", IP_ADDR_STRING), + ("HaveWins", BOOL), + ("PrimaryWinsServer", IP_ADDR_STRING), + ("SecondaryWinsServer", IP_ADDR_STRING), + ("LeaseObtained", c_ulong), + ("LeaseExpires", c_ulong), - ] +] ############################################################################### # GetNetworkParams constants and structures @@ -266,30 +293,36 @@ class IP_ADAPTER_INFO(Structure): NDIS_IF_MAX_STRING_SIZE = 256 + class IP_ADDRESS_STRING(Structure): _fields_ = [ ("String", c_char * 16), ] + class IP_MASK_STRING(Structure): _fields_ = [ ("String", c_char * 16), ] + class IP_ADDR_STRING(Structure): pass + + IP_ADDR_STRING._fields_ = [ - ("Next", POINTER(IP_ADDR_STRING)), - ("IpAddress", IP_ADDRESS_STRING), - ("IpMask", IP_MASK_STRING), - ("Context", DWORD), - ] + ("Next", POINTER(IP_ADDR_STRING)), + ("IpAddress", IP_ADDRESS_STRING), + ("IpMask", IP_MASK_STRING), + ("Context", DWORD), +] + class FIXED_INFO(Structure): _fields_ = [ ("HostName", c_char * (MAX_HOSTNAME_LEN + 4)), ("DomainName", c_char * (MAX_DOMAIN_NAME_LEN + 4)), - ("CurrentDnsServer", c_void_p), # Not used + ("CurrentDnsServer", c_void_p), # Not used ("DnsServerList", IP_ADDR_STRING), ("NodeType", UINT), ("ScopeId", c_char * (MAX_SCOPE_ID_LEN + 4)), @@ -298,7 +331,148 @@ class FIXED_INFO(Structure): ("EnableDns", UINT), ] -class WinUtilMixin(): + +class WinUtilMixin(diverterbase.DiverterPerOSDelegate): + def getNewDestinationIp(self, src_ip): + """Gets the IP to redirect to - loopback if loopback, external + otherwise. + + On Windows, and possibly other operating systems, if you redirect + external packets to a loopback address, they simply will not route. + + On Linux, FTP tests will fail if you do this, so it is overridden to + return 127.0.0.1. + """ + return self.loopback_ip if src_ip.startswith('127.') else self.external_ip + + def fix_gateway(self): + """Check if there is a gateway configured on any of the Ethernet + interfaces. If that's not the case, then locate configured IP address + and set a gateway automatically. This is necessary for VMWare Host-Only + DHCP server which leaves default gateway empty. + """ + fixed = False + + for adapter in self.get_adapters_info(): + + # Look for a DHCP interface with a set IP address but no gateway + # (Host-Only) + if self.check_ipaddresses_interface(adapter) and adapter.DhcpEnabled: + + (ip_address, netmask) = next( + self.get_ipaddresses_netmask(adapter)) + gw_address = ip_address[:ip_address.rfind('.')] + '.254' + + interface_name = self.get_adapter_friendlyname(adapter.Index) + + # Don't set gateway on loopback interfaces (e.g. Npcap Loopback + # Adapter) + if not "loopback" in interface_name.lower(): + + self.adapters_dhcp_restore.append(interface_name) + + cmd_set_gw = "netsh interface ip set address name=\"%s\" static %s %s %s" % ( + interface_name, ip_address, netmask, gw_address) + + # Configure gateway + try: + subprocess.check_call(cmd_set_gw, shell=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + except subprocess.CalledProcessError, e: + self.logger.error(" Failed to set gateway %s on interface %s." % ( + gw_address, interface_name)) + else: + self.logger.info(" Setting gateway %s on interface %s" % ( + gw_address, interface_name)) + fixed = True + + return fixed + + def fix_dns(self): + """Check if there is a DNS server on any of the Ethernet interfaces. If + that's not the case, then locate configured IP address and set a DNS + server automatically. + """ + fixed = False + + for adapter in self.get_adapters_info(): + + if self.check_ipaddresses_interface(adapter): + + ip_address = next(self.get_ipaddresses(adapter)) + dns_address = ip_address + + interface_name = self.get_adapter_friendlyname(adapter.Index) + + # Don't set DNS on loopback interfaces (e.g. Npcap Loopback + # Adapter) + if not "loopback" in interface_name.lower(): + + self.adapters_dns_restore.append(interface_name) + + cmd_set_dns = "netsh interface ip set dns name=\"%s\" static %s" % ( + interface_name, dns_address) + + # Configure DNS server + try: + subprocess.check_call(cmd_set_dns, + shell=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + except subprocess.CalledProcessError, e: + self.logger.error(" Failed to set DNS %s on interface %s." % ( + dns_address, interface_name)) + else: + self.logger.info(" Setting DNS %s on interface %s" % ( + dns_address, interface_name)) + fixed = True + + return fixed + + def get_pid_comm(self, pkt): + conn_pid, process_name = None, None + if pkt.proto and pkt.sport: + if pkt.proto == 'TCP': + conn_pid = self._get_pid_port_tcp(pkt.sport) + elif pkt.proto == 'UDP': + conn_pid = self._get_pid_port_udp(pkt.sport) + + if conn_pid: + process_name = self.get_process_image_filename(conn_pid) + return conn_pid, process_name + + def check_gateways(self): + + for adapter in self.get_adapters_info(): + for gateway in self.get_gateways(adapter): + if gateway != '0.0.0.0': + return True + else: + return False + + def check_ipaddresses(self): + + for adapter in self.get_adapters_info(): + if self.check_ipaddresses_interface(adapter): + return True + else: + return False + + def check_dns_servers(self): + + FixedInfo = self.get_network_params() + + if not FixedInfo: + return + + ip_addr_string = FixedInfo.DnsServerList + + if ip_addr_string and ip_addr_string.IpAddress.String: + return True + + else: + return False ########################################################################### # Service related functions @@ -337,8 +511,6 @@ def close_service_handle(self, sc_handle): return True - - ########################################################################### # Opens an existing service. # @@ -348,12 +520,14 @@ def close_service_handle(self, sc_handle): # _In_ DWORD dwDesiredAccess # ); - def open_service(self, sc_handle, service_name, dwDesiredAccess = SERVICE_ALL_ACCESS): + def open_service(self, sc_handle, service_name, + dwDesiredAccess=SERVICE_ALL_ACCESS): if not sc_handle: return - service_handle = windll.advapi32.OpenServiceA(sc_handle, service_name, dwDesiredAccess) + service_handle = windll.advapi32.OpenServiceA(sc_handle, service_name, + dwDesiredAccess) if service_handle == 0: self.logger.error('Failed to call OpenService') @@ -370,7 +544,7 @@ def open_service(self, sc_handle, service_name, dwDesiredAccess = SERVICE_ALL_AC # _Out_opt_ LPBYTE lpBuffer, # _In_ DWORD cbBufSize, # _Out_ LPDWORD pcbBytesNeeded - # ); + # ); def query_service_status_ex(self, service_handle): @@ -438,7 +612,8 @@ def start_service(self, service_handle): # _In_opt_ LPCTSTR lpDisplayName # ); - def change_service_config(self, service_handle, dwStartType = SERVICE_DISABLED): + def change_service_config(self, service_handle, + dwStartType=SERVICE_DISABLED): if windll.advapi32.ChangeServiceConfigA(service_handle, SERVICE_NO_CHANGE, dwStartType, SERVICE_NO_CHANGE, 0, 0, 0, 0, 0, 0, 0) == 0: self.logger.error('Failed to call ChangeServiceConfig') @@ -448,13 +623,12 @@ def change_service_config(self, service_handle, dwStartType = SERVICE_DISABLED): else: return True + def start_service_helper(self, service_name='Dnscache'): - def start_service_helper(self, service_name = 'Dnscache'): + sc_handle = None + service_handle = None - sc_handle = None - service_handle = None - - timeout = 5 + timeout = 5 sc_handle = self.open_sc_manager() @@ -472,64 +646,76 @@ def start_service_helper(self, service_name = 'Dnscache'): # Backup enable the service try: - subprocess.check_call("sc config %s start= auto" % service_name, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + subprocess.check_call("sc config %s start= auto" % + service_name, shell=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) except subprocess.CalledProcessError, e: - self.logger.error('Failed to enable the service %s. (sc config)', service_name) + self.logger.error( + 'Failed to enable the service %s. (sc config)', service_name) else: - self.logger.info('Successfully enabled the service %s. (sc config)', service_name) + self.logger.info( + 'Successfully enabled the service %s. (sc config)', service_name) else: - self.logger.info('Successfully enabled the service %s.', service_name) + self.logger.info('Successfully enabled the service %s.', + service_name) service_status = self.query_service_status_ex(service_handle) if service_status: - if not service_status.dwCurrentState in [SERVICE_RUNNING, SERVICE_START_PENDING]: + if not service_status.dwCurrentState in [SERVICE_RUNNING, SERVICE_START_PENDING]: # Start service - if self.start_service(service_handle): + if self.start_service(service_handle): # Wait for the service to start - while timeout: - timeout -= 1 - time.sleep(1) - - service_status = self.query_service_status_ex(service_handle) - if service_status.dwCurrentState == SERVICE_RUNNING: - self.logger.info('Successfully started the service %s.', service_name) - break - else: - self.logger.error('Timed out while trying to start the service %s.', service_name) + while timeout: + timeout -= 1 + time.sleep(1) + + service_status = self.query_service_status_ex( + service_handle) + if service_status.dwCurrentState == SERVICE_RUNNING: + self.logger.info( + 'Successfully started the service %s.', service_name) + break else: - self.logger.error('Failed to start the service %s.', service_name) + self.logger.error( + 'Timed out while trying to start the service %s.', service_name) else: - self.logger.error('Service %s is already running.', service_name) + self.logger.error( + 'Failed to start the service %s.', service_name) + else: + self.logger.error( + 'Service %s is already running.', service_name) # As a backup call net stop if service_status.dwCurrentState != SERVICE_RUNNING: try: - subprocess.check_call("net start %s" % service_name, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + subprocess.check_call("net start %s" % service_name, + shell=True, stdout=subprocess.PIPE, + stderr=subprocess.PIPE) except subprocess.CalledProcessError, e: - self.logger.error('Failed to start the service %s. (net stop)', service_name) + self.logger.error( + 'Failed to start the service %s. (net stop)', service_name) else: - self.logger.info('Successfully started the service %s.', service_name) - - - + self.logger.info('Successfully started the service %s.', + service_name) self.close_service_handle(service_handle) self.close_service_handle(sc_handle) - def stop_service_helper(self, service_name = 'Dnscache'): + def stop_service_helper(self, service_name='Dnscache'): - sc_handle = None - service_handle = None + sc_handle = None + service_handle = None - Control = SERVICE_CONTROL_STOP - dwControl = DWORD(Control) - timeout = 5 + Control = SERVICE_CONTROL_STOP + dwControl = DWORD(Control) + timeout = 5 sc_handle = self.open_sc_manager() @@ -547,57 +733,69 @@ def stop_service_helper(self, service_name = 'Dnscache'): # Backup disable the service try: - subprocess.check_call("sc config %s start= disabled" % service_name, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + subprocess.check_call("sc config %s start= disabled" % + service_name, shell=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) except subprocess.CalledProcessError, e: - self.logger.error('Failed to disable the service %s. (sc config)', service_name) + self.logger.error( + 'Failed to disable the service %s. (sc config)', service_name) else: - self.logger.info('Successfully disabled the service %s. (sc config)', service_name) + self.logger.info( + 'Successfully disabled the service %s. (sc config)', service_name) else: - self.logger.info('Successfully disabled the service %s.', service_name) + self.logger.info( + 'Successfully disabled the service %s.', service_name) service_status = self.query_service_status_ex(service_handle) if service_status: - if service_status.dwCurrentState != SERVICE_STOPPED: + if service_status.dwCurrentState != SERVICE_STOPPED: - # Send a stop code to the service - if self.control_service(service_handle, dwControl): + # Send a stop code to the service + if self.control_service(service_handle, dwControl): - # Wait for the service to stop - while timeout: - timeout -= 1 - time.sleep(1) + # Wait for the service to stop + while timeout: + timeout -= 1 + time.sleep(1) - service_status = self.query_service_status_ex(service_handle) - if service_status.dwCurrentState == SERVICE_STOPPED: - self.logger.info('Successfully stopped the service %s.', service_name) - break + service_status = self.query_service_status_ex( + service_handle) + if service_status.dwCurrentState == SERVICE_STOPPED: + self.logger.info( + 'Successfully stopped the service %s.', service_name) + break - else: - self.logger.error('Timed out while trying to stop the service %s.', service_name) else: - self.logger.error('Failed to stop the service %s.', service_name) + self.logger.error( + 'Timed out while trying to stop the service %s.', service_name) else: - self.logger.error('Service %s is already stopped.', service_name) + self.logger.error( + 'Failed to stop the service %s.', service_name) + else: + self.logger.error( + 'Service %s is already stopped.', service_name) # As a backup call net stop if service_status.dwCurrentState != SERVICE_STOPPED: try: - subprocess.check_call("net stop %s" % service_name, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + subprocess.check_call("net stop %s" % service_name, + shell=True, stdout=subprocess.PIPE, + stderr=subprocess.PIPE) except subprocess.CalledProcessError, e: - self.logger.error('Failed to stop the service %s. (net stop)', service_name) + self.logger.error( + 'Failed to stop the service %s. (net stop)', service_name) else: - self.logger.info('Successfully stopped the service %s.', service_name) + self.logger.info( + 'Successfully stopped the service %s.', service_name) self.close_service_handle(service_handle) self.close_service_handle(sc_handle) - - - ########################################################################### # Process related functions ########################################################################### @@ -627,20 +825,20 @@ def get_extended_tcp_table(self): for item in TcpTable.table[:TcpTable.dwNumEntries]: yield item - def get_pid_port_tcp(self, port): + def _get_pid_port_tcp(self, port): for item in self.get_extended_tcp_table(): lPort = socket.ntohs(item.dwLocalPort) lAddr = socket.inet_ntoa(struct.pack('L', item.dwLocalAddr)) - pid = item.dwOwningPid + pid = item.dwOwningPid if lPort == port: return pid else: return None - ################################################################################# + ########################################################################## # The GetExtendedUdpTable function retrieves a table that contains a list of UDP endpoints available to the application. # # DWORD GetExtendedUdpTable( @@ -665,20 +863,20 @@ def get_extended_udp_table(self): for item in UdpTable.table[:UdpTable.dwNumEntries]: yield item - def get_pid_port_udp(self, port): + def _get_pid_port_udp(self, port): for item in self.get_extended_udp_table(): lPort = socket.ntohs(item.dwLocalPort) lAddr = socket.inet_ntoa(struct.pack('L', item.dwLocalAddr)) - pid = item.dwOwningPid + pid = item.dwOwningPid if lPort == port: return pid else: return None - ############################################################################### + ########################################################################## # Retrieves the name of the executable file for the specified process. # # DWORD WINAPI GetProcessImageFileName( @@ -691,24 +889,49 @@ def get_process_image_filename(self, pid): process_name = None - hProcess = windll.kernel32.OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, False, pid) - if hProcess: + if pid == 4: + # Skip the inevitable errno 87, invalid parameter + process_name = 'System' + elif pid: + hProcess = windll.kernel32.OpenProcess( + PROCESS_QUERY_LIMITED_INFORMATION, False, pid) + if hProcess: - lpImageFileName = create_string_buffer(MAX_PATH) + lpImageFileName = create_string_buffer(MAX_PATH) - if windll.psapi.GetProcessImageFileNameA(hProcess, lpImageFileName, MAX_PATH) > 0: - process_name = os.path.basename(lpImageFileName.value) - else: - self.logger.error('Failed to call GetProcessImageFileNameA') + if windll.psapi.GetProcessImageFileNameA(hProcess, lpImageFileName, MAX_PATH) > 0: + process_name = os.path.basename(lpImageFileName.value) + else: + self.logger.error('Failed to call GetProcessImageFileNameA, %d' % + (ctypes.GetLastError())) - windll.kernel32.CloseHandle(hProcess) + windll.kernel32.CloseHandle(hProcess) return process_name + def setLastErrorNull(self): + """Workaround for WinDivert handle.send() LastError behavior. + + It looks a lot like WinDivert's handle.send(wdpkt) erroneously fails if + LastError is non-zero before invoking the method. Hence, in case of ANY + Windows APIs setting LastError to a nonzero value, this function is + available for the Windows Diverter to NULL LastError before invoking + handle.send(). + This was discovered in cases where GetProcessImageFileNameA() was + called on PID 4 (System): GetProcessImageFileNameA returned an error + value, and GetLastError() returned 87. Reliably when this happened, + handle.send(wdpkt) raised an exception that, when printed as a string, + read as follows: + [Error 87] The parameter is incorrect. - ############################################################################### + In these cases, calling SetLastError(0) before invoking handle.send() + yielded normal operation. + """ + ctypes.windll.kernel32.SetLastError(0) + + ########################################################################## # The GetAdaptersAddresses function retrieves the addresses associated with the adapters on the local computer. # # ULONG WINAPI GetAdaptersAddresses( @@ -723,10 +946,12 @@ def get_adapters_addresses(self): Size = ULONG(0) - windll.iphlpapi.GetAdaptersAddresses(AF_INET, 0, None, None, byref(Size)) + windll.iphlpapi.GetAdaptersAddresses(AF_INET, 0, None, None, + byref(Size)) AdapterAddresses = create_string_buffer(Size.value) - pAdapterAddresses = cast(AdapterAddresses, POINTER(IP_ADAPTER_ADDRESSES)) + pAdapterAddresses = cast(AdapterAddresses, + POINTER(IP_ADAPTER_ADDRESSES)) if not windll.iphlpapi.GetAdaptersAddresses(AF_INET, 0, None, pAdapterAddresses, byref(Size)) == NO_ERROR: self.logger.error('Failed calling GetAdaptersAddresses') @@ -763,7 +988,6 @@ def get_adapter_friendlyname(self, if_index): else: return None - ########################################################################### # The GetAdaptersInfo function retrieves adapter information for the local computer. # @@ -826,15 +1050,6 @@ def get_ipaddresses_index(self, index): if adapter.Index == index: return self.get_ipaddresses(adapter) - def check_gateways(self): - - for adapter in self.get_adapters_info(): - for gateway in self.get_gateways(adapter): - if gateway != '0.0.0.0': - return True - else: - return False - def get_ip_with_gateway(self): for adapter in self.get_adapters_info(): @@ -844,7 +1059,6 @@ def get_ip_with_gateway(self): else: return None - def check_ipaddresses_interface(self, adapter): for ipaddress in self.get_ipaddresses(adapter): @@ -853,14 +1067,6 @@ def check_ipaddresses_interface(self, adapter): else: return False - def check_ipaddresses(self): - - for adapter in self.get_adapters_info(): - if self.check_ipaddresses_interface(adapter): - return True - else: - return False - ########################################################################### # The GetNetworkParams function retrieves network parameters for the local computer. # @@ -893,29 +1099,13 @@ def get_dns_servers(self): yield ip_addr_string.IpAddress.String ip_addr_string = ip_addr_string.Next - def check_dns_servers(self): - - FixedInfo = self.get_network_params() - - if not FixedInfo: - return - - ip_addr_string = FixedInfo.DnsServerList - - if ip_addr_string and ip_addr_string.IpAddress.String: - return True - - else: - return False - - ########################################################################### # The GetBestInterface function retrieves the index of the interface that has the best route to the specified IPv4 address. # # DWORD GetBestInterface( # _In_ IPAddr dwDestAddr, # _Out_ PDWORD pdwBestIfIndex - # ); + # ); def get_best_interface(self, ip='8.8.8.8'): BestIfIndex = DWORD() @@ -934,7 +1124,7 @@ def check_best_interface(self, ip='8.8.8.8'): if not windll.iphlpapi.GetBestInterface(DestAddr, byref(BestIfIndex)) == NO_ERROR: return False - return True + return True # Return the best local IP address to reach defined IP address def get_best_ipaddress(self, ip='8.8.8.8'): @@ -956,7 +1146,7 @@ def get_best_ipaddress(self, ip='8.8.8.8'): # NETIO_STATUS WINAPI ConvertInterfaceIndexToLuid( # _In_ NET_IFINDEX InterfaceIndex, # _Out_ PNET_LUID InterfaceLuid - # ); + # ); # # NETIO_STATUS WINAPI ConvertInterfaceLuidToNameA( # _In_ const NET_LUID *InterfaceLuid, @@ -995,23 +1185,26 @@ def convert_interface_index_to_name(self, index): def notify_ip_change(self, adapter_name): if windll.dhcpcsvc.DhcpNotifyConfigChange(0, adapter_name, 0, 0, 0, 0, 0) == NO_ERROR: - self.logger.debug('Successfully performed adapter change notification on %s', adapter_name) + self.logger.debug( + 'Successfully performed adapter change notification on %s', adapter_name) else: - self.logger.error('Failed to notify adapter change on %s', adapter_name) + self.logger.error('Failed to notify adapter change on %s', + adapter_name) ########################################################################### # DnsFlushResolverCache def flush_dns(self): - + try: - subprocess.check_call('ipconfig /flushdns', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + subprocess.check_call( + 'ipconfig /flushdns', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) except subprocess.CalledProcessError, e: - self.logger.error( - "Failed to flush DNS cache. Local machine may use cached DNS results.") + self.logger.error("Failed to flush DNS cache. Local machine may " + "use cached DNS results.") else: self.logger.info('Flushed DNS cache.') - def get_reg_value(self, key, sub_key, value, sam = KEY_READ): + def get_reg_value(self, key, sub_key, value, sam=KEY_READ): try: handle = OpenKey(key, sub_key, 0, sam) @@ -1026,7 +1219,7 @@ def get_reg_value(self, key, sub_key, value, sam = KEY_READ): self.logger.error('Failed getting registry value %s.', value) return None - def set_reg_value(self, key, sub_key, value, data, type = REG_SZ, sam = KEY_WRITE): + def set_reg_value(self, key, sub_key, value, data, type=REG_SZ, sam=KEY_WRITE): try: handle = CreateKeyEx(key, sub_key, 0, sam) @@ -1042,29 +1235,35 @@ def set_reg_value(self, key, sub_key, value, data, type = REG_SZ, sam = KEY_WRIT ########################################################################### # Set DNS Server - def set_dns_server(self, dns_server = '127.0.0.1'): + def set_dns_server(self, dns_server='127.0.0.1'): key = HKEY_LOCAL_MACHINE sub_key = "SYSTEM\\CurrentControlSet\\Services\\Tcpip\\Parameters\\Interfaces\\%s" value = 'NameServer' - for adapter in self.get_active_ethernet_adapters(): + for adapter in self.get_active_ethernet_adapters(): # Preserve existing setting - dns_server_backup = self.get_reg_value(key, sub_key % adapter.AdapterName, value) + dns_server_backup = self.get_reg_value(key, sub_key % + adapter.AdapterName, value) - # Restore previous value or a blank string if the key was not present + # Restore previous value or a blank string if the key was not + # present if dns_server_backup: - self.adapters_dns_server_backup[adapter.AdapterName] = (dns_server_backup, adapter.FriendlyName) + self.adapters_dns_server_backup[adapter.AdapterName] = ( + dns_server_backup, adapter.FriendlyName) else: - self.adapters_dns_server_backup[adapter.AdapterName] = ('', adapter.FriendlyName) + self.adapters_dns_server_backup[adapter.AdapterName] = ( + '', adapter.FriendlyName) # Set new dns server value if self.set_reg_value(key, sub_key % adapter.AdapterName, value, dns_server): - self.logger.info('Set DNS server %s on the adapter: %s', dns_server, adapter.FriendlyName) + self.logger.info('Set DNS server %s on the adapter: %s', + dns_server, adapter.FriendlyName) self.notify_ip_change(adapter.AdapterName) else: - self.logger.error('Failed to set DNS server %s on the adapter: %s', dns_server, adapter.FriendlyName) + self.logger.error( + 'Failed to set DNS server %s on the adapter: %s', dns_server, adapter.FriendlyName) def restore_dns_server(self): @@ -1074,75 +1273,61 @@ def restore_dns_server(self): for adapter_name in self.adapters_dns_server_backup: - (dns_server, adapter_friendlyname) = self.adapters_dns_server_backup[adapter_name] + (dns_server, + adapter_friendlyname) = self.adapters_dns_server_backup[adapter_name] # Restore dns server value if self.set_reg_value(key, sub_key % adapter_name, value, dns_server): - self.logger.info('Restored DNS server %s on the adapter: %s', dns_server, adapter_friendlyname) + self.logger.info('Restored DNS server %s on the adapter: %s', + dns_server, adapter_friendlyname) else: - self.logger.error('Failed to restore DNS server %s on the adapter: %s', dns_server, adapter_friendlyname) + self.logger.error( + 'Failed to restore DNS server %s on the adapter: %s', dns_server, adapter_friendlyname) - ########################################################################### - # Check if user is an Administrator - def is_user_an_admin(self): - return ctypes.windll.shell32.IsUserAnAdmin() - - ########################################################################### - # Execute process and detach - def execute_detached(self, execute_cmd): - DETACHED_PROCESS = 0x00000008 - - # import pdb - # pdb.set_trace() - try: - pid = subprocess.Popen(execute_cmd.split(), creationflags=DETACHED_PROCESS).pid - except Exception, e: - self.logger.error('Error: Failed to execute command: %s', execute_cmd) - self.logger.error(' %s', e) - else: - return pid def test_process_list(): class Test(WinUtilMixin): - def __init__(self, name = 'WinUtil'): + def __init__(self, name='WinUtil'): self.logger = logging.getLogger(name) self = Test() - pid = self.get_pid_port_tcp(135) + pid = self._get_pid_port_tcp(135) if pid: - self.logger.info('pid: %d name: %s', pid, self.get_process_image_filename(pid)) + self.logger.info('pid: %d name: %s', pid, + self.get_process_image_filename(pid)) else: self.logger.error('failed to get pid for tcp port 135') - - pid = self.get_pid_port_udp(123) + pid = self._get_pid_port_udp(123) if pid: - self.logger.info('pid: %d name: %s', pid, self.get_process_image_filename(pid)) + self.logger.info('pid: %d name: %s', pid, + self.get_process_image_filename(pid)) else: self.logger.error('failed to get pid for udp port 123') - pid = self.get_pid_port_tcp(1234) + pid = self._get_pid_port_tcp(1234) if not pid: - self.logger.info('successfully returned None for unknown tcp port 1234') + self.logger.info('successfully returned None for unknown tcp port ' + '1234') - pid = self.get_pid_port_udp(1234) + pid = self._get_pid_port_udp(1234) if not pid: - self.logger.info('successfully returned None for unknown udp port 1234') + self.logger.info('successfully returned None for unknown udp port ' + '1234') -def test_interfaces_list(): +def test_interfaces_list(): class Test(WinUtilMixin): - def __init__(self, name = 'WinUtil'): + def __init__(self, name='WinUtil'): self.logger = logging.getLogger(name) self = Test() # for adapter in self.get_adapters_addresses(): - # self.logger.info('ethernet: %s enabled: %s index: %d friendlyname: %s name: %s', adapter.IfType == MIB_IF_TYPE_ETHERNET, adapter.OperStatus == IFOPERSTATUSUP, adapter.IfIndex, adapter.FriendlyName, adapter.AdapterName) - + # self.logger.info('ethernet: %s enabled: %s index: %d friendlyname: %s name: %s', adapter.IfType == MIB_IF_TYPE_ETHERNET, adapter.OperStatus == IFOPERSTATUSUP, adapter.IfIndex, adapter.FriendlyName, adapter.AdapterName) for dns_server in self.get_dns_servers(): self.logger.info('dns: %s', dns_server) @@ -1151,13 +1336,14 @@ def __init__(self, name = 'WinUtil'): self.logger.info('gateway: %s', gateway) for adapter in self.get_active_ethernet_adapters(): - self.logger.info('active ethernet index: %s friendlyname: %s name: %s', adapter.IfIndex, adapter.FriendlyName, adapter.AdapterName) + self.logger.info('active ethernet index: %s friendlyname: %s name: %s', + adapter.IfIndex, adapter.FriendlyName, adapter.AdapterName) def test_registry_nameserver(): class Test(WinUtilMixin): - def __init__(self, name = 'WinUtil'): + def __init__(self, name='WinUtil'): self.logger = logging.getLogger(name) self = Test() @@ -1167,7 +1353,6 @@ def __init__(self, name = 'WinUtil'): value = 'NameServer' data = '127.0.0.1' - data_tmp = self.get_reg_value(key, sub_key, value) self.logger.info('NameServer: %s', data_tmp) @@ -1187,7 +1372,7 @@ def __init__(self, name = 'WinUtil'): def test_registry_gateway(): class Test(WinUtilMixin): - def __init__(self, name = 'WinUtil'): + def __init__(self, name='WinUtil'): self.logger = logging.getLogger(name) self = Test() @@ -1202,15 +1387,15 @@ def __init__(self, name = 'WinUtil'): else: ip = self.get_reg_value(key, sub_key, 'Dhcp') - #self.logger - + # self.logger self.notify_ip_change('{cd17d5b5-bf83-44f5-8de7-d988e3db5451}') + def test_check_connectivity(): class Test(WinUtilMixin): - def __init__(self, name = 'WinUtil'): + def __init__(self, name='WinUtil'): self.logger = logging.getLogger(name) self = Test() @@ -1235,10 +1420,11 @@ def __init__(self, name = 'WinUtil'): else: self.logger.info('DNS server PASS') + def test_stop_service(): class Test(WinUtilMixin): - def __init__(self, name = 'WinUtil'): + def __init__(self, name='WinUtil'): self.logger = logging.getLogger(name) self = Test() @@ -1248,16 +1434,17 @@ def __init__(self, name = 'WinUtil'): def test_start_service(): class Test(WinUtilMixin): - def __init__(self, name = 'WinUtil'): + def __init__(self, name='WinUtil'): self.logger = logging.getLogger(name) self = Test() self.start_service_helper('Dnscache') + def test_get_best_ip(): class Test(WinUtilMixin): - def __init__(self, name = 'WinUtil'): + def __init__(self, name='WinUtil'): self.logger = logging.getLogger(name) self = Test() @@ -1269,24 +1456,22 @@ def __init__(self, name = 'WinUtil'): self.logger.info("IP with gateway address: %s" % ipaddress) - - def main(): pass - #test_process_list() + # test_process_list() - #test_interfaces_list() + # test_interfaces_list() - #test_registry_gateway() + # test_registry_gateway() + # test_check_connectivity() - #test_check_connectivity() - - #test_stop_service() - #test_start_service() + # test_stop_service() + # test_start_service() test_get_best_ip() + if __name__ == '__main__': main() diff --git a/fakenet/fakenet.py b/fakenet/fakenet.py index d56da61..548ae41 100644 --- a/fakenet/fakenet.py +++ b/fakenet/fakenet.py @@ -31,7 +31,7 @@ # FakeNet ############################################################################### -class Fakenet(): +class Fakenet(object): def __init__(self, logging_level = logging.INFO): @@ -151,7 +151,7 @@ def start(self): self.diverter_config['networkmode'] = 'singlehost' from diverters.windows import Diverter - self.diverter = Diverter(self.diverter_config, self.listeners_config, self.logging_level) + self.diverter = Diverter(self.diverter_config, self.listeners_config, ip_addrs, self.logging_level) elif platform_name.lower().startswith('linux'): if self.diverter_config['networkmode'].lower() == 'auto': @@ -194,7 +194,7 @@ def start(self): try: listener_provider_instance.start() except Exception, e: - self.logger.error('Error starting %s listener:', listener_config['listener']) + self.logger.error('Error starting %s listener on port %s:', listener_config['listener'], listener_config['port']) self.logger.error(" %s" % e) # Start the diverter @@ -227,8 +227,6 @@ def stop(self): if self.diverter: self.diverter.stop() - sys.exit(0) - def get_ips(ipvers): """Return IP addresses bound to local interfaces including loopbacks. @@ -272,11 +270,9 @@ def main(): | | / ____ \| . \| |____| |\ | |____ | | | |\ | |__| | |_|/_/ \_\_|\_\______|_| \_|______| |_| |_| \_|\_____| - Version 1.3 + Version 1.4.0 _____________________________________________________________ - Developed by - Peter Kacherginsky and Michael Bailey - FLARE (FireEye Labs Advanced Reverse Engineering) + Developed by FLARE Team _____________________________________________________________ """ @@ -288,6 +284,8 @@ def main(): action="store_true", dest="verbose", default=False, help="print more verbose messages.") parser.add_option("-l", "--log-file", action="store", dest="log_file") + parser.add_option("-f", "--stop-flag", action="store", dest="stop_flag", + help="terminate if stop flag file is created") (options, args) = parser.parse_args() @@ -300,19 +298,34 @@ def main(): fakenet = Fakenet(logging_level) fakenet.parse_config(options.config_file) + + if options.stop_flag: + options.stop_flag = os.path.expandvars(options.stop_flag) + fakenet.logger.info('Will seek stop flag at %s' % (options.stop_flag)) + fakenet.start() try: while True: time.sleep(1) + if options.stop_flag and os.path.exists(options.stop_flag): + fakenet.logger.info('Stop flag found at %s' % (options.stop_flag)) + break except KeyboardInterrupt: - fakenet.stop() + pass except: e = sys.exc_info()[0] fakenet.logger.error("ERROR: %s" % e) - fakenet.stop() + + fakenet.stop() + + # Delete flag only after FakeNet-NG has stopped to indicate completion + if options.stop_flag and os.path.exists(options.stop_flag): + os.remove(options.stop_flag) + + sys.exit(0) if __name__ == '__main__': main() diff --git a/fakenet/listeners/BITSListener.py b/fakenet/listeners/BITSListener.py index c284e47..0980b90 100644 --- a/fakenet/listeners/BITSListener.py +++ b/fakenet/listeners/BITSListener.py @@ -434,7 +434,7 @@ def do_BITS_POST(self): repr(e.internal_exception)) self.__send_response(headers, status_code = status_code) -class BITSListener(): +class BITSListener(object): def taste(self, data, dport): request_methods = ['BITS_POST',] diff --git a/fakenet/listeners/BannerFactory.py b/fakenet/listeners/BannerFactory.py index 929f4d0..c2de6b0 100644 --- a/fakenet/listeners/BannerFactory.py +++ b/fakenet/listeners/BannerFactory.py @@ -3,7 +3,7 @@ import string import datetime -class Banner(): +class Banner(object): """Act like a string, but actually get date/time components on the fly. Returned by BannerFactory.genBanner(). @@ -96,7 +96,7 @@ def fmt(self): return banner -class BannerFactory(): +class BannerFactory(object): def genBanner(self, config, bannerdict, defaultbannerkey='!generic'): """Select and initialize a banner. diff --git a/fakenet/listeners/DNSListener.py b/fakenet/listeners/DNSListener.py index c2b2e78..1b1b287 100644 --- a/fakenet/listeners/DNSListener.py +++ b/fakenet/listeners/DNSListener.py @@ -10,7 +10,7 @@ from . import * -class DNSListener(): +class DNSListener(object): def taste(self, data, dport): diff --git a/fakenet/listeners/FTPListener.py b/fakenet/listeners/FTPListener.py index 8c8ab9d..9551b8e 100644 --- a/fakenet/listeners/FTPListener.py +++ b/fakenet/listeners/FTPListener.py @@ -200,7 +200,7 @@ def rmdir(self, path): # Don't remove anything pass -class FTPListener(): +class FTPListener(object): def taste(self, data, dport): diff --git a/fakenet/listeners/HTTPListener.py b/fakenet/listeners/HTTPListener.py index c60990c..381cff5 100644 --- a/fakenet/listeners/HTTPListener.py +++ b/fakenet/listeners/HTTPListener.py @@ -29,7 +29,7 @@ 'application/xml': 'FakeNet.html' } -class HTTPListener(): +class HTTPListener(object): def taste(self, data, dport): diff --git a/fakenet/listeners/IRCListener.py b/fakenet/listeners/IRCListener.py index cdfab2c..cbe839a 100644 --- a/fakenet/listeners/IRCListener.py +++ b/fakenet/listeners/IRCListener.py @@ -29,7 +29,7 @@ '|------------------------------------------------------------------------|\n') } -class IRCListener(): +class IRCListener(object): def taste(self, data, dport): @@ -241,7 +241,10 @@ def irc_send_client_custom(self, nick, user, servername, message): self.request.sendall(":%s!%s@%s %s\r\n" % (nick, user, servername, message)) class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer): - pass + # Avoid [Errno 98] Address already in use due to TIME_WAIT status on TCP + # sockets, for details see: + # https://stackoverflow.com/questions/4465959/python-errno-98-address-already-in-use + allow_reuse_address = True ############################################################################### # Testing code diff --git a/fakenet/listeners/POPListener.py b/fakenet/listeners/POPListener.py index a1801ec..fedc57a 100644 --- a/fakenet/listeners/POPListener.py +++ b/fakenet/listeners/POPListener.py @@ -22,7 +22,7 @@ Your friend, Bob\r\n""" -class POPListener(): +class POPListener(object): # Once the TCP connection has been established, the POP server initiates # the conversation with +OK message. However, if the client connects diff --git a/fakenet/listeners/ProxyListener.py b/fakenet/listeners/ProxyListener.py index d88ac45..33b193d 100644 --- a/fakenet/listeners/ProxyListener.py +++ b/fakenet/listeners/ProxyListener.py @@ -17,7 +17,7 @@ BUF_SZ = 1024 IP = '0.0.0.0' -class ProxyListener(): +class ProxyListener(object): def __init__( @@ -127,67 +127,6 @@ def run(self): except Exception as e: self.logger.debug('Listener socket exception %s' % e.message) - -class ThreadedUDPClientSocket(threading.Thread): - - - def gen_endpoint_key(self, ip, port): - """e.g. 192.168.19.132/3030""" - return str(ip) + '/' + str(port) - - def __init__(self, ip, port, listener_q, remote_q, config, log, - listener_port, fwd_table, orig_src_ip, orig_src_port): - - super(ThreadedUDPClientSocket, self).__init__() - self.ip = ip - self.port = int(port) - self.listener_q = listener_q - self.remote_q = remote_q - self.config = config - self.logger = log - self.listener_port = int(listener_port) - self.orig_src_ip = orig_src_ip - self.orig_src_port = orig_src_port - self.fwd_table = fwd_table - self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) - self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - - def run(self): - - try: - - # if there is an existing UDP connection from this client, this - # packet needs to exit the proxy from the same port as the previous - # client so the listener knows which stream it belongs to. - connection_exists = False - skey = self.gen_endpoint_key(self.orig_src_ip, self.orig_src_port) - if skey in self.fwd_table: - connection_exists = True - self.port = self.fwd_table[skey] - - self.sock.bind((self.ip, self.port)) - if connection_exists == False: - self.port = self.sock.getsockname()[1] - self.fwd_table[skey] = self.port - self.logger.debug('Proxy connected to listener on port %s' - % self.port) - - while True: - readable, writable, exceptional = select.select([self.sock], - [], [], .001) - if not self.remote_q.empty(): - data = self.remote_q.get() - self.sock.sendto(data, ('localhost', self.listener_port)) - if readable: - data = self.sock.recv(BUF_SZ) - if data: - self.listener_q.put(data) - else: - self.sock.close() - exit(1) - except Exception as e: - self.logger.debug('Listener socket exception %s' % e) - class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer): daemon_threads = True @@ -320,7 +259,6 @@ class ThreadedUDPRequestHandler(SocketServer.BaseRequestHandler): def handle(self): - data = self.request[0] remote_sock = self.request[1] @@ -343,42 +281,15 @@ def handle(self): orig_src_ip, orig_src_port, 'UDP') if top_listener: - - # queue for data received from the listener - listener_q = Queue.Queue() - # queue for data received from remote - remote_q = Queue.Queue() - self.server.logger.debug('Likely listener: %s' % - top_listener.name) - listener_sock = ThreadedUDPClientSocket('localhost', 0, - listener_q, remote_q, self.server.config, - self.server.logger, top_listener.port, - self.server.fwd_table, orig_src_ip, orig_src_port) - listener_sock.daemon = True - listener_sock.start() - remote_sock.setblocking(0) - - # no peek option so process the data already recd - remote_q.put(data) - - try: - while True: - readable, writable, exceptional = select.select( - [remote_sock], [], [], .001) - if readable: - data = remote_sock.recv(BUF_SZ) - if data: - remote_q.put(data) - else: - self.server.logger.debug( - 'Closing remote socket connection') - return - if not listener_q.empty(): - data = listener_q.get() - remote_sock.sendto(data, (orig_src_ip, int(orig_src_port))) - except Exception as e: - pass - + sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + sock.bind(('localhost', 0)) + + sock.sendto(data, ('localhost', int(top_listener.port))) + reply = sock.recv(BUF_SZ) + self.server.logger.info('Received %d bytes.', len(data)) + sock.close() + remote_sock.sendto(reply, (orig_src_ip, int(orig_src_port))) else: self.server.logger.debug('No packet data') diff --git a/fakenet/listeners/RawListener.py b/fakenet/listeners/RawListener.py index 99d1f31..0fbe619 100644 --- a/fakenet/listeners/RawListener.py +++ b/fakenet/listeners/RawListener.py @@ -11,7 +11,7 @@ from . import * -class RawListener(): +class RawListener(object): def taste(self, data, dport): return 1 @@ -146,7 +146,10 @@ def handle(self): self.server.logger.error('Error: %s', e) class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer): - pass + # Avoid [Errno 98] Address already in use due to TIME_WAIT status on TCP + # sockets, for details see: + # https://stackoverflow.com/questions/4465959/python-errno-98-address-already-in-use + allow_reuse_address = True class ThreadedUDPServer(SocketServer.ThreadingMixIn, SocketServer.UDPServer): pass diff --git a/fakenet/listeners/SMTPListener.py b/fakenet/listeners/SMTPListener.py old mode 100755 new mode 100644 index 137af59..8c33225 --- a/fakenet/listeners/SMTPListener.py +++ b/fakenet/listeners/SMTPListener.py @@ -11,7 +11,7 @@ from . import * -class SMTPListener(): +class SMTPListener(object): def taste(self, data, dport): diff --git a/fakenet/listeners/TFTPListener.py b/fakenet/listeners/TFTPListener.py index 0efa7bf..c209800 100644 --- a/fakenet/listeners/TFTPListener.py +++ b/fakenet/listeners/TFTPListener.py @@ -31,7 +31,7 @@ BLOCKSIZE = 512 -class TFTPListener(): +class TFTPListener(object): def taste(self, data, dport): diff --git a/test/scripts/iptables-flush.sh b/test/scripts/iptables-flush.sh old mode 100755 new mode 100644 diff --git a/test/scripts/iptables-list.sh b/test/scripts/iptables-list.sh old mode 100755 new mode 100644 diff --git a/test/scripts/ncscript.sh b/test/scripts/ncscript.sh old mode 100755 new mode 100644 diff --git a/test/template.ini b/test/template.ini new file mode 100644 index 0000000..286524c --- /dev/null +++ b/test/template.ini @@ -0,0 +1,385 @@ +############################################################################### +# Fakenet Configuration + +[FakeNet] + +# Specify whether or not FakeNet should divert traffic. Disable if you want to +# just start listeners and direct traffic manually (e.g. modify DNS server) +DivertTraffic: Yes + +############################################################################### +# Diverter Configuration + +[Diverter] + +# Specify what mode of operation to use. Options: +# SingleHost - manipulate local traffic +# MultiHost - manipulate traffic from foreign hosts +# Auto - Use SingleMode on Windows or use MultiHost on Linux +# +# The current support for these modes on each supported platform is as follows: +# | Windows | Linux | +# -----------+------------+--------------+ +# SingleHost | Functional | Experimental | +# MultiHost | - | Functional | +NetworkMode: SingleHost +# NetworkMode: MultiHost +# NetworkMode: Auto + +# DebugLevel (Linux only as of this writing): specify fine-grained debug print +# flags to enable. Enabling all logging when verbose mode is selected results +# in an unacceptable overhead cost, hence this setting. +DebugLevel: NFQUEUE,IPTALBS,NONLOC,GENPKTV,PCAP + +# MultiHost mode only: Specify what interfaces the Linux Diverter should create +# an iptables rule for to redirect traffic destined for other hosts to the +# local networking stack. This allows FakeNet-NG to log and handle packets +# from remote machines that are destined for non-local IP addresses that are +# either hard-coded or were returned in responses from the DNSListener. Use '*' +# (no quotes) or 'any' (no quotes, case-sensitive!) to indicate that this rule +# should be applied to all interfaces. Comment out to leave unconfigured. +LinuxRedirectNonlocal: * + +# Set LinuxFlushIptables to Yes to have the Linux Diverter flush all iptables +# rules before adding its FakeNet-NG-specific rules to iptables. FakeNet-NG +# will restore all old rules when it exits, unless its termination is +# interrupted. If you disable this setting, and you accidentally interrupt the +# termination of FakeNet-NG (such as by hitting Ctrl+C more than once), then be +# prepared for network mayhem as the Diverter may receive each packet multiple +# times due to duplicate NFQUEUE rules. +LinuxFlushIptables: Yes + +# Incorporated so that users of the binary release may make this work for +# various Linux distros. On Ubuntu, this is `service dns-clean restart`. For +# other distributions, it may be `nscd -I hosts`. Check your manual for +# details. +LinuxFlushDNSCommand: service dns-clean restart + +# Specify whether or not to save captured traffic. You can also change +# the file prefix for the generated PCAPs. +DumpPackets: Yes +DumpPacketsFilePrefix: packets + +# DHCP server running under VMWare Host-Only networking does not configure +# interface gateway and DNS server. Gateway must be configured to allow +# Windows to attempt to route external traffic so that FakeNet could +# could intercept it. This option will automatically generate and set +# appropriate gateway and DNS addresses to allow normal operation. +FixGateway: Yes +FixDNS: Yes + +# Enable 'ModifyLocalDNS' to statically set DNS server to the local machine. +# Linux: Modifies (and restores) /etc/resolv.conf on Linux to make this an +# ephemeral change. +ModifyLocalDNS: Yes + +# Enable 'StopDNSService' to stop Windows DNS client to see the actual +# processes resolving domains. This is a no-op on Linux, until such time as DNS +# caching is observed to interfere with finding the pid associated with a DNS +# request. +StopDNSService: Yes + +# Enable 'RedirectAllTraffic' to optionally divert traffic going to ports not +# specifically listed in one of the listeners below. 'DefaultTCPListener' and +# 'DefaultUDPListener' will handle TCP and UDP traffic going to unspecified ports. +# +# NOTE: Setting default UDP listener will intercept all DNS traffic unless you +# enable a dedicated UDP port 53 DNS listener or add UDP port 53 to the +# 'BlackListPortsUDP' below so that system's default DNS server is used instead. + +RedirectAllTraffic: Yes +DefaultTCPListener: ProxyTCPListener +DefaultUDPListener: ProxyUDPListener + +# Specify TCP and UDP ports to ignore when diverting packets. +# For example, you may want to avoid diverting UDP port 53 (DNS) traffic +# when trying to intercept a specific process while allowing the rest to +# function normally +# +# NOTE: This setting is only honored when 'RedirectAllTraffic' is enabled. + +BlackListPortsTCP: 139 +BlackListPortsUDP: 67, 68, 137, 138, 443, 1900, 5355 + +# Specify processes to ignore when diverting traffic. Windows example used +# here. +# ProcessBlackList: java.exe + +# Specify processes to consider when diverting traffic (others will be +# ignored). Linux examples used here. +# ProcessWhiteList: wget, nc + +# Specify hosts to ignore when diverting traffic. +HostBlackList: 6.6.6.6 + +############################################################################### +# Listener Configuration +# +# Listener configuration consists of generic settings used by the diverter which +# are the same for all listeners and listener specific settings. +# +# NOTE: Listener section names will be used for logging. +# +# NOTE: Settings labels are not case-sensitive. +# +# The following settings are available for all listeners: +# * Enabled - specify whether or not the listener is enabled. +# * Port - TCP or UDP port to listen on. +# * Protocol - TCP or UDP +# * Listener - Listener name to handle traffic. +# * ProcessWhiteList - Only traffic from these processes will be modified +# and the rest will simply be forwarded. +# * ProcessBlackList - Traffic from all but these processes will be simply forwarded +# and the rest will be modified as needed. +# * HostWhiteList - Only traffic to these hosts will be modified and +# the rest will be simply forwarded. +# * HostBlackList - Traffic to these hosts will be simply forwarded +# and the rest will be modified as needed. +# * ExecuteCmd - Execute command on the first connection packet. This is feature is useful +# for extending FakeNet-NG's functionality (e.g. launch a debugger on the +# connecting pid to help with unpacking and decoding.) +# +# The following format string variables are made available: +# * {pid} - process id +# * {procname} - process executable name +# * {src_addr} - source address +# * {src_port} - source port +# * {dst_addr} - destination address +# * {dst_port} - destination port +# +# Listener entry which does not specify a specific listener service +# will still redirect all packets to the local machine on the specified port and +# subject to all the filters (processes, hosts, etc.). However, you must set-up a +# third party service (e.g. proxy servers) to accept these connections. This feature can be +# used to provide FakeNet-NG's passive traffic diverting and filtering capabilities to other +# applications. +# +# Listener specific settings: +# +# * Timeout - Set connection timeout for any listeners that support +# TCP connections (e.g. RawListener, DNSListener, HTTPListener +# SMTPListener). +# * UseSSL - Enable SSL support on the listener (RawListener, HTTPListener) +# * Webroot - Set webroot path for HTTPListener. +# * DumpHTTPPosts - Store HTTP Post requests for the HTTPListener. +# * DumpHTTPPostsFilePrefix - File prefix for the stored HTTP Post requests used by the HTTPListener. +# * BITSFilePrefix - File prefix for the stored BITS uploads used by the BITSListener. +# * TFTPFilePrefix - File prefix for the stored tftp uploads used by the TFTPListener. +# * DNSResponse - IP address to respond with for A record DNS queries. (DNSListener) +# * NXDomains - A number of DNS requests to ignore to let the malware cycle through +# all of the backup C2 servers. (DNSListener) +# * Banner - FTPListener, IRCListener: FTP or IRC banner to display. +# Valid settings are any banner string, or ! where +# is a valid key in the BANNERS dictionary within +# FTPListener.py or IRCListener.py, or !random to +# randomize among the banners in the BANNERS dictionary. +# The default value if none is specified is !generic, +# which selects the banner in the BANNERS dictionary going +# by that key. Banner string may specify the following +# escapes/insertions: +# {servername} - ServerName setting value +# {tz} - Time zone, currently hard-coded to 'UTC' +# * ServerName - FTPListener, IRCListener: FTP or IRC server name for +# insertion into selected default banners or into a +# user-specified banner string. Valid settings are any +# hostname string, !hostname to insert the actual hostname +# of the system, or !random to generate a random hostname +# between 1 and 15 characters (inclusive). + +[ProxyTCPListener] +Enabled: True +Protocol: TCP +Listener: ProxyListener +Port: 38926 +Listeners: HTTPListener, RawListener, FTPListener, DNSListener, POPListener, SMTPListener, TFTPListener, IRCListener, BITSListener +Hidden: False + +[ProxyUDPListener] +Enabled: True +Protocol: UDP +Listener: ProxyListener +Port: 38926 +Listeners: RawListener, DNSListener, TFTPListener, FTPListener +Hidden: False + +[RawTCPListener] +Enabled: True +Port: 1337 +Protocol: TCP +Listener: RawListener +UseSSL: No +Timeout: 10 +Hidden: False + +[HiddenRawTcpListener] +Enabled: True +Port: 12345 +Protocol: TCP +Listener: RawListener +UseSSL: No +Timeout: 10 +Hidden: True + +[RawUDPListener] +Enabled: True +Port: 1337 +Protocol: UDP +Listener: RawListener +UseSSL: No +Timeout: 10 +Hidden: False + +[FilteredListener] +Enabled: False +Port: 31337 +Protocol: TCP +Listener: RawListener +UseSSL: No +Timeout: 10 +ProcessWhiteList: ncat.exe, nc.exe +HostBlackList: 5.5.5.5 +Hidden: False + +[DNS Server] +Enabled: True +Port: 53 +Protocol: UDP +Listener: DNSListener +ResponseA: 192.0.2.123 +ResponseMX: mail.evil2.com +ResponseTXT: FAKENET +NXDomains: 0 +Hidden: False + +[HTTPListener80] +Enabled: True +Port: 80 +Protocol: TCP +Listener: HTTPListener +UseSSL: No +Webroot: defaultFiles/ +Timeout: 10 +#ProcessBlackList: dmclient.exe, OneDrive.exe, svchost.exe, backgroundTaskHost.exe, GoogleUpdate.exe, chrome.exe +DumpHTTPPosts: Yes +DumpHTTPPostsFilePrefix: http +Hidden: False + + +[HTTPListener8080_ProcessBlack] +Enabled: True +Port: 8080 +Protocol: TCP +Listener: HTTPListener +UseSSL: No +Webroot: defaultFiles/ +Timeout: 10 +ProcessBlackList: python, python.exe +DumpHTTPPosts: Yes +DumpHTTPPostsFilePrefix: http +Hidden: False + +[HTTPListener8081_ProcessWhite] +Enabled: True +Port: 8081 +Protocol: TCP +Listener: HTTPListener +UseSSL: No +Webroot: defaultFiles/ +Timeout: 10 +ProcessWhiteList: python, python.exe +DumpHTTPPosts: Yes +DumpHTTPPostsFilePrefix: http +Hidden: False + +[HTTPListener8082HostBlack] +Enabled: True +Port: 8082 +Protocol: TCP +Listener: HTTPListener +UseSSL: No +Webroot: defaultFiles/ +Timeout: 10 +HostBlackList: 6.6.6.6 +DumpHTTPPosts: Yes +DumpHTTPPostsFilePrefix: http +Hidden: False + +[HTTPListener8083HostWhite] +Enabled: True +Port: 8083 +Protocol: TCP +Listener: HTTPListener +UseSSL: No +Webroot: defaultFiles/ +Timeout: 10 +HostWhiteList: 6.6.6.6 +DumpHTTPPosts: Yes +DumpHTTPPostsFilePrefix: http +Hidden: False + + +[HTTPListener443] +Enabled: True +Port: 443 +Protocol: TCP +Listener: HTTPListener +UseSSL: Yes +Webroot: defaultFiles/ +DumpHTTPPosts: Yes +DumpHTTPPostsFilePrefix: http +Hidden: False + +[SMTPListener] +Enabled: True +Port: 25 +Protocol: TCP +Listener: SMTPListener +UseSSL: No +Hidden: False + +[FTPListener21] +Enabled: True +Port: 21 +Protocol: TCP +Listener: FTPListener +UseSSL: No +FTProot: defaultFiles/ +PasvPorts: 60000-60010 +Hidden: False +Banner: !generic +ServerName: !gethostname + +[FTPListenerPASV] +Enabled: True +Port: 60000-60010 +Protocol: TCP +Hidden: False + +[IRCServer] +Enabled: True +Port: 6667 +Protocol: TCP +Listener: IRCListener +UseSSL: No +Banner: !generic +ServerName: !gethostname +Timeout: 30 +Hidden: False + +[TFTPListener] +Enabled: True +Port: 69 +Protocol: UDP +Listener: TFTPListener +TFTPRoot: defaultFiles/ +Hidden: False +TFTPFilePrefix: tftp + +[POPServer] +Enabled: True +Port: 110 +Protocol: TCP +Listener: POPListener +UseSSL: No +Hidden: False + diff --git a/test/test.py b/test/test.py new file mode 100644 index 0000000..30957b4 --- /dev/null +++ b/test/test.py @@ -0,0 +1,895 @@ +import os +import re +import sys +import time +import ctypes +import signal +import socket +import pyping +import ftplib +import poplib +import hashlib +import smtplib +import logging +import binascii +import platform +import requests +import netifaces +import subprocess +import irc.client +import ConfigParser +from collections import OrderedDict + +logger = logging.getLogger('FakeNetTests') +logging.basicConfig(format='%(message)s', level=logging.INFO) + +def is_admin(): + result = False + try: + result = os.getuid() == 0 + except AttributeError: + result = ctypes.windll.shell32.IsUserAnAdmin() != 0 + return result + +def execute_detached(execute_cmd, winders=False): + DETACHED_PROCESS = 0x00000008 + cflags = DETACHED_PROCESS if winders else 0 + cfds = False if winders else True + shl = False if winders else True + + def ign_sigint(): + # Prevent KeyboardInterrupt in FakeNet-NG's console from + # terminating child processes + signal.signal(signal.SIGINT, signal.SIG_IGN) + + + preexec = None if winders else ign_sigint + + try: + pid = subprocess.Popen(execute_cmd, creationflags=cflags, + shell=shl, + close_fds = cfds, + preexec_fn = preexec).pid + except Exception, e: + logger.info('Error: Failed to execute command: %s', execute_cmd) + logger.info(' %s', e) + return None + else: + return pid + +def get_ips(ipvers): + """Return IP addresses bound to local interfaces including loopbacks. + + Parameters + ---------- + ipvers : list + IP versions desired (4, 6, or both); ensures the netifaces semantics + (e.g. netiface.AF_INET) are localized to this function. + """ + specs = [] + results = [] + + for ver in ipvers: + if ver == 4: + specs.append(netifaces.AF_INET) + elif ver == 6: + specs.append(netifaces.AF_INET6) + else: + raise ValueError('get_ips only supports IP versions 4 and 6') + + for iface in netifaces.interfaces(): + for spec in specs: + addrs = netifaces.ifaddresses(iface) + # If an interface only has an IPv4 or IPv6 address, then 6 or 4 + # respectively will be absent from the keys in the interface + # addresses dictionary. + if spec in addrs: + for link in addrs[spec]: + if 'addr' in link: + results.append(link['addr']) + + return results + +def get_external_ip(): + addrs = get_ips([4]) + for addr in addrs: + if not addr.startswith('127.'): + return addr + +class IrcTester(object): + def __init__(self, hostname, port=6667): + self.hostname = hostname + self.port = port + + self.nick = 'dr_evil' + self.join_chan = '#whatevs' + self.clouseau = 'inspector_clouseau' + self.safehouse = "I'm looking for a safe house." + self.pub_chan = '#evil_bartenders' + self.black_market = 'Black Market' + + def _irc_evt_handler(self, srv, evt): + """Check for each case and set the corresponding success flag.""" + if evt.type == 'join': + if evt.target.startswith(self.join_chan): + self.join_ok = True + elif evt.type == 'welcome': + if evt.arguments[0].startswith('Welcome to IRC'): + self.welcome_ok = True + elif evt.type == 'privmsg': + if (evt.arguments[0].startswith(self.safehouse) and + evt.source.startswith(self.clouseau)): + self.privmsg_ok = True + elif evt.type == 'pubmsg': + if (evt.arguments[0].startswith(self.black_market) and + evt.target == self.pub_chan): + self.pubmsg_ok = True + + def _irc_script(self, srv): + """Callback manages individual test cases for IRC.""" + # Clear success flags + self.welcome_ok = False + self.join_ok = False + self.privmsg_ok = False + self.pubmsg_ok = False + + # This handler should set the success flags in success cases + srv.add_global_handler('join', self._irc_evt_handler) + srv.add_global_handler('welcome', self._irc_evt_handler) + srv.add_global_handler('privmsg', self._irc_evt_handler) + srv.add_global_handler('pubmsg', self._irc_evt_handler) + + # Issue all commands, indirectly invoking the event handler for each + # flag + + srv.join(self.join_chan) + srv.process_data() + + srv.privmsg(self.pub_chan, self.black_market) + srv.process_data() + + srv.privmsg(self.clouseau, self.safehouse) + srv.process_data() + + srv.quit() + srv.process_data() + + if not self.welcome_ok: + raise FakeNetTestException('Welcome test failed') + + if not self.join_ok: + raise FakeNetTestException('Join test failed') + + if not self.privmsg_ok: + raise FakeNetTestException('privmsg test failed') + + if not self.pubmsg_ok: + raise FakeNetTestException('pubmsg test failed') + + return all([ + self.welcome_ok, + self.join_ok, + self.privmsg_ok, + self.pubmsg_ok + ]) + + def _run_irc_script(self, nm, callback): + """Connect to server and give control to callback.""" + r = irc.client.Reactor() + srv = r.server() + srv.connect(self.hostname, self.port, self.nick) + retval = callback(srv) + srv.close() + return retval + + def test_irc(self): + return self._run_irc_script('testnm', self._irc_script) + +class FakeNetTestException(Exception): + """A recognizable exception type indicating a known failure state based on + test criteria. HTTP test uses this, others may in the future, too. + """ + pass + +class FakeNetTester(object): + """Controller for FakeNet-NG that runs test cases""" + + def __init__(self, settings): + self.settings = settings + self.pid_fakenet = None + + def _setStopFlag(self): + with open(self.settings.stopflag, 'w') as f: + f.write('1') + + def _clearStopFlag(self): + if os.path.exists(self.settings.stopflag): + os.remove(self.settings.stopflag) + + def _confirmFakenetStopped(self): + return not os.path.exists(self.settings.stopflag) + + def _waitFakenetStopped(self, timeoutsec=None): + retval = False + + while True: + if self._confirmFakenetStopped(): + retval = True + break + time.sleep(1) + + if timeoutsec is not None: + timeoutsec -= 1 + if timeoutsec <= 0: + break + + return retval + + def _checkPid(self, pid): + retval = False + if self.settings.windows: + PROCESS_TERMINATE = 1 + p = ctypes.windll.kernel32.OpenProcess(PROCESS_TERMINATE, 0, pid) + retval = p != 0; + if p: + ctypes.windll.kernel32.CloseHandle(p) + else: + # https://stackoverflow.com/questions/568271/how-to-check-if-there-exists-a-process-with-a-given-pid-in-python + try: + os.kill(pid, 0) + except OSError: + pass + else: + retval = True + + return retval + + def _kill(self, pid): + if self.settings.windows: + PROCESS_TERMINATE = 1 + # Note, this will get a handle even after the process terminates, + # in which case TerminateProcess will simply return FALSE. + p = ctypes.windll.kernel32.OpenProcess(PROCESS_TERMINATE, 0, pid) + if p: + ok = ctypes.windll.kernel32.TerminateProcess(p, 1) + ctypes.windll.kernel32.CloseHandle(p) + else: + os.kill(pid, signal.SIGKILL) + + def stopFakenetAndWait(self, timeoutsec=None, kill=False): + if not self.pid_fakenet: + raise RuntimeError('FakeNet-NG not running, nothing to stop') + + self._setStopFlag() + stopped_responsive = self._waitFakenetStopped(timeoutsec) + + if not stopped_responsive: + self._clearStopFlag() + + if kill and self._checkPid(self.pid_fakenet): + self._kill(self.pid_fakenet) + + self.pid_fakenet = None + + return stopped_responsive + + def executeFakenet(self): + if self.pid_fakenet: + raise RuntimeError('FakeNet-NG already running, PID %d' % + (self.pid_fakenet)) + + os.chdir(self.settings.fndir) + + max_del_attempts = 3 + if os.path.exists(self.settings.logpath): + for i in range(1, max_del_attempts + 1): + try: + os.remove(self.settings.logpath) + except WindowsError: # i.e. log file locked by another process + logger.warning('Failed to delete %s, attempt %d' % + (self.settings.logpath, i)) + if i == max_del_attempts: + logger.error('Final attempt, re-raising exception') + raise + else: + logger.warning('Retrying in %d seconds...' % (i)) + time.sleep(i) + else: + break + + cmd = self.settings.genFakenetCmd() + logger.info('About to run %s' % (cmd)) + self.pid_fakenet = execute_detached(cmd, self.settings.windows) + if self.pid_fakenet: + logger.info('FakeNet started with PID %s' % (str(self.pid_fakenet))) + + return (self.pid_fakenet is not None) + + def delConfig(self): + if os.path.exists(self.settings.configpath): + os.remove(self.settings.configpath) + + def doTests(self, match_spec): + self.testGeneral(match_spec) + self.testNoRedirect(match_spec) + self.testBlacklistProcess(match_spec) + self.testWhitelistProcess(match_spec) + + def _printStatus(self, desc, passed): + status = 'Passed' if passed else 'FAILED' + punc = '[ + ]' if passed else '[!!!]' + logger.info('%s %s: %s' % (punc, status, desc)) + + def _tryTest(self, desc, callback, args, expected): + retval = None + try: + retval = callback(*args) + except Exception as e: + logger.info('Test %s: Uncaught exception of type %s: %s' % + (desc, str(type(e)), str(e))) + + passed = (retval == expected) + + return passed + + def _filterMatchingTests(self, tests, matchspec): + """Remove tests that match negative specifications (regexes preceded by + a minus sign) or do not match positive specifications (regexes not + preceded by a minus sign). + + Modifies the contents of the tests dictionary. + """ + negatives = [] + positives = [] + + if len(matchspec): + # If the user specifies a minus sign before a regular expression, + # match negatively (exclude any matching tests) + for spec in matchspec: + if spec.startswith('-'): + negatives.append(spec[1:]) + else: + positives.append(spec) + + # Iterating over tests first, match specifications second to + # preserve the order of the selected tests. Less efficient to + # compile every regex several times, but less confusing. + for testname, test in tests.items(): + + # First determine if it is to be excluded, in which case, + # remove it and do not evaluate further match specifications. + exclude = False + for spec in negatives: + if bool(re.search(spec, testname)): + exclude = True + if exclude: + tests.pop(testname) + continue + + # If the user ONLY specified negative match specifications, + # then admit all tests + if not len(positives): + continue + + # Otherwise, only admit if it matches a positive spec + include = True + for spec in positives: + if bool(re.search(spec, testname)): + include = True + break + if not include: + tests.pop(testname) + + return + + def _testGeneric(self, label, config, tests, matchspec=[]): + self._filterMatchingTests(tests, matchspec) + if not len(tests): + logger.info('No matching tests') + return False + + # If doing a multi-host test, then toggle the network mode + if not self.settings.singlehost: + config.multiHostMode() + + self.writeConfig(config) + + if self.settings.singlehost: + if not self.executeFakenet(): + self.delConfig() + return False + + sec = self.settings.sleep_after_start + logger.info('Sleeping %d seconds before commencing' % (sec)) + time.sleep(sec) + else: + logger.info('Waiting for you to transition the remote FakeNet-NG') + logger.info('system to run the %s test suite' % (label)) + logger.info('(Copy this config: %s)' % (self.settings.configpath)) + logger.info('') + while True: + logger.info('Type \'ok\' to continue, or \'exit\' to stop') + try: + ok = raw_input() + except EOFError: + ok = 'exit' + + if ok.lower() in ['exit', 'quit', 'stop', 'n', 'no']: + sys.exit(0) + elif ok.lower() in ['ok', 'okay', 'go', 'y', 'yes']: + break + + logger.info('-' * 79) + logger.info('Testing') + logger.info('-' * 79) + + # Do each test + for desc, (callback, args, expected) in tests.iteritems(): + logger.debug('Testing: %s' % (desc)) + passed = self._tryTest(desc, callback, args, expected) + + # Retry in case of transient error e.g. timeout + if not passed: + logger.debug('Retrying: %s' % (desc)) + passed = self._tryTest(desc, callback, args, expected) + + self._printStatus(desc, passed) + + time.sleep(0.5) + + logger.info('-' * 79) + logger.info('Tests complete') + logger.info('-' * 79) + + if self.settings.singlehost: + sec = self.settings.sleep_before_stop + logger.info('Sleeping %d seconds before transitioning' % (sec)) + time.sleep(sec) + + logger.info('Stopping FakeNet-NG and waiting for it to complete') + responsive = self.stopFakenetAndWait(15, True) + + if responsive: + logger.info('FakeNet-NG is stopped') + else: + logger.info('FakeNet-NG was no longer running or was stopped forcibly') + + time.sleep(1) + + self.delConfig() + + def _test_sk(self, proto, host, port, timeout=5): + """Test socket-oriented""" + retval = False + s = socket.socket(socket.AF_INET, proto) + s.settimeout(timeout) + + try: + s.connect((host, port)) + + teststring = 'Testing FakeNet-NG' + remaining = len(teststring) + + while remaining: + sent = s.send(teststring) + if sent == 0: + raise IOError('Failed to send all bytes') + remaining -= sent + + recvd = '' + remaining = len(teststring) + + while remaining: + chunk = s.recv(remaining) + if chunk == '': + raise IOError('Failed to receive all bytes') + remaining -= len(chunk) + recvd += chunk + + retval = (recvd == teststring) + + except socket.error as e: + logger.error('Socket error: %s (%s %s:%d)' % + (str(e), proto, host, port)) + except Exception as e: + logger.error('Non-socket Exception received: %s' % (str(e))) + + return retval + + def _test_icmp(self, host): + r = pyping.ping(host, count=1) + return (r.ret_code == 0) + + def _test_ns(self, hostname, expected): + return (expected == socket.gethostbyname(hostname)) + + def _test_smtp_ssl(self, sender, recipient, msg, hostname, port=None, timeout=5): + smtpserver = smtplib.SMTP_SSL(hostname, port, 'fake.net', None, None, timeout) + server.sendmail(sender, recipient, msg) + smtpserver.quit() + + def _test_smtp(self, sender, recipient, msg, hostname, port=None, timeout=5): + smtpserver = smtplib.SMTP(hostname, port, 'fake.net', timeout) + smtpserver.sendmail(sender, recipient, msg) + smtpserver.quit() + + return True + + def _test_pop(self, hostname, port=None, timeout=5): + pop3server = poplib.POP3(hostname, port, timeout) + pop3server.user('popuser') + pop3server.pass_('password') + msg = pop3server.retr(1) + + response = msg[0] + lines = msg[1] + octets = msg[2] + + if not response.startswith('+OK'): + msg = 'POP3 response does not start with "+OK"' + logger.error(msg) + return False + + if not 'Alice' in ''.join(lines): + msg = 'POP3 message did not contain expected string' + raise FakeNetTestException(msg) + return False + + return True + + def _util_irc(self, nm, hostname, port, nick, callback): + r = irc.client.Reactor() + srv = r.server() + srv.connect(hostname, port, nick) + retval = callback(srv) + srv.close() + return retval + + def _test_irc(self, hostname, port=6667): + irc_tester = IrcTester(hostname, port) + return irc_tester.test_irc() + + def _test_http(self, hostname, port=None): + """Test HTTP Listener""" + retval = False + + if port: + url = 'http://%s:%d/asdf.html' % (hostname, port) + else: + url = 'http://%s/asdf.html' % (hostname) + + try: + r = requests.get(url, timeout=3) + + if r.status_code != 200: + raise FakeNetTestException('Status code %d' % (r.status_code)) + + teststring = 'H T T P L I S T E N E R' + if teststring not in r.text: + raise FakeNetTestException('Test string not in response') + + retval = True + + except requests.exceptions.Timeout as e: + pass + + except FakeNetTestException as e: + pass + + return retval + + def _test_ftp(self, hostname, port=None): + """Note that the FakeNet-NG Proxy listener won't know what to do with + this client if you point it at some random port, because the client + listens silently for the server 220 welcome message which doesn't give + the Proxy listener anything to work with to decide where to forward it. + """ + fullbuf = '' + + m = hashlib.md5() + + def update_hash(buf): + m.update(buf) + + f = ftplib.FTP() + f.connect(hostname, port) + f.login() + f.set_pasv(False) + f.retrbinary('RETR FakeNet.gif', update_hash) + f.quit() + + digest = m.digest() + expected = binascii.unhexlify('a6b78c4791dc8110dec6c55f8a756395') + + return (digest == expected) + + def testNoRedirect(self, matchspec=[]): + config = self.makeConfig(singlehostmode=True, proxied=False, redirectall=False) + + domain_dne = self.settings.domain_dne + ext_ip = self.settings.ext_ip + arbitrary = self.settings.arbitrary + localhost = self.settings.localhost + + tcp = socket.SOCK_STREAM + udp = socket.SOCK_DGRAM + + t = OrderedDict() # The tests + + t['RedirectAllTraffic disabled external IP @ bound'] = (self._test_sk, (tcp, ext_ip, 1337), True) + t['RedirectAllTraffic disabled external IP @ unbound'] = (self._test_sk, (tcp, ext_ip, 9999), False) + + t['RedirectAllTraffic disabled arbitrary host @ bound'] = (self._test_sk, (tcp, arbitrary, 1337), False) + t['RedirectAllTraffic disabled arbitrary host @ unbound'] = (self._test_sk, (tcp, arbitrary, 9999), False) + + t['RedirectAllTraffic disabled named host @ bound'] = (self._test_sk, (tcp, domain_dne, 1337), False) + t['RedirectAllTraffic disabled named host @ unbound'] = (self._test_sk, (tcp, domain_dne, 9999), False) + + if self.settings.singlehost: + t['RedirectAllTraffic disabled localhost @ bound'] = (self._test_sk, (tcp, localhost, 1337), True) + t['RedirectAllTraffic disabled localhost @ unbound'] = (self._test_sk, (tcp, localhost, 9999), False) + + return self._testGeneric('No Redirect', config, t, matchspec) + + def testBlacklistProcess(self, matchspec=[]): + config = self.makeConfig() + config.blacklistProcess(self.settings.pythonname) + + arbitrary = self.settings.arbitrary + + tcp = socket.SOCK_STREAM + udp = socket.SOCK_DGRAM + + t = OrderedDict() # The tests + + if self.settings.singlehost: + t['Global blacklisted process test'] = (self._test_sk, (tcp, arbitrary, 9999), False) + + return self._testGeneric('Global process blacklist', config, t, matchspec) + + def testWhitelistProcess(self, matchspec=[]): + config = self.makeConfig() + config.whitelistProcess(self.settings.pythonname) + + arbitrary = self.settings.arbitrary + + tcp = socket.SOCK_STREAM + udp = socket.SOCK_DGRAM + + t = OrderedDict() # The tests + + if self.settings.singlehost: + t['Global whitelisted process test'] = (self._test_sk, (tcp, arbitrary, 9999), True) + + return self._testGeneric('Global process whitelist', config, t, matchspec) + + def testGeneral(self, matchspec=[]): + config = self.makeConfig() + + domain_dne = self.settings.domain_dne + ext_ip = self.settings.ext_ip + arbitrary = self.settings.arbitrary + blacklistedhost = self.settings.blacklistedhost + blacklistedtcp = self.settings.blacklistedtcp + blacklistedudp = self.settings.blacklistedudp + localhost = self.settings.localhost + dns_expected = self.settings.dns_expected + hidden_tcp = self.settings.hidden_tcp + no_service = self.settings.no_service + + sender = self.settings.sender + recipient = self.settings.recipient + smtpmsg = self.settings.smtpmsg + + tcp = socket.SOCK_STREAM + udp = socket.SOCK_DGRAM + + t = OrderedDict() # The tests + + t['TCP external IP @ bound'] = (self._test_sk, (tcp, ext_ip, 1337), True) + t['TCP external IP @ unbound'] = (self._test_sk, (tcp, ext_ip, 9999), True) + t['TCP arbitrary @ bound'] = (self._test_sk, (tcp, arbitrary, 1337), True) + t['TCP arbitrary @ unbound'] = (self._test_sk, (tcp, arbitrary, 9999), True) + t['TCP domainname @ bound'] = (self._test_sk, (tcp, domain_dne, 1337), True) + t['TCP domainname @ unbound'] = (self._test_sk, (tcp, domain_dne, 9999), True) + if self.settings.singlehost: + t['TCP localhost @ bound'] = (self._test_sk, (tcp, localhost, 1337), True) + t['TCP localhost @ unbound'] = (self._test_sk, (tcp, localhost, 9999), False) + + t['UDP external IP @ bound'] = (self._test_sk, (udp, ext_ip, 1337), True) + t['UDP external IP @ unbound'] = (self._test_sk, (udp, ext_ip, 9999), True) + t['UDP arbitrary @ bound'] = (self._test_sk, (udp, arbitrary, 1337), True) + t['UDP arbitrary @ unbound'] = (self._test_sk, (udp, arbitrary, 9999), True) + t['UDP domainname @ bound'] = (self._test_sk, (udp, domain_dne, 1337), True) + t['UDP domainname @ unbound'] = (self._test_sk, (udp, domain_dne, 9999), True) + if self.settings.singlehost: + t['UDP localhost @ bound'] = (self._test_sk, (udp, localhost, 1337), True) + t['UDP localhost @ unbound'] = (self._test_sk, (udp, localhost, 9999), False) + + t['ICMP external IP'] = (self._test_icmp, (ext_ip,), True) + t['ICMP arbitrary host'] = (self._test_icmp, (arbitrary,), True) + t['ICMP domainname'] = (self._test_icmp, (domain_dne,), True) + + t['DNS listener test'] = (self._test_ns, (domain_dne, dns_expected), True) + t['HTTP listener test'] = (self._test_http, (arbitrary,), True) + t['FTP listener test'] = (self._test_ftp, (arbitrary,), True) + t['POP3 listener test'] = (self._test_pop, (arbitrary, 110), True) + t['SMTP listener test'] = (self._test_smtp, (sender, recipient, smtpmsg, arbitrary), True) + + # Does not work, SSL error + t['SMTP SSL listener test'] = (self._test_smtp_ssl, (sender, recipient, smtpmsg, arbitrary), True) + + # Works on Linux, not on Windows + t['IRC listener test'] = (self._test_irc, (arbitrary,), True) + + t['Proxy listener HTTP test'] = (self._test_http, (arbitrary, no_service), True) + t['Proxy listener hidden test'] = (self._test_http, (arbitrary, hidden_tcp), True) + + t['TCP blacklisted host @ unbound'] = (self._test_sk, (tcp, blacklistedhost, 9999), False) + t['TCP arbitrary @ blacklisted unbound'] = (self._test_sk, (tcp, arbitrary, blacklistedtcp), False) + t['UDP arbitrary @ blacklisted unbound'] = (self._test_sk, (udp, arbitrary, blacklistedudp), False) + + if self.settings.singlehost: + t['Listener process blacklist'] = (self._test_http, (arbitrary, self.settings.listener_proc_black), False) + t['Listener process whitelist'] = (self._test_http, (arbitrary, self.settings.listener_proc_white), True) + t['Listener host blacklist'] = (self._test_http, (arbitrary, self.settings.listener_host_black), True) + t['Listener host whitelist'] = (self._test_http, (arbitrary, self.settings.listener_host_black), True) + + return self._testGeneric('General', config, t, matchspec) + + def makeConfig(self, singlehostmode=True, proxied=True, redirectall=True): + template = self.settings.configtemplate + return FakeNetConfig(template, singlehostmode, proxied, redirectall) + + def writeConfig(self, config): + logger.info('Writing config to %s' % (self.settings.configpath)) + config.write(self.settings.configpath) + +class FakeNetConfig: + """Convenience class to read/modify/rewrite a configuration template.""" + + def __init__(self, path, singlehostmode=True, proxied=True, redirectall=True): + self.rawconfig = ConfigParser.RawConfigParser() + self.rawconfig.read(path) + + if singlehostmode: + self.singleHostMode() + else: + self.multiHostMode() + + if not proxied: self.noProxy() + + self.setRedirectAll(redirectall) + + def blacklistProcess(self, process): self.rawconfig.set('Diverter', 'ProcessBlacklist', process) + def whitelistProcess(self, process): self.rawconfig.set('Diverter', 'ProcessWhitelist', process) + + def setRedirectAll(self, enabled): + if enabled: + self.rawconfig.set('Diverter', 'RedirectAllTraffic', 'Yes') + else: + self.rawconfig.set('Diverter', 'RedirectAllTraffic', 'No') + + def singleHostMode(self): self.rawconfig.set('Diverter', 'NetworkMode', 'SingleHost') + def multiHostMode(self): self.rawconfig.set('Diverter', 'NetworkMode', 'MultiHost') + + def noProxy(self): + self.rawconfig.remove_section('ProxyTCPListener') + self.rawconfig.remove_section('ProxyUDPListener') + self.rawconfig.set('Diverter', 'DefaultTCPListener', 'RawTCPListener') + self.rawconfig.set('Diverter', 'DefaultUDPListener', 'RawUDPListener') + + def write(self, path): + with open(path, 'w') as f: + return self.rawconfig.write(f) + +class FakeNetTestSettings: + """Test constants/literals, some of which may vary per OS, etc.""" + + def __init__(self, startingpath, singlehost=True): + self.singlehost = singlehost + self.startingpath = startingpath + self.configtemplate = os.path.join(startingpath, 'template.ini') + + # Where am I? Who are you? + self.platform_name = platform.system() + self.windows = (self.platform_name == 'Windows') + self.linux = (self.platform_name.lower().startswith('linux')) + + # Paths + self.configpath = self.genPath('%TEMP%\\fakenet.ini', '/tmp/fakenet.ini') + self.stopflag = self.genPath('%TEMP%\\stop_fakenet', '/tmp/stop_fakenet') + self.logpath = self.genPath('%TEMP%\\fakenet.log', '/tmp/fakenet.log') + self.fakenet = self.genPath('fakenet', 'python fakenet.py') + self.fndir = self.genPath('.', '$HOME/files/src/flare-fakenet-ng/fakenet') + + # For process blacklisting + self.pythonname = os.path.basename(sys.executable) + + # Various + self.ext_ip = get_external_ip() + self.arbitrary = '8.8.8.8' + self.blacklistedhost = '6.6.6.6' + self.blacklistedtcp = 139 + self.blacklistedudp = 67 + self.hidden_tcp = 12345 + self.no_service = 10 + self.listener_proc_black = 8080 # HTTP listener with process blacklist + self.listener_proc_white = 8081 # HTTP listener with process whitelists + self.listener_host_black = 8082 # HTTP listener with host blacklist + self.listener_host_white = 8083 # HTTP listener with host whitelists + self.localhost = '127.0.0.1' + self.dns_expected = '192.0.2.123' + self.domain_dne = 'does-not-exist-amirite.fireeye.com' + self.sender = 'from-fakenet@example.org' + self.recipient = 'to-fakenet@example.org' + self.smtpmsg = 'FakeNet-NG SMTP test email' + + # Behaviors + self.sleep_after_start = 4 + self.sleep_before_stop = 1 + + def genPath(self, winpath, unixypath): + if self.windows: + return os.path.expandvars(winpath) + else: + return os.path.expandvars(unixypath) + + def genFakenetCmd(self): + return ('%s -f %s -l %s -c %s' % + (self.fakenet, self.stopflag, self.logpath, self.configpath)) + +def is_ip(s): + pat = '^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$' + return bool(re.match(pat, s)) + +def main(): + if not is_admin(): + logger.error('Not an admin, exiting...') + sys.exit(1) + + if len(sys.argv) < 2: + logger.error('Usage: test.py [matchspec1 [matchspec2 [...] ] ]') + logger.error('') + logger.error('Valid where:') + logger.error(' here') + logger.error(' Any dot-decimal IP address') + logger.error('') + logger.error('Each match specification is a regular expression that') + logger.error('will be compared against test names, and any matches') + logger.error('will be included. Because regular expression negative') + logger.error('matching is complicated to use, you can just prefix') + logger.error('a match specification with a minus sign to indicate') + logger.error('that you would like to include only tests that do NOT') + logger.error('match the expression.') + sys.exit(1) + + # Validate where + where = sys.argv[1] + + singlehost = (where.lower() == 'here') + + if not singlehost and not is_ip(where): + logger.error('Invalid where: %s' % (where)) + sys.exit(1) + + # Will execute only tests matching *match_spec if specified + match_spec = sys.argv[2:] + + if len(match_spec): + logger.info('Only running tests that match the following ' + + 'specifications:') + for spec in match_spec: + logger.info(' %s' % (spec)) + + # Doit + startingpath = os.getcwd() + settings = FakeNetTestSettings(startingpath, singlehost) + if not singlehost: # was an IP, so record it + settings.ext_ip = where + tester = FakeNetTester(settings) + logger.info('Running with privileges on %s' % (settings.platform_name)) + tester.doTests(match_spec) + +if __name__ == '__main__': + main()