Skip to content
Permalink
Browse files

Refactor candidate 2 (#75)

# 1.4.0:
* Refactor FakeNet-NG to unify Windows and Linux packet handling
* Remove Proxy Listener UDP stream abstraction to prevent issue where
  subsequent clients do not receive response packets because the proxy listener
  continues to send them to the old (expired) ephemeral port for the previous
  client
* Stop flag command-line support for rudimentary IPC-based start/stop
  automation
* Integration test script for MultiHost and SingleHost mode
* Fixed Errno 98 (`TIME_WAIT`) issue with `RawTcpListener`
* WinDivert `GetLastError` exception work-around for [WinDivert issue
  #32](ffalcinelli/pydivert#32)
  • Loading branch information...
strictlymike committed May 1, 2018
1 parent dcdcfcb commit e54c7374a37660a26b17b8b572d7692f395391db
@@ -0,0 +1,14 @@
# Change Log

## 1.4.0:
* Refactor FakeNet-NG to unify Windows and Linux packet handling
* Remove Proxy Listener UDP stream abstraction to prevent issue where
subsequent clients do not receive response packets because the proxy listener
continues to send them to the old (expired) ephemeral port for the previous
client
* Stop flag command-line support for rudimentary IPC-based start/stop
automation
* Integration test script for MultiHost and SingleHost mode
* Fixed Errno 98 (`TIME_WAIT`) issue with `RawTcpListener`
* WinDivert `GetLastError` exception work-around for [WinDivert issue
#32](https://github.com/ffalcinelli/pydivert/issues/32)
@@ -0,0 +1,35 @@
# Contributors

This document credits those who conceptualized and/or implemented features for
FakeNet-NG.

## Legacy

FakeNet-NG is based on the original
[FakeNet](https://practicalmalwareanalysis.com/fakenet/) tool developed by
Andrew Honig and Michael Sikorski, which is still the tool of choice for
malware analysis on Windows XP.

## Windows

Peter Kacherginsky [implemented
FakeNet-NG](https://www.fireeye.com/blog/threat-research/2016/08/fakenet-ng_next_gen.html)
targeting modern versions of Windows.

## Linux and Core

Michael Bailey [implemented FakeNet-NG on
Linux](https://www.fireeye.com/blog/threat-research/2017/07/linux-support-for-fakenet-ng.html),
and later refactored FakeNet-NG to use this as the unified packet processing
logic for both Windows and Linux.

## Content-Based Protocol Detection

The original FakeNet-NG was able to automatically handle SSL; meanwhile, Joshua
Homan developed the original concept of using a protocol "taste" callback to
sample traffic and direct clients to the appropriate server ports. Matthew
Haigh, Michael Bailey, and Peter Kacherginsky conceptualized the Proxy Listener
and Hidden Listener mechanisms for introducing both of these content-based
protocol detection features to FakeNet-NG. Matthew Haigh then [implemented
Content-Based Protocol
Detection](https://www.fireeye.com/blog/threat-research/2017/10/fakenet-content-based-protocol-detection.html).
@@ -1,15 +1,9 @@
# FakeNet-NG Internals

## FakeNet-NG Linux Diverter Internals
This documentation was originally written for the Linux implementation, and
where specifics are called for, it currently references Linux.

The FakeNet-NG Diverter implementation for Linux uses
[netfilter](http://netfilter.org/) to examine, log, and redirect packets.
Netfilter is available at the kernel level through loadable kernel modules
(LKMs) and in userspace through the
[libnetfilter_queue](http://netfilter.org/projects/libnetfilter_queue/index.html)
library. Since FakeNet-NG is Python-based, it uses libnetfilter_queue via the
[python-netfilterqueue](https://github.com/kti/python-netfilterqueue/) wrapper
which is written in Cython.
## FakeNet-NG Diverter Internals

For purposes of this documentation, some rigorously-defined terms will be
repurposed or replaced:
@@ -21,6 +15,20 @@ repurposed or replaced:
we will use the more general term _conversation_ to represent the concept of
a pair of endpoints (again, using that term loosely) that are communicating.

FakeNet-NG can also operate in two modes on Linux:
* `SingleHost` - simulate the Internet for the local machine
* `MultiHost` - simulate the Internet by acting as the gateway for another
machine

Each implementation of FakeNet-NG ultimately relies on a driver or kernel
module that supports network hooking and a library that makes this accessible
from user space. The Windows Diverter uses
[PyDivert](https://github.com/ffalcinelli/pydivert) to control the
[WinDivert](https://reqrypt.org/windivert.html) driver. The Linux Diverter uses
[python-netfilterqueue](https://github.com/kti/python-netfilterqueue) to access
[libnetfilter_queue](https://netfilter.org/projects/libnetfilter_queue/) and in
turn [NetFilter](https://netfilter.org/).

### Traffic Flow Condition Evaluation

The simplest case for the Linux implementation of the FakeNet-NG Diverter is
@@ -30,10 +38,9 @@ we use `iptables` to implement a `REDIRECT` rule in the `PREROUTING` chain.
In this use case, FakeNet-NG implements only dynamic port forwarding (DPF)
using python-netfilterqueue.

The most complicated case is `SingleHost` mode (experimental), in which both
DPF and NAT must be controlled by FakeNet-NG to permit process blacklisting
and other configuration settings. In this case, FakeNet-NG uses
python-netfilterqueue to evaluate four conditions:
The more complicated case is `SingleHost` mode, in which both DPF and NAT must
be controlled by FakeNet-NG to permit process blacklisting and other
configuration settings. In this case, FakeNet-NG evaluates four conditions:
1. When a packet is produced, is it destined for a foreign IP address? (if so,
fix up its destination address to be a local address)
2. When a packet is about to be consumed, is it destined for an unbound port?
@@ -45,8 +52,8 @@ python-netfilterqueue to evaluate four conditions:
that has been NATted? (if so, fix up its source IP)

Given two processes `P1` and `P2`, here is a diagram of communication and
condition evaluation using the `INPUT` and `OUTPUT` chains provided by
Netfilter:
condition evaluation specific to Linux, using the `INPUT` and `OUTPUT` chains
provided by Netfilter:

```
(1) (2)
@@ -288,7 +295,6 @@ Or, in Python:
(not sport_bound and not dport_bound))
```


### Future

#### NetworkMode Auto for Linux
@@ -312,11 +318,12 @@ troubleshooting problems transferring the 24KB file `FakeNet.gif` over FTP.

This is fine for `MultiHost` mode because external interfaces (e.g. `eth0`)
frequently have a maximum transmittal unit (MTU) of 1500. However, for loopback
communications where the MTU is 65536, this causes errors. It is possible to
fix these errors by changing the buffer size to 65616, however this may be
overridden by future installations of python-netfilterqueue either via the
package management system specific to the Linux distribution, Pip, etc.
communications where the MTU may be something like 65536, this causes errors.
It is possible to fix these errors by changing the buffer size to 65616
(accounting for 80 bytes of overhead), however this could be overridden by
future installations of python-netfilterqueue either via the package management
system specific to the Linux distribution, Pip, etc.

A work-around for this issue is to send all NAT packets through an externally
facing IP address instead of 127.0.0.1 to avoid exposing ourselves to
`BufferSize < MTU` conditions such as in the transfer of large files.
facing IP address instead of 127.0.0.1 to avoid exposing traffic to `BufferSize
< MTU` conditions such as in the transfer of large files.
@@ -26,9 +26,26 @@ DivertTraffic: Yes
# NetworkMode: MultiHost
NetworkMode: Auto

# DebugLevel (Linux only as of this writing): specify fine-grained debug print
# flags to enable. Enabling all logging when verbose mode is selected results
# in an unacceptable overhead cost, hence this setting.
# DebugLevel: specify fine-grained debug print flags to enable. Enabling all
# logging when verbose mode is selected results in overwhelming output, hence
# this setting. Valid values (comma-separated) are:
#
# GENPKT Generic packet information
# GENPKTV Packet analysis, displays IP, TCP, UDP fields, very wide output
# CB Diverter packet handler callback start/finish logging
# NONLOC Nonlocal packet verbose logging
# DPF Dynamic port forwarding decisions
# DPFV Dynamic port forwarding table activity
# IPNAT NAT decisions
# MANGLE Packet mangling (modification) activity
# PCAP PCAP writes of original and mangled packets
# IGN Cases where packets are forwarded as is
# FTP FTP-specific logic
# IGN-FTP Cases where packets are forwarded as is due to FTP Active Mode
# MISC Miscellaneous
# NFQUEUE NetfilterQueue activity (Linux only)
# PROCFS Procfs read/write activity (Linux only)
# IPTABLES iptables firewall rule activity (Linux only)
DebugLevel: Off

# MultiHost mode only: Specify what interfaces the Linux Diverter should create
@@ -0,0 +1,42 @@
# Debug print levels for fine-grained debug trace output control
DNFQUEUE = (1 << 0) # netfilterqueue
DGENPKT = (1 << 1) # Generic packet handling
DGENPKTV = (1 << 2) # Generic packet handling with TCP analysis
DCB = (1 << 3) # Packet handlign callbacks
DPROCFS = (1 << 4) # procfs
DIPTBLS = (1 << 5) # iptables
DNONLOC = (1 << 6) # Nonlocal-destined datagrams
DDPF = (1 << 7) # DPF (Dynamic Port Forwarding)
DDPFV = (1 << 8) # DPF (Dynamic Port Forwarding) Verbose
DIPNAT = (1 << 9) # IP redirection for nonlocal-destined datagrams
DMANGLE = (1 << 10) # Packet mangling
DPCAP = (1 << 11) # Pcap write logic
DIGN = (1 << 12) # Packet redirect ignore conditions
DFTP = (1 << 13) # FTP checks
DMISC = (1 << 27) # Miscellaneous

DCOMP = 0x0fffffff # Component mask
DFLAG = 0xf0000000 # Flag mask
DEVERY = 0x0fffffff # Log everything, low verbosity
DEVERY2 = 0x8fffffff # Log everything, complete verbosity

DLABELS = {
DNFQUEUE: 'NFQUEUE',
DGENPKT: 'GENPKT',
DGENPKTV: 'GENPKTV',
DCB: 'CB',
DPROCFS: 'PROCFS',
DIPTBLS: 'IPTABLES',
DNONLOC: 'NONLOC',
DDPF: 'DPF',
DDPFV: 'DPFV',
DIPNAT: 'IPNAT',
DMANGLE: 'MANGLE',
DPCAP: 'PCAP',
DIGN: 'IGN',
DFTP: 'FTP',
DIGN | DFTP: 'IGN-FTP',
DMISC: 'MISC',
}

DLABELS_INV = {v.upper(): k for k, v in DLABELS.iteritems()}
Oops, something went wrong.

0 comments on commit e54c737

Please sign in to comment.
You can’t perform that action at this time.