Npcap 802.11 monitor mode FCS and capture stopping after malformed packet

[thoukydides]

I am attempting to use Npcap to capture raw IEEE 802.11 traffic from a Netgear A6200 in monitor mode using Wireshark. However, this fails in two ways (both illustrated in Capture.zip):

1. The captured packets all include an FCS, but the Radiotap Flags field is 0x00 indicating "FCS at end: False".
   This results in Wireshark incorrectly treating the FCS as being part of the payload. For Control frames this results in four undecoded bytes after the MAC header. For Management frames this results in an error (typically either "Malformed Packet" or "Packet size limited during capture") when the FCS is decoded as a Tagged parameter (information element).
   This is almost the opposite of nmap/nmap#1028, in which the Flags field has "FCS at end: True" but there is no FCS in the capture.

2. After a short time (typically a few minutes) the capture stops with a packet that has an invalid Radiotap header.
   This usually has a few (typically 1~16) spurious bytes inserted prior to the correct Radiotap header. Occasionally one extra packet is captured after the one with the Radiotap error.
   This may be the same as nmap/nmap#1001.

Both the Promiscuous and Monitor Mode options are ticked in the Capture Interfaces window.

I have tried using the "Assume packets have FCS" option, but this appears to be ignored. Based on https://www.wireshark.org/lists/wireshark-dev/201410/msg00079.html that is the intended behaviour, but it is not helpful in this case.

There are three flavors of 802.11 packet provided to the 802.11 dissector:

1. packets from sources that indicate whether the packet includes the FCS, such as some capture file formats;
2. packets that include metadata that indicates whether the FCS is included, such 802.11+radiotap radiotap packets;
3. packets that don't come from a source that indicates whether the packet includes the FCS.

The preference works only for packets of type 3), as it's unnecessary for packets of type 1) or 2).

The same behaviour occurs with Npcap installed in both WinPcap API-compatible and non-compatible modes.

These are the software and hardware versions being used:

• Npcap 0.94 (Windows installer)
• Wireshark version 2.4.2 (v2.4.2-0-gb6c63ae086) 64-bit
• Windows 10 Home version 1703 (OS Build 15063.674) 64-bit
• Netgear A6200 v1 Wi-Fi Adapter version 1.0.0.35, driver version 6.32.145.8

Here is the output from DiagReport:

• DiagReport-20171015-142335.txt

and here are the installation log files:

• setupapi.dev.log
• install.log
• NPFInstall.log

[thoukydides]

I also note that the Channel field, and sometimes the Rate field, are also incorrectly 0 in the Radiotap Headers for all captured packets...

[dmiller-nmap]

Thanks for this report! We are trying to track down a few bugs related to the 802.11 raw capture feature, but haven't been able to get detailed information. Can you please run the Windows Driver Verifier (included with Windows) and see if it produces a BSOD crash report?

1. Type Windows+R to open the Run dialog
2. Type "verifier.exe" and click OK
3. Choose "Create standard settings" and click Next
4. Choose "Select driver names from a list" and click Next
5. Scroll down and check the box next to "npcap.sys" by "Insecure.Com LLC." and click Finish
6. Reboot
7. Run your packet capture as normal.

Hopefully this will produce a BSOD crash and minidump. If so, follow the directions in the Npcap Guide to collect and submit it. If not, then run Verifier again and choose "Delete existing settings." Keeping the Verifier running is helpful for diagnostics, but can slow down your computer.

Please let us know how this works! Hopefully we can get it fixed soon.

[thoukydides]

After enabling Verifier for npcap.sys and rebooting I ran a few captures. They all ended when a packet with a misaligned Radiotap header appeared, and all captured packets had the later fields of the Radiotap headers set to zero. Unfortunately no BSOD.

Is there anything else that I can try to help debug these issues?

[dmiller-nmap]

Unfortunately, I didn't find the (possible/untested) fix for this until after Npcap 0.95 was released, so you'll have to wait a bit until I can get it tested and packaged. Npcap 0.95 still has this bug.

[thoukydides]

It looks encouragingly like you have found and fixed the cause of this problem. I look forward to trying the next release. Thanks for the update.

[dmiller-nmap]

Npcap 0.96 has been released, which ought to solve this issue. Please let us know so we can close this issue. https://nmap.org/npcap/

[thoukydides]

Npcap 0.96 appears to be an improvement; the capture no longer stops with an invalid Radiotap header. However, much of the Radiotap header is still incorrectly set to zero in Monitor mode:

• Flags = 0x00 (should have FCS at end set)
• Channel frequency = 0 (should be 2412 for 2.4GHz channel 1)
• Channel flags = 0x0000 (should have at least one PHY type indicated)

The missing FCS at end flag is still resulting in the final four bytes of each captured packet being incorrectly interpreted by Wireshark as being part of the payload. This results in the same [Malformed Packet] and [Packet size limited during capture] errors.

Here is the output from DiagReport:

• DiagReport-20171101-164307.txt
  and here are the installation log files:
• install.log
• NPFInstall.log

[dmiller-nmap]

Thanks for this analysis! I'll take a look at it.

[dmiller-nmap]

Ok, I've looked at the code (and you're welcome to peek, too: https://github.com/nmap/npcap/blob/v0.96/packetWin7/npf/npf/Read.c#L767). It looks like we try to get the relevant info from the lower layers by using the NET_BUFFER_LIST_INFO macro with the MediaSpecificInformation ID, resulting in a DOT11_EXTSTA_RECV_CONTEXT structure. This should have all the information we need, except that nothing indicates whether the FCS is included or not. The channel and frequency information is certainly supposed to be there. But other reports make it sound like this is dependent on the device driver implementation, and some drivers just don't do this properly. If some drivers don't include the FCS, then it's not hard to imagine that some might not set the physical ID or frequency info either.

I am open to more ideas on this, and I'm not giving up quite yet, but I would encourage you if possible to try a different WiFi Adapter. I believe @hsluoyz used the Netgear A6210 when developing this feature. I am still working on getting an adapter that works properly for testing.

[thoukydides]

Trying with a Netgear A6210 (instead of an A6200) is an improvement: the FCS at end flag is now being set, which matches the number of bytes provided for each packet. However, the Radiotap header still has issues:

• The Bad FCS flag is set for almost all packets, even though the captured payload is almost certainly correct for most frames.
• The Data Rate, Channel frequency, and Channel flags are always zero.
• The SSI Signal is the same for all packets in a capture, usually zero but not always.

as does the captured packet data:

• The FCS value provided at the end of the captured frame is definitely not the value that was transmitted over the air. This is obvious because the same value is reported for all frames of the same type and length, even though it should vary with the contents. For example, all Control frames (ACK, CTS, etc) have an FCS of 0x002c0000, and all Beacons of the same length have the same FCS (even from different APs). As expected this results in Wireshark reporting that the FCS is incorrect for almost all frames.

Fortunately, with this configuration I can at least disable Wireshark's Validate the FCS checksum if possible option. This makes the capture usable (the rest of the captured packet looks valid), although it is not possible to tell for sure whether the frames were actually correctly received.

I get the same results with both the generic Mediatek driver that Windows 10 installs automatically, and with the latest driver from Netgear's website (A6210_v1.0.0.36.exe, containing driver version 5.1.29.0).

I inspected the Radiotap header handling within npcap's source code previously before raising this bug report. The code looks plausible, but I only have limited experience with NDIS device drivers. As you suggest, this could easily be an issue with the Netgear hardware or device drivers rather than with npcap. I might try to get a build environment setup so that I can see what information the device driver is providing.

[dmiller-nmap]

I will try to get a debug build of Npcap 0.96 done so that you (or others) can get trace info. Building the driver yourself is possible, but then you have to set up a system to allow unsigned kernel drivers, which is troublesome. I will also create a page on SecWiki for recording the limitations of various adapters. Hopefully we can find some that work very well, since a primary purpose of this feature was to be able to do with commodity hardware what used to be possible only with AirPcap.

[dmiller-nmap]

The debug build is available at https://nmap.org/npcap/dist/npcap-0.96-debug.exe

I don't have a good set of instructions for debugging this issue; the driver prints trace statements to the kernel debug log, and I will be looking through these to check on the channel/frequency and data rate portions of the code, since those don't seem to be working with any of the adapters I have tested with.

I have started a page on SecWiki.org to track WiFi adapter support for raw 802.11 frame capture. I welcome your input on this page so that we can find the best adapters and drivers for this feature.

[thoukydides]

After installing the debug build I get the following message from Program Compatibility Assistant:

Should it be necessary to disable secure boot to use the debug build?

[thoukydides]

Using the "Disable driver signature enforcement" option I was able to capture kernel debug logs with both Wi-Fi adapters:

• Netgear A6200: NetgearA6200.log
• Netgear A6210: NetgearA6210.log

The log messages do not appear to contain any information about the Radiotap header, or the DOT11_EXTSTA_RECV_CONTEXT structure from which the information is obtained.

The only interesting things that I could spot are that

1. the Netgear A6200 driver runs on multiple CPU cores, but the A6210 version only runs on one; and
2. the Netgear A6210 driver does not appear to support OID_DOT11_DATA_RATE_MAPPING_TABLE.

The latter presumably explains why the Data Rate field is set to zero in the Radiotap header.

[guyharris]

See also issue #76, in which packets captured on a Netgear A6210 adapter don't appear to have an FCS but are flagged as having an FCS.

[guyharris]

Maybe what's at the end of the packet for the captures in this issue isn't an FCS - maybe it's just extra junk of some sort.

From a comment in Wireshark's code for reading NetMon captures:

It appears to be the case that management frames (and control and extension frames ?) may or may not have an FCS and data frames don't.

(Netmon capture files have been seen for this encapsulation having management frames either completely with or without an FCS. Also: instances have been seen where both Management and Control frames do not have an FCS).

NetMon has its own equivalent of the Npcap driver; it might use the same mechanism to get 802.11 packets that the Npcap driver does. Perhaps it's attempting to work around driver issues of that sort.

[guyharris]

I have tried using the "Assume packets have FCS" option, but this appears to be ignored. Based on https://www.wireshark.org/lists/wireshark-dev/201410/msg00079.html that is the intended behaviour, but it is not helpful in this case.

Yes, it is.

The FCS preferences for the 802.11 protocol control how the dissector should handle frames where no file or packet metadata indicates whether it has an FCS or not. That would be the case for packets where the radiotap header doesn't include the Flags field, so there is nothing in the radiotap header to indicate whether there's an FCS or not.

What would be appropriate for packets where the radiotap header does indicate whether the FCS is present, but indicates incorrectly, would be a radiotap preference to override the FCS bit and force "FCS present" or "FCS absent". I've just added such a preference in the Wireshark master branch for cases such as this and issue #76.

It Would Be Nice if Npcap could find out from the driver whether the packet had an FCS or not, but, sadly, we may not live in that universe. And It Would Also Be Nice if Windows drivers for Wi-Fi devices didn't have native Wi-FI drivers whose monitor mode implementations didn't suck in general.

[dmiller-nmap]

Merging with #76.