Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Npcap wrong data link type? #387

Closed
bitbanditorg opened this issue Apr 26, 2019 · 15 comments
Closed

Npcap wrong data link type? #387

bitbanditorg opened this issue Apr 26, 2019 · 15 comments

Comments

@bitbanditorg
Copy link

@bitbanditorg bitbanditorg commented Apr 26, 2019

When capturing on a WWAN link there is no Ethernet header and Wireshark cannot decode the saved pcap correctly. The reason is that in the capture file LinkType is ETHERNET (or DLT_EN10MB) but the packets don't have ethernet headers, they are starting from the IP header. From the saved file it cannot be determied subsequently how to interpret the packets if the LinkType is wrong. I need to analyze pcap files programatically that's why the correct LinkType is essential.

I cannot override the LinkType by calling pcap_set_datalink (to DLT_RAW), it returns an error since pcap_list_datalinks returns only ETHERNET (1). This is a Sierra modem where the PDN is activated by an AT command (SCACT=1) then the device driver associates an IP address to the network interface. The Interface Type is 243 (IF_TYPE_WWANPP).

Why other link types don't supported/listed only the ETHERNET even though it's not even an Ethernet interface?

Alternatively, is there a way to override the LinkType in the pcap_file_header programmatically before saving the file or I have to write a custom save function and manually generate a pcap header? I don't like to postprocess files with editcap.

Another weird phenomena that pcap_datalink_val_to_name or pcap_datalink_val_to_desc cannot translate DLT_RAW but return nil.

Thanks!

@guyharris
Copy link

@guyharris guyharris commented May 10, 2019

Do you supply NdisMediumWan in PacketGetNetType() calls for WWAN devices?

Currently, libpcap maps that to DLT_EN10MB; if all devices for which you supply NdisMediumWan in PacketGetNetType() calls provide raw IP packets with no link-layer headers, libpcap could just map that to DLT_RAW. If not, you may have to somehow figure out how to make PacketGetNetType() supply the appropriate NdisMedium value for WWAN devices, and tweak pcap-npf's pcap_activate_npf() appropriately.

Another weird phenomena that pcap_datalink_val_to_name or pcap_datalink_val_to_desc cannot translate DLT_RAW but return nil.

That seems... unlikely. That works in libpcap dating back at least as far as libpcap 1.8.0.

Does DLT_RAW have the value 12, or 14, in your program? (Yes, there are two different values, for binary compatibility reasons; OpenBSD uses 14, every other OS uses 12 - including Windows, unless __OpenBSD__ is somehow getting defined on Windows. That's why we have LINKTYPE_ values on the tcpdump.org link-layer header types page, and write LINKTYPE_ values, not DLT_ values, to pcap and pcapng files, and map between those types in APIs.)

What is returned if you pass the numerical values 12 and 14 to pcap_datalink_val_to_name() and pcap_datalink_val_to_desc()? In WinPcap and Npcap, 12 should give you "DLT_RAW" and "Raw IP" for 12 and NULL for 14.

@dmiller-nmap
Copy link
Contributor

@dmiller-nmap dmiller-nmap commented May 10, 2019

It appears we get this value via a OID_GEN_MEDIA_IN_USE query, which is not supported by NDIS 6.0 and later drivers. I think we need to update to OID_GEN_PHYSICAL_MEDIUM, which I'm guessing may solve a lot of compatibility problems with WAN, VPN, and WiFi drivers.

@bitbanditorg
Copy link
Author

@bitbanditorg bitbanditorg commented Jun 11, 2019

I see that (from the libcap source) pcap_activate_npf() calls PacketGetNetType() internally and maps NdisMediumWan to DLT_EN10MB. However, I use a WWAN (wireless WAN) interface which is not handled specifically by the large switch (type.LinkType) statement but setting the default which is DLT_EN10MB. So it seems a libpcap issue and not winpcap/npcap. Maybe the following lines should be added to libpcap:

case NdisMediumWirelessWan:
	p->linktype = DLT_RAW;
	break;

To be honest, I used the LINKTYPE_RAW value, which is 101. To check which value is appropriate to pcap_datalink_val_to_name() I called it with 12, 14 and 101. Only 12 gives the expected DLT_RAW value.

After investigating the pcap.c source I see pcap_datalink_val_to_name() uses an internal dlt_choices array to translate values to names. For RAW it uses the following macro: DLT_CHOICE(RAW, "Raw IP").
This is the macro: #define DLT_CHOICE(code, description) { #code, description, DLT_ ## code }
That means, DLT_RAW is 12.

In pcap_common.c there is linktype_map structure which maps DLT_RAW to LINKTYPE_RAW. But pcap_datalink_val_to_name() doesn't translate LINKTYPE_ values, only DLT_ values. However, I found there's a linktype_to_dlt() function which can translate the LINKTYPE_ value into DLT_ value, but this is used internally. It seems I don't have to use LINKTYPE_ constants since pcap_set_datalink uses DLT_ values.

I can use DLT_RAW for pcap_set_datalink() to override DLT_EN10MB but according to the above findings in case of a WWAN interface the only supported type is DLT_EN10MB and that is why I cannot set it to DLT_RAW. This is unfortunate. How else can I override the datalink type before saving a live capture?
I suppose, if WWAN will be handled as DLT_RAW (or a more appropriate value other than Ethernet) then this pcap_set_datalink() will not be required.

I have to apologize, it seems all of my problems related to libpcap only.

@guyharris
Copy link

@guyharris guyharris commented Jun 11, 2019

In libpcap, the mapping of platform-dependent link-layer types (that aren't DLT_s) to DLT_s is done in platform-dependent code - in this case, it's pcap-npf.c, as you found in the issue you filed against libpcap, the-tcpdump-group/libpcap#824, which has been fixed. (For platforms using BPF as a capture mechanism - rather than, for example, just as a filtering mechanism - the platform-dependent link-layer types are DLTs.)

LINKTYPE_s were added because not all DLT_xxxx defines have the same numerical value on all platforms (DLT_RAW happens to be one of them - 12 on most platforms, 14 on OpenBSD), and shouldn't be changed on those platforms for binary-compatibility reasons, but a given link-layer header type as recorded in pcap (and pcapng) files should have the same numerical value on all platforms. libpcap's APIs currently deal with DLT_ values, with LINKTYPE_ values not used in the API, so pcap_datalink_val_to_name() takes a DLT_ value, rather than a LINKTYPE_ value, as an argument; future additions to the API to fully handle pcapng will use LINKTYPE_ values.

@guyharris
Copy link

@guyharris guyharris commented Jun 11, 2019

It appears we get this value via a OID_GEN_MEDIA_IN_USE query, which is not supported by NDIS 6.0 and later drivers. I think we need to update to OID_GEN_PHYSICAL_MEDIUM, which I'm guessing may solve a lot of compatibility problems with WAN, VPN, and WiFi drivers.

NPF_GetPhysicalMedium() now appears to use OID_GEN_PHYSICAL_MEDIUM; has that update been done?

The current tip of the master Npcap branch appears to use OID_GEN_MEDIA_IN_USE only 1) on loopback adapters and 2) on 802.11 adapters. The Microsoft documentation says of NDIS 6 and OID_GEN_MEDIA_IN_USE:

NDIS 6.0 and later miniport drivers do not receive this OID request. NDIS handles this OID with a cached value that miniport drivers supply during initialization.

so the OID should still work from code above the driver (whether userland or kernel, presumably).

@bitbanditorg
Copy link
Author

@bitbanditorg bitbanditorg commented Jun 20, 2019

Is is possible to build new official npcap versions with the new libpcap version? The npcap 0.996 version still does not include the feature fix from the-tcpdump-group/libpcap#824. Or maybe the build environment does not include NdisMediumWirelessWan?

@dmiller-nmap
Copy link
Contributor

@dmiller-nmap dmiller-nmap commented Jun 20, 2019

@guyharris I'll have to look at this again. The use of OID_GEN_MEDIA_IN_USE is in AdInfo.cpp, part of Packet.dll used to get adapter info for things like pcap_findalldevs. I think this is what ought to be replaced with the newer OID.

@bitbanditorg Npcap tracks the latest release of libpcap, and only backports patches for issues of particular necessity to Npcap users. However, we also now build wpcap.dll from libpcap using CMake, which means that you should be able to fairly easily build it yourself and drop the DLL in place. We won't really be able to offer you support in that case, but it might help in your particular situation.

@dmiller-nmap
Copy link
Contributor

@dmiller-nmap dmiller-nmap commented Mar 12, 2020

Closing this issue, since Npcap 0.9984 and later include libpcap 1.9.1, which has this fix.

@bitbanditorg
Copy link
Author

@bitbanditorg bitbanditorg commented Apr 1, 2020

I think this issue still doesn't fixed, even in the latest (0.9989) version. I tested it on a WWAN interface and only the Ethernet (1) type is supported. I believe the binary is targeted to an old Windows SDK. Dumping out the OS Version from the DLLs the target seems to be Windows Vista. The WWAN interface is supported from Windows 7. The libpcap fix solved the issue by a conditional compiler option: #ifdef NdisMediumWirelessWan. If you target the DLL to an earlier Windows SDK the fix will be omitted. Since Vista is not supported I think the target SDK version should be updated.

@guyharris
Copy link

@guyharris guyharris commented Apr 1, 2020

The libpcap fix solved the issue by a conditional compiler option: #ifdef NdisMediumWirelessWan.

It solved the problem of handling WWAN interfaces by adding a case in the switch statement in pcap_activate_npf().

It solved the problem of allowing libpcap to be compiled with, for example, WinPcap, in case somebody wants to or needs to use the WinPcap driver but wants a newer version of libpcap, by protecting that case with an #ifdef.

If you target the DLL to an earlier Windows SDK the fix will be omitted.

Yes - as Microsoft's page for the NDIS_MEDIUM enumeration says:

Note Starting with Windows 7, this media type is supported and can
be used for Mobile Broadband.

where "this media type" refers to NdisMediumWirelessWan.

@dmiller-nmap
Copy link
Contributor

@dmiller-nmap dmiller-nmap commented Apr 3, 2020

@guyharris In ntddndis.h, NdisMediumWirelessWan is a member of typedef enum _NDIS_MEDIUM NDIS_MEDIUM, so it can't be checked with #ifdef. The header does guard some enum values with checks for NTDDI_VERSION, but this isn't one of them. Furthermore, I found that it is included in copies of ntddndis.h bundled in other software dating back at least to 2004 (the last "changelog" date in the file is 1996). I think it is safe to include this unconditionally. The next Npcap build of wpcap.dll will have this change incorporated.

guyharris referenced this issue in the-tcpdump-group/libpcap Apr 3, 2020
1) As noted by Daniel Miller in nmap/nmap#1573, NdisMediumWirelessWan is
a member of an enum, not a #define, so you can't test its presence with

2) It should be present in Windows 7 or later SDKs; if that means that
any sufficiently recent build environment should have it, it might not
be worth worrying about older build environments, especially given that
we require a bunch of C99isms to be supported, so you'll need a newer
Visual Studio version anyway.

If it turns out to be an issue, we could add CMake checks for
NdisMediumWirelessWan.
@guyharris
Copy link

@guyharris guyharris commented Apr 3, 2020

I've removed the #ifdef/#endif in the-tcpdump-group/libpcap@bdb86fe.

@bitbanditorg
Copy link
Author

@bitbanditorg bitbanditorg commented Apr 8, 2020

I can confirm that the new 0.9990 version is able to change the data link type on WWAN interface. However, there is a new problem introduced: I cannot transfer data on the WWAN interface if this new version is installed. Seems only Windows 7 is affected. Even normal ethernet interfaces are affacted. I have to uncheck the "Npcap Packet Driver" on the network adapter properties list. If I check it back the problem didn't show up until the PC is restarted. This problem should be unrelated to this issue, it may deserves an independent ticket.

@guyharris
Copy link

@guyharris guyharris commented Apr 8, 2020

I can confirm that the new 0.9990 version is able to change the data link type on WWAN interface.

Meaning that, with 0.9990, the data link type is now DLT_RAW rather than DLT_EN10MB?

This problem should be unrelated to this issue,

It is - this issue was an issue with libpcap, the other problem is probably some in-kernel networking stack problem either with Npcap or with Windows 7.

it may deserves an independent ticket.

It does.

@Permission13
Copy link

@Permission13 Permission13 commented Apr 15, 2020

There are a few posts mentioning that independent problem.
I guess we should take this nmap/npcap#109 one as it also backreferences the potential history of that bug.

@fyodor fyodor transferred this issue from nmap/nmap May 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants