Redo the way interface are enumerated?

[guyharris]

In libpcap, pcap_findalldevs() calls pcap_platform_finddevs() to find all the regular interfaces (as well as calling the "find devices" routines for various pcap "modules", if configured in), and the NPF version (in pcap-npf.c) calls PacketGetAdapterNames() twice - once to get the size of the adapter list so that it can allocate a buffer for the list, and once to read the list into the buffer.

PacketGetAdapterNames() calls PacketPopulateAdaptersInfoList(). PacketPopulateAdaptersInfoList() always empties out the current list and then calls, in order:

• PacketGetAdaptersNPF();
• PacketGetAdaptersIPH();
• PacketAddAdapterNPF() to add the loopback adapter, if loopback support is configured in.;
• PacketGetAdaptersAirpcap(), if the AirPcap DLL is loaded;
• PacketGetAdaptersDag(), if the Endace DAG card support is available;

to fill in the list.

PacketGetAdaptersNPF() digs through the Registry to find adapters; for each adapter it finds, it calls PacketAddAdapterNPF().

PacketAddAdapterNPF() first iterates over the list to see if it's already been added, and just returns if it has. Otherwise, it attempts to open the adapter, using PacketOpenAdapterNPF() and, if that succeeds, adds it. (I've omitted some details such as the INFO_FLAG_DONT_EXPORT flag.)

PacketAddAdapterNPF() tries to find the IPv4 and IPv6 addresses for the adapter.

To find the IPv4 addresses, it calls PacketGetAddressesFromRegistry(). PacketGetAddressesFromRegistry() calls IsIPv4Enabled(), which calls...

...GetAdaptersAddresses().

To get all the IPv6 addresses for the adapter, it calls PacketAddIP6Addresses().

PacketAddIP6Addresses() gets the adapter's IPv6 addresses by...

...calling GetAdaptersAddresses(), looking for the adapter in the list, and getting the IPv6 addresses from that list.

So we end up calling GetAdaptersAddresses() in this process. We end up calling it twice per interface in this process.

Oh, and then pcap_platform_finddevs()() calls packet_add_if_npf() per interface; that then calls PacketGetNetInfoEx(), which again calls PacketGetAddressesFromRegistry() and PacketAddIP6Addresses(), so there are even more per-adapter GetAdaptersAddresses() calls.

PacketGetAdaptersIPH() calls GetAdaptersInfo(), which, as @akontsevoy notes in this comment on issue 26, is probably a wrapper for GetAdaptersAddresses() since Windows XP. I'm not sure we need PacketGetAdaptersIPH() and PacketGetAdaptersNPF(), unless there are Npcap-usable adapters that PacketGetAdaptersIPH() would find but PacketGetAdaptersNPF() wouldn't find and Npcap-usable adapters that PacketGetAdaptersNPF() would find but PacketGetAdaptersIPH() would find.

This code needs some work. I have a version of libpcap's pcap-npf.c where pcap_platform_finddevs() calls GetAdaptersAddresses() directly and iterates once through the list it provides. Unfortunately, it currently has to call PacketOpenAdapter() to check whether the interface is supported by Npcap, and that ends up calling PacketFindAdInfo(), which ends up calling PacketPopulateAdaptersInfoList() if the list is empty. That's used to determine whether the device is a regular NPF device, a DAG device, or an AirPcap device.

The current master branch of libpcap includes direct support for AirPcap devices (tested on a Windows VM; it works). If we were to remove the AirPcap support from Npcap, and leave that up to libpcap, that would leave only the DAG devices. Endace currently doesn't support Windows, only Linux and FreeBSD, and libpcap already has DAG support, so that might be another case to leave up to libpcap; were Endace to support Windows again, they'll probably either contribute whatever changes are needed to pcap-dag.c or I'd nag them into doing so. :-)

So the first question is whether, were packet.dll not to provide PacketGetAdapterNames() at all, the list of adapters would be necessary.

If not, it could be removed, and PacketOpenAdapter() could just try opening the device through the driver, with the new pcap-npf.c used; that would probably be significantly faster (the profiling I did with a test program that repeatedly called pcap_findalldevs() found a lot of the time in pcap_findalldevs() is spent in repeated calls to GetAdaptersAddresses()).

If it is necessary, perhaps it could just be a list of names, not full interface information.

Alternatively, a PacketGetAdapterInfo() routine could be provided, which just hands back a buffer full of IP_ADAPTER_ADDRESSES structures fetched from a single GetAdaptersAddresses() call. It could allocate a buffer large enough for all the adapters found by GetAdaptersAddresses() plus an extra entry, added at the beginning, for the loopback device, if present. In addition, it could adjust the "next interface" pointers so that the list in that buffer doesn't include interfaces not supported by Npcap. It would return both a pointer to the raw buffer, for the caller to hand to a packet.dll routine to free, and a pointer to the first interface supported by Npcap. That would let the caller decide what information it wants; some of the information in the IP_ADAPTER_ADDRESSES structure wouldn't be useful to pcap_findalldevs(), but might be useful to future libpcap APIs that would return more information (e.g., many of the options in a pcapng Interface Description Block).

[dmiller-nmap]

Related #61

[dmiller-nmap]

Helpful comment listing the things libpcap gathers per interface: #26 (comment)

[gvanem]

I can add to this that running e.g. findalldevstest.exe with a libpcap built with the NPcap-SDK but without
a NPcap driver/service installed, I get this total bogus error:

Error in pcap_lookupdev: PacketGetAdapterNames: The data area passed to a system call is too small.  (122)

(122 = ERROR_INSUFFICIENT_BUFFER).

AFAICS since PacketPopulateAdaptersInfoList() does not preserve it's error-code.
The test on if(!g_AdaptersInfoList) then sets a totally unrelated ERROR_INSUFFICIENT_BUFFER.

I modified PacketPopulateAdaptersInfoList() to return an error, and then findalldevstest.exe returns this bit more sensible error:

Error in pcap_findalldevs: PacketGetAdapterNames: The system cannot find the path specified.  (3)

which relates to opening the non-existing symlink \\.\Global\NPCAP\Loopback.