Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wireshark is not able to see ethernet adapters after changing from WinPCAP to NPCAP on Windows server 2016 #1899

Open
pc500 opened this issue Jan 26, 2020 · 7 comments

Comments

@pc500
Copy link

@pc500 pc500 commented Jan 26, 2020

Wireshark is not able to see ethernet adapters after changing from WinPCAP to NPCAP on Windows server 2016 for Broadcom Netextreme Virtual Function ethernet adapter.

Attached is the Diagreport. I can reproduce on a fresh empty VM and provide remote access if desired.

C:\Program Files\Wireshark>dumpcap.exe -D

  1. \Device\NPF_Loopback (Adapter for loopback traffic capture)

C:\Program Files\Wireshark>


Network Adapter(s) Info:


Name InterfaceDescription ifIndex Status MacAddress LinkSpeed


Ethernet Broadcom NetXtreme-E Virtual Function 3 Up 00-00-17-02-79-0E 25 Gbps

Caption : [00000002] Broadcom NetXtreme-E Virtual Function
GUID : {CC75C87B-6F35-4560-8556-EA0329DA6CB7}
Index : 2
InterfaceIndex : 3
Manufacturer : Broadcom Corporation
NetConnectionID : Ethernet
PNPDeviceID : PCI\VEN_14E4&DEV_16DC&SUBSYS_16D7108E&REV_00\3&13C0B0C5&1&18
diagreport.txt

@dmiller-nmap

This comment has been minimized.

Copy link

@dmiller-nmap dmiller-nmap commented Feb 11, 2020

We've confirmed this issue, but we don't know why it's happening. Neither Npcap nor the "NDIS Sample LightWeight Filter" driver from Microsoft/Window-driver-samples can bind to the VF adapter. I can't debug the system where we're seeing this currently, so I don't know if our FilterAttach function is even being called. I'm hoping to reach out to more knowledgeable NDIS people to see if there's something obvious I'm missing. (CC @jtippet)

@pc500

This comment has been minimized.

Copy link
Author

@pc500 pc500 commented Feb 11, 2020

I am provide a system where this reproduces if it assist in any ways.

@jtippet

This comment has been minimized.

Copy link

@jtippet jtippet commented Feb 12, 2020

Ah this is sort of by design, at least from the OS's point of view. The VF doesn't bind directly to most drivers -- instead, it slots into a purpose-built protocol driver that is bundled with netvsc (the Hyper-V network adapter). netvsc exposes the VF's traffic up through its own datapath. It does this so it can provide seamless and transparent failover if the VM is migrated to a host that doesn't have a VF.

Because it isn't just a regular Ethernet+TCPIP situation, the VF's network interface is designed to not bind all the usual filters by default.

This is accomplished by setting the UpperRange and LowerRange in a VF's INF to distinct tokens. Usually an Ethernet adapter would have UpperRange=ndis5 (ignore the name; it's not actually NDIS 5.x) and LowerRange=ethernet. But a VF has UpperRange=ndisvf and LowerRange=iovvf.

If you want, you can modify the LWF's INF to bind to VF adapters too. I believe you'd only have to append iovvf to your INF's FilterMediaTypes. Example: FilterMediaTypes=ethernet,bluetooth,iovvf. But before you rush out and do that -- be advised that a VF stack is not quite the usual TCPIP stack. You should test it a bit to make sure that your LWF doesn't break things.

Silently capturing traffic shouldn't cause any problems, but setting a OID_GEN_CURRENT_PACKET_FILTER does theoretically perturb the state of the system, so it could theoretically break something that wasn't expecting it.

Also, @pc500, keep in mind that even without any changes to npcap, you can still capture traffic from the VF right now: just attach to the netvsc (Hyper-V) interface. The netvsc adapter will show all the traffic to/from the VF. In other words, this is your network architecture looks like this (zoomed in from here):

             +-----------------+              
             |                 |              
             |      TCPIP      |              
             |                 |              
             +--------|--------+              
                      |                       
                      |   (A)                    
                      |                       
             +--------|--------+              
             |                 |              
             |     netvsc      |              
             |                 |              
             +-----------------+              
              --/            \--              
           --/                  \---  (B)      
+---------/------+          +-------\--------+
|                |          |                |
|   synthetic    |          |       VF       |
|                |          |                |
+----------------+          +----------------+

You're asking to capture packets at point (B), which is reasonable, but requires a change to the npcap driver INF. Capturing at point (A) would give you largely the same results as capturing at point (B), but can be done right now. The only real difference between (A) and (B) is that, if the VF is revoked, (A) will continue to show traffic while (B) will show a dead link.

@pc500

This comment has been minimized.

Copy link
Author

@pc500 pc500 commented Feb 12, 2020

Hi @jtippet The guest operating system cannot see the netsvc interface. This is not actually running on Hyper-V, although it is a virtual machine guest. I can only see one ("single") interface on the VM.

image

My understanding is it's a virtual function mapped directly from the hardware NIC, and only that PCI address is exposed to the guest OS. The hypervisor passes through the VF function as hardware and is otherwise not involved in the networking, similar to the following:

image

More info on this setup is here:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_deployment_and_administration_guide/sect-pci_devices-pci_passthrough

@dmiller-nmap

This comment has been minimized.

Copy link

@dmiller-nmap dmiller-nmap commented Feb 12, 2020

Thanks so much for the insight, @jtippet! As @pc500 points out, in this setup the Hyper-V Virtual Ethernet Adapter is not present, so I think Npcap will be unable to capture like this.

We have 2 ways forward that we may pursue, but they will require lots more development and testing:

  1. Expand the existing WFP capturing mechanism that we currently use for Loopback capture to cover all interfaces. This is something I've been wanting to do for a while, though it has limitations compared to the LWF mechanism in that it does not provide link-layer headers and has some known problems interfering with certain setups (#1529, #1789).
  2. Explore binding to the iovvf edge of these adapters. Since we have now been warned that our usual means of controlling the capture with OID_GEN_CURRENT_PACKET_FILTER and other things may disrupt the system, this might be a "read-only, you get what you get" type of capture, which we would have to test to see how libpcap and other APIs can handle it.
@jtippet

This comment has been minimized.

Copy link

@jtippet jtippet commented Feb 12, 2020

@pc500 - Oh, that's interesting. I honestly have no idea how a VF would look if it was exposed from another hypervisor. The VF architecture in Windows is (for better or worse) somewhat coupled to netvsc, to provide seamless failover. My comments about INFs and bindings may or may not apply, depending on how Broadcom and RedHat decided to expose the VF to the guest OS. You can check quickly by looking at the registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\<some number>\Ndi\Interfaces

where <some number> is whichever key has all the Broadcom stuff under it. If there's a value named LowerRange and it has only iovvf, then my comments about bindings apply. Although, as you pointed out, there's no workaround by tapping the netvsc interface.

@dmiller-nmap - I don't want to discourage you from binding to iovvf interfaces. I'm just saying that you should try it out before committing a PR that adds a claim that the LWF supports iovvf. In general, we add distinct binding tokens when there's something different about the interface that LWFs may have to take into consideration. But, if the LWF doesn't really interact with the NIC much, it's often a non-event. For comparison, ndiscap, the built-in packet capture LWF, binds to ethernet,wlan,ppip,vmnetextension,bluetooth. (Actually, the last token was only added in the most recent preview build of the OS.) Of those, only ppip required a special code change in the LWF.

@tamwoza

This comment has been minimized.

Copy link

@tamwoza tamwoza commented Feb 13, 2020

I'm having similar issues with npcap and Suricata. Do we know what method WinPcap used to allow it to be attached to these types of adapters? Could a similar method be implemented for npcap when installed with the WinPCAP compatibily mode?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.