Npcap's pcap_sendqueue_transmit Causes Windows10 Blue Screen

[WangChengZhang]

I need to let the gigabit ethernet adapter work with the full speed(about 950Mbit/s). So I use the pcap_sendqueue_transmit function.
But:
The npcap causes bluescreen.
The wpcap works well but lose some packets probably.

[dmiller-nmap]

Thanks for the report! We'll look into it. Does this happen every time you use pcap_sendqueue_transmit, or only when you approach the limits of your network device?

[dmiller-nmap]

I'm doing some initial testing, and while I am running into problems getting it to work, I have not seen any blue screen errors. Can you get a minidump crash dump and DiagReport output so we can diagnose the problem? https://nmap.org/npcap/guide/npcap-users-guide.html#npcap-issues

[dmiller-nmap]

@WangChengZhang Would you mind sharing a bit of your sample code that uses pcap_sendqueue_transmit? I suspect there is a problem with the order in which the sendqueue functions are being called. Obviously, we don't want Npcap to crash no matter how the user program is written, so this would help us find a reproducible case for your blue screen crash.

[WangChengZhang]

Okay Daniel...

Some explanations:

1.For this sender,I use 1 circular linked list with 3 pcap_send_queue to store data. And I use 2 threads filling and sending data. Within the main thread,I fill the queue by calling pcap_sendqueue_queue. Within the sending thread,I send the queue by calling pcap_sendqueue_transmit.

2.Each time before I calling pcap_sendqueue_transmit I will call pcap_sendpacket first. Because I need reopen the adapter in the case of an error. Unfortunately,pcap_sendqueue_transmit won't return false in some cases(like computer wakes up from sleep and restarts the adapter). So I need to call pcap_sendpacket judging the state of adapter.

[dmiller-nmap]

Thanks for the update. I'll see what I can figure out. I wonder if there's a better way to determine if the adapter handle is still valid?

[dmiller-nmap]

Still investigating. I think that in the case where there is a power event that causes Npcap handles to be invalidated, GetLastError can be used to get the error set by the DeviceIoControl function within Packet.dll. You can also check pcap_geterr to see if there is an error set. In this case, pcap_sendqueue_transmit will return a number less than the length of the sendqueue, which I see you are already checking for.

We will work on improving the errors that are returned, but also look for a cause on the BSOD crash.

[WangChengZhang]

Thanks, I'll try it.

[WangChengZhang]

@dmiller-nmap , I have checked the code of npcap recently.
I didn't find any difference of NPF_IoControl function in Packet.c between winpcap and npcap projects. Maybe the blue screen is cause by the difference of vs2015 Compiler？:

case BIOCSENDPACKETSSYNC:
		SyncWrite = TRUE;

	case BIOCSENDPACKETSNOSYNC:
		TRACE_MESSAGE(PACKET_DEBUG_LOUD, "BIOCSENDPACKETSNOSYNC");

		NdisAcquireSpinLock(&Open->WriteLock);
		if (Open->WriteInProgress)
		{
			NdisReleaseSpinLock(&Open->WriteLock);
			//
			// Another write operation is currently in progress
			//
			SET_FAILURE_UNSUCCESSFUL();
			break;
		}
		else
		{
			Open->WriteInProgress = TRUE;
		}
		NdisReleaseSpinLock(&Open->WriteLock);

		WriteRes = NPF_BufferedWrite(Irp,
			(PUCHAR) Irp->AssociatedIrp.SystemBuffer,
			IrpSp->Parameters.DeviceIoControl.InputBufferLength,
			SyncWrite);

		NdisAcquireSpinLock(&Open->WriteLock);
		Open->WriteInProgress = FALSE;
		NdisReleaseSpinLock(&Open->WriteLock);

		if (WriteRes != -1)
		{
			SET_RESULT_SUCCESS(WriteRes);
		}
		else
		{
			SET_FAILURE_UNSUCCESSFUL();
		}
		break;

And I found that: pcap_sendqueue_transmit will call PacketSendPackets in Packet32.cpp to send packets. But it will break and return error once DeviceIoControl return FALSE. Can you consider of adding a interface which can resend the packet till succeed or time out?

do{
			// Send the data to the driver
			//TODO Res is NEVER checked, this is REALLY bad.
			Res = (BOOLEAN)DeviceIoControl(AdapterObject->hFile,
				(Sync)?BIOCSENDPACKETSSYNC:BIOCSENDPACKETSNOSYNC,
				(PCHAR)PacketBuff + TotBytesTransfered,
				Size - TotBytesTransfered,
				NULL,
				0,
				&BytesTransfered,
				NULL);

			// Exit from the loop on error
			if(Res != TRUE)
				break;

			TotBytesTransfered += BytesTransfered;

			// Exit from the loop if we have transferred everything
			if(TotBytesTransfered >= Size)
				break;

			// calculate the time interval to wait before sending the next packet
			TargetTicks.QuadPart = StartTicks.QuadPart +
			(LONGLONG)
			((((struct timeval*)((PCHAR)PacketBuff + TotBytesTransfered))->tv_sec - BufStartTime.tv_sec) * 1000000 +
			(((struct timeval*)((PCHAR)PacketBuff + TotBytesTransfered))->tv_usec - BufStartTime.tv_usec)) *
			(TimeFreq.QuadPart) / 1000000;
			
			// Wait until the time interval has elapsed
			while( CurTicks.QuadPart <= TargetTicks.QuadPart )
				QueryPerformanceCounter(&CurTicks);

		}
		while(TRUE);

[dmiller-nmap]

In the crash dumps @PiAil provided on #370, the backtrace comes directly from rt640x64.sys, a RealTek NIC driver. This doesn't mean that it's not a problem with Npcap, just that the crash dumps don't point to any particular place within Npcap. The crash is DRIVER_IRQL_NOT_LESS_OR_EQUAL with a read operation.

Pure speculation: Maybe Npcap is injecting NBLs allocated from a pageable pool, and when the NIC driver gets around to sending them, they've been paged out?

[dmiller-nmap]

I've tried to reproduce this crash and have not been able to. My testing was done on a Windows 10 VM in Hyper-V, with Npcap 0.99-r9 running with standard Driver Verifier settings. I captured a speed test on fast.com, resulting in a 13.8 MB pcap file with 27573 packets. Then I played that back on the network with sendcap.exe. It reported a packet rate of 28000 packets per second, so significantly faster than the original packets were received.

@WangChengZhang are you able to submit a crash minidump file that we could inspect?

@PiAil Does the crash happen on different hardware, especially a network card from a different manufacturer?

[PiAil]

I tried with two different network card (ASIX AX88179 USB 3.0 to Gigabit Ethernet Adapter and Intel(R) Ethernet Connection (2) I219-V). I also tried with both my dll called with ctypes and with sendcap.exe and got similar results.

With the first card, I got a BSoD, here a minidump:
020419-15937-01.zip

With the second one, a big freeze, a reboot, and a minidup:
020119-7328-01.zip

The file I'm trying to send is bigger than yours (~150 MB). I don't know the limit but it seems that with small file, the packets are sent without any problem.

An exemple of my file:
test.zip

Note: It is an UDP flood with spoofed IP for learning purpose

[dmiller-nmap]

Thanks for the additional info! I'm still not able to trigger with your file example, but that may be because of mismatching MAC addresses. The first minidump has a short stack trace without Npcap in it, but the second one shows a crash in the NIC driver with Npcap functions in the stack trace. I do have a further question based on this stack trace, though: Are you sending the packets on the Npcap Loopback Adapter? The function in the stack trace is NPF_LoopbackSendNetBufferLists, which ought to be called only when the adapter being used is the Npcap Loopback Adapter, so that might indicate a problem.

[PiAil]

Nop, I'm not using the Npcap Loopback Adapter, but the one linked to my NIC (found with iflist.exe).

[dmiller-nmap]

Ok, I've audited all the places the internal Loopback flag is used and set, as well as all the code paths that lead to that function, and I don't see anything wrong. This might not be even relevant to your issue, so I'm going to suggest a partial workaround to see if we can get a better crash dump:

Please reinstall the latest Npcap, but install without loopback capture support. Then try to send your packet dump. If it crashes, we've narrowed it down to a different portion of the code (and maybe we will get a helpful crash dump). If it does not crash, then we've proven the problem is related to the loopback code, and we can investigate further there.