Kernel Panic on macOS 10.15 when using pcap_inject

[Sn0wfreezeDev]

Hi,

we are using libpcap in the owl project that uses it to inject frames into the WiFi chip.

General Info:

libpcap version is 1.9.1

macOS 10.15.3
Darwin 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan 9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64

Statement of the problem

Our recent build seems to work fine on Linux, but we get a Kernel panic on macOS.
We discovered that the kernel Panic originates after we call the function pcap_inject.
I tried it with the most recent release and the the master branch.
The code for this function can be found here:

libpcap/pcap.c

Line 4030 in 028ce66

pcap_inject(pcap_t *p, const void *buf, size_t size)

Steps to reproduce

Install owl as described on a Mac.
Run it with
sudo owl -i en0
Right after the setup of putting the WiFi chip into monitor mode the first packet should be sent in the function wlan_send located in daemon/io.c.
It's possible to set a breakpoint and check that the kernel panic happens when calling pcap_inject.

Crash Logs

panic(cpu 8 caller 0xffffff800f0652fa): Kernel trap at 0xffffff7f9062ebac, type 14=page fault, registers:
CR0: 0x0000000080010033, CR2: 0x0000000000000bbe, CR3: 0x0000000568038113, CR4: 0x00000000003626e0
RAX: 0x0000000000000bae, RBX: 0xffffff805e6ad800, RCX: 0xffffff7f9062eb90, RDX: 0x0000000000000000
RSP: 0xffffffa3cfd0ba80, RBP: 0xffffffa3cfd0ba90, RSI: 0xffffff83c3047800, RDI: 0xffffff805c4b4000
R8:  0x0000000000000000, R9:  0x0000000000000000, R10: 0xffffff800f8117d8, R11: 0xffffff800f642440
R12: 0xffffffa3cfd0bb84, R13: 0xffffff80615b80a0, R14: 0xffffff83c3047800, R15: 0xffffff80615b80a0
RFL: 0x0000000000010246, RIP: 0xffffff7f9062ebac, CS:  0x0000000000000008, SS:  0x0000000000000010
Fault CR2: 0x0000000000000bbe, Error code: 0x0000000000000000, Fault CPU: 0x8, PL: 0, VF: 0

Backtrace (CPU 8), Frame : Return Address
0xffffffa3cfd0b4e0 : 0xffffff800ef3bb2b 
0xffffffa3cfd0b530 : 0xffffff800f0734d5 
0xffffffa3cfd0b570 : 0xffffff800f064f4e 
0xffffffa3cfd0b5c0 : 0xffffff800eee2a40 
0xffffffa3cfd0b5e0 : 0xffffff800ef3b217 
0xffffffa3cfd0b6e0 : 0xffffff800ef3b5fb 
0xffffffa3cfd0b730 : 0xffffff800f6d2aa9 
0xffffffa3cfd0b7a0 : 0xffffff800f0652fa 
0xffffffa3cfd0b920 : 0xffffff800f064ff8 
0xffffffa3cfd0b970 : 0xffffff800eee2a40 
0xffffffa3cfd0b990 : 0xffffff7f9062ebac 
0xffffffa3cfd0ba90 : 0xffffff7f8fcbddc4 
0xffffffa3cfd0bad0 : 0xffffff800f6427c5 
0xffffffa3cfd0bb30 : 0xffffff7f8fcbdea8 
0xffffffa3cfd0bb50 : 0xffffff7f8fcbd2a0 
0xffffffa3cfd0bbb0 : 0xffffff7f8fcbd4ad 
0xffffffa3cfd0bbf0 : 0xffffff7f9062ec8c 
0xffffffa3cfd0bc20 : 0xffffff7f9062eb7b 
0xffffffa3cfd0bc40 : 0xffffff800f1e7a1f 
0xffffffa3cfd0bcd0 : 0xffffff800f1d6b54 
0xffffffa3cfd0bd50 : 0xffffff800f1caf52 
0xffffffa3cfd0bdc0 : 0xffffff800f1bbfb9 
0xffffffa3cfd0be20 : 0xffffff800f4b7dd0 
0xffffffa3cfd0bee0 : 0xffffff800f4b7b90 
0xffffffa3cfd0bf40 : 0xffffff800f59b08a 
0xffffffa3cfd0bfa0 : 0xffffff800eee3206 
      Kernel Extensions in backtrace:
         com.apple.iokit.IONetworkingFamily(3.4)[DADDF78F-DD4E-359E-AE63-446D90F3ADDA]@0xffffff7f8fcaa000->0xffffff7f8fcd9fff
         com.apple.iokit.IO80211FamilyV2(1200.12.2b1)[98A64913-DDC9-33A0-8C9F-7888D50EC84C]@0xffffff7f905a5000->0xffffff7f90706fff
            dependency: com.apple.driver.corecapture(1.0.4)[5C9461C8-9B50-3D42-BFE3-3DB943A0C506]@0xffffff7f9031c000
            dependency: com.apple.driver.AppleMobileFileIntegrity(1.0.5)[A243D030-19AC-30AA-AC70-6C786DF9E6CE]@0xffffff7f90022000
            dependency: com.apple.kec.corecrypto(1.0)[4A7262FB-5D8B-35A9-B10C-8889A7108153]@0xffffff7f8ff3c000
            dependency: com.apple.iokit.IOSkywalkFamily(1)[DF2AAB7C-08DA-36D1-A5C4-8DF2E3A139E2]@0xffffff7f9039a000
            dependency: com.apple.iokit.IONetworkingFamily(3.4)[DADDF78F-DD4E-359E-AE63-446D90F3ADDA]@0xffffff7f8fcaa000

BSD process name corresponding to current thread: owl
Boot args: amfi_get_out_of_my_way=0x0 chunklist-security-epoch=0 -chunklist-no-rev2-dev

Mac OS version:
19D76

Kernel version:
Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64
Kernel UUID: A8DDE75C-CD97-3C37-B35D-1070CC50D2CE
Kernel slide:     0x000000000ec00000
Kernel text base: 0xffffff800ee00000
__HIB  text base: 0xffffff800ed00000
System model name: MacBookPro15,1 (Mac-937A206F2EE63C01)
System shutdown begun: NO

System uptime in nanoseconds: 14628019008185
last loaded kext at 13984175433110: net.sf.tuntaposx.tun	1.0 (addr 0xffffff7f933ca000, size 36864)
last unloaded kext at 12085201385586: |SCSITaskUserClient	422.0.2 (addr 0xffffff7f927db000, size 28672)
loaded kexts:
net.sf.tuntaposx.tun	1.0
net.sf.tuntaposx.tap	1.0
... 

If there is more information that I can submit to help discovering the issue I am willing to do so.

[Sn0wfreezeDev]

I managed to get a symbolicated crash log, which I will attach below.
Furthermore, this issue seems to be related to a subset of Macs. It seems to occur only on Macs that have a T2 security chip. Older models without this chip can use these features normally.
The same issue occurs on the bettercap projects with similar Mac models since 2018.
There is a similar issue open there:
bettercap/bettercap#448

In the end this looks like a macOS bug caused by Apple that is difficult to fix from the outside. I am open for other recommendations though.

panic(cpu 2 caller 0xffffff801ae652fa): Kernel trap at 0xffffff7f9c42ebac, type 14=page fault, registers:
CR0: 0x0000000080010033, CR2: 0x000000000000001b, CR3: 0x000000085c0d1149, CR4: 0x00000000003626e0
RAX: 0x000000000000000b, RBX: 0xffffff806a07c800, RCX: 0xffffff7f9c42eb90, RDX: 0x0000000000000000
RSP: 0xffffffa400303a80, RBP: 0xffffffa400303a90, RSI: 0xffffff83cee63000, RDI: 0xffffff80685a4000
R8:  0x0000000000000000, R9:  0x0000000000000000, R10: 0x00000000ffffffff, R11: 0xffffff801b442440
R12: 0xffffffa400303b84, R13: 0xffffff806e2e54a0, R14: 0xffffff83cee63000, R15: 0xffffff806e2e54a0
RFL: 0x0000000000010246, RIP: 0xffffff7f9c42ebac, CS:  0x0000000000000008, SS:  0x0000000000000010
Fault CR2: 0x000000000000001b, Error code: 0x0000000000000000, Fault CPU: 0x2, PL: 0, VF: 0

Backtrace (CPU 2), Frame : Return Address
0xffffffa4003034e0 : 0xffffff801ad3bb2b mach_kernel : _handle_debugger_trap + 0x47b
0xffffffa400303530 : 0xffffff801ae734d5 mach_kernel : _kdp_i386_trap + 0x155
0xffffffa400303570 : 0xffffff801ae64f4e mach_kernel : _kernel_trap + 0x4ee
0xffffffa4003035c0 : 0xffffff801ace2a40 mach_kernel : _return_from_trap + 0xe0
0xffffffa4003035e0 : 0xffffff801ad3b217 mach_kernel : _DebuggerTrapWithState + 0x17
0xffffffa4003036e0 : 0xffffff801ad3b5fb mach_kernel : _panic_trap_to_debugger + 0x21b
0xffffffa400303730 : 0xffffff801b4d2aa9 mach_kernel : _panic + 0x61
0xffffffa4003037a0 : 0xffffff801ae652fa mach_kernel : _sync_iss_to_iks + 0x2aa
0xffffffa400303920 : 0xffffff801ae64ff8 mach_kernel : _kernel_trap + 0x598
0xffffffa400303970 : 0xffffff801ace2a40 mach_kernel : _return_from_trap + 0xe0
0xffffffa400303990 : 0xffffff7f9c42ebac com.apple.iokit.IO80211FamilyV2 : __ZN16IO80211Interface15bpfOutputPacketEP6__mbufPv + 0x1c
0xffffffa400303a90 : 0xffffff7f9babddc4 com.apple.iokit.IONetworkingFamily : __ZN18IOGatedOutputQueue11gatedOutputEP8OSObjectPS_P11IOMbufQueuePj + 0x68
0xffffffa400303ad0 : 0xffffff801b4427c5 mach_kernel : __ZN13IOCommandGate13attemptActionEPFiP8OSObjectPvS2_S2_S2_ES2_S2_S2_S2_ + 0xb5
0xffffffa400303b30 : 0xffffff7f9babdea8 com.apple.iokit.IONetworkingFamily : __ZN18IOGatedOutputQueue6outputEP11IOMbufQueuePj + 0x2c
0xffffffa400303b50 : 0xffffff7f9babd2a0 com.apple.iokit.IONetworkingFamily : __ZN18IOBasicOutputQueue7dequeueEv + 0x84
0xffffffa400303bb0 : 0xffffff7f9babd4ad com.apple.iokit.IONetworkingFamily : __ZN18IOBasicOutputQueue7enqueueEP6__mbufPv + 0xc7
0xffffffa400303bf0 : 0xffffff7f9c42ec8c com.apple.iokit.IO80211FamilyV2 : __ZN16IO80211Interface9bpfOutputEjP6__mbuf + 0xc6
0xffffffa400303c20 : 0xffffff7f9c42eb7b com.apple.iokit.IO80211FamilyV2 : __ZL10_bpfOutputP7__ifnetjP6__mbuf + 0x1e
0xffffffa400303c40 : 0xffffff801afe7a1f mach_kernel : _bpfwrite + 0x71f
0xffffffa400303cd0 : 0xffffff801afd6b54 mach_kernel : _spec_write + 0x374
0xffffffa400303d50 : 0xffffff801afcaf52 mach_kernel : _VNOP_WRITE + 0x92
0xffffffa400303dc0 : 0xffffff801afbbfb9 mach_kernel : _utf8_normalizestr + 0x969
0xffffffa400303e20 : 0xffffff801b2b7dd0 mach_kernel : _write_nocancel + 0x310
0xffffffa400303ee0 : 0xffffff801b2b7b90 mach_kernel : _write_nocancel + 0xd0
0xffffffa400303f40 : 0xffffff801b39b08a mach_kernel : _unix_syscall64 + 0x28a
0xffffffa400303fa0 : 0xffffff801ace3206 mach_kernel : _hndl_unix_scall64 + 0x16
      Kernel Extensions in backtrace:
         com.apple.iokit.IONetworkingFamily(3.4)[DADDF78F-DD4E-359E-AE63-446D90F3ADDA]@0xffffff7f9baaa000->0xffffff7f9bad9fff
         com.apple.iokit.IO80211FamilyV2(1200.12.2b1)[98A64913-DDC9-33A0-8C9F-7888D50EC84C]@0xffffff7f9c3a5000->0xffffff7f9c506fff
            dependency: com.apple.driver.corecapture(1.0.4)[5C9461C8-9B50-3D42-BFE3-3DB943A0C506]@0xffffff7f9c11c000
            dependency: com.apple.driver.AppleMobileFileIntegrity(1.0.5)[A243D030-19AC-30AA-AC70-6C786DF9E6CE]@0xffffff7f9be22000
            dependency: com.apple.kec.corecrypto(1.0)[4A7262FB-5D8B-35A9-B10C-8889A7108153]@0xffffff7f9bd3c000
            dependency: com.apple.iokit.IOSkywalkFamily(1)[DF2AAB7C-08DA-36D1-A5C4-8DF2E3A139E2]@0xffffff7f9c19a000
            dependency: com.apple.iokit.IONetworkingFamily(3.4)[DADDF78F-DD4E-359E-AE63-446D90F3ADDA]@0xffffff7f9baaa000

BSD process name corresponding to current thread: owl
Boot args: keepsyms=1 chunklist-security-epoch=0 -chunklist-no-rev2-dev

Mac OS version:
19D76

Kernel version:
Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64
Kernel UUID: A8DDE75C-CD97-3C37-B35D-1070CC50D2CE
Kernel slide:     0x000000001aa00000
Kernel text base: 0xffffff801ac00000
__HIB  text base: 0xffffff801ab00000
System model name: MacBookPro15,1 (Mac-937A206F2EE63C01)
System shutdown begun: NO

[guyharris]

If the stack traces valid, it's panicking in a routine for which the comment is

/* Synchronize a thread's x86_kernel_state (if any) with the given

• x86_saved_state_t obtained from the trap/IPI handler; called in
• kernel_trap() prior to entering the debugger, and when receiving
• an "MP_KDP" IPI. Called with null saved_state if an incoming IPI
• was detected from the kernel while spinning with interrupts masked.
  */

The call is from kernel_trap(), for which the comment is

/*

• Trap from kernel mode. Only page-fault errors are recoverable,
• and then only in special circumstances. All other errors are
• fatal. Return value indicates if trap was handled.
  */

The recoverable page fault errors may be ones where the kernel is copying data from or to userland and is taking a page fault because a userland page is paged out (or zero-fill-on-demand or copy-on-write or...), or is not readable/writable.

The packet should already have been copied into userland from bpfwrite() calling bpf_movein(), assuming that the code path in xnu-6153.81.5~1 is the same as in xnu-6153.11.26, so it's probably a page fault other than a copy-from-userland fault.

Unfortunately, IO80211Family isn't in the open-source part of macOS, so I don't know what's going wrong, but you should probably submit a bug report at http://feedbackassistant.apple.com. You may have to sign up from an Apple developer account.

[guyharris]

Apple seem to have screwed up many aspects of the interaction between BPF and the Wi-Fi driver in newer MacBook Pros; monitor mode now requires some Special Magic to be done - the sniffer in Wireless Diagnostics does some form of Special Magic before running tcpdump (yes, it's tcpdump that does the sniffing work there). I think the AirPort group needs some more adult supervision by the networking group.

[Sn0wfreezeDev]

I sent feedback to Apple about this issue with the number FB7671413.
If someone wanted to reproduce this issue easily I made a sample implementation using Xcode and the pre-installed version of libpcacp on macOS Catalina.

I attached the project to this message. Be aware, it will lead to a kernel panic on any Mac with a T2 chip.
Xcode-Sample-project.zip