Crash when calling enumerate from different threads (not simultaneously) - arm64 macOS

[afilini]

The following python script should make the interpreter crash when run on a mac M1. This has been tested with multiple python3 versions (if i remember correctly 3.9, 3.10 and 3.11).

It seems to be working fine on x86 macos (at least in the VM I use to test).

Essentially, calling enumerate from different threads that are correctly synchronized using a mutex causes the crash. See the example code here:

from hwilib.commands import enumerate
import threading

def e():
	m.acquire()
	print(enumerate())
	m.release()

m = threading.Lock()

a = threading.Thread(target=e, args=())
b = threading.Thread(target=e, args=())
c = threading.Thread(target=e, args=())

a.start()
b.start()
c.start()

a.join()
b.join()
c.join()

I had it work a couple of times, but the vast majority of times it will crash in the same way, somewhere in the CoreFoundation os lib.

Debugger backtrace

Process 5879 resuming
No module named 'hwilib.devices.jadepy.jade_ble'
BLE scanning/connectivity will not be available
[]
Process 5879 stopped
* thread #4, stop reason = EXC_BREAKPOINT (code=1, subcode=0x19dcaee54)
    frame #0: 0x000000019dcaee54 CoreFoundation`_CFAssertMismatchedTypeID + 112
CoreFoundation`:
->  0x19dcaee54 <+112>: brk    #0x1

CoreFoundation`:
    0x19dcaee58 <+0>:   adrp   x8, 425
    0x19dcaee5c <+4>:   add    x8, x8, #0x3e9            ; "Unable to allocate memory for CF runtime type registration"
    0x19dcaee60 <+8>:   adrp   x9, 374339
(lldb) bt
* thread #4, stop reason = EXC_BREAKPOINT (code=1, subcode=0x19dcaee54)
  * frame #0: 0x000000019dcaee54 CoreFoundation`_CFAssertMismatchedTypeID + 112
    frame #1: 0x000000019db26d48 CoreFoundation`CFRunLoopAddSource + 1040
    frame #2: 0x00000001a0c9a914 IOKit`IOHIDDeviceScheduleWithRunLoop + 92
    frame #3: 0x00000001a0c9f8c0 IOKit`__IOHIDManagerDeviceApplier + 636
    frame #4: 0x00000001a0c5c09c IOKit`__IOHIDManagerDeviceAdded + 768
    frame #5: 0x00000001a0c5bc70 IOKit`__IOHIDManagerSetDeviceMatching + 372
    frame #6: 0x00000001a0c5baa4 IOKit`IOHIDManagerSetDeviceMatchingMultiple + 272
    frame #7: 0x0000000101f4b4ec hid.cpython-39-darwin.so`hid_enumerate + 272
    frame #8: 0x0000000101f4a884 hid.cpython-39-darwin.so`__pyx_pw_3hid_1enumerate + 192
    frame #9: 0x0000000100497ef8 Python`cfunction_call + 60
    frame #10: 0x0000000100448a4c Python`_PyObject_MakeTpCall + 132
    frame #11: 0x0000000100540498 Python`call_function + 268
    frame #12: 0x0000000100539dd8 Python`_PyEval_EvalFrameDefault + 23044
    frame #13: 0x00000001005333d4 Python`_PyEval_EvalCode + 452
    frame #14: 0x0000000100449638 Python`_PyFunction_Vectorcall + 344
    frame #15: 0x000000010054040c Python`call_function + 128
    frame #16: 0x00000001005385bc Python`_PyEval_EvalFrameDefault + 16872
    frame #17: 0x0000000100449598 Python`_PyFunction_Vectorcall + 184
    frame #18: 0x000000010054040c Python`call_function + 128
    frame #19: 0x00000001005385bc Python`_PyEval_EvalFrameDefault + 16872
    frame #20: 0x0000000100449598 Python`_PyFunction_Vectorcall + 184
    frame #21: 0x000000010054040c Python`call_function + 128
    frame #22: 0x0000000100539dd8 Python`_PyEval_EvalFrameDefault + 23044
    frame #23: 0x00000001005333d4 Python`_PyEval_EvalCode + 452
    frame #24: 0x0000000100449638 Python`_PyFunction_Vectorcall + 344
    frame #25: 0x000000010054040c Python`call_function + 128
    frame #26: 0x0000000100539dd8 Python`_PyEval_EvalFrameDefault + 23044
    frame #27: 0x0000000100449718 Python`_PyFunction_Vectorcall + 568
    frame #28: 0x000000010054040c Python`call_function + 128
    frame #29: 0x00000001005385bc Python`_PyEval_EvalFrameDefault + 16872
    frame #30: 0x0000000100449598 Python`_PyFunction_Vectorcall + 184
    frame #31: 0x0000000100534d30 Python`_PyEval_EvalFrameDefault + 2396
    frame #32: 0x0000000100449598 Python`_PyFunction_Vectorcall + 184
    frame #33: 0x000000010054040c Python`call_function + 128
    frame #34: 0x0000000100538de0 Python`_PyEval_EvalFrameDefault + 18956
    frame #35: 0x0000000100449598 Python`_PyFunction_Vectorcall + 184
    frame #36: 0x000000010054040c Python`call_function + 128
    frame #37: 0x0000000100538de0 Python`_PyEval_EvalFrameDefault + 18956
    frame #38: 0x0000000100449598 Python`_PyFunction_Vectorcall + 184
    frame #39: 0x000000010044c118 Python`method_vectorcall + 388
    frame #40: 0x00000001005f838c Python`t_bootstrap + 120
    frame #41: 0x0000000100597e84 Python`pythread_wrapper + 48
    frame #42: 0x000000019da6d06c libsystem_pthread.dylib`_pthread_start + 148

[prusnak]

AFAIk hidapi is not thread-safe on neither of supported platforms

[afilini]

Do you have any links/documentation where they talk about thread safety? I find it very weird that it crashes even when the threads are properly synchronized. And it only happens in one specific os/arch combination, not on x86 macos or arm linux.

[prusnak]

Hm, I might be wrong. I only found this 11 years old comment: signal11/hidapi#56 (comment)

On Linux and Mac, HIDAPI is fully thread-safe.

On Windows, it mostly is. You should be able to read and write at the same time like you're saying. One thing that won't work is using hid_close() to stop a hid_read() which is on another thread.

[prusnak]

@afilini do you see the same failure with the following code?

import hid
import threading

def e():
    m.acquire()
    print(hid.enumerate())
    m.release()

m = threading.Lock()

a = threading.Thread(target=e, args=())
b = threading.Thread(target=e, args=())
c = threading.Thread(target=e, args=())

a.start()
b.start()
c.start()

a.join()
b.join()
c.join()

If yes, we can close this issue and open a new one in https://github.com/trezor/cython-hidapi because the issue is not in hwi but rather in cython-hidapi.

[afilini]

Yep, same issue

[achow101]

Closing as this is an usptream issue.

[prusnak]

Created trezor/cython-hidapi#148 to continue there