New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ctypes memory error on Apple Silicon with external libffi #86854
Comments
Building python 3.9.1 on Apple Silicon compiled against a external (non-os-provided) libffi makes the following code return a MemoryError: Test: import ctypes
@ctypes.CFUNCTYPE(None, ctypes.c_int, ctypes.c_char_p)
def error_handler(fif, message):
pass I have tracked this down to the following code in malloc_closure.c: #if USING_APPLE_OS_LIBFFI && HAVE_FFI_CLOSURE_ALLOC
if (__builtin_available(macos 10.15, ios 13, watchos 6, tvos 13, *)) {
ffi_closure_free(p);
return;
}
#endif and #if USING_APPLE_OS_LIBFFI && HAVE_FFI_CLOSURE_ALLOC
if (__builtin_available(macos 10.15, ios 13, watchos 6, tvos 13, *)) {
return ffi_closure_alloc(size, codeloc);
}
#endif In fact, while the __builtin_available() call should be guarded by USING_APPLE_OS_LIBFFI, the call to ffi_closure_alloc() should only be guarded by HAVE_FFI_CLOSURE_ALLOC, as this is set as the result of an independent check in setup.py and should be used with external libffi when supported. The following code does work instead: #if HAVE_FFI_CLOSURE_ALLOC
#if USING_APPLE_OS_LIBFFI
if (__builtin_available(macos 10.15, ios 13, watchos 6, tvos 13, *)) {
#endif
ffi_closure_free(p);
return;
#if USING_APPLE_OS_LIBFFI
}
#endif
#endif
#if HAVE_FFI_CLOSURE_ALLOC
#if USING_APPLE_OS_LIBFFI
if (__builtin_available(macos 10.15, ios 13, watchos 6, tvos 13, *)) {
#endif
return ffi_closure_alloc(size, codeloc);
return;
#if USING_APPLE_OS_LIBFFI
}
#endif
#endif |
Could you please sign the CLA? (See the PR for details on that) The PR looks sane. Out of interest: why do you use an external version of libffi? AFAIK the system copy of libffi contains some magic sauce to work nicer with signed binaries. |
Thanks for your quick feedback! I signed the CLA after submitting the PR, but I think it takes a bit of time to percolate through the system. As for the "why", until 3.9.1 conda-forge had been successfully using an external ffi (with 3.9.0 + osx-arm64 patches) and then suddenly it broke. For the time being conda-forge is using the system ffi, which does have other advantages as well, having all the latest patches. However, I was curious as to _why_ it broke, and that led me to discover this bug, and it seemed straightforward to fix. When/if conda-forge will switch back to external ffi is TBD, but if that decision is made this issue (at least) will be taken care of. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: