Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lib/hostip.c: SCDynamicStoreCopyProxies is not safe to run in a fork #11252

Closed
stanhu opened this issue Jun 5, 2023 · 3 comments
Closed

lib/hostip.c: SCDynamicStoreCopyProxies is not safe to run in a fork #11252

stanhu opened this issue Jun 5, 2023 · 3 comments

Comments

@stanhu
Copy link
Contributor

stanhu commented Jun 5, 2023 •
edited

I did this

The Elasticsearch Ruby client uses the Typhoeus Ruby gem, which uses libcurl. If you do something like this:

require 'typhoeus'

fork {
  Typhoeus.head('https://wwww.google.com')
}

The process crashes with this warning:

objc[73067]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[73067]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

I expected the following

No error.

For simple applications, you can work around the problem by making the libcurl call outside of the fork:

require 'typhoeus'
 
Typhoeus.head('http://localhost')

fork {
  Typhoeus.head('http://localhost')
}

However, this doesn't always work. As discussed in the links below, these macOS system calls are not thread-safe. Under some circumstances calling SCDynamicStoreCopyProxiesWithOptions can crash with a backtrace as the following:

Target 0: (ruby) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x109be0b22)
  * frame #0: 0x0000000186ed66a0 libsystem_trace.dylib`_os_log_preferences_refresh + 64
    frame #1: 0x0000000186ed710c libsystem_trace.dylib`os_log_type_enabled + 712
    frame #2: 0x00000001871ea430 CoreFoundation`-[CFPrefsSearchListSource alreadylocked_copyValueForKey:] + 204
    frame #3: 0x00000001871ea344 CoreFoundation`-[CFPrefsSource copyValueForKey:] + 52
    frame #4: 0x00000001871ea2f8 CoreFoundation`__76-[_CFXPreferences copyAppValueForKey:identifier:container:configurationURL:]_block_invoke + 32
    frame #5: 0x00000001871e3260 CoreFoundation`__108-[_CFXPreferences(SearchListAdditions) withSearchListForIdentifier:container:cloudConfigurationURL:perform:]_block_invoke + 376
    frame #6: 0x000000018735fc78 CoreFoundation`-[_CFXPreferences withSearchListForIdentifier:container:cloudConfigurationURL:perform:] + 384
    frame #7: 0x00000001871e2b30 CoreFoundation`-[_CFXPreferences copyAppValueForKey:identifier:container:configurationURL:] + 168
    frame #8: 0x00000001871e2a4c CoreFoundation`_CFPreferencesCopyAppValueWithContainerAndConfiguration + 112
    frame #9: 0x0000000187e23f24 SystemConfiguration`SCDynamicStoreCopyProxiesWithOptions + 180
    frame #10: 0x000000019d29c01c libcurl.4.dylib`Curl_resolv + 288
    frame #11: 0x000000019d2cd40c libcurl.4.dylib`resolve_server + 320
    frame #12: 0x000000019d2cba80 libcurl.4.dylib`Curl_connect + 5764
    frame #13: 0x000000019d2b3344 libcurl.4.dylib`multi_runsingle + 580
    frame #14: 0x000000019d2b2fdc libcurl.4.dylib`curl_multi_perform + 124
    frame #15: 0x000000019d290bcc libcurl.4.dylib`curl_easy_perform + 276
    frame #16: 0x0000000111a68044 ffi_c.bundle`ffi_call_SYSV + 68
    frame #17: 0x0000000111a639fc ffi_c.bundle`ffi_call_int + 1560
    frame #18: 0x0000000111a633d8 ffi_c.bundle`ffi_call + 52
    frame #19: 0x0000000111a56ef0 ffi_c.bundle`call_blocking_function(data=<unavailable>) at Call.c:336:5
    frame #20: 0x0000000105167cd0 libruby.3.1.dylib`rb_nogvl + 268
    frame #21: 0x0000000111a56ec8 ffi_c.bundle`rbffi_do_blocking_call(data=<unavailable>) at Call.c:344:5
    frame #22: 0x0000000105019af8 libruby.3.1.dylib`rb_vrescue2 + 368
    frame #23: 0x0000000105019960 libruby.3.1.dylib`rb_rescue2 + 44
    frame #24: 0x0000000111a5715c ffi_c.bundle`rbffi_CallFunction(argc=<unavailable>, argv=<unavailable>, function=0x000000019d290ab8, fnInfo=0x00006000037bad60) at Call.c:387:9
    frame #25: 0x0000000111a5ace8 ffi_c.bundle`attached_method_invoke(cif=<unavailable>, mretval=0x000000016b2cdc90, parameters=<unavailable>, user_data=<unavailable>) at MethodHandle.c:174:26
    frame #26: 0x0000000111a63f08 ffi_c.bundle`ffi_closure_SYSV_inner + 988
    frame #27: 0x0000000111a681b4 ffi_c.bundle`.Ldo_closure + 20
    frame #28: 0x00000001051b64e4 libruby.3.1.dylib`vm_call_cfunc_with_frame + 232
    frame #29: 0x00000001051b8c08 libruby.3.1.dylib`vm_sendish + 1336
    frame #30: 0x000000010519b53c libruby.3.1.dylib`vm_exec_core + 8128
  1. https://blog.phusion.nl/2017/10/13/why-ruby-app-servers-break-on-macos-high-sierra-and-what-can-be-done-about-it/
  2. https://bugs.python.org/issue33725

This system call was introduced in #7121 by @zajdee. I'm wondering: can we avoid calling this system call more than once? For example, I'm wondering if we can move this into curl_global_init() instead.

curl/libcurl version

curl 7.88.1 (x86_64-apple-darwin22.0) libcurl/7.88.1 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.51.0
Release-Date: 2023-02-20
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets

operating system

macOS Ventura 13.4
stanhu added a commit to stanhu/curl that referenced this issue Jun 5, 2023
@stanhu
src/hostip.c: Move macOS-specific calls into global init call
eb5772f 
curl#7121 introduced a macOS system call
to `SCDynamicStoreCopyProxies`, which is invoked every time an IP
address needs to be resolved.

However, this system call is not thread-safe, and macOS will kill the
process if the system call is run first in a fork. To make it possible
for the parent process to call this once and prevent the crash, only
invoke this system call in the global initialization routine.

Closes curl#11252
@zajdee
Copy link

zajdee commented Jun 5, 2023

We could move it to a global initialization function if that's what makes it safer.

stanhu added a commit to stanhu/curl that referenced this issue Jun 5, 2023
@stanhu
src/hostip.c: Move macOS-specific calls into global init call
544f615 
curl#7121 introduced a macOS system call
to `SCDynamicStoreCopyProxies`, which is invoked every time an IP
address needs to be resolved.

However, this system call is not thread-safe, and macOS will kill the
process if the system call is run first in a fork. To make it possible
for the parent process to call this once and prevent the crash, only
invoke this system call in the global initialization routine.

Closes curl#11252
@stanhu stanhu mentioned this issue Jun 5, 2023
Closed
stanhu added a commit to stanhu/curl that referenced this issue Jun 5, 2023
@stanhu
src/hostip.c: Move macOS-specific calls into global init call
1d3c869 
curl#7121 introduced a macOS system call
to `SCDynamicStoreCopyProxies`, which is invoked every time an IP
address needs to be resolved.

However, this system call is not thread-safe, and macOS will kill the
process if the system call is run first in a fork. To make it possible
for the parent process to call this once and prevent the crash, only
invoke this system call in the global initialization routine.

Closes curl#11252
stanhu added a commit to stanhu/curl that referenced this issue Jun 5, 2023
@stanhu
src/hostip.c: Move macOS-specific calls into global init call
d21e2b2 
curl#7121 introduced a macOS system call
to `SCDynamicStoreCopyProxies`, which is invoked every time an IP
address needs to be resolved.

However, this system call is not thread-safe, and macOS will kill the
process if the system call is run first in a fork. To make it possible
for the parent process to call this once and prevent the crash, only
invoke this system call in the global initialization routine.

Closes curl#11252
stanhu added a commit to stanhu/curl that referenced this issue Jun 5, 2023
@stanhu
src/hostip.c: Move macOS-specific calls into global init call
0f93c7e 
curl#7121 introduced a macOS system call
to `SCDynamicStoreCopyProxies`, which is invoked every time an IP
address needs to be resolved.

However, this system call is not thread-safe, and macOS will kill the
process if the system call is run first in a fork. To make it possible
for the parent process to call this once and prevent the crash, only
invoke this system call in the global initialization routine.

Closes curl#11252
@jay
Copy link
Member

jay commented Jun 5, 2023

Isn't this a forking issue, not a libcurl issue? I don't think we can guarantee forking behavior. See also #788 #6968

@stanhu
Copy link
Contributor Author

stanhu commented Jun 5, 2023 •
edited

Ultimately it is a forking issue, but I'd argue that #11254 would be useful anyways because:

  1. It avoids extra macOS system calls for every IP lookup.
  2. Consolidates macOS-specific initialization in a separate file/function.

stanhu added a commit to stanhu/curl that referenced this issue Jun 9, 2023
@stanhu
src/hostip.c: Move macOS-specific calls into global init call
ddae69e 
curl#7121 introduced a macOS system call
to `SCDynamicStoreCopyProxies`, which is invoked every time an IP
address needs to be resolved.

However, this system call is not thread-safe, and macOS will kill the
process if the system call is run first in a fork. To make it possible
for the parent process to call this once and prevent the crash, only
invoke this system call in the global initialization routine.

In addition, this change is beneficial because it:

1. Avoids extra macOS system calls for every IP lookup.
2. Consolidates macOS-specific initialization in a separate file.

Closes curl#11252
@Totktonada Totktonada mentioned this issue Jun 30, 2023
Closed
@bagder bagder closed this as completed in c730859 Jul 9, 2023
bch pushed a commit to bch/curl that referenced this issue Jul 19, 2023
@stanhu @bch
hostip.c: Move macOS-specific calls into global init call
80739cb 
curl#7121 introduced a macOS system call
to `SCDynamicStoreCopyProxies`, which is invoked every time an IP
address needs to be resolved.

However, this system call is not thread-safe, and macOS will kill the
process if the system call is run first in a fork. To make it possible
for the parent process to call this once and prevent the crash, only
invoke this system call in the global initialization routine.

In addition, this change is beneficial because it:

1. Avoids extra macOS system calls for every IP lookup.
2. Consolidates macOS-specific initialization in a separate file.

Fixes curl#11252
Closes curl#11254
ptitSeb pushed a commit to wasix-org/curl that referenced this issue Sep 25, 2023
@stanhu @ptitSeb
hostip.c: Move macOS-specific calls into global init call
21c84fe 
curl#7121 introduced a macOS system call
to `SCDynamicStoreCopyProxies`, which is invoked every time an IP
address needs to be resolved.

However, this system call is not thread-safe, and macOS will kill the
process if the system call is run first in a fork. To make it possible
for the parent process to call this once and prevent the crash, only
invoke this system call in the global initialization routine.

In addition, this change is beneficial because it:

1. Avoids extra macOS system calls for every IP lookup.
2. Consolidates macOS-specific initialization in a separate file.

Fixes curl#11252
Closes curl#11254
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

src/hostip.c: Move macOS-specific calls into global init call stanhu/curl
3 participants
@stanhu @jay @zajdee