lib/hostip.c: SCDynamicStoreCopyProxies is not safe to run in a fork

[stanhu]

I did this

The Elasticsearch Ruby client uses the Typhoeus Ruby gem, which uses libcurl. If you do something like this:

require 'typhoeus'

fork {
  Typhoeus.head('https://wwww.google.com')
}

The process crashes with this warning:

objc[73067]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[73067]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

I expected the following

No error.

For simple applications, you can work around the problem by making the libcurl call outside of the fork:

require 'typhoeus'
 
Typhoeus.head('http://localhost')

fork {
  Typhoeus.head('http://localhost')
}

However, this doesn't always work. As discussed in the links below, these macOS system calls are not thread-safe. Under some circumstances calling SCDynamicStoreCopyProxiesWithOptions can crash with a backtrace as the following:

Target 0: (ruby) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x109be0b22)
  * frame #0: 0x0000000186ed66a0 libsystem_trace.dylib`_os_log_preferences_refresh + 64
    frame #1: 0x0000000186ed710c libsystem_trace.dylib`os_log_type_enabled + 712
    frame #2: 0x00000001871ea430 CoreFoundation`-[CFPrefsSearchListSource alreadylocked_copyValueForKey:] + 204
    frame #3: 0x00000001871ea344 CoreFoundation`-[CFPrefsSource copyValueForKey:] + 52
    frame #4: 0x00000001871ea2f8 CoreFoundation`__76-[_CFXPreferences copyAppValueForKey:identifier:container:configurationURL:]_block_invoke + 32
    frame #5: 0x00000001871e3260 CoreFoundation`__108-[_CFXPreferences(SearchListAdditions) withSearchListForIdentifier:container:cloudConfigurationURL:perform:]_block_invoke + 376
    frame #6: 0x000000018735fc78 CoreFoundation`-[_CFXPreferences withSearchListForIdentifier:container:cloudConfigurationURL:perform:] + 384
    frame #7: 0x00000001871e2b30 CoreFoundation`-[_CFXPreferences copyAppValueForKey:identifier:container:configurationURL:] + 168
    frame #8: 0x00000001871e2a4c CoreFoundation`_CFPreferencesCopyAppValueWithContainerAndConfiguration + 112
    frame #9: 0x0000000187e23f24 SystemConfiguration`SCDynamicStoreCopyProxiesWithOptions + 180
    frame #10: 0x000000019d29c01c libcurl.4.dylib`Curl_resolv + 288
    frame #11: 0x000000019d2cd40c libcurl.4.dylib`resolve_server + 320
    frame #12: 0x000000019d2cba80 libcurl.4.dylib`Curl_connect + 5764
    frame #13: 0x000000019d2b3344 libcurl.4.dylib`multi_runsingle + 580
    frame #14: 0x000000019d2b2fdc libcurl.4.dylib`curl_multi_perform + 124
    frame #15: 0x000000019d290bcc libcurl.4.dylib`curl_easy_perform + 276
    frame #16: 0x0000000111a68044 ffi_c.bundle`ffi_call_SYSV + 68
    frame #17: 0x0000000111a639fc ffi_c.bundle`ffi_call_int + 1560
    frame #18: 0x0000000111a633d8 ffi_c.bundle`ffi_call + 52
    frame #19: 0x0000000111a56ef0 ffi_c.bundle`call_blocking_function(data=<unavailable>) at Call.c:336:5
    frame #20: 0x0000000105167cd0 libruby.3.1.dylib`rb_nogvl + 268
    frame #21: 0x0000000111a56ec8 ffi_c.bundle`rbffi_do_blocking_call(data=<unavailable>) at Call.c:344:5
    frame #22: 0x0000000105019af8 libruby.3.1.dylib`rb_vrescue2 + 368
    frame #23: 0x0000000105019960 libruby.3.1.dylib`rb_rescue2 + 44
    frame #24: 0x0000000111a5715c ffi_c.bundle`rbffi_CallFunction(argc=<unavailable>, argv=<unavailable>, function=0x000000019d290ab8, fnInfo=0x00006000037bad60) at Call.c:387:9
    frame #25: 0x0000000111a5ace8 ffi_c.bundle`attached_method_invoke(cif=<unavailable>, mretval=0x000000016b2cdc90, parameters=<unavailable>, user_data=<unavailable>) at MethodHandle.c:174:26
    frame #26: 0x0000000111a63f08 ffi_c.bundle`ffi_closure_SYSV_inner + 988
    frame #27: 0x0000000111a681b4 ffi_c.bundle`.Ldo_closure + 20
    frame #28: 0x00000001051b64e4 libruby.3.1.dylib`vm_call_cfunc_with_frame + 232
    frame #29: 0x00000001051b8c08 libruby.3.1.dylib`vm_sendish + 1336
    frame #30: 0x000000010519b53c libruby.3.1.dylib`vm_exec_core + 8128

1. https://blog.phusion.nl/2017/10/13/why-ruby-app-servers-break-on-macos-high-sierra-and-what-can-be-done-about-it/
2. https://bugs.python.org/issue33725

This system call was introduced in #7121 by @zajdee. I'm wondering: can we avoid calling this system call more than once? For example, I'm wondering if we can move this into curl_global_init() instead.

curl/libcurl version

curl 7.88.1 (x86_64-apple-darwin22.0) libcurl/7.88.1 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.51.0
Release-Date: 2023-02-20
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets

operating system

macOS Ventura 13.4

[zajdee]

We could move it to a global initialization function if that's what makes it safer.

[jay]

Isn't this a forking issue, not a libcurl issue? I don't think we can guarantee forking behavior. See also #788 #6968

[stanhu]

Ultimately it is a forking issue, but I'd argue that #11254 would be useful anyways because:

1. It avoids extra macOS system calls for every IP lookup.
2. Consolidates macOS-specific initialization in a separate file/function.