libcurl compiled on OS X 10.11 fails to load on 10.8 (and earlier) #1330

markovicpp · 2017-03-16T09:32:50Z

I did this

Compiled libcurl on OS X 10.11 with base SDK 10.11 and deployment target 10.8 but the product linking the libcurl stopped working in OS X 10.8 with following message:

Symbol not found: _connectx
Expected in: /usr/lib/libSystem.B.dylib

The problem can be backtraced to introduction of TCP Fast open on OS X.
Officially connectx() is introduced in Darwin 15.0.0 (OS X 10.11 / iOS 9.0).
It looks like the condition based on CONNECT_DATA_IDEMPOTENT value assumes that the library is never going to be used on system with lower version than the base SDK.

I expected the following

The product compiled with base SDK 10.11 and linked with libcurl will work on OS X 10.8 with proper use availability macros.

curl/libcurl version

curl 7.52.1 (x86_64-apple-darwin15.5) libcurl/7.52.1 SecureTransport zlib/1.2.8

operating system

OS X 10.8

bagder · 2017-03-16T09:46:46Z

I suppose the mac build needs a an option like "lowest version I want this to work on"...

markovicpp · 2017-03-16T10:28:52Z

Something similar is done in lib/md5.c, this condition should work:

#if (defined(__MAC_OS_X_VERSION_MIN_REQUIRED) && \
            (__MAC_OS_X_VERSION_MIN_REQUIRED >= 101100)) || \
    (defined(__IPHONE_OS_VERSION_MIN_REQUIRED) && \
            (__IPHONE_OS_VERSION_MIN_REQUIRED >= 90000))

bagder · 2017-03-16T16:18:56Z

You up to providing a full suggested fix as a pull request? I'd appreciate it!

markovicpp · 2017-03-16T20:36:04Z

Keen to do it, but it seems to be a little bit more complicated than I thought so.
I need to do some research about it, and find out why connectx() is not declared as weak-linked symbol. If it was so, the fix would be much easier with better backwards compatibility.

The connectx() function call appeared in Darwin 15.0.0 That covers OS X 10.11, iOS 9 and tvOS 9. Because connectx is not declared with weak_import attribute it’s not possible to build libcurl on OS X 10.11 and later and target systems which don’t have _connectx symbol declared in libsystem_kernel.dylib (i.e. OS 10.8 and earlier). Solution is to use connectx only on platforms that officially support it i.e. by defining CFLAGS="-mmacosx-version-min=10.11" in configure step. Note: It is possible to conditionally use connectx() in libcurl targeting range of systems based on availability determined during runtime using dlsym(). [Bug: curl#1330]

…earlier The connectx() function call appeared in Darwin 15.0.0 That covers OS X 10.11, iOS 9 and tvOS 9. connectx() is not declared with weak_import attribute in sys/socket.h, but it’s allowed to redeclare it with missing availability and weak_import attributes which opens possibility to build libcurl that supports range of OS versions with or without connectx() functionality. Because libcurl uses -Wpartial-availability to warn about symbols not available for all system targets, use of weak linked symbols need to be done within pragma blocks isolating those specific cases. [Bug: curl#1330]

The connectx() function call appeared in Darwin 15.0.0 That covers OS X 10.11, iOS 9 and tvOS 9. connectx() is not declared with weak_import attribute in sys/socket.h, but it’s allowed to redeclare it with missing availability and weak_import attributes which opens possibility to build libcurl that supports range of OS versions with or without connectx() functionality. Because libcurl uses -Wpartial-availability to warn about symbols not available for all system targets, use of weak linked symbols need to be done within pragma blocks isolating those specific cases. [Bug: curl#1330]

markovicpp · 2017-03-20T10:40:32Z

I’ve created two possible solutions to this problem:

The first one is simple compile time decision which does not allow targetting older systems and using --tcp-fastopen on newer using single library. #1336

The second one redeclares connectx() function with missing attributes and checks for the its availability during the runtime. This way it’s possible to use same libcurl library on range on systems and enable --tcp-fastopen one those which have support for connectx(). #1339

bagder · 2017-03-21T16:35:34Z

I think we should decide on which way to go and kill of the other PR. It is highly confusing I think to drive both ways at once.

markovicpp · 2017-03-21T18:22:56Z

I definitely agree, I took the opportunity to work on two approaches and see where it might lead.

I think that despite the PR #1339 offers more flexibility, it’s becoming quite complex workaround requiring runtime check of the symbol availability and system version due to non-standard handling of connectx() function in Darwin libc.

PR #1336 offers simpler compile time but mutes tcp-fastopen feature by default (until it’s configured with targetting minimum system 10.11).

stale · 2017-09-17T18:57:53Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

The connectx() function call appeared in Darwin 15.0.0. That covers OS X 10.11, iOS 9 and tvOS 9. Because connectx is not declared with weak_import attribute it's not possible to build libcurl on OS X 10.11 and later and target systems which don't have _connectx symbol declared in libsystem_kernel.dylib (i.e. OS X 10.8 and earlier). Solution is to use connectx only on platforms that officially support it i.e. by defining CFLAGS="-mmacosx-version-min=10.11" in configure step. Note: It is possible to conditionally use connectx() in libcurl targeting range of systems based on availability determined during runtime using dlsym(). Fixes curl#1330 Closes curl#1336

The previous fix curl#1788 worked just for Xcode 9. This commit extends the fix to older Xcode versions effectively by not using connectx function. Fixes curl#1330 Fixes curl#2080 Closes curl#1336

The previous fix #1788 worked just for Xcode 9. This commit extends the fix to older Xcode versions effectively by not using connectx function. Fixes #1330 Fixes #2080 Closes #1336 Closes #2082

bagder added the build label Mar 16, 2017

stale bot added the stale label Sep 17, 2017

yuyichao mentioned this issue Sep 23, 2017

build failure on macOS JuliaLang/julia#23819

Closed

bitbytedog mentioned this issue Sep 27, 2017

libcurl built on iOS11 Xcode9, fails to build because connectx is unavailable on iOS8 Cogosense/iOSCurlFramework#2

Closed

stale bot closed this as completed Oct 1, 2017

iamed2 mentioned this issue Oct 3, 2017

curl build broken with XCode 9.0 JuliaLang/julia#23980

Closed

markovicpp mentioned this issue Nov 15, 2017

macOS: Fix missing connectx function with Xcode version older than 9.0 #2082

Closed

lock bot locked as resolved and limited conversation to collaborators May 6, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libcurl compiled on OS X 10.11 fails to load on 10.8 (and earlier) #1330

libcurl compiled on OS X 10.11 fails to load on 10.8 (and earlier) #1330

markovicpp commented Mar 16, 2017

bagder commented Mar 16, 2017

markovicpp commented Mar 16, 2017

bagder commented Mar 16, 2017

markovicpp commented Mar 16, 2017

markovicpp commented Mar 20, 2017 •

edited by jay

bagder commented Mar 21, 2017

markovicpp commented Mar 21, 2017

stale bot commented Sep 17, 2017

libcurl compiled on OS X 10.11 fails to load on 10.8 (and earlier) #1330

libcurl compiled on OS X 10.11 fails to load on 10.8 (and earlier) #1330

Comments

markovicpp commented Mar 16, 2017

I did this

I expected the following

curl/libcurl version

operating system

bagder commented Mar 16, 2017

markovicpp commented Mar 16, 2017

bagder commented Mar 16, 2017

markovicpp commented Mar 16, 2017

markovicpp commented Mar 20, 2017 • edited by jay

bagder commented Mar 21, 2017

markovicpp commented Mar 21, 2017

stale bot commented Sep 17, 2017

markovicpp commented Mar 20, 2017 •

edited by jay