Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-7039] can't use -static-stdlib (or -static-executable) with any Dispatch using app #662

Closed
weissi opened this issue Feb 20, 2018 · 29 comments
Assignees

Comments

@weissi
Copy link
Member

@weissi weissi commented Feb 20, 2018

Previous ID SR-7039
Radar rdar://problem/38913393
Original Reporter @weissi
Type Bug
Status Resolved
Resolution Done
Additional Detail from JIRA
Votes 6
Component/s libdispatch
Labels Bug
Assignee @spevans
Priority Medium

md5: b75d9276b711df9f7b981f99923ed483

is duplicated by:

  • SR-7085 Link error related to libdispatch when using -static-stdlib on Linux

relates to:

  • SR-9368 libdispatch_static unit tests are disabled
  • SR-10625 tests for -static-*

Issue Description:

repro

mkdir /tmp/test && cd /tmp/test && swift package init --type=executable
cat > Sources/test/main.swift <<EOF
import Dispatch
let q = DispatchQueue(label: "q")
q.sync { print("hello") }
EOF
swift build -Xswiftc -static-stdlib

expected

works, just like without the -static-stdlib

actual

$ rm -rf .build && /usr/local/swift/swift-4.0.2-RELEASE-ubuntu16.04/usr/bin/swift build -Xswiftc -static-stdlib
Compile Swift Module 'test' (1 sources)
Linking ./.build/x86_64-unknown-linux/debug/test
/usr/bin/ld.gold: error: cannot find -ldispatch
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCMa'
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCACSS5label_AA0A3QoSV3qosAC10AttributesV10attributesAC20AutoreleaseFrequencyO011autoreleaseI0ACSg6targettcfcfA0_'
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCACSS5label_AA0A3QoSV3qosAC10AttributesV10attributesAC20AutoreleaseFrequencyO011autoreleaseI0ACSg6targettcfcfA1_'
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCACSS5label_AA0A3QoSV3qosAC10AttributesV10attributesAC20AutoreleaseFrequencyO011autoreleaseI0ACSg6targettcfcfA2_'
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCACSS5label_AA0A3QoSV3qosAC10AttributesV10attributesAC20AutoreleaseFrequencyO011autoreleaseI0ACSg6targettcfcfA3_'
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCACSS5label_AA0A3QoSV3qosAC10AttributesV10attributesAC20AutoreleaseFrequencyO011autoreleaseI0ACSg6targettcfC'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
error: terminated(1): /usr/local/swift/swift-4.0.2-RELEASE-ubuntu16.04/usr/bin/swift-build-tool -f /tmp/test/.build/debug.yaml main

cause

libdispatch.a isn't shipped even though it could be. Since this patch libdispatch.a should work but it needs to be compiled with

./configure --enable static=yes
@belkadan
Copy link

@belkadan belkadan commented Feb 21, 2018

I was pretty sure we fixed this in 4.1. Can you check there?

@belkadan
Copy link

@belkadan belkadan commented Feb 27, 2018

dgrove-oss (JIRA User), any insights?

@swift-ci
Copy link

@swift-ci swift-ci commented Feb 27, 2018

Comment by David Grove (JIRA)

At least in the snapshot build I just grabbed, there was a libdisaptch.so, but not a libdispatch.a. I'd speculate there's still a missing bit of CMake configuration that is causing libdispatch.a to either not be built or to not be installed.

@compnerd any thoughts?

dgrove$ find . -name "dispatch"
./swift-DEVELOPMENT-SNAPSHOT-2018-02-26-a-ubuntu16.04/usr/lib/swift/dispatch
./swift-DEVELOPMENT-SNAPSHOT-2018-02-26-a-ubuntu16.04/usr/lib/swift/dispatch/dispatch.h
./swift-DEVELOPMENT-SNAPSHOT-2018-02-26-a-ubuntu16.04/usr/lib/swift/linux/libdispatch.so

@compnerd
Copy link
Collaborator

@compnerd compnerd commented Mar 4, 2018

shared builds are the default, and I wouldn't be surprised if we weren't enabling static builds. Passing -DBUILD_SHARED_LIBS=NO to the cmake invocation should help.

@weissi
Copy link
Member Author

@weissi weissi commented Mar 27, 2018

@swift-ci create

@weissi
Copy link
Member Author

@weissi weissi commented Mar 27, 2018

this is still an issue. For example, compiling NIO (https://github.com/apple/swift-nio) fails with

 undefined reference to '_Block_copy'
 undefined reference to '_NSConcreteStackBlock'
 undefined reference to '_T08Dispatch0A13WorkItemFlagsVACSu8rawValue_tcfC'
 undefined reference to '_T08Dispatch0A3QoSV11unspecifiedACvau'
 undefined reference to '_T08Dispatch0A4TimeV17uptimeNanosecondss6UInt64Vvg'
 undefined reference to '_T08Dispatch0A4TimeV1loiSbAC_ACtFZ'
 undefined reference to '_T08Dispatch0A4TimeV3nowACyFZ'
 undefined reference to '_T08Dispatch0A4TimeVACs6UInt64V17uptimeNanoseconds_tcfC'
 undefined reference to '_T08Dispatch0A5GroupC4waityyF'
 undefined reference to '_T08Dispatch0A5GroupC5enteryyF'
 undefined reference to '_T08Dispatch0A5GroupC5leaveyyF'
 undefined reference to '_T08Dispatch0A5GroupC6notifyyAA0A3QoSV3qos_AA0A13WorkItemFlagsV5flagsAA0A5QueueC5queueyyXB7executetF'
 undefined reference to '_T08Dispatch0A5GroupCACycfc'
 undefined reference to '_T08Dispatch0A5GroupCMa'
 undefined reference to '_T08Dispatch0A5GroupCN'
 undefined reference to '_T08Dispatch0A5QueueC10AttributesVAEs6UInt64V8rawValue_tcfC'
 undefined reference to '_T08Dispatch0A5QueueC4syncyyyc7execute_tF'
 undefined reference to '_T08Dispatch0A5QueueC5asyncyAA0A5GroupCSg5group_AA0A3QoSV3qosAA0A13WorkItemFlagsV5flagsyyXB7executetF'
 undefined reference to '_T08Dispatch0A5QueueC6globalAcA0A3QoSV0D6SClassO3qos_tFZ'
 undefined reference to '_T08Dispatch0A5QueueCACSS5label_AA0A3QoSV3qosAC10AttributesV10attributesAC20AutoreleaseFrequencyO011autoreleaseI0ACSg6targettcfC'
 undefined reference to '_T08Dispatch0A5QueueCMa'
 undefined reference to '_T08Dispatch0A5QueueCN'
 undefined reference to '_T08Dispatch0A8WorkItemC4waityyF'
 undefined reference to '_T08Dispatch0A8WorkItemC7performyyF'
 undefined reference to '_T08Dispatch0A8WorkItemCAcA0A3QoSV3qos_AA0abC5FlagsV5flagsyyXB5blocktcfc'
 undefined reference to '_T08Dispatch0A8WorkItemCMa'
 undefined reference to '_T08Dispatch0A8WorkItemCN'
 undefined reference to '_T08Dispatch0A9SemaphoreC4waityyF'
 undefined reference to '_T08Dispatch0A9SemaphoreC6signalSiyF'
 undefined reference to '_T08Dispatch0A9SemaphoreCACSi5value_tcfc'
 undefined reference to '_T08Dispatch0A9SemaphoreCMa'

@compnerd
Copy link
Collaborator

@compnerd compnerd commented Apr 1, 2018

Just a note, setting {{BUILD_SHARED_LIBS}} to {{NO}} does in fact build the static libdispatch.

@weissi
Copy link
Member Author

@weissi weissi commented Apr 20, 2018

@compnerd thanks but BUILD_SHARED_LIBS=NO implies that we then wouldn't build the shared libs. Is there a way to build both and can we enable that by default?

Still fails on 4.1 release

root@9c37f1ac9294:/tmp/test# swiftc -version
Swift version 4.1 (swift-4.1-RELEASE)
Target: x86_64-unknown-linux-gnu
root@9c37f1ac9294:/tmp/test# swift build -Xswiftc -static-stdlib
Linking ./.build/x86_64-unknown-linux/debug/test
/usr/bin/ld.gold: error: cannot find -ldispatch
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCMa'
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCACSS5label_AA0A3QoSV3qosAC10AttributesV10attributesAC20AutoreleaseFrequencyO011autoreleaseI0ACSg6targettcfcfA0_'
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCACSS5label_AA0A3QoSV3qosAC10AttributesV10attributesAC20AutoreleaseFrequencyO011autoreleaseI0ACSg6targettcfcfA1_'
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCACSS5label_AA0A3QoSV3qosAC10AttributesV10attributesAC20AutoreleaseFrequencyO011autoreleaseI0ACSg6targettcfcfA2_'
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCACSS5label_AA0A3QoSV3qosAC10AttributesV10attributesAC20AutoreleaseFrequencyO011autoreleaseI0ACSg6targettcfcfA3_'
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '_T08Dispatch0A5QueueCACSS5label_AA0A3QoSV3qosAC10AttributesV10attributesAC20AutoreleaseFrequencyO011autoreleaseI0ACSg6targettcfC'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
error: terminated(1): /usr/bin/swift-build-tool -f /tmp/test/.build/debug.yaml main output:

@compnerd
Copy link
Collaborator

@compnerd compnerd commented May 6, 2018

Unfortunately, there is no way to do that currently. CMake only builds a single type of library by default. I think that doing two CMake invocations is probably the best solution here as it avoids any custom machinery for building both types of libraries. I will note that it is possible to generate both libraries, but requires adding custom logic in the CMake build.

@weissi
Copy link
Member Author

@weissi weissi commented May 13, 2018

@compnerd the old build system supported that so this is a regression, should we just do two invocations for now to fix this?

@swift-ci
Copy link

@swift-ci swift-ci commented May 23, 2018

Comment by Matt Wright (JIRA)

This seems like you wheelhouse?

@compnerd
Copy link
Collaborator

@compnerd compnerd commented May 26, 2018

@weissi - yeah, I think that we should do two invocations for now.

@weissi
Copy link
Member Author

@weissi weissi commented May 26, 2018

Ok, perfect, thank you!

@swift-ci
Copy link

@swift-ci swift-ci commented May 29, 2018

Comment by Aleksei Cherepanov (JIRA)

@compnerd could you show how to build in this way?

@swift-ci
Copy link

@swift-ci swift-ci commented Jun 23, 2018

Comment by Joel Saltzman (JIRA)

ftp27 (JIRA User) I think they mean append it to the build commands here:

cmake -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DBUILD_SHARED_LIBS=NO <path-to-source>

from here at the very bottom: https://github.com/apple/swift-corelibs-libdispatch/blob/master/INSTALL.md

Also, I'm not sure what to pass in addition to "swift build -Xswiftc -static-stdlib" to link to the lib

@weissi
Copy link
Member Author

@weissi weissi commented Aug 25, 2018

@compnerd any news on this one?

@kevints
Copy link
Mannequin

@kevints kevints mannequin commented Oct 1, 2018

PR: #393

@compnerd
Copy link
Collaborator

@compnerd compnerd commented Nov 28, 2018

I believe that this should be working with apple/swift#20793.

@keith
Copy link

@keith keith commented Feb 12, 2019

It looks like most of the PRs and issues related to this are closed, but it looks like the swift snapshots for linux do not have a static version of libdispatch included. I might be misunderstanding the intent here but should they now? If not does anyone know about an issue related to that specifically?

@weissi
Copy link
Member Author

@weissi weissi commented Feb 13, 2019

@keith they really should. We need to reopen then

@weissi
Copy link
Member Author

@weissi weissi commented Feb 13, 2019

Still doesn’t work to work

@weissi
Copy link
Member Author

@weissi weissi commented Apr 1, 2019

Still not fixed in Swift 5.0

root@51bec2f18d8f:/tmp/test# swift -version
Swift version 5.0 (swift-5.0-RELEASE)
Target: x86_64-unknown-linux-gnu
root@51bec2f18d8f:/tmp/test# mkdir /tmp/test && cd /tmp/test && swift package init --type=executable
root@51bec2f18d8f:/tmp/test# cat > Sources/test/main.swift <<EOF
> import Dispatch
> let q = DispatchQueue(label: "q")
> q.sync { print("hello") }
> EOF
root@51bec2f18d8f:/tmp/test# swift build -Xswiftc -static-stdlib
/usr/bin/ld.gold: error: cannot find -lswiftDispatch
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '$s8Dispatch0A5QueueCMa'
/tmp/test/Sources/test/main.swift:2: error: undefined reference to '$s8Dispatch0A5QueueC5label3qos10attributes20autoreleaseFrequency6targetACSS_AA0A3QoSVAC10AttributesVAC011AutoreleaseG0OACSgtcfC'
<compiler-generated>:0: error: undefined reference to '$s8Dispatch0A3QoSV11unspecifiedACvau'
<compiler-generated>:0: error: undefined reference to '$s8Dispatch0A5QueueC10AttributesVN'
<compiler-generated>:0: error: undefined reference to '$s8Dispatch0A5QueueC10AttributesVN'
<compiler-generated>:0: error: undefined reference to '$s8Dispatch0A5QueueC10AttributesV8rawValueAEs6UInt64V_tcfC'
<compiler-generated>:0: error: undefined reference to '$s8Dispatch0A5QueueC10AttributesV8rawValueAEs6UInt64V_tcfC'
<compiler-generated>:0: error: undefined reference to '$s8Dispatch0A5QueueC10AttributesV8rawValueAEs6UInt64V_tcfC'
<compiler-generated>:0: error: undefined reference to '$s8Dispatch0A5QueueC10AttributesV8rawValueAEs6UInt64V_tcfC'
clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

@spevans
Copy link
Contributor

@spevans spevans commented May 28, 2019

The following PRs should fix this for -static-stdlib including a swift-integration-tests test to catch future breakage.

apple/swift#25085
#489
apple/swift-integration-tests#58

@spevans
Copy link
Contributor

@spevans spevans commented May 31, 2019

These 2 PRs fix -static-executable for Linux and also include some swift-integration-tests checks for -static-executable with Dispatch

apple/swift#15183
apple/swift-integration-tests#59

@spevans
Copy link
Contributor

@spevans spevans commented May 31, 2019

Fixed in swift-DEVELOPEMENT-2019-05-30-a-ubuntu18.04 for -static-stdlib

$ cat Sources/test/main.swift 
import Dispatch
let q = DispatchQueue(label: "q")
q.sync { print("hello") }

$ ~/swift-DEVELOPMENT-SNAPSHOT-2019-05-30-a-ubuntu18.04/usr/bin/swiftc -O -o test -static-stdlib -lbsd -lDispatchStubs Sources/test/main.swift 
$ ldd test 
    linux-vdso.so.1 (0x00007ffe18dc3000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9097ac1000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f90978be000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f90976ba000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f909731c000)
    libatomic.so.1 => /usr/lib/x86_64-linux-gnu/libatomic.so.1 (0x00007f9097114000)
    libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f9096eff000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f9096b76000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f909695e000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f909656d000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f9099c5b000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f9096365000)
$ ls -l test
-rwxrwxr-x 1 spse spse 35512208 May 31 21:00 test
$ ./test 
hello
 

@swift-ci
Copy link

@swift-ci swift-ci commented Oct 18, 2019

Comment by Oleh (JIRA)

Swift version 5.1.1 (swift-5.1.1-RELEASE)

Target: x86_64-unknown-linux-gnu

/usr/bin/ld.gold: error: cannot find -lswiftDispatch

/usr/bin/ld.gold: error: cannot find -lFoundation

@spevans
Copy link
Contributor

@spevans spevans commented Oct 18, 2019

Unfortunately this fix went into the master branch after the swift-5.1-branch was created, which is why it is not in the 5.1 release.

Note the Foundation hasnt been fixed, even in master. Only Dispatch has been fixed at the this time.

@spevans
Copy link
Contributor

@spevans spevans commented Nov 15, 2020

Should be fixed in swift-5.3.1 now

$ swift --version
Swift version 5.3.1 (swift-5.3.1-RELEASE)
Target: x86_64-unknown-linux-gnu
$ swift run -Xswiftc -static-stdlib
[3/3] Linking test
hello
$ ldd .build/debug/test
        linux-vdso.so.1 (0x00007ffc27f10000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fded74d8000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fded74d3000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fded74cd000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fded737e000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fded719d000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fded7182000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fded6f8e000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fded95ef000)
$  

Should also work for -static-executable as well

 $ swift run -Xswiftc -static-executable
 [3/3] Linking test
 hello
 $ ldd .build/debug/test
         not a dynamic executable
 $
  

@swift-ci
Copy link

@swift-ci swift-ci commented Nov 15, 2020

Comment by Alexander Skvortsov (JIRA)

I can confirm that the fix works on my project, thanks!

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 5, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants