New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discard swift.ld in favor of portable solution and support gold linker in Swift #1157

Merged
merged 2 commits into from Feb 18, 2016

Conversation

Projects
None yet
10 participants
@hpux735
Contributor

hpux735 commented Feb 2, 2016

This pull request addresses the conformances and metadata sections differently than the former Linux solution. Unlike the former solution, the size of the sections are not stored as part of the sections themselves. Rather, section start and end markers are stored in the binary, and are used to compute the size of these sections at load time.

This solution is more portable and allows the use of the gold linker, which is far more stable than BFD on ARM targets. Specifically, the gold linker does not generate R_ARM_COPY relocations, which break protocol conformances, though BFD does. Although gold does address some of these issues, -Bsymbolic is still required because swift still generates R_ARM_REL32 relocations, which are illegal. Finally, because the JIT images also contain the illegal relocations, RuntimeDyld refuses to process them, and as such the interpreter tests are disabled on ARM..

In further support of the use of the gold linker on arm-linux, we added a switch to swift for the selection of linker, and under arm, the gold linker is selected as the default.

This work was the topic of an RFC on the swift-dev mailing list:
https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160111/000822.html

@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 2, 2016

I think the FreeBSD port is also using the linker script, as does the Cygwin port.

/cc @dcci @tinysun212 What do you think?

@modocache

View changes

test/DebugInfo/ASTSection_ObjC.swift Outdated
@@ -12,6 +12,9 @@
// REQUIRES: executable_test
// REQUIRES: objc_interop
// UNSUPPORTED: OS=linux-gnu
// UNSUPPORTED: OS=linux-gnueabihf
// UNSUPPORTED: OS=freebsd

This comment has been minimized.

@modocache

modocache Feb 2, 2016

Collaborator

Are these OS's not already covered by the REQUIRES: objc_interop requirement above? I believe #1032 disabled this test for all Linux variants.

This comment has been minimized.

@hpux735

hpux735 Feb 2, 2016

Contributor

Yep, you're right. Removed.

@modocache

View changes

test/Misc/expression_too_complex.swift Outdated
@@ -1,4 +1,4 @@
// RUN: %target-parse-verify-swift -solver-memory-threshold 5000
// RUN: %target-parse-verify-swift -solver-memory-threshold 4000

This comment has been minimized.

@modocache

modocache Feb 2, 2016

Collaborator

What's the intention behind making this change? Does using gold have an effect on %target-parse-verify-swift, or was this included by accident?

This comment has been minimized.

@hpux735

hpux735 Feb 2, 2016

Contributor

The reasoning is that the 5000 threshold was sufficient memory to process the expression. Reducing it to 4000 allows the test to pass on ARM, and it still passes on x86_64.

This comment has been minimized.

@modocache

modocache Feb 2, 2016

Collaborator

Neat! 👌

@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 2, 2016

By the way, the failing test count in arm-linux is down to 5 (from the 40-or-so prior to this patch)


Testing Time: 601.26s


Failing Tests (5):
Swift :: 1_stdlib/VarArgs.swift
Swift :: Driver/Dependencies/one-way-external.swift
Swift :: Driver/sdk.swift
Swift :: IRGen/c_layout.sil
Swift :: IRGen/class_resilience.swift

Expected Passes : 1699
Expected Failures : 67
Unsupported Tests : 676
Unexpected Failures: 5

@modocache

This comment has been minimized.

Collaborator

modocache commented Feb 2, 2016

@hpux735 That's great! Kudos to you and @tienex, this is fantastic work. I'm pulling it down now to give it a whirl.

I've looked into the c_layout.sil failure and wrote up what I found in #1103. To fix the failure, you'll probably need to modify the IRGen/c_layout.sil test for the default char-signedness of your platform--which might be unsigned in your case.

@tienex

This comment has been minimized.

Contributor

tienex commented Feb 2, 2016

@modocache Yes, that's correct, 'char' is unsigned on Linux/arm, the very same issue you have, I see you did already some work on it! Cool. The other failure that can be fixed quickly is 'Driver/Dependencies/one-way-external.swift', it is failing because it tries to run touch with year 3000, which fails on Linux/arm because of 32-bits time_t, changing it to 2037 makes it pass. SDK seems just a matter of CHECK order. This leaves VarArgs and class_resiliance to be fixed.

@modocache

This comment has been minimized.

Collaborator

modocache commented Feb 2, 2016

The following changes get IRGen/class_resilience.swift passing on Android armv7. I'm not sure why the index passed to getelementptr would be offset by three here--perhaps you have some insight here, @tienex?

diff --git a/test/IRGen/class_resilience.swift b/test/IRGen/class_resilience.swift
index 6868574..b9782fa 100644
--- a/test/IRGen/class_resilience.swift
+++ b/test/IRGen/class_resilience.swift
@@ -232,11 +232,11 @@ public class MyResilientChild : MyResilientParent {
 // CHECK:             [[SIZE_METADATA:%.*]] = call %swift.type* @_TMaV16resilient_struct4Size()
 // CHECK:             call void @swift_initClassMetadata_UniversalStrategy(
 // CHECK-native:      [[METADATA_PTR:%.*]] = bitcast %swift.type* [[METADATA]] to [[INT]]*
-// CHECK-native-NEXT: [[FIELD_OFFSET_PTR:%.*]] = getelementptr inbounds [[INT]], [[INT]]* [[METADATA_PTR]], [[INT]] 12
+// CHECK-native-NEXT: [[FIELD_OFFSET_PTR:%.*]] = getelementptr inbounds [[INT]], [[INT]]* [[METADATA_PTR]], [[INT]] 15
 // CHECK-native-NEXT: [[FIELD_OFFSET:%.*]] = load [[INT]], [[INT]]* [[FIELD_OFFSET_PTR]]
 // CHECK-native-NEXT: store [[INT]] [[FIELD_OFFSET]], [[INT]]* @_TWvdvC16class_resilience26ClassWithResilientProperty1sV16resilient_struct4Size
 // CHECK-native-NEXT: [[METADATA_PTR:%.*]] = bitcast %swift.type* [[METADATA]] to [[INT]]*
-// CHECK-native-NEXT: [[FIELD_OFFSET_PTR:%.*]] = getelementptr inbounds [[INT]], [[INT]]* [[METADATA_PTR]], [[INT]] 13
+// CHECK-native-NEXT: [[FIELD_OFFSET_PTR:%.*]] = getelementptr inbounds [[INT]], [[INT]]* [[METADATA_PTR]], [[INT]] 16
 // CHECK-native-NEXT: [[FIELD_OFFSET:%.*]] = load [[INT]], [[INT]]* [[FIELD_OFFSET_PTR]]
 // CHECK-native-NEXT: store [[INT]] [[FIELD_OFFSET]], [[INT]]* @_TWvdvC16class_resilience26ClassWithResilientProperty5colorVs5Int32
 // CHECK:             ret %swift.type* [[METADATA]]
@@ -249,11 +249,11 @@ public class MyResilientChild : MyResilientParent {
 // CHECK:             [[RECTANGLE_METADATA:%.*]] = call %swift.type* @_TMaV16resilient_struct9Rectangle()
 // CHECK:             call void @swift_initClassMetadata_UniversalStrategy(
 // CHECK-native:      [[METADATA_PTR:%.*]] = bitcast %swift.type* [[METADATA]] to [[INT]]*
-// CHECK-native-NEXT: [[FIELD_OFFSET_PTR:%.*]] = getelementptr inbounds [[INT]], [[INT]]* [[METADATA_PTR]], [[INT]] 11
+// CHECK-native-NEXT: [[FIELD_OFFSET_PTR:%.*]] = getelementptr inbounds [[INT]], [[INT]]* [[METADATA_PTR]], [[INT]] 14
 // CHECK-native-NEXT: [[FIELD_OFFSET:%.*]] = load [[INT]], [[INT]]* [[FIELD_OFFSET_PTR]]
 // CHECK-native-NEXT: store [[INT]] [[FIELD_OFFSET]], [[INT]]* @_TWvdvC16class_resilience33ClassWithResilientlySizedProperty1rV16resilient_struct9Rectangle
 // CHECK-native-NEXT: [[METADATA_PTR:%.*]] = bitcast %swift.type* [[METADATA]] to [[INT]]*
-// CHECK-native-NEXT: [[FIELD_OFFSET_PTR:%.*]] = getelementptr inbounds [[INT]], [[INT]]* [[METADATA_PTR]], [[INT]] 12
+// CHECK-native-NEXT: [[FIELD_OFFSET_PTR:%.*]] = getelementptr inbounds [[INT]], [[INT]]* [[METADATA_PTR]], [[INT]] 15
 // CHECK-native-NEXT: [[FIELD_OFFSET:%.*]] = load [[INT]], [[INT]]* [[FIELD_OFFSET_PTR]]
 // CHECK-native-NEXT: store [[INT]] [[FIELD_OFFSET]], [[INT]]* @_TWvdvC16class_resilience33ClassWithResilientlySizedProperty5colorVs5Int32
 // CHECK:             ret %swift.type* [[METADATA]]
@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 2, 2016

CC @slavapestov for the class_resilience.swift test.

@jckarter

This comment has been minimized.

Member

jckarter commented Feb 2, 2016

(Sorry, missed that you are William Dillon.) We can avoid runtime cost by using .s or .ll files instead of C++ files to define these symbols: https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160118/000914.html Did that not work?

@tienex

This comment has been minimized.

Contributor

tienex commented Feb 2, 2016

@jckarter We tried, but the .ll approach resulted in a too complex expression, while the .s variant has to special case arm because of assembly syntax. In the end, the major problem is that using subtraction in the assembly produces PC-relative relocations, they work on ARM with -Bsymbolic, but will fail with gold on x86-64 because the relocation is unsupported in PIC model or results in being evaluated to zero using BFD linker.

@jckarter

This comment has been minimized.

Member

jckarter commented Feb 2, 2016

Which relocation is it that gold doesn't support—was it R_X86_64_PC64? Was just running into this in #1154 while testing on Linux. Is that something that can be fixed in gold?

@tienex

This comment has been minimized.

Contributor

tienex commented Feb 2, 2016

@jckarter Exactly, it produces an R_X86_64_PC64 relocation, @compnerd may help here, he has some good knowledge of gold.

@slavapestov

This comment has been minimized.

Member

slavapestov commented Feb 2, 2016

@modocache @tienex Perhaps the failing class_resilience test is caused by differences in alignment? Was it failing on Linux/ARM before this patch too?

@jckarter

This comment has been minimized.

Member

jckarter commented Feb 2, 2016

@tienex The R_X86_64_PC64 relocation comes from the .end - .start calculation. We already make assumptions that Swift images will be small code model (<2GB), and .end should always be ahead of .start, so it should be possible to assume it's positive and 32-bit, and pad it out to 64 bits with a literal zero:

.align 8
.long (.swift2_protocol_conformances_end - .swift2_protocol_conformances_start) - 8
.long 0

which should produce a PC32 relocation instead.

@modocache

This comment has been minimized.

Collaborator

modocache commented Feb 2, 2016

@slavapestov Yes, the failure occurs prior to this patch as well.

@slavapestov

This comment has been minimized.

Member

slavapestov commented Feb 2, 2016

@modocache Oh, the problem is obvious.

Those CHECK lines only run when no ObjC runtime is present. We have CI for 32-bit and 64-bit Darwin platforms, but on Linux we're only testing 64-bit. So I think you can safely replace those offsets with {{12|15}}, etc.

@tienex

This comment has been minimized.

Contributor

tienex commented Feb 2, 2016

@jckarter I'll perform more tests tomorrow, I'm on the GMT+1 timezone, no offense :-) We wanted to avoid to clutter the file with ifdefs supporting arm and little/big endian configurations, also we would need to change the top-level cmake file to enable support for assembly files, by default cmake doesn't know how to handle them, and we found some issues with cmake 2.8 which seems to be the minimum version supported by swift, @hpux735 can give you more information regarding this issue.

@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 2, 2016

@jckarter We were able to eventually get CMake to play nice with .S files, though the solution was a little hackish. CMake versions subsequent to 2.8 handle it more cleanly.

@jckarter

This comment has been minimized.

Member

jckarter commented Feb 2, 2016

I wouldn't worry too much about big endian for now; there are far deeper endianness sins in IRGen and the runtime.

@jckarter

This comment has been minimized.

Member

jckarter commented Feb 2, 2016

…though that's another thing an .ll file could abstract away. A zeroext of an i32 relative reference should get assembled as a zero-padded 32-bit relocation too.

@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 2, 2016

@slavapestov I applied your fix for class_resilience. Thanks!

@tinysun212

This comment has been minimized.

Contributor

tinysun212 commented Feb 2, 2016

Cygwin port is not using the linker script swift.ld. It is reading the size of the sections directly from the COFF section header structure which is loaded in memory.

@tienex

This comment has been minimized.

Contributor

tienex commented Feb 2, 2016

@jckarter I performed some tests, even in the case of R_X86_64_PC32 I get gold complain about it, unless I use -Bsymbolic, I could use .quad instead of .long and I would have the same result. BFD accepts it, but like R_X86_64_PC64 produces a zero value, should I assume -Bsymbolic is the default now? If so, we should make it also the default for non-ARM platforms when invoking clang++ from swift during linking shared objects, in this PR we specify it only for ARM/Thumb targets.

@jckarter

This comment has been minimized.

Member

jckarter commented Feb 2, 2016

Yeah, -Bsymbolic was supposed to be the default when building for all ELF, but that got accidentally reverted when we merged the arm-linux branch.

@jckarter

This comment has been minimized.

Member

jckarter commented Feb 2, 2016

I had added it back in #1154, and everything seems to work with BFD using PC64 relocations from metadata to nominal type descriptors, though gold still crashes when linking apps.

@jckarter

This comment has been minimized.

Member

jckarter commented Feb 2, 2016

Longer-term, it might be a good idea to investigate alternatives to -Bsymbolic; if we can mark only Swift-generated symbols to be eagerly resolved, that would be a little bit more hygienic for mixed-language projects. AIUI some awful C++ and Fortran features depend on load-time fixup to work as expected under ELF.

@tienex

This comment has been minimized.

Contributor

tienex commented Feb 2, 2016

@jckarter that's why we ended up with the non-assembly version, it worked independently of -Bsymbolic

@jckarter

This comment has been minimized.

Member

jckarter commented Feb 2, 2016

@tienex Sure, but the behavior of -Bsymbolic is the right thing for Swift-generated symbols.

@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 14, 2016

@swift-ci Please test

@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 14, 2016

@gribozavr Thanks! There's a companion PR for foundation apple/swift-corelibs-foundation#249

@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 14, 2016

I reproduced the failure on Darwin, and fixed it.

@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 14, 2016

@swift-ci Please test

@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 14, 2016

@jrose-apple Does everything look good from your side?

@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 14, 2016

@hpux735 I have ran the tests on Linux/x86_64, and it looks like we would also need a companion PR at least for xctest, and possibly the package manager and possibly lldb (for the JIT'ed code).

You can reproduce the buildbot configuration with:

./utils/build-script --preset=buildbot_linux_1404 install_destdir=/tmp/install installable_package=/tmp/swift.tar.gz
@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 15, 2016

Thanks for spending the time looking into it, @gribozavr . I tried to reproduce your issues with both my branch and the apple/master branch. The errors pasted below occur on the apple master branches as well as mine. Here were my steps to verify (my branch is the only provided example, the apple branch is just using update-checkout --clone as is):

  477  rm -rf *
  478  git clone https://github.com/hpux735/swift.git
  479  cd swift/
  480  ./utils/update-checkout --clone
  481  cd ..
  482  rm -rf swift-corelibs-foundation/
  483  git clone https://github.com/hpux735/swift-corelibs-foundation.git
  484  cd swift-corelibs-foundation/
  485  git checkout gold
  486  cd ..
  487  cd swift
  488  git checkout gold
  489  ./utils/build-script --preset=buildbot_linux_1404 install_destdir=/tmp/install installable_package=/tmp/swift.tar.gz
  490  history

The results from testing are these two errors which seem to be lit being braindead, and have been present for quite a long time.:

******************** TEST 'Swift :: Driver/environment.swift' FAILED ********************
Script:
--
env SDKROOT= '/home/wdillon/swift-build/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheBaTruO' -target x86_64-unknown-gnu-linux -L/foo/ -driver-use-frontend-path /home/wdillon/swift-build/swift/test/Driver/Inputs/print-var.sh /home/wdillon/swift-build/swift/test/Driver/environment.swift LD_LIBRARY_PATH | FileCheck /home/wdillon/swift-build/swift/test/Driver/environment.swift
--
Exit Code: 1

Command Output (stderr):
--
/home/wdillon/swift-build/swift/test/Driver/environment.swift:5:11: error: expected string not found in input
// CHECK: {{^/foo/:[^:]+/lib/swift/linux$}}
          ^
<stdin>:1:1: note: scanning from here
/foo/:/home/wdillon/swift-build/swift/test/Driver/lib/swift/linux:/usr/local/cuda-7.0/lib64:
^
<stdin>:1:36: note: possible intended match here
/foo/:/home/wdillon/swift-build/swift/test/Driver/lib/swift/linux:/usr/local/cuda-7.0/lib64:
********************
FAIL: Swift :: Driver/options-interpreter.swift (463 of 7952)
******************** TEST 'Swift :: Driver/options-interpreter.swift' FAILED ********************
Script:
--
not env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -deprecated-integrated-repl -emit-module 2>&1 | FileCheck -check-prefix=IMMEDIATE_NO_MODULE /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
not env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -emit-module 2>&1 | FileCheck -check-prefix=IMMEDIATE_NO_MODULE /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix INTERPRET /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift a b c | FileCheck -check-prefix ARGS /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### -parse-stdlib /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix PARSE_STDLIB /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### -parse-stdlib | FileCheck -check-prefix PARSE_STDLIB /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### -target x86_64-apple-macosx10.9 -resource-dir /RSRC/ /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-RESOURCE-DIR-ONLY /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### -target x86_64-unknown-linux-gnu -resource-dir /RSRC/ /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-RESOURCE-DIR-ONLY-LINUX /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### -target x86_64-apple-macosx10.9 -L/foo/ /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-L /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### -target x86_64-apple-macosx10.9 -L/foo/ -L/bar/ /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-L2 /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env DYLD_LIBRARY_PATH=/abc/ '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -### -target x86_64-apple-macosx10.9 -L/foo/ -L/bar/ /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-L2-ENV /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### -target x86_64-apple-macosx10.9 /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-NO-FRAMEWORKS /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env DYLD_FRAMEWORK_PATH=/abc/ '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -### -target x86_64-apple-macosx10.9 /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-NO-FRAMEWORKS /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### -target x86_64-apple-macosx10.9 -F/foo/ /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-F /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### -target x86_64-apple-macosx10.9 -F/foo/ -F/bar/ /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-F2 /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env DYLD_FRAMEWORK_PATH=/abc/ '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -### -target x86_64-apple-macosx10.9 -F/foo/ -F/bar/ /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-F2-ENV /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env DYLD_FRAMEWORK_PATH=/abc/ '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -### -target x86_64-apple-macosx10.9 -F/foo/ -F/bar/ -L/foo2/ -L/bar2/ /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-COMPLEX /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env SDKROOT= '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -module-cache-path '/tmp/swift-testsuite-clang-module-cacheAaHAPZ' -### -target x86_64-unknown-linux-gnu -L/foo/ /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-L-LINUX /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
env LD_LIBRARY_PATH=/abc/ '/home/wdillon/swift-generic/build/buildbot_linux/swift-linux-x86_64/bin/swift' -### -target x86_64-unknown-linux-gnu -L/foo/ -L/bar/ /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift | FileCheck -check-prefix=CHECK-LINUX-COMPLEX /home/wdillon/swift-generic/swift/test/Driver/options-interpreter.swift
--
Exit Code: 1

Command Output (stderr):
--
/home/wdillon/swift-build/swift/test/Driver/options-interpreter.swift:21:35: error: expected string not found in input
// CHECK-RESOURCE-DIR-ONLY-LINUX: # LD_LIBRARY_PATH=/RSRC/linux{{$}}
                                  ^
<stdin>:1:1: note: scanning from here
/home/wdillon/swift-build/build/buildbot_linux/swift-linux-x86_64/bin/swift -frontend -interpret /home/wdillon/swift-build/swift/test/Driver/options-interpreter.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -module-cache-path /tmp/swift-testsuite-clang-module-cacheBaTruO -resource-dir /RSRC/ -module-name main # LD_LIBRARY_PATH=/RSRC/linux:/usr/local/cuda-7.0/lib64:
^
<stdin>:1:327: note: possible intended match here
/home/wdillon/swift-build/build/buildbot_linux/swift-linux-x86_64/bin/swift -frontend -interpret /home/wdillon/swift-build/swift/test/Driver/options-interpreter.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -module-cache-path /tmp/swift-testsuite-clang-module-cacheBaTruO -resource-dir /RSRC/ -module-name main # LD_LIBRARY_PATH=/RSRC/linux:/usr/local/cuda-7.0/lib64:

Looking at them with my (admittedly very permissive) expression parser, I don't understand why these tests fail in the first place.

@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 15, 2016

@hpux735 I think these issues are related to your local configuration. Something is adding /usr/local/cuda-7.0/lib64 to your paths.

@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 15, 2016

Thanks @gribozavr . I was able to reproduce what I think was the error you're referencing:

Test Case 'ManifestTests.testManifestLoading' started.
fatal error: Unexpected TOMLItem: file /home/wdillon/swift-gold/swiftpm/Sources/dep/Manifest.swift, line 78
Current stack trace:
/home/wdillon/swift-gold/build/buildbot_linux/swift-linux-x86_64/lib/swift/linux/x86_64/libswiftCore.so(swift_reportError+0x8b) [0x7f7f12a3f58b]
/home/wdillon/swift-gold/build/buildbot_linux/swift-linux-x86_64/lib/swift/linux/x86_64/libswiftCore.so(_swift_stdlib_reportFatalErrorInFile+0x67) [0x7f7f12a50277]
/home/wdillon/swift-gold/build/buildbot_linux/swift-linux-x86_64/lib/swift/linux/x86_64/libswiftCore.so(+0x27296c) [0x7f7f129a096c]
/home/wdillon/swift-gold/build/buildbot_linux/swift-linux-x86_64/lib/swift/linux/x86_64/libswiftCore.so(+0x272cab) [0x7f7f129a0cab]
/home/wdillon/swift-gold/build/buildbot_linux/swift-linux-x86_64/lib/swift/linux/x86_64/libswiftCore.so(function signature specialization <Arg[0] = Exploded, Arg[1] = Exploded, Arg[2] = Exploded> of Swift._assertionFailed (Swift.StaticString, Swift.String, Swift.StaticString, Swift.UInt, flags : Swift.UInt32) -> ()+0x75) [0x7f7f129a0df5]
/home/wdillon/swift-gold/build/buildbot_linux/swiftpm-linux-x86_64/.bootstrap/bin/dep-tests() [0x4a883a]
/home/wdillon/swift-gold/build/buildbot_linux/swiftpm-linux-x86_64/.bootstrap/bin/dep-tests() [0x4a5572]
/home/wdillon/swift-gold/build/buildbot_linux/swiftpm-linux-x86_64/.bootstrap/bin/dep-tests() [0x4a9b6c]
/home/wdillon/swift-gold/build/buildbot_linux/swiftpm-linux-x86_64/.bootstrap/bin/dep-tests() [0x44f300]
/home/wdillon/swift-gold/build/buildbot_linux/swiftpm-linux-x86_64/.bootstrap/bin/dep-tests() [0x44f589]
/home/wdillon/swift-gold/build/buildbot_linux/swiftpm-linux-x86_64/.bootstrap/bin/dep-tests() [0x45130a]
/home/wdillon/swift-gold/build/buildbot_linux/swiftpm-linux-x86_64/.bootstrap/bin/dep-tests() [0x40d698]
/home/wdillon/swift-gold/build/buildbot_linux/swiftpm-linux-x86_64/.bootstrap/bin/dep-tests() [0x44f179]
/home/wdillon/swift-gold/build/buildbot_linux/xctest-linux-x86_64/libXCTest.so(+0x16083) [0x7f7f12522083]
/home/wdillon/swift-gold/build/buildbot_linux/xctest-linux-x86_64/libXCTest.so(+0x16c2e) [0x7f7f12522c2e]
/home/wdillon/swift-gold/build/buildbot_linux/xctest-linux-x86_64/libXCTest.so(+0x14cba) [0x7f7f12520cba]
/home/wdillon/swift-gold/build/buildbot_linux/xctest-linux-x86_64/libXCTest.so(+0x154b5) [0x7f7f125214b5]
/home/wdillon/swift-gold/build/buildbot_linux/xctest-linux-x86_64/libXCTest.so(+0x15f9f) [0x7f7f12521f9f]
/home/wdillon/swift-gold/build/buildbot_linux/xctest-linux-x86_64/libXCTest.so(+0x14cba) [0x7f7f12520cba]
/home/wdillon/swift-gold/build/buildbot_linux/xctest-linux-x86_64/libXCTest.so((extension in XCTest):XCTest.XCTestCase.invokeTest () -> ()+0x150) [0x7f7f12520780]
/home/wdillon/swift-gold/build/buildbot_linux/xctest-linux-x86_64/libXCTest.so(+0x180a7) [0x7f7f125240a7]
/home/wdillon/swift-gold/build/buildbot_linux/xctest-linux-x86_64/libXCTest.so(+0x14cba) [0x7f7f12520cba]
/home/wdillon/swift-gold/build/buildbot_linux/xctest-linux-x86_64/libXCTest.so(XCTest.XCTMain (Swift.Array<XCTest.XCTestCase>) -> ()+0x5b) [0x7f7f125236db]
/home/wdillon/swift-gold/build/buildbot_linux/swiftpm-linux-x86_64/.bootstrap/bin/dep-tests() [0x495c7c]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f7f11323ec5]
/home/wdillon/swift-gold/build/buildbot_linux/swiftpm-linux-x86_64/.bootstrap/bin/dep-tests() [0x40c979]
bootstrap: error: tests failed with exit status -4
./utils/build-script: command terminated with a non-zero exit status 1, aborting
./utils/build-script: command terminated with a non-zero exit status 1, aborting

I'll look into it.

With regard to the other two tests that were failing, it's probably not great that environment-defined variables can break them.

@jrose-apple

This comment has been minimized.

Member

jrose-apple commented Feb 15, 2016

@gribozavr My comments were all addressed, LGTM. :-)

@jrose-apple

This comment has been minimized.

Member

jrose-apple commented Feb 15, 2016

@hpux735 Lit filters out most environment variables but it can't filter out PATH because it expects to be able to use things like "cp" and "ls" unqualified. I'm not sure what to do about LD_LIBRARY_PATH because it's possible some systems depend on it just to run things. We could either:

  • take it out of the environment variable whitelist, or
  • update this test to use env -i on every line.

But either way we can do that in a separate PR.

@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 15, 2016

@jrose-apple Thanks! I agree that it's inappropriate for this PR 👍

@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 18, 2016

@gribozavr would you mind taking a look at the changes in the other PRs and the current version of this. I think we've addressed your comments, but there is some new weirdness with the Evolution validation tests. These are new since the last time we exchanged about this, and I don't know if they're supposed to fail. Either way, they're preventing the buildbot from finishing. In my testing, I deleted the tests and everything went though fine.

@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 18, 2016

CC @slavapestov for Evolution tests.

@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 18, 2016

I'm running tests on Ubuntu 14.04.

@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 18, 2016

Oops! Sorry @slavapestov I just saw that I was a couple hundred commits behind master. I rebased and ran it again and the evolution tests pass. Also, @gribozavr , the patch to swift package manager was just merged in (I works with or without this PR).

Anyway, I just re-tested with your build script, and all the tests pass. The only exception is swift-package-tests :: repl/test-repl-glibc.py post-install.

@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 18, 2016

I tested a combination of:

on Ubuntu 14.04 x86_64 and found no issues. I couldn't reproduce swift-package-tests :: repl/test-repl-glibc.py.

Merging.

gribozavr added a commit that referenced this pull request Feb 18, 2016

Merge pull request #1157 from hpux735/gold
Discard swift.ld in favor of portable solution and support gold linker in Swift

@gribozavr gribozavr merged commit 220a2b9 into apple:master Feb 18, 2016

@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 18, 2016

Wow. I want to sincerely thank @tienex @gribozavr @compnerd @modocache @jckarter and @jrose-apple for all your help! This was one heck of a PR. As far as I can tell, it has the most comments of any non-joke PR.

@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 18, 2016

@hpux735 This is huge, thank you! Could you add an item to the changelog?

@hpux735

This comment has been minimized.

Contributor

hpux735 commented Feb 18, 2016

@gribozavr Of course, under the Swift3 header?

@gribozavr

This comment has been minimized.

Collaborator

gribozavr commented Feb 18, 2016

@hpux735 Yes.

@hpux735 hpux735 deleted the hpux735:gold branch Feb 18, 2016

@modocache modocache referenced this pull request Mar 7, 2016

Closed

Enable cross-compiling #1398

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment