Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-387] Undef. symbols when linking SwiftPM #5328

Closed
hpux735 opened this issue Dec 26, 2015 · 24 comments
Closed

[SR-387] Undef. symbols when linking SwiftPM #5328

hpux735 opened this issue Dec 26, 2015 · 24 comments
Assignees

Comments

@hpux735
Copy link
Contributor

@hpux735 hpux735 commented Dec 26, 2015

Previous ID SR-387
Radar None
Original Reporter @hpux735
Type Sub-task
Status Resolved
Resolution Cannot Reproduce
Additional Detail from JIRA
Votes 0
Component/s Package Manager
Labels Sub-task, StarterBug, arm, armhf, linux
Assignee @hpux735
Priority Medium

md5: c7d1f5db28cce77c7cc686cf6e7bddd9

Parent-Task:

  • SR-40 Port Swift to Linux on Raspberry Pi

relates to:

  • SR-662 swiftpm doesn't correctly order .a files when constructing swiftc args

Issue Description:

Linking fails while compiling the swift 'swift_build' module:

Compiling Swift Module 'swift_build' (3 sources)
Link /home/wdillon/build/Ninja-ReleaseAssert/swiftpm-linux-armv7/.bootstrap/bin/swift-build
bootstrap: note: building self-hosted 'swift-build': env SWIFTC=/home/wdillon/build/Ninja-ReleaseAssert/swift-linux-armv7/bin/swiftc SWIFT_BUILD_TOOL=/home/wdillon/build/Ninja-ReleaseAssert/llbuild-linux-armv7/bin/swift-build-tool SWIFT_BUILD_PATH=/home/wdillon/build/Ninja-ReleaseAssert/swiftpm-linux-armv7 SWIFTPM_EMBED_RPATH=$ORIGIN/../lib/swift/linux /home/wdillon/build/Ninja-ReleaseAssert/swiftpm-linux-armv7/.bootstrap/bin/swift-build
LLVM ERROR: Program used external function '_TMaC18PackageDescription7Package' which could not be resolved!
error: ExitStatus(1, ["/home/wdillon/build/Ninja-ReleaseAssert/swift-linux-armv7/bin/swiftc", "--driver-mode=swift", "-I", "/home/wdillon/build/Ninja-ReleaseAssert/swiftpm-linux-armv7/.bootstrap/lib/swift/pm", "-L", "/home/wdillon/build/Ninja-ReleaseAssert/swiftpm-linux-armv7/.bootstrap/lib/swift/pm", "-lPackageDescription", "/home/wdillon/swiftpm/Package.swift"])
bootstrap: error: build failed with exit status 1
utils/build-script: command terminated with a non-zero exit status 1, aborting
@gottesmm
Copy link
Member

@gottesmm gottesmm commented Dec 26, 2015

Does this happen with optimization enabled?

@hpux735
Copy link
Contributor Author

@hpux735 hpux735 commented Dec 27, 2015

Good question. Optimization is enabled (-O3) during cmark, LLVM, Swift itself (some of Swift is -O2). It appears that the standard library is optimized (-O). Foundation is built without explicit optimization set, but -Onone is used in debug. I'm not able to use the debug build mode on these platforms because there isn't enough RAM for linking.

Does this help? What should it be?

@gottesmm
Copy link
Member

@gottesmm gottesmm commented Dec 27, 2015

If you build without optimization and without debug symbols does the crash happen as well?

My interest is whether or not our dead function elimination is being too aggressive and miscompiling the module.

@mxcl
Copy link
Contributor

@mxcl mxcl commented Jan 15, 2016

Looks like this is failure during the manifest parsing. `swift build` executes `swiftc` linking to a dynamic library `libPackageDescription`, but apparently a symbol from that library is missing or at least cannot be found.

/cc @ddunbar in case he has any ideas.

This is holding up using the PM on Raspberry Pi, which obviously is super cool so it would be nice to fix it.

@hpux735
Copy link
Contributor Author

@hpux735 hpux735 commented Jan 15, 2016

I just noticed that I missed the second comment from Michael. I'm trying to build it now w/o optimizations. The only real way I could think to do this was to symlink from the Ninja-ReleaseAssert to Ninja-DebugAssert the llvm and swift products. Then I compiled llbuild using debug. I tried to build swiftpm, but I got the same error:

{{
Compiling Swift Module 'swift_build' (3 sources)
Link /mnt/temp/build/Ninja-DebugAssert/swiftpm-linux-armv7/.bootstrap/bin/swift-build
bootstrap: note: building self-hosted 'swift-build': env SWIFTC=/mnt/temp/build/Ninja-DebugAssert/swift-linux-armv7/bin/swiftc SWIFT_BUILD_TOOL=/mnt/temp/build/Ninja-DebugAssert/llbuild-linux-armv7/bin/swift-build-tool SWIFT_BUILD_PATH=/mnt/temp/build/Ninja-DebugAssert/swiftpm-linux-armv7 SWIFTPM_EMBED_RPATH=$ORIGIN/../lib/swift/linux /mnt/temp/build/Ninja-DebugAssert/swiftpm-linux-armv7/.bootstrap/bin/swift-build
LLVM ERROR: Program used external function '_TMaC18PackageDescription7Package' which could not be resolved!
error: ExitStatus(1, ["/mnt/temp/build/Ninja-DebugAssert/swift-linux-armv7/bin/swiftc", "--driver-mode=swift", "-I", "/mnt/temp/build/Ninja-DebugAssert/swiftpm-linux-armv7/.bootstrap/lib/swift/pm", "-L", "/mnt/temp/build/Ninja-DebugAssert/swiftpm-linux-armv7/.bootstrap/lib/swift/pm", "-lPackageDescription", "/mnt/temp/swiftpm/Package.swift"])
bootstrap: error: build failed with exit status 1
./utils/build-script: command terminated with a non-zero exit status 1, aborting
}}

@hpux735
Copy link
Contributor Author

@hpux735 hpux735 commented Feb 16, 2016

This is a problem with the JIT and LLVM's ExecutionEngine not understanding REL32 and GOT-PREL relocations. There is an effort on the upstream LLVM side to incorporate changes that fix this issue. I'll link that commit here when it's finished so it can be cherry-picked or merged into swift-llvm.

@hpux735
Copy link
Contributor Author

@hpux735 hpux735 commented Mar 21, 2016

I I haven't had any luck getting the patch into LLVM. I guess I'll leave the patch here, re-open the bug, and un-assign myself.

diff --git a/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp b/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp
index e09b71a..fbba6d8 100644
--- a/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp
+++ b/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp
@@ -480,6 +480,9 @@ void RuntimeDyldELF::resolveARMRelocation(const SectionEntry &Section,
     *TargetPtr |= Value & 0xFFF;
     *TargetPtr |= ((Value >> 12) & 0xF) << 16;
     break;
+  case ELF::R_ARM_REL32:
+    *TargetPtr += Value - FinalAddress;
+    break;
     // Write 24 bit relative value to the branch instruction.
   case ELF::R_ARM_PC24: // Fall through.
   case ELF::R_ARM_CALL: // Fall through.
@@ -1319,6 +1322,19 @@ relocation_iterator RuntimeDyldELF::processRelocationRef(
                           RelType, 0);
         Section.advanceStubOffset(getMaxStubSize());
       }
+    } else if (RelType == ELF::R_ARM_GOT_PREL) {
+      uint32_t GOTOffset = allocateGOTEntries(SectionID, 1);
+
+      RelocationEntry GOTRE(SectionID, Offset, ELF::R_ARM_REL32, GOTOffset);
+      addRelocationForSection(GOTRE, GOTSectionID);
+
+      // Fill in the value of the symbol we're targeting into the GOT
+      RelocationEntry RE = computeGOTOffsetRE(SectionID, GOTOffset,
+                                              Value.Offset, ELF::R_ARM_ABS32);
+      if (Value.SymbolName)
+        addRelocationForSymbol(RE, Value.SymbolName);
+      else
+        addRelocationForSection(RE, Value.SectionID);
     } else {
       uint32_t *Placeholder =
         reinterpret_cast<uint32_t*>(computePlaceholderAddress(SectionID, Offset));

Also available here: https://github.com/hpux735/swift-llvm/tree/arm

@gottesmm
Copy link
Member

@gottesmm gottesmm commented Mar 21, 2016

The person who should look at this is lang hames.

@gottesmm
Copy link
Member

@gottesmm gottesmm commented Mar 21, 2016

I just talked with Lang. Is your patch on llvm-commits?

@hpux735
Copy link
Contributor Author

@hpux735 hpux735 commented Mar 21, 2016

It isn't. We never got that far. I tried working through the LLVM mailing list and that got no where, then I tried to convince someone I know with commit access, but that never got anywhere either.

@gottesmm
Copy link
Member

@gottesmm gottesmm commented Mar 22, 2016

William here is what I would suggest. I already spoke with Lang. He said, send it to the commits list, CC him on it. He will review it. lhames AT you know where.

@gottesmm
Copy link
Member

@gottesmm gottesmm commented Mar 22, 2016

Also feel free to just post the title here. I can find it and CC him.

@hpux735
Copy link
Contributor Author

@hpux735 hpux735 commented Mar 22, 2016

Great! Thanks @gottesmm, I'll do that tomrrow. 🙂

@hpux735
Copy link
Contributor Author

@hpux735 hpux735 commented Mar 22, 2016

@gottesmm
Copy link
Member

@gottesmm gottesmm commented Mar 22, 2016

Just an FYI, this is not the end of the pipeline William ; ).

To get that in you are going to need to go back and forth with Lang on the commits list until the patch is ready. For instance, you have no tests. That patch will definitely need tests to be accepted into LLVM.

Lang is a really nice guy. I told him you are new to LLVM and will need guidance/him to commit the change when it is ready.

@hpux735
Copy link
Contributor Author

@hpux735 hpux735 commented Mar 22, 2016

Thanks Michael 🙂

Yah, that makes sense. Lang is helping me figure out how to write a test for it. Testing is good (of course), but it sure can be a hurdle to learn the testing strategy for a new contributor (especially someone new to and working across swift, Foundation, and swiftpm).

@gottesmm
Copy link
Member

@gottesmm gottesmm commented Mar 22, 2016

Absolutely. No worries. You are helping me, help you, help me, help you, etc.

@gottesmm
Copy link
Member

@gottesmm gottesmm commented Mar 22, 2016

Another suggestion. LLVM has a very active chat room at irc.oftc.net #llvm. It is a great place to get help in situations like this and reduce the turn around time.

@hpux735
Copy link
Contributor Author

@hpux735 hpux735 commented Mar 22, 2016

Cool, that's a great heads-up 🙂

@swift-ci
Copy link
Contributor

@swift-ci swift-ci commented Jul 1, 2016

Comment by Joe (JIRA)

@hpux735: I'm thinking this should go ahead and be closed; swift build is working on Swift 3.0 for our BBB and Pi 2/3 builds.

@hpux735
Copy link
Contributor Author

@hpux735 hpux735 commented Jul 1, 2016

I'm not comfortable closing it until it's fixed without a patched LLVM. But I could be overridden if others feel strongly.

@abertelrud
Copy link
Contributor

@abertelrud abertelrud commented Aug 4, 2016

I agree that this issue should be kept open until it's fixed without patches. Have there been any updates on getting the patch into LLVM?

@ddunbar
Copy link
Member

@ddunbar ddunbar commented Aug 15, 2016

This has been done right, at least I hear people talking about using swift build on the Pi.

@hpux735
Copy link
Contributor Author

@hpux735 hpux735 commented Aug 16, 2016

@ddunbar Yes and no. It requires my fork of llvm. I don't have time to learn the testing infrastructure for LLVM that would be required for a patch upstream. I don't mind if this is marked resolved, though.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 4, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants