[SR-7441] ppc64le build started breaking for swift compiler.

[swift-ci]

Previous ID	SR-7441
Radar	None
Original Reporter	asowani (JIRA User)
Type	Bug

Attachment: Download

Environment

ppc64le running Ubuntu 16.04 distribution.

Additional Detail from JIRA

Votes	0
Component/s	Compiler
Labels	Bug
Assignee	None
Priority	Medium

md5: e8e5748ebca1db8526bf68baa425ac9a

Issue Description:

I was rebuilding swift to validate my ppc64le related changes done in Swift and related repositories. I am observing crashes during swift build. The crashes are happening in swift/stdlib/public/core/Builtin.swift file and I was able to trace the cause to a new addition made under
commit 3be2faf which seems to be a new implementation for 64-bit StringGuts.

H

ere is the stack trace I am seeing in the console:

— bootstrap: note: building runtime v4 target: PackageDescription4:
Compile Swift Module ‘PackageDescription’ (9 sources)
Link PackageDescription
— bootstrap: note: building self-hosted ‘swift-build’: env SWIFT_EXEC=/root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/powerpc64le-unknown-linux/release/swiftc SWIFTPM_BUILD_DIR=/root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/powerpc64le-unknown-linux/release/swift-build-stage1 --disable-sandbox -Xlinker -rpath -Xlinker $ORIGIN/…/lib/swift/linux -Xlinker -L -Xlinker /root/latest/swift-source/buildbot_incremental/foundation-linux-powerpc64le/Foundation -Xlinker -rpath -Xlinker /root/latest/swift-source/buildbot_incremental/foundation-linux-powerpc64le/Foundation -Xswiftc -I/root/latest/swift-source/buildbot_incremental/foundation-linux-powerpc64le/Foundation -Xswiftc -I/root/latest/swift-source/buildbot_incremental/foundation-linux-powerpc64le/Foundation/usr/lib/swift -Xlinker -L -Xlinker /root/latest/swift-source/buildbot_incremental/xctest-linux-powerpc64le -Xlinker -rpath -Xlinker /root/latest/swift-source/buildbot_incremental/xctest-linux-powerpc64le -Xswiftc -I/root/latest/swift-source/buildbot_incremental/xctest-linux-powerpc64le -Xlinker -L/root/latest/swift-source/buildbot_incremental/libdispatch-linux-powerpc64le/src/.libs -Xswiftc -I/root/latest/swift-source/buildbot_incremental/libdispatch-linux-powerpc64le/src -Xswiftc -I/root/latest/swift-source/buildbot_incremental/libdispatch-linux-powerpc64le/src/swift -Xswiftc -I/root/latest/swift-source/swift-corelibs-libdispatch -Xcc -fblocks -Xswiftc -enable-testing --configuration release --build-tests
Fatal error: file /root/latest/swift-source/swift/stdlib/public/core/Builtin.swift, line 466
Current stack trace:
0 libswiftCore.so 0x00003fff83344f90 swift_stdlib_reportFatalErrorInFile + 276
1 libswiftCore.so 0x00003fff83288b28 + 5933864
2 libswiftCore.so 0x00003fff8328859c + 5932444
3 libswiftCore.so 0x00003fff82e944f8 + 1787128
4 libswiftCore.so 0x00003fff832888b8 + 5933240
5 libswiftCore.so 0x00003fff832884f4 + 5932276
6 libswiftCore.so 0x00003fff82e944f8 + 1787128
7 libswiftCore.so 0x00003fff831cd3b4 + 5166004
8 libswiftCore.so 0x00003fff831cd214 + 5165588
9 libswiftCore.so 0x00003fff82e944f8 + 1787128
10 libswiftCore.so 0x00003fff82e936c0 fatalErrorMessage(__::file:line🎏) + 212
11 libswiftCore.so 0x00003fff8317b790 + 4831120
12 libswiftCore.so 0x00003fff83193840 + 4929600
13 libswiftCore.so 0x00003fff8312e308 + 4514568
14 libswiftCore.so 0x00003fff832f794c + 6388044
15 libstdc++.so.6 0x00003fff81bc2f00 __once_proxy + 52
16 libpthread.so.0 0x00003fff81e322f4 + 74484
17 libswiftCore.so 0x00003fff832f7850 swift_once + 120
18 libswiftCore.so 0x00003fff8312e380 static CommandLine.arguments.getter + 52
19 swift-build-stage1 0x00000000258dd360 + 316256
20 libc.so.6 0x00003fff8181309c + 143516
21 libc.so.6 0x00003fff818131e0 __libc_start_main + 184
— bootstrap: error: build failed with exit status -5
utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting

I tried to start swift build in debug mode, but my VM does not have enough resources to complete the build task.

[belkadan]

cc @milseman

[milseman]

So tracing through the logic, this is a little weird. asowani (JIRA User) can you post the git hash where you have the stack trace from? Can you verify what is in /root/latest/swift-source/swift/stdlib/public/core/Builtin.swift, line 466? If you can't finish a full Debug build, can you try with `build-script -r`, which will keep some debug info around and assertions too?

If the same as master at https://github.com/apple/swift/blob/master/stdlib/public/core/Builtin.swift#L466, then it seems like it's seeing a tagged bit pattern. This may be an issue on platforms where our runtime doesn't have support for tagged patterns in bridge objects, which we're leveraging to represent String literals and small strings. CC @jckarter and @gparker42.

[jckarter]

That seems likely. We only specifically use tagged patterns on platforms we've opted into them, like x86_64 and arm64, and the default target info without any customization would not assume anything about the address space and not use high tag bits at all. Maybe we just need to add or customize a SwiftTargetInfo for ppc64le in IRGen.

[milseman]

We'd like to use the same pattern for all 64-bit ABIs. Should we just change the value of SWIFT_ABI_DEFAULT_OBJC_RESERVED_BITS_MASK based on 64-bit or something like that? What do you think is cleanest?

[jckarter]

Making it consistent would be cleanest for sure, but I don't know what the constraints if any on PPC64 platforms in the wild are. We have some guarantees that Apple ARM64 and x86_64 platforms won't use more than the low 56 bits of address space for user mode, but I don't know whether ppc platforms make any similar promises.

[milseman]

asowani (JIRA User), do you know what ppc64le's guarantees concerning user mode address spaces are?

[swift-ci]

Comment by Atul Sowani (JIRA)

I re-built swift using "-r" option.When the build crashed, I ran it through gdb which yielded following:

warning: Could not find DWO CU /root/latest/swift-source/buildbot_incremental/swift-linux-powerpc64le/module-cache/6HHA0BVHXTL9/SwiftShims-1IPGKDU80LD57.pcm(0x8748742808ee24fd) referenced by CU at offset 0x2e5388 [in module /root/latest/swift-source/buildbot_incremental/swift-linux-powerpc64le/lib/swift/linux/libswiftCore.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0xea888 [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0xee6ff [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0xf01af [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0xf1007 [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0xfbe3b [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0x120198 [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/SwiftGlibc-18508AN2IPOUC.pcm(0xf4a5f30bbc7a2f09) referenced by CU at offset 0x1201c2 [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0x167b45 [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0x17d4e4 [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0x18763a [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0x194217 [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0x1d24b5 [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/CoreFoundation-14RWYRSPC07C0.pcm(0x9ba24b865b2db6a3) referenced by CU at offset 0x206c31 [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/swift-corelibs-foundation/../buildbot_incremental/foundation-linux-powerpc64le/1BAND5YGR9OX6/SwiftShims-1IPGKDU80LD57.pcm(0x99f198308ae37944) referenced by CU at offset 0x20bcc2 [in module /root/latest/swift-source/buildbot_incremental/swiftpm-linux-powerpc64le/.bootstrap/bin/../lib/swift/linux/libFoundation.so]
warning: Could not find DWO CU /root/latest/swift-source/buildbot_incremental/swift-linux-powerpc64le/module-cache/3JLXVOYZ8I8ZK/SwiftShims-1IPGKDU80LD57.pcm(0x34e1e7c6995cb4b9) referenced by CU at offset 0x2471 [in module /root/latest/swift-source/buildbot_incremental/swift-linux-powerpc64le/lib/swift/linux/libswiftGlibc.so]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
Fatal error: file /root/latest/swift-source/swift/stdlib/public/core/Builtin.swift, line 466
Current stack trace:
0 libswiftCore.so 0x00003fffb7e000f0 _swift_stdlib_reportFatalErrorInFile + 276
1 libswiftCore.so 0x00003fffb7d40e88 <unavailable> + 5967496
2 libswiftCore.so 0x00003fffb7d408fc <unavailable> + 5966076
3 libswiftCore.so 0x00003fffb7944688 <unavailable> + 1787528
4 libswiftCore.so 0x00003fffb7d40c18 <unavailable> + 5966872
5 libswiftCore.so 0x00003fffb7d40854 <unavailable> + 5965908
6 libswiftCore.so 0x00003fffb7944688 <unavailable> + 1787528
7 libswiftCore.so 0x00003fffb7c84554 <unavailable> + 5195092
8 libswiftCore.so 0x00003fffb7c843b4 <unavailable> + 5194676
9 libswiftCore.so 0x00003fffb7944688 <unavailable> + 1787528
10 libswiftCore.so 0x00003fffb7943c40 fatalErrorMessage(:_:file:line:flags🙂 + 212
11 libswiftCore.so 0x00003fffb7c32830 <unavailable> + 4859952
12 libswiftCore.so 0x00003fffb7c4aa20 <unavailable> + 4958752
13 libswiftCore.so 0x00003fffb7be4b58 <unavailable> + 4541272
14 libswiftCore.so 0x00003fffb7db2a4c <unavailable> + 6433356
15 libstdc++.so.6 0x00003fffb66c2f00 __once_proxy + 52
16 libpthread.so.0 0x00003fffb69422f4 <unavailable> + 74484
17 libswiftCore.so 0x00003fffb7db2950 swift_once + 120
18 libswiftCore.so 0x00003fffb7be4c30 static CommandLine.arguments.getter + 52
19 swift-build-stage1 0x000000002004d0d0 <unavailable> + 315600
20 libc.so.6 0x00003fffb631309c <unavailable> + 143516
21 libc.so.6 0x00003fffb63131e0 __libc_start_main + 184

[swift-ci]

Comment by Atul Sowani (JIRA)

@milseman I am using master branch for build. /root/latest/swift-source/swift/stdlib/public/core/Builtin.swift:466 is the first _sanityCheck statement in the following code snippet:

@inline(__always)
@inlinable
public func _bridgeObject(fromTagged x: UInt) -> Builtin.BridgeObject {
_sanityCheck(x & _objCTaggedPointerBits != 0)
let object: Builtin.BridgeObject = Builtin.valueToBridgeObject❌
_sanityCheck(_isTaggedObject(object))
return object
}

[swift-ci]

Comment by Atul Sowani (JIRA)

@jckarter @milseman - I am checking for user mode address space guarantees on ppc64le.

[swift-ci]

Comment by Atul Sowani (JIRA)

BTW, would it help if I setup a CI for Swift on ppc64le?

[milseman]

I think that would be immensely helpful. I don't know if there's currently a policy for new CI platforms. CC @tkremenek and najacque (JIRA User).

[swift-ci]

Comment by Nicole Jacque (JIRA)

asowani (JIRA User) yes please! I will send you an email to discuss.

[swift-ci]

Comment by Atul Sowani (JIRA)

@jckarter @milseman What I learned from Power Linux kernel developers is that Power supports 4PB of user address space which is equivalent to 52 bits. If you need any specific information, please let me know.

Since I am not an expert in kernel, is it possible for you/Swift community to interact with Power Linux kernel experts on a public forum? If yes, I can initiate the dialog there and provide details to both communities.

Thanks!

[milseman]

Are they able to comment on a thread at forums.swift.org?

For the address space, I think we should do something like the following:

diff --git a/stdlib/public/SwiftShims/System.h b/stdlib/public/SwiftShims/System.h
index abe42c119d2..6cce9985d48 100644
--- a/stdlib/public/SwiftShims/System.h
+++ b/stdlib/public/SwiftShims/System.h
@@ -62,6 +62,8 @@
 #define SWIFT_ABI_DEFAULT_OBJC_WEAK_REFERENCE_MARKER_MASK 0
 #define SWIFT_ABI_DEFAULT_OBJC_WEAK_REFERENCE_MARKER_VALUE 0
 
+#define SWIFT_ABI_DEFAULT_64BIT_SPARE_BITS_MASK 0xFF00000000000007ULL
+
 /*********************************** i386 *************************************/
 
 // Heap objects are pointer-aligned, so the low two bits are unused.
@@ -100,7 +102,8 @@
 #define SWIFT_ABI_DARWIN_X86_64_LEAST_VALID_POINTER 0x100000000ULL
 
 // Only the bottom 56 bits are used, and heap objects are eight-byte-aligned.
-#define SWIFT_ABI_X86_64_SWIFT_SPARE_BITS_MASK 0xFF00000000000007ULL
+#define SWIFT_ABI_X86_64_SWIFT_SPARE_BITS_MASK                                 \
+  SWIFT_ABI_DEFAULT_64BIT_SPARE_BITS_MASK
 
 // Objective-C reserves the high and low bits for tagged pointers.
 // Systems exist which use either bit.
@@ -126,7 +129,8 @@
 
 // TBI guarantees the top byte of pointers is unused.
 // Heap objects are eight-byte aligned.
-#define SWIFT_ABI_ARM64_SWIFT_SPARE_BITS_MASK 0xFF00000000000007ULL
+#define SWIFT_ABI_ARM64_SWIFT_SPARE_BITS_MASK                                  \
+  SWIFT_ABI_DEFAULT_64BIT_SPARE_BITS_MASK
 
 // Objective-C reserves just the high bit for tagged pointers.
 #define SWIFT_ABI_ARM64_OBJC_RESERVED_BITS_MASK 0x8000000000000000ULL
@@ -147,11 +151,13 @@
 /*********************************** powerpc64 ********************************/
 
 // Heap objects are pointer-aligned, so the low three bits are unused.
-#define SWIFT_ABI_POWERPC64_SWIFT_SPARE_BITS_MASK 0x0000000000000007ULL
+#define SWIFT_ABI_POWERPC64_SWIFT_SPARE_BITS_MASK                              \
+  SWIFT_ABI_DEFAULT_64BIT_SPARE_BITS_MASK
 
 /*********************************** s390x ************************************/
 
 // Top byte of pointers is unused, and heap objects are eight-byte aligned.
-#define SWIFT_ABI_S390X_SWIFT_SPARE_BITS_MASK 0x0000000000000007ULL
+#define SWIFT_ABI_S390X_SWIFT_SPARE_BITS_MASK                                  \
+  SWIFT_ABI_DEFAULT_64BIT_SPARE_BITS_MASK
 
 #endif /* SWIFT_ABI_SYSTEM_H */

This puts Swift's generic 64-bit ABI at up to 56 bits of address space (needed for String to have same representation across these platforms). Swift's generic ABI also assumes that the first 4kb of addresses are outside of user-space address space (which is unchanged). Just to double check, is this ok for PPC64?

We should also check with the s390x people. Anyone know who to contact for that?

[milseman]

Opened: #16425