Swift Potentially Uses the Wrong Swift Driver

[thekief]

I recently opened an issue on the Swift repo (swiftlang/swift#65461) but realised after some debugging that it probabl better fits here.

When trying to build an existing application with a custom toolchain based on the Swift 5.8 Release (swiftlang/swift@f9611bc) the application will not compile due to the Swift Package Manager trying to disallow the new driver.

I noticed that when executing swiftc -version the output seems to vastly differ. With Xcode's swiftc I get following output:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -version
swift-driver version: 1.75.2 Apple Swift version 5.8 (swiftlang-5.8.0.124.2 clang-1403.0.22.11.100)
Target: arm64-apple-macosx13.0

while with the custom compiler I get following output:

/Library/Developer/Toolchains/swift-LOCAL-2023-05-02-a.xctoolchain/usr/bin/swift-frontend -version
Apple Swift version 5.8 (swift-5.8-RELEASE)
Target: arm64-apple-macosx13.0

To me it looks like there are some issues with the custom compiled toolchain choosing the wrong driver. Is there any easy-fix for this?

[artemcm]

@thekief , in your second example, you are not invoking the driver from the swift-LOCAL-2023-05-02-a.xctoolchain toolchain, but the compiler frontend: swift-frontend. Whereas from the Xcode toolchain you are invoking the driver swiftc.

[thekief]

Sorry, my bad. I put the wrong command there. Here is the corrected one:

/Library/Developer/Toolchains/swift-LOCAL-2023-05-02-a.xctoolchain/usr/bin/swiftc -version
Apple Swift version 5.8 (swift-5.8-RELEASE)
Target: arm64-apple-macosx13.0

Anyway, my problem still applied. I did some debugging with dtruss and noticed that the my custom toolchain does not check for swift-driver while the stock one does:

351- 1234/0x252e:     16496       1      0 sysctl([CTL_KERN, 4, 0, 0, 0, 0] (2), 0x16BC19140, 0x16BC18DA0, 0x0, 0x0)		 = 0 0
352- 1234/0x252e:     16500       4      3 sysctl([CTL_HW, 1, 0, 0, 0, 0] (2), 0x16BC19240, 0x16BC18DA0, 0x0, 0x0)		 = 0 0
353: 1234/0x252e:     16505       3      2 access("/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver\0", 0x0, 0x0)		 = 0 0
354: 1234/0x252e:     16507       2      1 access("/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver\0", 0x0, 0x0)		 = 0 0
355- 1234/0x2530:     17970:        0:       0 fork()		 = 0 0
356- 1234/0x2530:     17971       1      0 mprotect(0x104EB4000, 0x8000, 0x1)	

I also noticed this details in the README:

https://github.com/apple/swift-driver/blob/221b5eadad03a6da5bbc51c34264a12ce4b4c0fb/README.md?plain=1#L14

Previous snapshots of Swift worked without any problems in that regard. Could it be that using a self-built "release" disables forwarding executions to the driver?

[artemcm]

Could you please clarify how your build a custom toolchain, exactly?

[thekief]

I used the official instructions aka:

swift/utils/build-toolchain <prefix>

[thekief]

Sure, sorry for leaving out this details. My bundle prefix is customCompiler.

Is there anything special you have in mind? Are there some names that are considered "forbidden"?

[artemcm]

@thekief do you have a swift-driver executable adjacent to /Library/Developer/Toolchains/swift-LOCAL-2023-05-02-a.xctoolchain/usr/bin/swiftc?

I am having trouble using build-toolchain script locally, but I just triggered a toolchain build on a release/5.8 PR here:
swiftlang/swift#65587

And the resulting toolchain seems to have the new driver correctly installed and it does get used when calling swiftc from this toolchain.

[thekief]

Thank you for your help there. I had a look at your build and I noticed that some files on my side were kind of missing and broken most likely due to some issues in the build process.

We had some issues during our build times in the night and so far restarting/resuming the build process never caused any issues. That is the first time for me here. I will test the toolchains internally and potentially close the ticket soon

[finagolfin]

@thekief, the trunk and 5.9 snapshots might be a good comparison to your custom toolchain builds, ie you should end up with the same files as in there.

[MaxDesiatov]

@thekief is this issue still reproducible for you?

[thekief]

i will inform you until next Monday. Currently, we switched back to Xcode 14, circumventing any problems with this. I will write about the outcome as soon as possible.

[MaxDesiatov]

@thekief have you had a chance to reproduce this again?

[thekief]

(Potentially) related to this one here:

swiftlang/swift-package-manager#6970

I kinda forgot about this issue. Thank you for the reminder

[MaxDesiatov]

I don't think that SwiftPM issue is related in any way, unless some details in the description of this one were omitted.

[thekief]

Sorry for the bad phrasing:

I wanted to retest it with the new version mentioned in the other ticket. As I cannot build the new version, I have no update on it yet.

[thekief]

I was able to build release/5.9.0 and the issue does not occure anymore.