Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force enabling SwiftCompilerSources in the build #73417

Merged
merged 3 commits into from
May 7, 2024

Conversation

eeckstein
Copy link
Member

@eeckstein eeckstein commented May 3, 2024

This PR is a follow-up of #73247. Now that SwiftCompilerSources work on all supported platforms, we can make sure the compiler is always built with SwiftCompilerSources.
This means that bootstrapping=off cannot be used anymore.

Also, hosttools is now the default bootstrapping mode on all platforms.
This change also enables SwiftCompilerSources in lldb on Windows (#73322)

Before this PR can be merged, apple/llvm-project#8683 must land in llvm-project.

@eeckstein
Copy link
Member Author

@swift-ci test

Copy link
Collaborator

@compnerd compnerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really ready? I thought that we had pieces of the swiftcompilersources disabled on Windows still.

@@ -910,6 +897,10 @@ set(SWIFT_MAIN_INCLUDE_DIR "${SWIFT_SOURCE_DIR}/include")
set(SWIFT_SHIMS_INCLUDE_DIR "${SWIFT_SOURCE_DIR}/stdlib/public/SwiftShims")
set(SWIFT_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/include")

if (NOT BOOTSTRAPPING_MODE)
Copy link
Member

@etcwilde etcwilde May 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still need to be able to have bootstrapping mode off until we can reliably cross-compile the compiler. Otherwise there is no way to bring up new platforms. (Unless you're okay restricting features to what is available and works with the 5.10 compiler.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is anybody checking that building with bootstrapping off then using that to build a full Swift compiler still works? For example, the regular bootstrapping modes are no longer checked on CI, and have been broken for months on linux.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have been checking. It takes two build-script invocations to get the complete toolchain built. The first stage 0 compiler doesn't have macros or SwiftCompilerSources and it won't pass the test suite. You use that one as the host-tools compiler to bring up a stage 1 compiler with macros and SwiftCompilerSources support. That one will (or should) pass its tests. Finally, I like to do one more stage 2 build with the stage 1 compiler, which compounds issues in either of the other two. Then I run the tests, validation tests, stress tests, and long tests on that stage 2 compiler.

Unfortunately, Swift 6 doesn't build at the moment since there are some unguarded macros in the stdlib, so you can't get through the stage-0 build, but that should be reversible at the moment. You're right though, we really need some level of testing. Either that a stage-0 bootstrapping=off build works, or that we can build the 5.10 compiler, and then build the latest compiler with the 5.10 compiler or we won't be able to bring up Swift on new platforms (at least until we can cross-compile reliably).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have been cross-compiling the Swift compiler to run natively on Android since Swift 5.2, termux/termux-packages#5134, so works fine for me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, @eeckstein, if Evan is still using the compiler with bootstrapping off, can't we keep that mode around until his cross-compilation changes are in? I don't use it and I'm all for your first change in this patch of changing the default mode to HOSTTOOLS, but why disable being able to turn it off right now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still possible to turn if off by not providing a host compiler

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I didn't understand your comment below before, you simply turned it into an undocumented internal option.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exactly

@eeckstein
Copy link
Member Author

eeckstein commented May 6, 2024

I thought that we had pieces of the swiftcompilersources disabled on Windows still.

Only two small things are disabled: an instruction simplification (#73253) and unit tests (#73252). Both should be fixed soon (cc @ahatanaka, @nate-chandler)

@eeckstein
Copy link
Member Author

We still need to be able to have bootstrapping mode off until we can reliably cross-compile the compiler.

@etcwilde I should have mentioned that this is still possible. There is an escape hatch: bootstrapping mode is turned off if no host compiler is found:

message(WARNING "BOOSTRAPPING set to OFF because no Swift compiler is defined")

@eeckstein
Copy link
Member Author

For bringing swift on a new platform we have two short-term solutions:

  1. build the 5.10 compiler on the new platform (as @etcwilde said)
  2. don't build with bootstrapping by not providing a host compiler. This is not tested in CI, but as long as we are not breaking this intentionally it should not be hard to get it working - in worst case with a few manual workarounds and fixes.

The long-term solution will be cross compiling. I don't see another option.

@eeckstein
Copy link
Member Author

@swift-ci test

@finagolfin
Copy link
Contributor

The long-term solution will be cross compiling. I don't see another option.

I think it is the solution now, as spelled out on the forum by Doug years ago. Hopefully, Evan's work later this year on retooling the CMake support for cross-compilation will make it easier on those porting to a new platform.

This was already the intention before, but it didn't work because cmake's `option` only works for boolean variables.
Using `set` instead of `option` fixes that.

This change also enables SwiftCompilerSources in lldb on Windows.
apple#73322
This means that `bootstrapping=off` cannot be used anymore.
Now that SwiftCompilerSources are always enabled, `swift_in_compiler` is not needed anymore.
Set this feature unconditionally for tests which still include `REQUIRES: swift_in_compiler`.
@eeckstein
Copy link
Member Author

@swift-ci test

@eeckstein eeckstein merged commit ade04a7 into apple:main May 7, 2024
5 checks passed
@eeckstein eeckstein deleted the force-bootstrapping branch May 7, 2024 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants