Skip to content

Swift: rework resource dir #14629

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 3, 2023
Merged

Swift: rework resource dir #14629

merged 3 commits into from
Nov 3, 2023

Conversation

redsun82
Copy link
Contributor

This fetches the resource directory directly from the released
toolchains, allowing us to stop prebuilding and assembling them.
Moreover insertion of our resource directory is moved to the lua
tracing configuration (solving a TODO) and enhanced. Now all options
that start with the original resource directory (either explicit or
implied) are redirected to our resource directory.

This solves a problem where -I <original resource dir>/some/path was
passed to the extractor and did not work.

This works around the 5.9 linux compatibility problem by including the
PackageDescription swift modules in the in-dist toolchain. Copying the
toolchain and fixing the -I flag was not enough as for some reason
compilation of PackageDescription.swiftinterface was causing a crash
in the SIL pass. We work around that by pre-compiling those modules
during the build and including .swiftmodule files in the resource
directory.

TODO (apart from testing):

  • the libraries included in the macOS toolchain are now fat (they were
    intel only before), occupying more space. We should see if we need to
    trim them down.
  • there might be other swiftinterface files causing problems on linux
    lurking around...
  • if we go with this, we can simplify and trim down the prebuilding we
    do leaving out the resource directory.

@github-actions github-actions bot added the Swift label Oct 30, 2023
…olchain

This fetches the resource directory directly from the released
toolchains, allowing us to stop prebuilding and assembling them.
Moreover insertion of our resource directory is moved to the lua
tracing configuration (solving a `TODO`) and enhanced. Now all options
that start with the original resource directory (either explicit or
implied) are redirected to our resource directory.

This solves a problem where `-I <original resource dir>/some/path` was
passed to the extractor and did not work.

This works around the 5.9 linux compatibility problem by including the
`PackageDescription` swift modules in the in-dist toolchain. Copying the
toolchain and fixing the `-I` flag was not enough as for some reason
compilation of `PackageDescription.swiftinterface`  was causing a crash
in the SIL pass. We work around that by pre-compiling those modules
during the build and  including `.swiftmodule` files in the resource
directory.

TODO (apart from testing):
* the libraries included in the macOS toolchain are now fat (they were
  intel only before), occupying more space. We should see if we need to
  trim them down.
* there might be other swiftinterface files causing problems on linux
  lurking around...
* if we go with this, we can simplify and trim down the prebuilding we
  do leaving out the resource directory.
@redsun82 redsun82 force-pushed the redsun82/rework-resource-dir branch from d85bad4 to 567aa11 Compare October 31, 2023 15:21
@redsun82 redsun82 marked this pull request as ready for review October 31, 2023 16:48
@redsun82 redsun82 requested review from a team as code owners October 31, 2023 16:48
Copy link
Contributor

@AlexDenisov AlexDenisov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both QA and DCA runs look good :shipit:


the libraries included in the macOS toolchain are now fat (they were
intel only before), occupying more space. We should see if we need to
trim them down.

I don't think we should thin anything. The main size contributions at this point are just new dylibs, it seems we won't win much by thinning what's in there.

there might be other swiftinterface files causing problems on linux
lurking around...

Let's keep an eye on it!

if we go with this, we can simplify and trim down the prebuilding we
do leaving out the resource directory.

Let's keep that part in place for a moment, it might still be useful for local debugging.

@redsun82 redsun82 merged commit 506b911 into main Nov 3, 2023
@redsun82 redsun82 deleted the redsun82/rework-resource-dir branch November 3, 2023 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants