Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build: migrate to file lists for driver invocations #6573

Merged
merged 1 commit into from May 27, 2023

Conversation

compnerd
Copy link
Collaborator

When invoking the driver use file lists rather than passing all the sources on the command line. This is particularly import for Windows where the command line limit is 32k. With deep dependency sets and large projects, it is possible to overrun this limit and fail to invoke the command. The driver itself will invoke the frontend using response files if the invocation will overflow the command line limit.

@compnerd compnerd force-pushed the listing branch 2 times, most recently from b0768f3 to ca904ed Compare May 18, 2023 15:46
@compnerd
Copy link
Collaborator Author

It seems that changing https://github.com/apple/swift-package-manager/pull/6573/files#diff-31b280ef997c175a0c39198547ce45d473e9fb89ac71aaa5ed2ad7e4a92473c8R549 to a file list breaks the ability to build targets individually? 😕

@compnerd
Copy link
Collaborator Author

@swift-ci please smoke test

@MaxDesiatov
Copy link
Member

@swift-ci test windows

Copy link
Member

@tomerd tomerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few follow up questions

@compnerd
Copy link
Collaborator Author

@swift-ci please smoke test

@neonichu
Copy link
Member

neonichu commented May 26, 2023

The failures in self-hosted look legitimate, these jobs run some additional integration tests which are in separate package (but in the same repo, honestly a bit of a weird setup might be worth changing at some point). Easiest way to repro locally is using Utilities/build-using-self. There are probably some tests which need a small update.

When invoking the driver use file lists rather than passing all the
sources on the command line. This is particularly import for Windows
where the command line limit is 32k. With deep dependency sets and large
projects, it is possible to overrun this limit and fail to invoke the
command. The driver itself will invoke the frontend using response files
if the invocation will overflow the command line limit.

Fixes: apple#6402
@compnerd
Copy link
Collaborator Author

The self-host failure was due to the expectations changing due to the use of a file list.

@compnerd
Copy link
Collaborator Author

@swift-ci please smoke test

@compnerd compnerd merged commit 8bf6bd7 into apple:main May 27, 2023
5 checks passed
@compnerd compnerd deleted the listing branch May 27, 2023 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SPM does not use response files for invoking the driver
4 participants