-
Notifications
You must be signed in to change notification settings - Fork 806
[SYCL][E2E] Add functionality to split build and run of e2e tests #15728
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Dry run of e2e test builds with no logic for setting triple or treating tests as unsupported: https://github.com/intel/llvm/actions/runs/11388738602 68 tests failed, so currently that should be the max amount of unsupported tests for building with the |
Here is the results of the latest dry-run, after removing the In total there were 63 Failures. 6 are addressed by #15789, another 9 are addressed by #15787. This leaves 48 tests that would need changes.
The remaining 7 are fixed in a different manner:
10 tests are marked as XPASS, these should be addressed by #15891 |
MOVED TO #16016 (to make this not a sycl-devops-pr)
Adds functionality to split the compilation and execution of e2e tests across separate machines.
This functionality is enabled via two new lit features
run-mode
andbuild-mode
. By default both of these are enabled and tests run just as they have before. We can call lit with the flag--param split-mode=
with eitherbuild
orrun
to set these features.When using
split-mode=build
only thebuild-mode
feature is added. In this mode all tests that can be built on the system will be built. When running a lit test in this mode two key things change:RUN:
lines will be executed unless they contain%{run}
,%{run-unfiltered-devices}
or%if run-mode
.run-mode
in a requires statement.When using
split-mode=run
only therun-mode
is set. In this case tests are not built, they are only executed. Deciding whether a test is supported in this mode works the same way as when thesplit-mode
parameter is not set. To only execute the tests we ignore all litRUN:
lines unless they contain%{run}
,%{run-unfiltered-devices}
or%if run-mode
. Since we do not build the tests in this mode, for any test to pass we must have ran the tests in eitherbuild
or non-split mode previously.Tests that are marked as
REQUIRES: run-mode
, will ignore the run line filtering from the run only mode. This is done since we know in this case we will not have an executable binary from the build only mode.Some notes/current limitations:
REQUIRES: run-mode
SYCL E2E
action, selecting the "Linux, build" runner, and adding--param split-mode=build
toLIT_OPTS
. This pr does not have the needed CI changes to be able to use the run only mode.