Skip more OS-independent bundle acceptance tests on PRs#5879
Merged
Conversation
Add a GOOSOnPR filter in test.toml, mirroring the existing GOOS filter but consulted only on GitHub pull_request events. It ANDs with GOOS, inherits down the directory tree, is opt-out (absent = runs everywhere), and surfaces in out.test.toml. The event name comes from GITHUB_EVENT_NAME, so no workflow changes are needed. Skip the following bundle subtrees on Windows/macOS for PRs (each verified to have no OS-specific behavior; all still run on Linux for PRs and on every OS on push to main): invariant, resources, resource_deps, run_as, migrate, state, bundle_tag, lifecycle, summary. Co-authored-by: Isaac
Follow-up to the GOOSOnPR filter: skip additional bundle subtrees on Windows/macOS for PRs (all still run on Linux for PRs and on every OS on push to main). - templates/default-python/combinations: the notebook/DLT/python matrix tests OS-independent bundle logic; the OS-specific wheel build is covered on every OS by templates/default-python/integration_classic. - bundle/validate: config validation and diagnostics. - bundle/variables: variable resolution and interpolation. - bundle/telemetry: telemetry payload assertions (output is OS-normalized). - bundle/deployment: bind/unbind and deploy/destroy logic. The MSYS_NO_PATHCONV and leading "//" are Git Bash path-mangling workarounds (test scaffolding), not OS-specific behavior under test; output is identical on every OS. - bundle/integration_whl: the OS surface is the wheel build (venv layout, python3 alias); it is identical across variants, so keep a symmetric set covering both cluster modes x base/dynamic (serverless, serverless_dynamic_version, interactive_cluster, interactive_cluster_dynamic_version) and skip the rest. Also document the GOOSOnPR contract in .agent/rules/testing.md: OS-specific tests must re-enable themselves on PRs, and MSYS/CRLF/venv scaffolding is not evidence of OS-specific behavior under test. Co-authored-by: Isaac
Collaborator
Integration test reportCommit: e5a4eb8
10 interesting tests: 4 RECOVERED, 4 SKIP, 2 flaky
Top 11 slowest tests (at least 2 minutes):
|
janniklasrose
approved these changes
Jul 9, 2026
Collaborator
Integration test reportCommit: b78ea1e
43 interesting tests: 21 flaky, 15 FAIL, 4 RECOVERED, 2 SKIP, 1 KNOWN
Top 50 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to the
GOOSOnPRfilter (#5876): skip more bundle subtrees on Windows/macOS PR runs. All still run on Linux for PRs and on every OS on push tomain.Each subtree was verified to have no OS-specific behavior under test (not just no OS markers —
MSYS_NO_PATHCONV, leading//, and CRLF-normalizingsedare portability scaffolding that make output OS-identical, not evidence of OS sensitivity):bundle/deployment— bind/unbind and deploy/destroy logicbundle/validate— config validation and diagnosticsbundle/variables— variable resolution and interpolationbundle/telemetry— telemetry payload assertions (output is OS-normalized)templates/default-python/combinations— the notebook/DLT/python matrix; the OS-specific wheel build stays covered bytemplates/default-python/integration_classicbundle/integration_whl— the OS surface is the wheel build (venv layout,python3alias), identical across variants; keep a symmetric set covering both cluster modes × base/dynamic (serverless,serverless_dynamic_version,interactive_cluster,interactive_cluster_dynamic_version) and skip the restAlso documents
GOOSOnPRin.agent/rules/testing.md: tests run on every OS by default, some directories recursively opt out for speed, and an OS-specific test sitting under such a directory should override the inherited skip.This pull request and its description were written by Isaac.