You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In large-scale projects or projects heavily integrated with third-party services, it is often necessary to skip certain tests when specific conditions are not met. For instance, if a required database is not available, running integration tests dependent on this database should be skipped to save time and avoid irrelevant test failures.
Existing solutions
Rust issue #68007: A long-standing issue suggests runtime test skipping but hasn't progressed. Need fundamental change on libtest.
Community crate test-with: This uses a proc macro to add #[ignore] at compile time. While useful, it lacks the flexibility.
Proposal
Enhance nextest to support conditional test skipping based on the output of a setup script. The configuration may look like:
[[profile.default.scripts]]
filter = "test(~integration)"run-if = 'check-db'# alternatively, we can implement an option `skip-if-fail`# setup = 'check-db'# skip-if-fail = true
[script.check-db]
command = 'cargo run -p check-db'
When the script outputs 'false', the tests in the specified group (filtered by "test(~integrate)") should be marked as SKIP.
Thanks for the report! This seems like a good idea. The way I'd probably do this is: currently, setup scripts are either pass or fail. We'd want to add a third "skip" state for them.
As for how to indicate this: the most obvious answer is a special exit code, but that can be a bit brittle. So maybe we should have people write something like "skip" out to a file, along with maybe a message that can be displayed.
To set expectations: We might need this at some point at work, but otherwise I don't imagine I'll get to this soon.
Motivation
In large-scale projects or projects heavily integrated with third-party services, it is often necessary to skip certain tests when specific conditions are not met. For instance, if a required database is not available, running integration tests dependent on this database should be skipped to save time and avoid irrelevant test failures.
Existing solutions
Rust issue #68007: A long-standing issue suggests runtime test skipping but hasn't progressed. Need fundamental change on
libtest
.Community crate test-with: This uses a proc macro to add
#[ignore]
at compile time. While useful, it lacks the flexibility.Proposal
Enhance nextest to support conditional test skipping based on the output of a setup script. The configuration may look like:
When the script outputs 'false', the tests in the specified group (filtered by "test(~integrate)") should be marked as SKIP.
On CI, we can run
cargo nextest run --run-ignored all
to force them to run.The text was updated successfully, but these errors were encountered: