feat(test): Auto-redact elapsed time #13973
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
r? @weihanglo rustbot has assigned @weihanglo. Use |
rustbot
added
A-testing-cargo-itself
weihanglo
reviewed
May 28, 2024
There was a problem hiding this comment.
Regex support looks quite useful, but what's the benefit of this over [..] in this specific test case?
There was a problem hiding this comment.
I should have asked this in the previous PR. I saw it as a dependency update so forgot to ask.
There was a problem hiding this comment.
When you first create a snapshot, there is no [..] so it might work for that run or for your machine but then fail. Or if you changed output near enough to a [..] you run into this when updating the snapshot.
By making this a redaction, it automatically gets applied when generating the snapshot, removing these extra steps (that you likely only discover after the fact and have to debug)
There was a problem hiding this comment.
Thanks for the clarification. Updated this in the PR description.
weihanglo
approved these changes
May 28, 2024
|
@bors r+ |
bors
added
S-waiting-on-bors
bors
added a commit
that referenced
this pull request
May 28, 2024
feat(test): Auto-redact elapsed time By making this a redaction, it automatically gets applied when generating the snapshot, removing these extra steps (that you likely only discover after the fact and have to debug)
|
💔 Test failed - checks-actions |
bors
added
S-waiting-on-review
|
@bors retry |
bors
added
S-waiting-on-bors
|
☀️ Test successful - checks-actions |
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
May 28, 2024
Update cargo 5 commits in a8d72c675ee52dd57f0d8f2bae6655913c15b2fb..431db31d0dbeda320caf8ef8535ea48eb3093407 2024-05-24 03:34:17 +0000 to 2024-05-28 18:17:31 +0000 - Include `lints.rust.unexpected_cfgs.check-cfg` in the fingerprint (rust-lang/cargo#13958) - feat(test): Auto-redact elapsed time (rust-lang/cargo#13973) - chore: Update to snapbox 0.6 (rust-lang/cargo#13963) - fix: check if rev is full commit sha for github fast path (rust-lang/cargo#13969) - test: switch from `drop` to `let _` due to nightly rustc change (rust-lang/cargo#13964) r? ghost
bors
added a commit
that referenced
this pull request
Jun 2, 2024
refactor: Transition direct assertions from cargo-test-support to snapbox ### What does this PR try to resolve? Cargo has a bespoke testing framework for functional tests - Extra stuff for us to maintain - Don't leverage benefits from contributions related to other projects - Less incentive to be thoroughly documented UI tests are written using snapbox. The latest release of snapbox (#13963) was geared at supporting cargo's needs in the hope that we can consolidate on testing frameworks. Besides having a single set of semantics, benefits we'd gain include - Updating of test snapshots - Fancier redacting of test output (e.g. #13973) This is the first incremental step in this direction. This replaces direct assertions with snapbox assertions. This still leaves all of the CLI output assertions. These will be done incrementally. ### How should we test and review this PR? ### Additional information
bors
added a commit
that referenced
this pull request
Jun 2, 2024
refactor: Transition direct assertions from cargo-test-support to snapbox ### What does this PR try to resolve? Cargo has a bespoke testing framework for functional tests - Extra stuff for us to maintain - Don't leverage benefits from contributions related to other projects - Less incentive to be thoroughly documented UI tests are written using snapbox. The latest release of snapbox (#13963) was geared at supporting cargo's needs in the hope that we can consolidate on testing frameworks. Besides having a single set of semantics, benefits we'd gain include - Updating of test snapshots - Fancier redacting of test output (e.g. #13973) This is the first incremental step in this direction. This replaces direct assertions with snapbox assertions. This still leaves all of the CLI output assertions. These will be done incrementally. ### How should we test and review this PR? ### Additional information
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
By making this a redaction,
it automatically gets applied when generating the snapshot,
removing these extra steps
(that you likely only discover after the fact and have to debug)