New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testsuite: Require failing commands to check output. #6497

Merged
merged 1 commit into from Dec 29, 2018

Conversation

Projects
None yet
5 participants
@ehuss
Copy link
Contributor

ehuss commented Dec 29, 2018

This requires that commands that expect an error (using with_status()) also check some part of the output, to ensure that the test is working correctly. As evidence by the few tests fixed here, it is dangerous to assume that the correct error will trigger.

Some of these check rustc error messages which is generally undesirable since they aren't always stable, but I think in the few cases they shouldn't be much of a problem. If that seems too risky, I can make them more generic.

@rust-highfive

This comment has been minimized.

Copy link

rust-highfive commented Dec 29, 2018

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

@dwijnand

This comment has been minimized.

Copy link
Member

dwijnand commented Dec 29, 2018

SGTM. We can always rethink the verification if it's too imposing.

@bors: r+

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 29, 2018

📌 Commit f58d107 has been approved by dwijnand

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 29, 2018

⌛️ Testing commit f58d107 with merge 35b2b04...

bors added a commit that referenced this pull request Dec 29, 2018

Auto merge of #6497 - ehuss:test-require-output-check, r=dwijnand
testsuite: Require failing commands to check output.

This requires that commands that expect an error (using `with_status()`) also check some part of the output, to ensure that the test is working correctly. As evidence by the few tests fixed here, it is dangerous to assume that the correct error will trigger.

Some of these check rustc error messages which is generally undesirable since they aren't always stable, but I think in the few cases they shouldn't be much of a problem. If that seems too risky, I can make them more generic.
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Dec 29, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: dwijnand
Pushing 35b2b04 to master...

@bors bors merged commit f58d107 into rust-lang:master Dec 29, 2018

3 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
homu Test successful
Details

@ehuss ehuss referenced this pull request Jan 3, 2019

Merged

Update cargo #57315

bors added a commit to rust-lang/rust that referenced this pull request Jan 4, 2019

Auto merge of #57315 - ehuss:update-cargo, r=alexcrichton
Update cargo

24 commits in 0d1f1bbeabd5b43a7f3ecfa16540af8e76d5efb4..34320d212dca8cd27d06ce93c16c6151f46fcf2e
2018-12-19 14:45:14 +0000 to 2019-01-03 19:12:38 +0000
- Display environment variables for rustc commands (rust-lang/cargo#6492)
- Fix a very minor race condition in `cargo fix`. (rust-lang/cargo#6515)
- Add a high-level overview of how `fix` works. (rust-lang/cargo#6516)
- Add dependency `registry` to `cargo metadata`. (rust-lang/cargo#6500)
- Fix fingerprint calculation for patched deps. (rust-lang/cargo#6493)
- serialize version directly (rust-lang/cargo#6512)
- use DYLD_FALLBACK_LIBRARY_PATH for dylib_path_envvar on macOS (rust-lang/cargo#6355)
- Fix error message when resolving dependencies (rust-lang/cargo#6510)
- use PathBuf in cargo metadata (rust-lang/cargo#6511)
- Fixed link to testsuite in CONTRIBUTING.md (rust-lang/cargo#6506)
- Update display of contents of Cargo.toml (rust-lang/cargo#6501)
- Update display of contents of Cargo.toml (rust-lang/cargo#6502)
- Fixup cargo install's help message (rust-lang/cargo#6495)
- testsuite: Require failing commands to check output. (rust-lang/cargo#6497)
- Delete unnecessary 'return' (rust-lang/cargo#6496)
- Fix new unused patch warning. (rust-lang/cargo#6494)
- Some minor documentation changes. (rust-lang/cargo#6481)
- Add `links` to `cargo metadata`. (rust-lang/cargo#6480)
- Salvaged semver work (rust-lang/cargo#6476)
- Warn on unused patches. (rust-lang/cargo#6470)
- don't write a an incorrect rustc version to the fingerprint file (rust-lang/cargo#6473)
- Rewrite `login` and registry cleanups. (rust-lang/cargo#6466)
- [issue#6461] Fix cargo commands list (rust-lang/cargo#6462)
- Restrict registry names to same style as package names. (rust-lang/cargo#6469)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment