Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add error-chain errors. #4090

Merged
merged 12 commits into from
May 31, 2017
Merged

Add error-chain errors. #4090

merged 12 commits into from
May 31, 2017

Conversation

jluner
Copy link
Contributor

@jluner jluner commented May 24, 2017

Fixes #4209

Convert CargoResult, CargoError into an implementation provided by error-chain. The previous is_human machinery is mostly removed; now errors are displayed unless of the Internal kind, verbose mode will print all errors.

Convert CargoResult, CargoError into an implementation provided by error-chain. The previous is_human machinery is mostly removed; now errors are displayed unless of the Internal kind, verbose mode will print all errors.
@rust-highfive
Copy link

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @brson (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@jluner
Copy link
Contributor Author

jluner commented May 24, 2017

@alexcrichton As discussed on the issue thread, here's the PR for review.

Currently http_auth_offered fails because it looks like it used the wrong kind of credential.helper. I'll try to figure out how I did that. After that, it looks like custom_build_script_failed had an extra caused-by line, as do two dependency tests. There might be more behind them, but that's where I'm calling it a night.

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great, thanks so much for taking this on @jluner!

cc @matklad

@@ -100,7 +101,7 @@ fn run_unit_tests(options: &TestOptions,
shell.status("Running", cmd.to_string())
})?;

if let Err(e) = cmd.exec() {
if let Err(CargoError(CargoErrorKind::ProcessErrorKind(e), .. )) = cmd.exec() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it may accidentally ignore all other errors, is that intended?

@@ -526,7 +527,7 @@ fn add_plugin_deps(rustc: &mut ProcessBuilder,
let mut search_path = env::split_paths(&search_path).collect::<Vec<_>>();
for id in build_scripts.plugins.iter() {
let key = (id.clone(), Kind::Host);
let output = build_state.get(&key).chain_error(|| {
let output = build_state.get(&key).ok_or_else(|| {
internal(format!("couldn't find libs for plugin dep {}", id))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we could get rid of internal and human as function (I think error-chain just supports strings strings in these closures), do you think that'd be feasible? Or maybe happen in a future PR?

human(format!("failed to load checksum `.cargo-checksum.json` \
of {} v{}",
pkg.package_id().name(),
pkg.package_id().version()))

})?;
let cksum: Checksum = serde_json::from_str(&cksum).chain_error(|| {
let cksum: Checksum = serde_json::from_str(&cksum)
.map_err(CargoError::from).chain_err(|| {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this map_err necessary? Or is an extra foreign_links just needed?

@@ -132,7 +133,7 @@ impl GitRemote {
}
fs::create_dir_all(dst)?;
let repo = git2::Repository::init_bare(dst)?;
fetch(&repo, &url, "refs/heads/*:refs/heads/*", cargo_config)?;
fetch(&repo, &url, "refs/heads/*:refs/heads/*", cargo_config)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(trailing whitespace)

@@ -231,20 +232,22 @@ impl<'a> GitCheckout<'a> {
fn clone_repo(source: &Path, into: &Path) -> CargoResult<git2::Repository> {
let dirname = into.parent().unwrap();

fs::create_dir_all(&dirname).chain_error(|| {
fs::create_dir_all(&dirname).map_err(CargoError::from).chain_err(|| {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to above, these map_err instances shouldn't be necessary, right?

let object = self.repo.find_object(self.revision.0, None)?;
self.repo.reset(&object, git2::ResetType::Hard, None)?;
self.repo.find_object(self.revision.0, None)
.and_then(|obj| {self.repo.reset(&obj, git2::ResetType::Hard, None)})?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this use ? instead of and_then?

self().chain_error(callback)
error_chain! {
types {
CargoError, CargoErrorKind, CargoResultExt, CargoResult;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think eventually we'll want to not override the names here, but I like the idea of leaving this here for now to reduce the churn in the PR

_ => false
}
},
_ => false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make sure we explicitly list it out, mind expanding out this _ to match everything?

// NetworkError chain
error_chain!{
types {
NetworkError, NetworkErrorKind, NetworkResultExt, NetworkResult;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was it possible to fold this into the main error_chain chain?

@jluner
Copy link
Contributor Author

jluner commented May 24, 2017 via email

@bors
Copy link
Collaborator

bors commented May 24, 2017

☔ The latest upstream changes (presumably #4088) made this pull request unmergeable. Please resolve the merge conflicts.

Copy link
Member

@matklad matklad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks awesome, @jluner !

internal(format!("failed to deserialize json"))
})?;
let old_fingerprint = serde_json::from_str(&old_fingerprint_json)
.chain_err(|| {internal(format!("failed to deserialize json"))})?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{} can be dropped.

src/cargo/lib.rs Outdated
print(err.to_string(), shell);
}
}
else {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that it really matters, but usually we format else as } else { in Cargo.

@@ -1003,7 +1004,8 @@ impl TomlManifest {

let dep = replacement.to_dependency(spec.name(), cx, None)?;
let dep = {
let version = spec.version().chain_error(|| {
let version = spec.version().ok_or_else(||
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The brace probably wants to be on the previous line :)

tests/doc.rs Outdated
assert!(stderr.contains("☃"), "no snowman\n{}", stderr);
assert!(stderr.contains("unknown start of token"), "no message\n{}", stderr);
}else{
assert!(false, "an error kind other than ProcessErrorKind was encountered\n");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for \n because we don't show stderr afterwords.

src/cargo/lib.rs Outdated
fn print(error: String, shell: &mut MultiShell) {
let _ = shell.err().say("\nCaused by:", BLACK);
let _ = shell.err().say(format!(" {}", error), BLACK);
}

unsafe fn extend_lifetime(r: &Error) -> &'static Error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could have a safer signature:

    unsafe fn extend_lifetime(r: &Error) -> &(Error + 'static) {
        std::mem::transmute::<&Error, &Error>(r)
    }

It would be nice to add some short comment here, explaining why we need unsafe.

Convert CargoResult, CargoError into an implementation provided by error-chain. The previous is_human machinery is mostly removed; now errors are displayed unless of the Internal kind, verbose mode will print all errors.
@alexcrichton
Copy link
Member

These things can be removed, but not without changing cargo's behavior, so my preference is to work it after I make sure I didn't break anything

Ok sounds great, we can pursue future refactorings after this PR. This is already a fantastic cleanup as it is :)

* rebased
* removed `human` (deferring removing `internal` to a later PR)
* cargo_test.rs - fails on other error kinds
* unnecessary `map_err(CargoError::from)` removed
* fold NetworkError entirely into CargoError
* added justification comment for `extend_lifetime`
* various formatting goofs

The following tests are currently failing:
* `http_auth_offered`
* `custom_build_script_failed`
* `build_deps_for_the_right_arch`
* `dep_with_bad_submodule`
* `update_with_shared_deps`
* `finds_author_email`
* `finds_author_user`
* `finds_author_user_escaped`
* `finds_author_username`
* `finds_git_author`
* `exit_code`
@jluner
Copy link
Contributor Author

jluner commented May 25, 2017

  • rebased
  • removed human (deferring removing internal to a later PR)
  • cargo_test.rs - fails on other error kinds
  • unnecessary map_err(CargoError::from) removed
  • fold NetworkError entirely into CargoError
  • added justification comment for extend_lifetime
  • various formatting goofs

The following tests are currently failing:

  • http_auth_offered
  • custom_build_script_failed
  • build_deps_for_the_right_arch
  • dep_with_bad_submodule
  • update_with_shared_deps
  • finds_author_email
  • finds_author_user
  • finds_author_user_escaped
  • finds_author_username
  • finds_git_author
  • exit_code

I might have done the rebase + push thing wrong? This PR is showing all the commits between my first checkin and tonight, and I don't think it's supposed to work that way.

@jluner
Copy link
Contributor Author

jluner commented May 25, 2017

I just cloned rust-lang/cargo/master locally and ran tests, to make sure I don't bang my head against a wall fixing a test I didn't break, and http_auth_offered, build_deps_for_the_right_arch, and the finds_* tests all failed. I assume these are all sensitive to environment somehow (git settings, arch), so I won't worry about them for now, and I'll focus on the other test failures.

@alexcrichton
Copy link
Member

@jluner oh that's odd, in theory all these tests should be isolated from the environment but we may have forgotten something to keep isolating them!

@alexcrichton
Copy link
Member

Also feel free to just push to this branch and let CI take care of running tests, it may take a little longer but should get the job done!

@jluner
Copy link
Contributor Author

jluner commented May 26, 2017

Dumb question time:

How do I read the CI test results?

I see that AppVeyor is the Windows builds and Travis is non-Windows. Travis shows lots of green entries; did it just skip trying other configurations after the first one built? Also, the travis failure seems to be formatting stuff, so I guess that has to be fixed before it will run tests?

AppVeyor's errors appear to be failing to download something, rather than actually running the tests:

info: downloading component 'rust-std'
error: component download failed for rust-std-x86_64-pc-windows-msvc
info: caused by: could not download file from 'https://static.rust-lang.org/dist/2017-03-03/rust-std-nightly-x86_64-pc-windows-msvc.tar.gz' to 'C:\Users\appveyor\.rustup\downloads\9a9ed6218e17b975ae48735e5c85f224e4a53b0729c7c99d149de19b5d935b43.partial'
info: caused by: error during download
info: caused by: [18] Transferred a partial file (transfer closed with 32046374 bytes remaining to read)
Command exited with code 1

On the topic of formatting: Is there a specific cargo style guide I should reference? I initially tried running cargo fmt on crates-io, and that changed enough things that I figured it would just be noise, so I reverted.

@alexcrichton
Copy link
Member

Ah yeah no worries, it's not always trivial to do so :)

On PRs we test only one configuration on AppVeyor and Travis, everything else is basically skipped. An error like you're looking at looks like a spurious error so I'll just retry it. The automation basically runs cargo test so you can just poke around the logs and look for the failing command or build (if any)

@alexcrichton
Copy link
Member

Also nah we don't yet run rustfmt (I wouldn't recommend doing so) but I'd just in general follow the surrounding style. I can comment on any particular places as well

@alexcrichton
Copy link
Member

So for example:

@bors
Copy link
Collaborator

bors commented May 28, 2017

☔ The latest upstream changes (presumably #4107) made this pull request unmergeable. Please resolve the merge conflicts.

@jluner
Copy link
Contributor Author

jluner commented May 28, 2017

Looks like CI is happy. What's the next step to getting error-chain to land before moving onto the next pass?

@alexcrichton
Copy link
Member

Nice! I'll take a final pass at reviewing and then we can r+ to land this.

Copy link
Member

@alexcrichton alexcrichton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Holy cow I'm blown away this didn't even change the test suite, very nice work! I'm super excited to land this!

I've left some mostly-just-minor comments, but if you've got any questions don't hesitate to ask. And it goes without saying, but thank you so much for taking this on @jluner!

@@ -309,7 +311,7 @@ fn execute_external_subcommand(config: &Config, cmd: &str, args: &[String]) -> C
let command = match path {
Some(command) => command,
None => {
return Err(human(match find_closest(config, cmd) {
return Err(CargoError::from(match find_closest(config, cmd) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where possible I tend to find format!(...).into() a little more ergonomic than wrapping with CargoError::from, but that's not always possible due to inference issues.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In many cases the compiler was just not figuring it out. I can do another pass to make sure into doesn't work. Is there a preference between a turbofished into and from?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah unfortunately into doesn't work w/ turbofish as the type parameter is in the wrong place, if .into() doesn't work then CargoError::from is fine, no worries!

src/bin/cargo.rs Outdated
Err(CliError::code(code))
} else {
Err(CliError::new(Box::new(err), 101))
if let CargoError(CargoErrorKind::ProcessErrorKind(ref perr), ..) = err {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a kind method that should be able to extract this, perhaps that could be used here?

let mut path = url.path_segments().chain_error(|| {
human(format!("pkgid urls must have a path: {}", url))
let mut path = url.path_segments().ok_or_else(|| {
CargoError::from(format!("pkgid urls must have a path: {}", url))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think maybe .into() would work here? (just as an example)

src/cargo/lib.rs Outdated
@@ -30,6 +32,8 @@ extern crate url;

use std::io;
use std::fmt;
use std::error::Error;
use error_chain::ChainedError;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While you're at it mind throwing a newline above this import of error_chain to separate the std and non-std imports?

let result = cmd.exec();

match result {
Err(CargoError(CargoErrorKind::ProcessErrorKind(e), .. )) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the kind method be used here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one's a little more awkward, because I have to check that the Result is an Err and then check the kind as well. Unpacking it might get verbose. I could if let Some(foo) = result.err() and then if let blah = foo.kind(). Or I suppose I could result.err().map(kind).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upon review, we need to move the ProcessError out here, but kind gives a borrow.

pkg.package_id().name(),
pkg.package_id().version()))
let cksum: Checksum = serde_json::from_str(&cksum)
.chain_err(|| {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this call to chain_err stay on the previous line? As-is the indentation looks a little odd

fn description(&self) -> &str { self.error.description() }
}
impl CargoError {
pub fn to_internal(self) -> Self {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be called into_internal to signify the transfer of ownership?

}
impl CargoError {
pub fn to_internal(self) -> Self {
//This is actually bad, because it loses the cause information for foreign_link
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to preserve self.0 somehow?

Copy link
Contributor Author

@jluner jluner May 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore original comment, I was on the wrong track.

I'll look into preserving self.0 somehow.


fn maybe_spurious(err: &CargoError) -> bool {
match err.kind() {
&CargoErrorKind::Git(ref git_err) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The indentation here looks a little off?

use git2;

fn maybe_spurious(err: &CargoError) -> bool {
match err.kind() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be modified to iterate over the error to see if anything along the way is spurious? In theory if there's any spurious error in the chain then this operation should be retried

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would need a trick like the one in lib.rs that walks the chain to print. There, I could convince myself that the transmute-to-static trick is safe because the error was moved into the method and consumed entirely during printing. Here, we start with a &CargoError. I don't have the confidence to say that transmuting the chain to static and downcasting here is similarly ok, just because I'm still too new to Rust. It seems like it might be ok? Maybe? I can at least put it in, and people wiser than I can review, and then just pull it back out if it's no good.

Fix some formatting items.
Changes Internal error kind to preserve the original error.
Changes network retry logic to inspect full error chain for spurious
errors.
@alexcrichton
Copy link
Member

Alright this is looking great! Let's land this and continue to iterate in future PRs.

@bors: r+

@bors
Copy link
Collaborator

bors commented May 31, 2017

📌 Commit def249f has been approved by alexcrichton

@bors
Copy link
Collaborator

bors commented May 31, 2017

⌛ Testing commit def249f with merge 705c30b...

@bors
Copy link
Collaborator

bors commented May 31, 2017

💔 Test failed - status-travis

@jluner
Copy link
Contributor Author

jluner commented May 31, 2017

error: failed to run custom build command for `backtrace-sys v0.1.10`

process didn't exit successfully: `/tmp//target/release/build/backtrace-sys-3989d5c503a9238f/build-script-build` (exit code: 101)

Looks like it might be a cross-compilation issue? Or environment? Other jobs got past that successfully.

@alexcrichton
Copy link
Member

I'll try to land #4113 first which should unblock this by just removing those configurations, they're all tested upstream in rust-lang/rust anyway.

@alexcrichton
Copy link
Member

@bors: retry

@bors
Copy link
Collaborator

bors commented May 31, 2017

⌛ Testing commit def249f with merge 03c0a41...

bors added a commit that referenced this pull request May 31, 2017
Add error-chain errors.

Fixes #4209

Convert CargoResult, CargoError into an implementation provided by error-chain. The previous is_human machinery is mostly removed; now errors are displayed unless of the Internal kind, verbose mode will print all errors.
@bors
Copy link
Collaborator

bors commented May 31, 2017

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 03c0a41 to master...

@bors bors merged commit def249f into rust-lang:master May 31, 2017
@alexcrichton
Copy link
Member

Woo!

Thanks so much again for this @jluner! This is amazing 💖

@jluner
Copy link
Contributor Author

jluner commented Jun 1, 2017

Exciting! For a first contribution experience, this went well (modulo hardware problems locally). My next few questions are:
If I push later changes back up to my fork, does that mess with this pull request, or is it "closed"?
What path forward are you looking for with error changes? Do you have an idea of what "verbose" should mean after cargo prints all error messages by default? Should some errors still be masked with a generic "cargo error occurred" message using the is_human logic that's still in place?

I'm happy to keep contributing on this issue. Judging by my rate of productivity, I may have to end up handing some of it off or pausing work towards the middle or end of June.

@alexcrichton
Copy link
Member

Oh typically with github and git you'll do development on branches vs the master branch itself, but yeah it's fine to push to your master branch and it won't mess up this PR (I think). It certainly won't auto-affect this repo, so no worries!

The two code-wise changes I could see are:

  • Move to the idiomatic Result and Error names of error-chain rather than CargoFoo
  • Move to the idiomatic use util::errors::* rather than selective imports

More broadly though I've long wanted for a deeper revamp with how Cargo treats errors. My basic thinking is that for every possible error you get out of Cargo we know how it'll be printed, so you've got an amount of context that you'll print out and an amount of context that you'll hide. A good example of this is network errors which should never hide anything, whereas if rustc fails we shouldn't print out the rustc command that failed, just that the crate failed (and rustc's output tells you why).

On the other hand though we'll have bugs in Cargo where an error bubbles up that we didn't expect (e.g. an internal error today). In cases like this Cargo should sort of "ICE" but not quite as hard. I'm thinking that Cargo dumps all error context, a pretty severe message about how this error is a bug in Cargo, and maybe something like a data file in target with everything we can find (args, executable, version, fs layout, etc) and asks you to submit that as a bug report to Cargo itself. This way we'll have an avenue for getting bug reports about errors we should make better.

Finally I'm also thinking that all errors log all their context to the target directory if we hide anything. That way if you want to inspect more you can poke around and try to find more traces or something like that.

That's all pretty vague though and not super well thought out in my head. I'm sure others will have thoughts on this that will want to be voiced before any code actually gets written! Overall I've been not super satisfied with how Cargo's error handling story plays out, I fear that it hides too much by default which ends up being a pain :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants