Error reporting, major refactors

[oberien]

This got larger than expected. To make this easier I have refactored the Generator into an Iter module and to enforce the StackElement::Context (which previously needed to be put on the stack by the caller).

Even if we print diagnostics / errors, I try to recover from then by skipping over the events resulting in diagnostics and continuing with the next ones. No matter how many diagnostics got printed, I still try to produce a pdf.

There are three types of errors:

1. Internal logic errors: They just panic and unwind the program.
2. Non-recoverable expected errors (Fatal errors): Like when the latex output stream errors while writing to it. They are bubbled up to the main and printed there.
3. Diagnostics: We try to recover from them and continue generating.

For a testrun, execute cargo run -- examples/errors.md.

Fix #21

Example of an ICE output (this particular one should probably be fixed :) ):

- bench.md:214208:1
       |
214208 | ![Representation example](img/representation-overview.png)
       | -- defined here
       |
error: cause: No such file or directory (os error 2)

error: internal compiler error: error converting url to path
note: please report this
note: in heradoc file None name Some(backtrace::backtrace::trace_unsynchronized::h4edc62c197f1da24) line None address Some(0x55682b1aca6d)
note: in heradoc file None name Some(heradoc::diagnostics::Diagnostics::bug::h93d19dd5adb6e56f) line None address Some(0x55682b1ea7e5)
note: in heradoc file None name Some(heradoc::resolve::Resolver::resolve::h9d40ca2a9539900f) line None address Some(0x55682b1b5718)
note: in heradoc file None name Some(heradoc::generator::iter::Iter::next::h7b763bfe69fe5de2) line None address Some(0x55682b24ff81)
note: in heradoc file None name Some(heradoc::generator::iter::Iter::peek::ha0e91824631af9c0) line None address Some(0x55682b25477a)
note: in heradoc file None name Some(heradoc::generator::Generator<B,W>::generate_body::ha9a7b0303c2dce06) line None address Some(0x55682b23c948)
note: in heradoc file None name Some(heradoc::generator::Generator<B,W>::generate::h6c7617fc6ab18829) line None address Some(0x55682b233be5)
note: in heradoc file None name Some(heradoc::backend::generate::ha0f89092eff9bc78) line None address Some(0x55682b1f41e4)
note: in heradoc file None name Some(heradoc::main::h5c534d9e94344157) line None address Some(0x55682b1b8140)
note: in heradoc file None name Some(std::rt::lang_start::{{closure}}::h95a941aaf9dbbd15) line None address Some(0x55682b1a4c02)
note: in heradoc file Some("src/libstd/rt.rs") name Some("{{closure}}") line Some(49) address Some(0x55682b54bd92)
note: in heradoc file Some("src/libstd/panicking.rs") name Some("do_call<closure,i32>") line Some(293) address Some(0x55682b54bd92)
note: in heradoc file Some("src/libpanic_unwind/lib.rs") name Some("__rust_maybe_catch_panic") line Some(87) address Some(0x55682b555be9)
note: in heradoc file Some("src/libstd/panicking.rs") name Some("try<i32,closure>") line Some(272) address Some(0x55682b54c95c)
note: in heradoc file Some("src/libstd/panic.rs") name Some("catch_unwind<closure,i32>") line Some(388) address Some(0x55682b54c95c)
note: in heradoc file Some("src/libstd/rt.rs") name Some("lang_start_internal") line Some(48) address Some(0x55682b54c95c)
note: in heradoc file None name Some("main") line None address Some(0x55682b1bc281)
note: in heradoc file None name Some("__libc_start_main") line None address Some(0x7f17ad25b222)
note: in heradoc file None name Some("_start") line None address Some(0x55682b18719d)
note: in heradoc file None name None line None address None

[HeroicKatora]

Sweet. I'll try to get to it, as a first impression there are some opportunities for some cleanup. For example extracting Range<usize> into a newtype wrapper to make the import cleaner, add helper methods for interfacing with diagnostics()..with_info_section(..) and possibly adding additional information to the source location (intermediate files?) without requiring more rewrites.

[HeroicKatora]

@oberien Also please push your changes to codespan-reporting, cargo can't seem to find the respective commits (or are they https://github.com/oberien/codespan/commit/dc7b13739165fc58721415ce3120d05cf117ca81 after a rebase&force-push from upstream?).

[oberien]

cargo can't seem to find the respective commits

Oops, after brendanzab/codespan#55 was merged, I rebased brendanzab/codespan#56 onto the new master, but forgot to update the revision hash in Cargo.toml. Should be fixed now.

there are some opportunities for some cleanup

Oh absolutely, most likely. I wrote this in 3 days, and there's probably some stuff which I just hacked in without properly integrating it.

extracting Range<usize> into a newtype wrapper to make the import cleaner

I'm not sure if that actually simplifies a lot of stuff, because especially in the frontend / cskvp we need to perform a lot of range calculations. I don't think that a newtype would help here, in fact it might even lower the user experience (with us being the user).
There is the codespan::Span, but I decided against it, because it's a hassle to work with. That's why I only convert to it last second in the Diagnostics.
I do see some advantages of redefining Range<usize> as SourceRange, which we can make Copy (and not implement Iterator on). That might clean somes tuff up (especially all the clones throughout the code).

add helper methods for interfacing with diagnostics()..with_info_section(..)

I'm not sure I'm following. I find the diagnostics API pretty nice actually :) (also having rewritten it thrice to better fit the needs)

adding additional information to the source location

Again I'm not sure I'm following.

Anyways, I'm going to wait on your review, that might clear things up :)

[HeroicKatora]

Forgot the major comment on the PR: I really, really like having diagnostics!

[HeroicKatora]

Makes sure the tests work neatly

[oberien]

I think I fixed all your requests, so this PR should be ready for a second round of reviews. Sorry that this became so huge, we should definitely think about #80 in the long run to keep these kinds of PRs smaller. Also I don't really know how I would split up this PR into smaller ones.

[HeroicKatora]

I'm no longer sure if emitting diagnostics immediately into its own defined writer is optimal. If you run the tests from the resolve-error branch you currently get output printed even if successful because the stream opened in Diagnostics does not respect the redirection performed by the test harness and some tests have diagnostics even if they succeed. Maybe it would be better to buffer them into an intermediate and/or don't tie the construction of the output stream into the Diagnostics constructors.

And sorry that this appears so late in the progress and you literally changed hundreds of lines concerning exactly this :/

[HeroicKatora]

Alternatively, we could change the owner semantics of the current DiagnosticsBuilder aggregator and pass it by mutable reference in these cases, then delete the events before they would be printed or simply forget it since they only get printed by an explicit call to emit.

[oberien]

I'll take a look into it after some sleep :) If we can improve it, I think it's worth it to investigate other possibilities. I was wondering as well why it printed out the diagnostics, considering that the rustc test framework captures it. I think that StandardStream::stderr will open a completely new stderr stream instead of using filedescriptor 2. That maybe creates a new filedescriptor which rustc can't capture. I find this weird as I was expecting it to just wrap std::io::Stderr.

One idea I also had in the beginning was to collect all diagnostics in the Diagnostics and print them explicitly / on drop. I figured out that it was way easier to just directly print them.

Maybe we can just use a termcolor::NoColor<std::io::Stderr> or `termcolor::Ansistd::io::Stderr for tests?

[HeroicKatora]

Maybe we can just use a termcolor::NoColorstd::io::Stderr or `termcolor::Ansistd::io::Stderr for tests?

Something like that could also work, simply provide another constructor for tests. This is also more of a code-cleanup than a blocking concern for the PR as normal program experience is not itself affected for now. Should we ever get a case where we error but later in processing can completely 'unerror' or 'unwarn' then this becomes more important. But for now, just fixing it for tests is likely fine.

[oberien]

Actually it seems like termcolor::StandardStream internally uses std::io::Stderr: https://docs.rs/termcolor/1.0.4/src/termcolor/lib.rs.html#224-229

I have no idea why it shows up in the tests.

[oberien]

Actually it seems like cargo test only captures stdout and not stderr. So I guess we could just use StandardStream::stdout for unit tests.

[oberien]

Apparently cargo test only captures stuff printed through println and eprintln?! Because even when I switch to StandardStream::stdout, it won't get captured. Will need more reading into, but this is pretty bad for cargo test…

[oberien]

I think we can move this discussion to after this PR has landed. For the time being it's fine and it can always be refactored. I've opened #81 to track the diagnostics issue.

[oberien]

Thanks for the reviews!