Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"doctests working group" for T-rustdoc? #43

Open
QuietMisdreavus opened this issue Jan 9, 2019 · 7 comments
Open

"doctests working group" for T-rustdoc? #43

QuietMisdreavus opened this issue Jan 9, 2019 · 7 comments

Comments

@QuietMisdreavus
Copy link
Collaborator

There's been a lot of activity in trying to improve doctests recently. Most notably, a couple contributors outside the Rustdoc Team have really stepped up to help out: @euclio and @rep-nop. It would be cool to coordinate improvements to doctests so we can plan out our changes and not step on each other's toes.

(I'll try to fill this thread with some ideas that i remember seeing or wanting, but full discussion may have to wait until the All-Hands on February 4-8 so i have dedicated time to think about it. In the meantime, feel free to pitch ideas here!)

@GuillaumeGomez
Copy link
Member

👍

@euclio
Copy link

euclio commented Jan 10, 2019

I'd like to remove the panic from doc test error handling. Currently, if you have a doc test that fails or panics, you get output with 2 panics in it:

running 1 test
test src/lib.rs - Foo (line 1) ... FAILED

failures:

---- src/lib.rs - Foo (line 1) stdout ----
thread 'src/lib.rs - Foo (line 1)' panicked at 'test executable failed:

thread 'main' panicked at 'explicit panic', src/lib.rs:3:1
note: Run with `RUST_BACKTRACE=1` for a backtrace.

', librustdoc/test.rs:367:17
note: Run with `RUST_BACKTRACE=1` for a backtrace.


failures:
    src/lib.rs - Foo (line 1)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

@QuietMisdreavus
Copy link
Collaborator Author

A couple large items that i can think of:

  • I'd like to rework how the code conversion happens. Right now, there are still a lot of text-based checks or conversions, and the entire process happens before any macros have a chance to run in the doctest. This means that some situations can still confound rustdoc (attributes spread across multiple lines, macros which create the main function on their own) and cause a doctest to fail when it actually should create a proper executable. I see two ways to fix this, but they won't be easy:
    • Create a new AST in libsyntax that allows expressions in module scope, so that we can parse "standard" doctests to an AST while still allowing doctest with complex macros to have a chance to expand them. This will likely require a contact on the compiler team to tell us how infeasible it is, or how would be the best way around it.
    • Run a kind of "two-phase" macro, to eagerly perform main-wrapping before macro expansion (as a proc-macro or equivalent), then to inspect the AST after macro expansion to see whether the expanded code inserted a main function of its own. This may be a little more cumbersome to reason about, but it's also a lot easier to implement, as it requires no extra work in libsyntax.
  • Once rustdoc: Add option to persist doc test executables rust-lang/rust#56189 is merged, we should start considering what is necessary to help people cross-compile doctests and run them in the target environment.
    • @cramertj has requested that we compile doctests into a single executable, like how rustc does for unit tests. This will require some new structural work to mimic the generated main function that libtest creates (or perhaps work to just use that code directly). I think it'll be less work than i originally thought, but it will still require some study and footwork to put it together, especially since the current model is to compile the test as part of its "execution", but this will require a sort of two-phase model, where compilation and execution are handled separately. (Granted, this is currently the model for unit tests, so it's not that much of a stretch, mentally.)
    • A couple of the items i mentioned in the roadmap thread also apply here, namely, the ability to save the source instead of/in addition to the executable, and possibly the ability to run tests with a "runner" script. These will help bridge the gap to being able to run doctests on target systems, but without the "single executable" feature, it may be more trouble than it's worth.

@cramertj
Copy link

cc @rtsuk

@cramertj
Copy link

This will require a sort of two-phase model, where compilation and execution are handled separately.

Yes, I also need this because I want to be able to cross-compile doctests and run them on a different platform from the host compiler.

@QuietMisdreavus
Copy link
Collaborator Author

Yes, I also need this because I want to be able to cross-compile doctests and run them on a different platform from the host compiler.

Right, and this is quite close to being possible once the --persist-doctests flag is ready, though it will compile each doctest separately. Creating the "single executable" model is another layer on top of that, where we'll have to separate the compilation and execution in the code itself, rather than just skipping execution if we cross-compile the doctest.

@repnop
Copy link

repnop commented Jan 14, 2019

Sounds pretty reasonable to me 👍

I'd be more than happy to tackle some of the remaining text-based stuff you mention since I've already done it once!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants