Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rustdoc: some fatal lexing errors are not buffered #56885

Closed
euclio opened this issue Dec 16, 2018 · 3 comments · Fixed by #68326
Closed

rustdoc: some fatal lexing errors are not buffered #56885

euclio opened this issue Dec 16, 2018 · 3 comments · Fixed by #68326
Labels
T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.

Comments

@euclio
Copy link
Contributor

euclio commented Dec 16, 2018 •
edited
Loading

Discovered while working on #56884.

For code like the following:

/// ```
/// "unterminated
/// ```
pub struct Foo;

rustdoc will fail to output any documentation for the struct and outputs a fatal error. I would expect the lexing error to be caught and emitted without stopping rendering.
@euclio euclio mentioned this issue Dec 16, 2018
Merged
@GuillaumeGomez GuillaumeGomez added the T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. label Dec 16, 2018
Centril added a commit to Centril/rust that referenced this issue Jan 15, 2019
@Centril
Rollup merge of rust-lang#56884 - euclio:codeblock-diagnostics, r=Qui…
6b04db4 
…etMisdreavus

rustdoc: overhaul code block lexing errors

Fixes rust-lang#53919.

This PR moves the reporting of code block lexing errors from rendering time to an early pass, so we can use the compiler's error reporting mechanisms. This dramatically improves the diagnostics in this situation: we now de-emphasize the lexing errors as a note under a warning that has a span and suggestion instead of just emitting errors at the top level.

Additionally, this PR generalizes the markdown -> source span calculation function, which should allow other rustdoc warnings to use better spans in the future.

Last, the PR makes sure that the code block is always emitted in the docs, even if it fails to highlight correctly.

Of note:
- The new pass unfortunately adds another pass over the docs to gather the doc blocks for syntax-checking. I wonder if this could be combined with the pass that looks for testable blocks? I'm not familiar with that code, so I don't know how feasible that is.
- `pulldown_cmark` doesn't make it easy to find the spans of the code blocks, so the code that calculates the spans is a little nasty. It works for all the test cases I threw at it, but I wouldn't be surprised if an edge case would break it. Should have a thorough review.
- This PR worsens the state of rust-lang#56885, since those certain fatal lexing errors are now emitted before docs get generated at all.
bors added a commit that referenced this issue Jan 20, 2019
@bors
Auto merge of #56884 - euclio:codeblock-diagnostics, r=QuietMisdreavus
846ea58 
rustdoc: overhaul code block lexing errors

Fixes #53919.

This PR moves the reporting of code block lexing errors from rendering time to an early pass, so we can use the compiler's error reporting mechanisms. This dramatically improves the diagnostics in this situation: we now de-emphasize the lexing errors as a note under a warning that has a span and suggestion instead of just emitting errors at the top level.

Additionally, this PR generalizes the markdown -> source span calculation function, which should allow other rustdoc warnings to use better spans in the future.

Last, the PR makes sure that the code block is always emitted in the docs, even if it fails to highlight correctly.

Of note:
- The new pass unfortunately adds another pass over the docs to gather the doc blocks for syntax-checking. I wonder if this could be combined with the pass that looks for testable blocks? I'm not familiar with that code, so I don't know how feasible that is.
- `pulldown_cmark` doesn't make it easy to find the spans of the code blocks, so the code that calculates the spans is a little nasty. It works for all the test cases I threw at it, but I wouldn't be surprised if an edge case would break it. Should have a thorough review.
- This PR worsens the state of #56885, since those certain fatal lexing errors are now emitted before docs get generated at all.
@GuillaumeGomez
Copy link
Member

Closing this issue since it has been fixed by #56884.

@euclio
Copy link
Contributor Author

euclio commented Mar 9, 2019 •
edited
Loading

This wasn't fixed by #56884. Current output:

$ rustdoc +nightly test.rs
error: unterminated double quote string
 --> <doctest>:1:1
  |
1 | "unterminated
  | ^^^^^^^^^^^^^^

This is a problem with the lexer.

@GuillaumeGomez GuillaumeGomez reopened this Mar 9, 2019
@GuillaumeGomez
Copy link
Member

Ah my bad. Reopening.

@euclio euclio mentioned this issue Apr 23, 2019
Closed
@euclio euclio mentioned this issue Aug 5, 2019
Closed
@ollie27 ollie27 mentioned this issue Jan 17, 2020
Merged
@bors bors closed this as completed in 0dc2557 Jan 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

rustdoc: Catch fatal errors when syntax highlighting ollie27/rust
2 participants
@euclio @GuillaumeGomez