rustdoc: add header map to the table of contents

[notriddle]

Summary

Add header sections to the sidebar TOC.

Preview

• http://notriddle.com/rustdoc-html-demo-9/toc/rust/std/index.html
• http://notriddle.com/rustdoc-html-demo-9/toc/rust-derive-builder/derive_builder/index.html

Motivation

Some pages are very wordy, like these.

crate	word count
std::option	2,138
derive_builder	2,403
tracing	3,912
regex	8,412

This kind of very long document is more navigable with a table of contents, like Wikipedia's or the one GitHub recently added for READMEs.

In fact, the use case is so compelling, that it's been requested multiple times and implemented in an extension:

• rustdoc table of content #80858
• Add a toc to each std lib generated pages #28056
• rustdoc: provide summary views #14475
• https://rust.extension.sh/#show-table-of-content

(Some of these issues ask for more than this, so don’t close them.)

It's also been implemented by hand in some crates, because the author really thought it was needed. Protip: for a more exhaustive list, run site:docs.rs table of contents, though some of them are false positives.

• https://docs.rs/figment/0.10.14/figment/index.html#table-of-contents
• https://docs.rs/csv/1.3.0/csv/tutorial/index.html#table-of-contents
• https://docs.rs/axum/0.7.4/axum/response/index.html#table-of-contents
• https://docs.rs/regex-automata/0.4.5/regex_automata/index.html#table-of-contents

Unfortunately for these hand-built ToCs, because they're just part of the docs, there's no consistent way to turn them off if the reader doesn't want them. It's also more complicated to ensure they stay in sync with the docs they're supposed to describe, and they don't stay with you when you scroll like Wikipedia's does now.

Guide-level explanation

When writing docs for a top-level item, the first and second level of headers will be shown in an outline in the sidebar. In this context, "top level" means "not associated".

This means, if you're writing very long guides or explanations, and you want it to have a table of contents in the sidebar for its headings, the ideal place to attach it is usually the module or crate, because this page has fewer other things on it (and is the ideal place to describe "cross-cutting concerns" for its child items).

If you're reading documentation, and want to get rid of the table of contents, open the Settings panel and checkmark "Hide table of contents."

Reference-level explanation

Top-level items have an outline generated. This works for potentially-malformed header trees by pairing a header with the nearest header with a higher level. For example:

## A
# B
# C
## D
## E

A, B, and C are all siblings, and D and E are children of C.

Rustdoc only presents two layers of tree, but it tracks up to the full depth of 6 while preparing it.

That means that these two doc comment both generate the same outline:

/// # First
/// ## Second
struct One;
/// ## First
/// ### Second
struct Two;

Drawbacks

The biggest drawback is adding more stuff to the sidebar.

My crawl through docs.rs shows this to, surprisingly, be less of a problem than I thought. The manually-built tables of contents, and the pages with dozens of headers, usually seem to be modules or crates, not types (where extreme scrolling would become a problem, since they already have methods to deal with).

The best example of a type with many headers is vec::Vec, which still only has five headers, not dozens like axum::extract.

Rationale and alternatives

Why in the existing sidebar?

The method links and the top-doc header links have more in common with each other than either of them do with the "In [parent module]" links, and should go together.

Why limited to two levels?

The sidebar is pretty narrow, and I don't want too much space used by indentation. Making the sidebar wider, while it has some upsides, also takes up more space on middling-sized screens or tiled WMs.

Why not line wrap?

That behaves strangely when resizing.

Prior art

Doc generators that have TOC for headers

https://hexdocs.pm/phoenix/Phoenix.Controller.html is very close, in the sense that it also has header sections directly alongside functions and types.

Another example, referenced as part of the early sidebar discussion that added methods, Ruby will show a table of contents in the sidebar (for example, on the ARGF class). According to their changelog, they added it in 2013.

Haskell seems to mix text and functions even more freely than Elixir. For example, this Naming conventions is plain text, and is immediately followed by functions. And the Pandoc top level has items split up by function, rather than by kind. Their TOC matches exactly with the contents of the page.

Doc generators that don't have header TOC, but still have headers

Elm, interestingly enough, seems to have the same setup that Rust used to have: sibling navigation between modules, and no index within a single page. They keep Haskell's habit of named sections with machine-generated type signatures, though.

PHP, like elm, also has a right-hand sidebar with sibling navigation. However, PHP has a single page for a single method, unlike Rust's page for an entire "class." So even though these pages have headers, it's never more than ten at most. And when they have guides, those guides are also multi-page.

Unresolved questions

• Writing recommendations for anyone who wants to take advantage of this.
• Right now, it does not line wrap. That might be a bad idea: a lot of these are getting truncated.
• Split sidebars, which I tried implementing, are not required. The TOC can be turned off, if it's really a problem. Too complex.

Future possibilities

I would like to do a better job of distinguishing global navigation from local navigation. Rustdoc has a pretty reasonable information architecture, if only we did a better job of communicating it.

This PR aims, mostly, to help doc authors help their users by writing docs that can be more effectively skimmed. But it doesn't do anything to make it easier to tell the TOC and the Module Nav apart.

[rustbot]

r? @jsha

(rustbot has picked a reviewer for you, use r? to override)

[rustbot]

Some changes occurred in src/librustdoc/clean/types.rs

cc @camelid

Some changes occurred in GUI tests.

cc @GuillaumeGomez

Some changes occurred in HTML/CSS/JS.

cc @GuillaumeGomez, @jsha

[the8472]

In fact, the use case is so compelling, that it's been requested multiple times and implemented in an extension:

At least some of the requests (e.g. #14475, #40273) are about method signature summary tables, not the headings for the prose.

Additionally - as a desktop user who frequently fullscreens browser windows - I would prefer if such navigation aids were placed in the vast empty space on the right side rather on the very narrow left bar.

[notriddle]

At least some of the requests are about method signature summary tables, not the headings for the prose.

Some of the requests are a bit unclear about what they want. 80858 has a bunch of comments that talk about header outlines, but was closed as a duplicate of 14475.

Additionally - as a desktop user who frequently fullscreens browser windows - I would prefer if such navigation aids were placed in the vast empty space on the right side rather on the very narrow left bar.

I agree. I've tried a few different implementations of this, before deciding that the details were complicated enough to not block this on them. It's the same asymmetry that motivates poor mobile-first design: a small-screen layout on a big-screen is a lot better than a big-screen layout on a small-screen, so the three-column ("split sidebar") layout has to be in addition to the two-column layout, but can't replace it.

Is it a blocking concern for this PR? If not, I'll open another PR that implements it.

[the8472]

Regarding the issues. I want to avoid issues getting closed when this isn't the thing some of them are asking for. And also make it clear that what is being asked for is more complicated.

Is it a blocking concern for this PR?

No.

[notriddle]

Added a note to the PR

(Some of these issues ask for more than this, so don’t close them.)

[GuillaumeGomez]

My only issue with the current version is it's actually not obvious what I'm reading at the top of the sidebar (also I'm a bit sad because I really liked the version with a new sidebar).

It's also a bit strange to have the table of contents followed directly by section links like "Structs" as there is no real visual marker to indicate it's not the table of content anymore.

[notriddle]

My only issue with the current version is it's actually not obvious what I'm reading at the top of the sidebar (also I'm a bit sad because I really liked the version with a new sidebar).

Would an explicit label for the "Sections" be a good idea, like how Elixir does it?

Or Wikipedia, where the whole thing is the "Contents"?

I hesitated, originally, because it added another line of text, but maybe it's worth it?

It's also a bit strange to have the table of contents followed directly by section links like "Structs" as there is no real visual marker to indicate it's not the table of content anymore.

That's still the table of contents. They're programmatically generated, instead of coming from the Markdown text, but they're still part of the current page.

also I'm a bit sad because I really liked the version with a new sidebar

I agree. #120818 does that.

[GuillaumeGomez]

Would an explicit label for the "Sections" be a good idea, like how Elixir does it?

I think I prefer the non-wikipedia approach, but both look much better than the current situation!

That's still the table of contents. They're programmatically generated, instead of coming from the Markdown text, but they're still part of the current page.

Then maybe add a bit more of spacing to make it more obvious it's not the same thing?

I agree. #120818 does that.

👍

[notriddle]

Then maybe add a bit more of spacing to make it more obvious it's not the same thing?

Why is it important that they're not "the same thing?" If somebody saw the page for the first time, what kind of mistake would they make if we failed to get across to them the difference between the sections that list child items and the sections that contain hand-written docs?

[GuillaumeGomez]

My issue here is that it takes me (not sure about others, so might be irrelevant) much more time to find the methods/items list when I need to go through the TOC first. There is no visual marker to help there too.

[Folyd]

Good to know librustdoc support TOC in the doc page. Just a suggestion, why not put the TOC to the right? Rust Search Extension is an example :)
https://rust.extension.sh/#show-table-of-content