eRFC: Add JSON backend to Rustdoc

[P1n3appl3]

Rendered

This RFC is an extension of one from 2018 started by @QuietMisdreavus. I've been working on an implementation with mentorship from @tmandry and @betamos and some of the necessary refactoring has already been done in coordination with @GuillaumeGomez.

I'm still working on the reference level docs (pardon the TODOs) but I'd love feedback on the proposed design and unresolved questions.

[tmandry]

I've been working with @P1n3appl3 on this and have to say that I've been very impressed with how much attention to detail they put into it. This is definitely one of the most well-documented RFCs I've seen, which of course is fitting for a rustdoc RFC :)

[anp]

What would it take to generate rustdoc's current HTML from a JSON IR? It could be a way to ensure that changes are applied uniformly.

[P1n3appl3]

If we wanted to use the JSON output as the "source of truth" and make the HTML renderer consume that, we'd just slap serde derives on the clean types so that we could round trip them rather than creating another set of types. The issue then is that you probably wouldn't want to make that JSON a public interface since it would have to expose some compiler internal types that the HTML renderer happens to use, and it would still be completely unstable.

I think it's worth keeping the JSON types as a separate representation because they have a different purpose than the clean types. Since they're supposed to be a relatively stable public interface, they don't need to be updated to add support every time an unstable feature is added to the compiler. It also allows us to drop some information for the sake of making the JSON easier to handle.

[jyn514]

I was going to comment about being able to find that out from the items listed, but I realized you can't - if all the items are public, then this tells you that there were no private items, as opposed to hiding the private items.

No change necessary, just reasoning it out :)

[jyn514]

How will this work for generics in qualified paths (std::vec::Vec<T>)? Or for associated items (std::vec::Vec::into_iter, <std::vec::Vec as IntoIterator>::IntoIter? Or opaque types (fn f() -> impl Iterator)?

More things that could break this at QPath.

[P1n3appl3]

These paths are only to the actual items corresponding to IDs, so if you saw Vec<T> in some function signature it'd give you the ID for Vec and when you look it up in paths you'd get the ItemSummary with path = ["std", "vec", "Vec"]. The info about the generic params would be contained at the use site (in the Type representation for Vec<T>). If it's a local type then you can look it up in index to get more info about what generic params it takes and their bounds, but for something like Vec you don't get that info in your crates json blob. You'd have to generate the json for std and go find it in there yourself (probably by using the path from ItemSummary).

For reference here's what you get in the example above (with most fields removed for brevity)

pub fn f<T>(arg: Vec<T>) { ... }

{
  "name": "f",
  "kind": "function",
  "inner": {
    "decl": {
      "inputs": [
        [
          "arg",
          {
            "kind": "resolved_path",
            "inner": {
              "name": "Vec",
              "id": "5:4236",
              "args": {
                "angle_bracketed": {
                  "args": [ { "type": { "kind": "generic", "inner": "T" } } ],
                  "bindings": []
                }
              },
              "param_names": []
            }
          }
        ]
      ],
      "output": null
    },
    "generics": {
      "params": [
        { "name": "T", "kind": { "type": { "bounds": [] } } }
      ],
      "where_predicates": []
    }
  }
}

[jyn514]

Ok, that makes sense. How would you handle <T as Trait>::AssociatedItem though? Normalize the type before generating the path?

[P1n3appl3]

In that case the "kind" for the type would be "qualified_path" which has the necessary info.

[jyn514]

What would this be useful for?

[P1n3appl3]

Even though we don't output the source files themselves like the HTML backend, an alternative frontend could get the source and use these to link to it.

[jyn514]

This seems like a weird thing to expose in the public API (and is also very underspecified). What's the rationale for including it?

[P1n3appl3]

Similar to constants, you might want to show the value a static is assigned to:

pub static controller: Controller = Controller::new("some publicly useful arg", 42);

[jyn514]

This seems like it's mostly for pretty printing, so I think it should be marked as unstable and subject to change. Rustc has no stability guarantee on -Z unpretty=* anywhere else, I don't think we should add it here.

[jyn514]

What about macros 2.0?

[P1n3appl3]

Forgot they were a thing. Due to my unfamiliarity with those and proc macros I've left them out for now.

[jyn514]

This is not sufficient to describe function signatures in Rust - you could have arbitrary patterns.

fn f((_, x): (usize, usize)) { ... }
fn g(MyStruct { field1: usize, .. }) { ... }

[P1n3appl3]

Woah didn't know you could destructure stuff like that in function args. I think it's still fine though, patterns just get stringified like names so you end up with ["(_, x)", <Type representation of tuple of usizes>] and ["MyStruct { field1: usize, .. }", <Type representation of MyStruct>] for those examples. It would be a problem if the consumer of the json wanted to know the names x and field1 because they'd have to parse arbitrary patterns, but I can't think of a case where they need that info off the top of my head.

[jyn514]

I'm ok with having that in the output, but it should be marked as unstable and subject to change.

[tmandry]

+1 to marking the format of the value as unstable and subject to change, but I think we could stabilize its existence and that it's a reasonable thing to put in the rendered docs. (Here and elsewhere)

[jyn514]

Yeah, that seems reasonable. "You can display this to the end-user but you can't depend on parsing it".

[jyn514]

It is not possible in general to resolve intra-doc links with the format specified in this RFC. That requires knowing which paths are in scope where the item is documented, not all of which are documented or public.

[P1n3appl3]

I didn't think about the fact that being able to write unqualified paths means that scoping info is required. Maybe it would be possible to resolve them to their fully qualified module paths and modify the markdown to output those ("docstring referencing [Iterator]" -> "docstring referencing [Iterator](trait@foo::Iterator)" or something)? Alternatively we could have another field with the list of IDs or fully qualified paths corresponding to each link in the markdown block in order.

[jyn514]

trait@foo::Iterator

This is still ambiguous, because you could have multiple crates with the same name, e.g. tokio 0.1 and tokio 0.2.

[jyn514]

Maybe an alternative is to have intra-doc links as a separate output field? Something like

{
    "intra-doc-links": {
        "struct@foo::Iterator": 0,
   }
}

where the value is the ID. That assumes the linked item is part of the output though, which isn't always true for cross-crate re-exports.

[P1n3appl3]

In that case how about we include a list of IDs corresponding to the resolved items from the intradoc links in the docstring. That way the consumer can use those to look up the items in path or index and generate links to them using their crate_num which solves the versioning issue.

[jyn514]

Another general thing I want to mention besides nitpicks: A lot of the format here seems to be based around clean::Item. However I'm not particularly happy with clean; @GuillaumeGomez and I have discussed refactoring rustdoc to use queries directly instead, because it avoids overhead and caches more effectively. Are we sure that all these fields make sense even without clean::Item?

[P1n3appl3]

Yeah the representation was chosen both because it was easy to implement with a bunch of From impls on the clean types and because clean seemed pretty close to a usable public interface. I think even if the clean types didn't exist and we built up the json manually with queries it would end up looking pretty similar.

Are there specific things about clean::Item that you don't like? The main concern I had with it is that it's kinda polymorphic with ItemEnum and I thought it might be simpler for a module to just have separate lists with all it's structs/enums/functions/etc. But having a big flattened collection of every item seemed useful enough that it's worth representing them that way so you can look them up easily.

[jyn514]

Are there specific things about clean::Item that you don't like?

Yes, but they're not really relevant to this RFC. clean::Item shouldn't have to exist, rustdoc should use queries and compute the information on-demand for when it needs it. I don't have any particular changes to Item data structure that I'd suggest.

[jyn514]

What happens if there are broken links? For the following code, I think rustdoc will internally store [<Def Id of f>], so you'd have no way to know whether that should belong to f or to MissingItem.

/// Link to [`MissingItem`], [`f`]
pub fn f() {}

I think a map from the name to the id is more resilient. Alternatively we could store null to mean an unresolved link, but that requires the consumer of the output to do some more work on their end.

[P1n3appl3]

Yeah a mapping does sound better, i was worried at first that we couldn't use a map because of duplicate keys but that isn't an issue because there's no way to have the same name resolve to different stuff.

[jyn514]

Hmm now that you say that I'm not actually sure it's true ... consider:

pub struct S;
/// Link to [S]
pub fn f() {}
pub mod inner {
    pub struct S;
    /// Link to [S]
    pub fn g();
}

Maybe we should include the ID of the item being documented as one of the keys?

[P1n3appl3]

I think we're still fine, this mapping is just for a single item's docstring so in that context it could only refer to one of them.

[jyn514]

Should we explain what inherent means?

pub struct S;
impl S { ... }

[P1n3appl3]

yeah i definitely had to look that up when i read the code.

[jyn514]

Suggested change

	`synthetic` | bool | Whether this is an impl that's implied by the compiler (for autotraits).
	`synthetic` | bool | Whether this is an impl that's implied by the compiler (for autotraits, e.g. `Send` or `Sync`).

[jyn514]

Suggested change

	`attrs` | [String] | The attributes (other than doc comments) on the Item, rendered as strings (e.g. `["#[inline]", "#[test]"]`).
	`attrs` | [String] | The [unstable](#Unstable) stringified attributes (other than doc comments) on the Item (e.g. `["#[inline]", "#[test]"]`).

[matklad]

I suggest adding format version to the JSON output, and to the command line flag, requesting the output, similar to how cargo metadata does this. The data model here is complex, so we might want to tweak it at some point. Given that this is all for tools consumption, specifying the version doesn't seem to onerous.

This might be overly cautious, however -- with cargo metadata, we haven't used the ability to break the format yet, and just piled "better" fields on top (ie, there are dependencies and deps).

[jyn514]

This might be overly cautious

The worst that happens is that the version is permanently set to 1 🤷 I think having a format version is a great idea.

[Darksonn]

This seems like it would be great for detecting breaking changes in crates. One thing I would find incredibly useful, which it doesn't seem like the JSON output currently exposes, would be when the anonymous return type of an impl Trait or async fn also implements Send and Sync. It would be amazing if I could get rid of this.

// Does the future returned by this function implement Send?
async fn my_async_fn() {
    ...
}
// Does the future returned by this function implement Send? What if T is not Send?
async fn my_async_fn<T>() -> T {
    ...
}

[jyn514]

One thing I would find incredibly useful, which it doesn't seem like the JSON output currently exposes, would be when the anonymous return type of an impl Trait or async fn also implements Send and Sync.

Type inference is mutually incompatible with ignoring type errors in function bodies: rust-lang/rust#75100. So this is not possible, or at least extremely difficult.

Is there a reason you can't return impl Trait + Send + Sync?

[Darksonn]

That's unfortunate.

Is there a reason you can't return impl Trait + Send + Sync?

We can't do that on an async fn without converting it to an -> impl Future. It also wouldn't detect accidentically adding Sync where it wasn't before, and requires adding Send + Sync bounds on generic parameters that are stored in the returned type (common for futures).

[tmandry]

One thing I would find incredibly useful, which it doesn't seem like the JSON output currently exposes, would be when the anonymous return type of an impl Trait or async fn also implements Send and Sync.

Hm, that does sound useful. Theoretically I think it's possible to do, but it sounds like the implementation would be difficult right now. I think this would be a useful thing to add in the future.

That does raise the question of what the process should be for extending the JSON output in non-breaking ways in the future. An FCP on the PR feels to me like the right amount of "weight" to give something like this. What do people on @rust-lang/rustdoc think?

[Manishearth]

That does raise the question of what the process should be for extending the JSON output in non-breaking ways in the future.

Honestly? I don't expect there to be many consumers of the JSON data. I'd like to not have it be stable, but be careful about changes (and perhaps version the JSON data). This might be a controversion proposal.

[jyn514]

If we keep it permanently unstable, what's the point of the RFC? Or are you suggesting we stabilize the existence of a JSON backend but not the content? That doesn't seem very useful ...

Theoretically I think it's possible to do, but it sounds like the implementation would be difficult right now.

Getting off-topic wrt the RFC, but I want to push back against this. Even the current half-broken implementation for ignoring type errors is straining the query system to its limits. Difficult is an understatement - you'd need to mangle typeck as badly as I mangled name resolution in rust-lang/rust#73566. It would be at least multi-month project.

[Manishearth]

If we keep it permanently unstable, what's the point of the RFC? Or are you suggesting we stabilize the existence of a JSON backend but not the content? That doesn't seem very useful ...

Yeah, we stabilize the existence of the backend but not the format. This can still be done in a useful way by effectively having a version field as metadata, and we can stabilize that.

I'm wary of stabilizing the format with no upgrade path because this might preclude us implementing some rustdoc features in the future.

[jyn514]

Yeah, we stabilize the existence of the backend but not the format. This can still be done in a useful way by effectively having a version field as metadata, and we can stabilize that.

Oh, I think I see. We stabilize the flag and say if this has a version: 1 field then we guarentee it has the contents described in this RFC, but we can change the version in any new release of rustdoc.

Yeah, that sounds good to me :) And maybe require an FCP for any new version of the JSON to make sure it's not changing needlessly.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[workingjubilee]

Non-objection, but thought it might be worth noting: It may be worth investigating JSON-LD and its suitability with respect to this goal, especially where this may or may not concern docs.rs and rust-lang's hosted standard library documentation.

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.

[Manishearth]

Merged, tracking issue at rust-lang/rust#76578 . Please help fill it in with steps!

[kornelski]

There's a way to use this data beyond documentation, to verify whether crate's public API adheres to semver compatibility:

https://github.com/Enselic/cargo-public-api

And since this is becoming the way to learn about crate's types, I wonder if it could be extended to also include information about the ABI (such as type sizes, alignment, field offsets). For libraries semver-compat of their ABI may be as important as API.

https://internals.rust-lang.org/t/for-a-library-author-how-maintenance-of-a-stable-abi-would-look-like/16769/3?u=kornel

[jyn514]

ABI is inherently unstable and changes with new versions of the rust compiler. I don't think that information belongs in rustdoc. If someone wants to make a custom rustc_driver that emits ABI info, that seems fine.

[kornelski]

I realize that's a scope creep from perspective of rustdoc, but from perspective of a general-purpose compiler-info-exporting tool (that this is close to becoming), it's not a big stretch.

#[repr(C)] ABIs are stable already, and Rust will eventually get its own stable ABI too.

[GuillaumeGomez]

Also you can't rely on rustdo for that because it only provides the items which match the provided cfg. So if you have a crate providing implementations or different types depending on the system, unless you generate for all systems, you will not be able to really check if there is a semver change.

[kornelski]

Of course, that's an inherent limitation of the language. The combination of cfg's can be arbitrarily complex and mutually-exclusive, so I don't think it's even possible to get information for all combinations at once. But it still makes sense to gather it in multiple passes for a set of supported platforms.

[Manishearth]

(I do think there is a potential extension to cfg where you can say what kind of CFG something is: swapping out impls, making APIs available on certain CFGs, or something else)

(I also think we may need a holistic way to start talking about "public api" for both docs and diagnostics; e.g. there's a case to be made for doc(hidden) being respected by diagnostics; but there's also a case to be made that we really need a new holistic feature for that)

[GuillaumeGomez]

Of course, that's an inherent limitation of the language. The combination of cfg's can be arbitrarily complex and mutually-exclusive, so I don't think it's even possible to get information for all combinations at once. But it still makes sense to gather it in multiple passes for a set of supported platforms.

This is not the right approach I think. Also, this is a limitation which appears later on in the compilation stages and rustdoc relies on them (the later stages), meaning that the information you're looking for has already been stripped. Getting this information is complicated but not impossible, however, rustdoc is definitely not the right solution for it.

[LoveIsGrief]

Is there anything that could consume this JSON? Would rustdoc itself be able to convert the JSON to HTML?

[Enselic]

There are currently 12 packages on crates.io that consume rustdoc JSON: https://crates.io/crates/rustdoc-types/reverse_dependencies

And many more if you count dependents of dependents, private projects, projects that don't depend on rustdoc-types, etc.

[grantshandy]

@LoveIsGrief - Is there anything that could consume this JSON? Would rustdoc itself be able to convert the JSON to HTML?

AFAIK one of the main motivations would be alternate documentation viewers (e.g. terminal / nicely formatted LaTeX documents?). Personally, going out of the terminal to view documentation is a bit of a pain. Ideally I'd like a nice vim-keybinded TUI for quickly searching across a crate's documentation. I am thinking of creating one but I'm limited by this RFC.

[lorenzolewis]

At Tauri we're also keeping a close eye on this. We bridge a lot of our Rust APIs over to JS using FFI so they can be accessed on either the JavaScript side or the Rust side. Since that means we would have a lot of people reading both JavaScript and Rust documentation at the same time we're looking to build our own reference docs on the website to make it easier for people to switch between the two languages while still having a similar UI/UX.