New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consider uploading RA LSIF data to sourcegraph, from bors builds. #101446
Comments
cc @rust-lang/rustdoc this would be a good alternative to rustdoc's go-to-definition feature |
If it can be integrated into rustdoc, why not. |
I think it’s supposed to replace rustdoc’s entire source code view, so we would basically want to add a way to disable rustdoc’s own source code pages, while having the “source” links point to Sourcegraph instead. |
I guess we would just keep the source code rendering for the code examples in the docs and generate sourcegraph instead of the source code pages indeed. Potential issues we need to check at this point (that I can think of) would be:
|
I would prefer if the rustdoc pages shipped in the rust-doc component don't have any external dependencies, like sourcegraph. The primary reason to use the rust-doc component is offline usage I think. Using sourcegraph is incompatible with this. Also we already have goto definition inside rustdoc source pages behind an unstable feature. I would prefer extending this over replacing it with sourcegraph. |
Oh, I thought we could generate source graph pages to be used locally. Did I misunderstood maybe? |
AFAIK sourcegraph is a hosted service to which you can upload analysis data. You can also self-host it, but it needs a full database (both postgresql and redis) and a webserver. See https://docs.sourcegraph.com/dev/setup/quickstart |
Oh ok. So definitely not a good lead for the rustdoc jump-to-def feature indeed. However it could be nice on docs.rs. |
This is really cool! I think one of the big questions for jump-to-definition support in rustdoc is "why can't why use rust-analyzer for some of this?" Part of the answer has been that rust-analyzer needs to run live, and we want rustdoc output to work on a "basic" web server that only knows how to serve files (or on file:/// URLs). SourceGraph uses Language Server Index Format (LSIF) to extract info from a language server and make it available in a static format. And there are rust-analyzer bindings for LSIF already. Indeed as @bjorn3 says, if we wanted to run SourceGraph itself, that requires a live backend. But I believe the output of LSIF is static, so we could generate LSIF data at doc build time, and either write our own JS to process it for navigation or see if SourceGraph's JS is appropriately licensed. One of the interesting design tidbits here: doing truly precise code navigation essentially requires compiling the code (including version resolution). GitHub's code navigation and Google Code Search don't do this so they can't be quite correct; the hosted version of SourceGraph has a clever workaround: since you're probably building each commit of your code in CI anyhow, it asks you to build the relevant indexes in CI while you're building your code, and push it to SourceGraph! At any rate, @GuillaumeGomez, SourceGraph at least deserves some discussion in the RFC. |
Yes I'll mention it too! |
cc @tjdevries |
To be clear, I believe I made this issue about better supporting Sourcegraph, as a 3rd party service. That is, if Rust users are already using Sourcegraph, or considering using it, having the LSIF data automatically provided to it would greatly increase the usefulness of that service. I don't see a strong connection to However, I'm not sure I would want to strongly push for that, and there may be other ways to handle the situation, such as browser extensions for custom URL rewrite rules. Rewrite rules should be pretty easy for And in general (replace all "https://docs.rs/$CRATE/$VERSION/src/$CRATE/$PATH.rs.html#$LINES"
⇓⇓⇓
"https://sourcegraph.com/crates/$CRATE@$VERSION/-/blob/src/$PATH.rs?L$LINES" And yes Perhaps all we need for EDIT: looks like you only get that level of control when you run your own instance: https://docs.sourcegraph.com/code_navigation/how-to/enable_auto_indexing (though maybe there are separate docs for auto-indexing the public |
Thanks for the ping @matklad :) this is a super interesting thread. I can try to write up my thoughts sometime in the next few weeks, but I'm mostly away from my computer at the moment since my wife and I just had a baby girl 😍 . I'll tag in @olafurpg in case he wants to chime in at all -- otherwise I'll respond later. Super excited about this! |
Sourcegraph supports what they call "Precise code navigation" using data from tools such as
rust-analyzer
, and there's even convenience tools like https://github.com/sourcegraph/lsif-rust-action for uploading such data from CI.I don't personally use sourcegraph yet, and have been relying a lot on the GitHub CodeSearch (still in private beta, I believe), but having a publicly available service with better source code understanding seems really good and I'll try to remember it for the future.
I was looking around and found that at least one of the
rust-lang
repos happens to have it:https://sourcegraph.com/github.com/rust-lang/rustc-demangle@2811a1ad6f7c8bead2ef3671e4fdc10de1553e96/-/blob/src/v0.rs?L65
You can see how well hovering and clicking around works, it's a lot of fun frankly.
For
rust-lang/rust
it would likely be harder than thatsourcegraph/lsif-rust-action
helper (at the very least customrust-analyzer
configuration might be needed), but it might still be doable enough that we could maybe try it and see what happens (as long as no "paid plan" is required, I don't know the full details on the sourcegraph side).cc @bjorn3 (who mentioned the LSIF support, which got me looking at sourcegraph)
cc @rust-lang/infra
The text was updated successfully, but these errors were encountered: