Basic code intel: ideas for improvements

[chrismwendt]

Jump-to-definition can't disambiguate between symbols with the same name

Say you have 2 files that both define:

// a.go
type Foo int
// b.go
type Foo int

Jump-to-definition on a reference to Foo would show 2 results.

Ideas for improvements

• Filter by imported packages
• Use scope information from ctags and some rudimentary type inference (or even just look at the token to the left of the ., such as cli in cli.Main) to determine the enclosing class

See #3234

Find-references is inaccurate

User report

The most important thing IMHO is "Find references" should consider scopes of the objects, i.e. when using it on a local variable or a member, it shouldn't give results for all variables with the same name. Or at least order the results based on some heuristic, e.g. file path distance.

It includes references that appear in comments, but shouldn't. @KattMingMing

Ideas for improvements

• Boosting results by file path distance (from the user above). Done in sourcegraph/code-intel-extensions@27d3b89
• Filter out results from packages that do not import the package where the symbol is defined
• Filter out references if they appear in comments or strings

More discussion #3418

Elaborating on filtering by imports

We could do more computation in basic-code-intel in the browser. Here's a sketch:

• User clicks "Find references" on token Foo
• Internally, basic code intel does a j2d on Foo and discovers it is defined in pkg1/Foo.java
• Internally, basic code intel does a text search for Foo and finds lots of results
• For each search result, do another j2d and check if that is the same as the original

The main problem is "lots of results", which could be in the tens or hundreds. We could limit it to the top N, or prompt users to click a button to make basic code intel try harder.

We could do more indexing up front. Here's a sketch:

• Sourcegraph analyzes all of the files in all repositories and stores metadata about each identifier (name + definition location)
• User clicks "Find references" ... (same as above)
• Instead of doing a text search, it does a symbol search for references to that definition location

The main problem is compute and storage resources for all of these references.

Doesn't work well for dependencies

User report

Tooltips and jump-to-definition (and even basic type inference) don't seem to work for any symbols that are imported from other libraries (even Prelude).

Ideas for improvements

• Use import statements, package.jsons, and the respective package registry to find relevant repositories to search in for dependency symbols

Jump-to-definition and find-references don't work on local variables

Currently, basic-code-intel cannot provide jump-to-definition for anything but top-level variables/functions, and there are lots of false-positives because it's not scope-aware.

Proposal: use tree-sitter

Use tree-sitter to build a syntax tree with scope information for a given file in order to reduce false positives for references provide jump-to-definition within the given scope.

Domain-specific code intelligence

• Database table creation <-> column referenced in SELECT/INSERT/etc.
• HTTP router/handler <-> GET/POST requests to that endpoint

Custom hover docs

What I was asking for is structured documentation like this one (note the special syntax @see). It should be rendered like this (note the subsection “See Also” under section “Overview”).

From #4 (comment)

Currently, Sourcegraph RegexLexer.state is not a link:

[chrismwendt]

Basic code intel now ranks references by file path similarity sourcegraph/code-intel-extensions@27d3b89

[FranklinYu]

For section “Custom hover docs”, actually @see isn’t the only tag I was looking for. A full list is available here; if full support is too complicated, I would expect at least:

• @param
• @return
• @raise

As they are very often useful as tooltip. Note that all of them takes a type, which Sourcegraph should render as a link.

[aidaeology]

Closing this issue as it stands it is not actionable for us, team will discuss outstanding ideas and if still applicable we will create new separate scoped issues to track development.