LSP: Support document symbols

[PgBiel]

Fixes #3252

The main implementation is mostly done. All that is left is:

1. DONE: For selection_range, I imagine that the correct range would be one pointing to the identifier for each symbol (e.g. a range around Name in pub type Name); right now I just duplicated the full symbol's range. I will look into this soon (draft in the meantime).
2. Code style nitpicks might be in order. Feel free to ask for any changes in this regard.

It may also be desirable to write the full function signature alongside function symbols instead of just their types (at least, that's how rust-analyzer does it), but, in principle, the type has been working fine for an MVP. We could change this later if needed.

By the way, the #[allow(deprecated)] are because DocumentSymbol contains a deprecated field, but we have to specify it anyway to construct it.

Showcase

The source code below:

import gleam/option.{type Option}

pub type A

pub type B {
    C

    D(Option(Int))

    E(something: Option(Int))

    F(Int, something: Option(Int))
}

pub const my_const = 5

pub fn super_func(a: Int) -> Int {
    a + 5
}

produces the following outline on VSCode:

[PgBiel]

I've fixed symbol locations such that:

1. They now properly span the whole symbol, including a custom type's constructors or a function's body, so VSCode can now properly show the full path of symbols to your current cursor location.
   

2. The symbol's selection_range now only spans its name (whenever available, which should be always for top-level definitions). This is consistent with what the LSP standards expect, and also with what rust-analyzer does. This ensures clicking the symbol on the outline takes you to its name and not to pub fn or similar.

   • To do this, I had to add name_location / label_location / alias_location fields to a few symbols and properly populate them in the parser. Perhaps it could be more consistent to have a selection_location field common to those types, like rust-analyzer does, which is always present and is used to (later) instruct the LSP on what to do. But I think the current, less generalized impl will do for now.

P.S.: The clippy failure is unrelated to this PR.

[PgBiel]

Included doc comments as part of the symbol ranges as well now. Seems to match the LSP spec's recommendation, as well as what rust-analyzer does (for comparison).

[PgBiel]

By the way, let me know if I should have used ast::visit for any part of the implementation. In particular, I wanted to avoid introducing those more fine-grained location fields to the AST structures to be filled during parsing, but I couldn't think of a better way. I'm open to suggestions regarding any possible improvements here.

[lpil]

Wonderful! Thank you! I've left some notes inline

[lpil]

Lovely work! I've left a few notes inline, and we'll need tests for the document symbols part.

[PgBiel]

I'll look into adding some tests soon :)

[lpil]

Lovely work!! Could you add some tests and also add that comment to each of the #[allow attributes please 🙏

[PgBiel]

Sure, I'll do it soon!
Sorry, had run out of time 😅
But I definitely want to move this PR across the finish line :)

[PgBiel]

Some initial notes from tests:

1. Initial /// from first line of doc comments isn't being included in symbol spans; (fixed) - now "CommentDoc" tokens have a marker_start field indicating where their /// started, which we use for doc position now
2. pub const isn't being included in the location (it seems to be generally expected that the constant's location field corresponds to its name) (fixed) - now constructor.location includes everything from pub const to right before the equals;
3. Constructor over multiple lines is not spanning the end parens, but rather stopping at the start of the line containing it (fixed) - now we just use constructor.location for the constructor's location;
4. ⚠️ Constructor selection range seems to be across the whole constructor instead of just its name (this causes VSCode errors); (fixed) - after rebasing to main, the meaning of constructor.location changed to mean the whole constructor, having to switch my previous code to name_location for the name
5. Spans for constructor args aren't including docs (fixed) - for some reason analyse was removing doc info from all constructor args, so fixed that

Other stuff before undrafting:

• Add some more doc comments
• Perhaps refactor existing _location fields for consistency Seems unnecessary, at least for this PR
• Check it all again

[PgBiel]

@lpil There we go, had to make several changes when rebasing on main and upon adding tests, but I think it should be fine now. Feel free to take a look :)

[lpil]

Thank you. I've left some more notes inline.

I think something has gone awry with the rebase, there's conflicts which mean I cannot rebase this branch into main.

[PgBiel]

I think something has gone awry with the rebase, there's conflicts which mean I cannot rebase this branch into main.

That's probably due to merge commits; you'll have to squash and merge instead. (I tried to manually squash locally, but it's a bit difficult at this point :p)

[PgBiel]

There we go!

The CI is failing as it seems you've added an explicit check against merge commits, but other than that it's fine. I believe I owe a bit of an explanation: The reason I've used merge commits to keep the branch up-to-date with Gleam's main is that it was much simpler (for me, as a contributor): for each update, I only had to resolve the conflicts considering the latest version of the PR; I didn't have to update each and every commit since the PR was created (in a traditional rebase), saving a lot of time (as I had to update the branch multiple times). Then, by squashing and merging at the end (at least up to the latest merge commit, but just pressing "Squash and merge" is of course easier), no merge commits will make it to Gleam's main branch, avoiding any conflicts and applying the patch in the same way. (This is the policy we generally follow at Typst, for instance.)

With that said, it may of course be preferable to be able to rebase the PR to main and keep the commit history depending on your preferred policies, though in principle I don't see problems with either model, so long as the contributor can deliver their changes.

Regarding this PR, you should be able to squash and merge as needed, given the above. Let me know if you face any problems or have any concerns.

[lpil]

Thank you!!!!