LSP: user can easily discover, define client commands

[justinmk]

Problem

LSP servers have various custom capabilities and commands that are not part of the LSP spec, e.g.:

• "go to source definition" (typescript) #28328
• "update imports" #20784
• "open docs" (rust) #1921

The current situation with nvim-lspconfig is that 10+ lines of code are needed to define each command or custom capability. That's unsustainable, we can't maintain that for 100s of LSP servers.

See also:

• Commands interface nvim-lspconfig#1937
• workspace/didCreateFiles notification not sent to language server after a code action creates a file #26045 (comment)
• feat(lsp): add vim.lsp.config #18506

Expected behavior

Reduce friction for users to use custom (off-spec) server capabilities/commands:

1. Provide a way to discover/iterate through server capabilities custom (off-spec) commands.
2. Improve the commands interface and/or provide util functions so that defining commands/aliases to call custom (off-spec) server commands requires 2x less code.

Questions

• vim.lsp.get_active_clients()[1].server_capabilities and vim.lsp.get_active_clients()[1].commands already exist. What is the remaining friction? Why do we get requests like this? Do we simply need some hints in the documentation that guide users? Do we need a util function that makes it easy for users to define mappings around server commands/capabilities?
  • Answer: see LSP: user can easily discover, define client commands #28329 (comment)

Possible interface

From neovim/nvim-lspconfig#1937 (comment) (the names don't matter right now, they are just pseudo-code; the point is that we need a generic interface for (1) iterating custom shit from the LSP server and (2) enabling users to create their own mappings to this stuff):

some sort of "sub-command" entrypoint so that langserver-specific commands can be discovered. E.g.

• :Lsp <tab> would complete available langserver-specific commands.
• :LspCapabilties <tab> would show... something.

[mfussenegger]

We need a general mechanism that makes it easy for users and scripts (such as nvim-lspconfig) to:

1. iterate through discovered server capabilities / commands
2. define aliases to customer server commands and capabilities, in a non-verbose way

I think there's a bit of a misunderstanding here.

LSP defines two types of commands:

• server commands. These are advertised within server_capabilities.executeCommandProvider.commands and they (usually) work without any special client logic. The workflow is usually something like:

  • user triggers code actions
  • user selects code action to execute
  • client tells server to run the command
  • server sends workspaceEdit to the client

  We cannot provide a generic way for users to call these commands directly, because many of them require arguments or context (e.g given via the code actions). The arguments required are command specific

• client commands. The specification doesn't specify any concrete client commands, but it acknowledges that clients can implement custom client commands. These are not advertised by the server in server_capabilities. Typically the client informs the server that it supports them via settings or init_options in the initialize request. These always require some sort of custom client logic, and there's no way to have a general mechanism to use them.

The current situation with nvim-lspconfig is that 10+ lines of code are needed to define each command or custom capability. That's unsustainable, we can't maintain that for 100s of LSP servers.

Some of the client commands could become part of the specification. But for some of them there are issues with no clear path forward (E.g. microsoft/language-server-protocol#1641)

Other than that, I don't see what else neovim core could do. Other than delegating more of the work to language specific plugins.

[justinmk]

I think there's a bit of a misunderstanding here.

Yes, I'm just feeling around here. I updated the title + description.

LSP defines two types of commands ...

• server commands server_capabilities.executeCommandProvider.commands ... user triggers code actions.
  We cannot provide a generic way for users to call these commands directly

✅ Those are discovered and presented via "code actions" menu, correct? So that's already covered.

• client commands. ... These are not advertised by the server in server_capabilities. Typically the client informs the server that it supports them via settings or init_options in the initialize request.

Using rust-analyzer experimental/externalDocs as an example: is there any way to list all of its experimental/xx methods?

Are these goals tractable:

• reduce the amount of boilerplate needed to define commands, by at least 2x.
• reduce the amount of documentation needed to explain how users can create custom commands
  • provide concise (but fully-working) examples

Example

Code required to add custom rust-analyzer commands:

Before

code

local function reload_workspace(bufnr)
  bufnr = util.validate_bufnr(bufnr)
  local clients = vim.lsp.get_active_clients { name = 'rust_analyzer', bufnr = bufnr }
  for _, client in ipairs(clients) do
    vim.notify 'Reloading Cargo Workspace'
    client.request('rust-analyzer/reloadWorkspace', nil, function(err)
      if err then
        error(tostring(err))
      end
      vim.notify 'Cargo workspace reloaded'
    end, 0)
  end
end

local function open_docs(bufnr)
  bufnr = util.validate_bufnr(bufnr)
  vim.lsp.buf_request(bufnr, 'experimental/externalDocs', vim.lsp.util.make_position_params(), function(err, url)
    if err then
      error(tostring(err))
    else
      vim.fn['netrw#BrowseX'](url, 0)
    end
  end)
end

vim.lsp.start{
  ...,
  commands = {
    CargoReload = {
      function()
        reload_workspace(0)
      end,
      description = 'Reload current cargo workspace',
    },
    RustOpenDocs = {
      function()
        open_docs(0)
      end,
      description = 'Open documentation for the symbol under the cursor in default browser',
    },
  },
}

After

Assuming we provide a util function like:

vim.lsp.command(name:string, on_done:function)
-- Works like buf_request_all ?
vim.lsp.command_all(name:string, on_done:function)

The code would now look like:

TODO

[mfussenegger]

Are these goals tractable:

reduce the amount of boilerplate needed to define commands, by at least 2x.
reduce the amount of documentation needed to explain how users can create custom commands
provide concise (but fully-working) examples

We could document some examples, but I don't think you can reduce the amount of boilerplate by 2x.

Using rust-analyzer experimental/externalDocs as an example: is there any way to list all of its experimental/xx methods?

This isn't a client command. It's just a extra LSP method. There is no discovery mechanism for these. Just the documentation of the language server - or in many cases not even that, but reverse engineering of vscode extensions.

The main problems here are:

• You need to know the arguments required, and logic to create them.
• You need to know the response type
• You need custom logic to process the response.

This is all specific to the method.

And the open_docs example is also kinda broken, it should use client.request like the reload_workspace example. vim.lsp.buf_request would make a request using all clients attached to a buffer, which means other servers not supporting it would receive the request too, and very likely fail.

Client commands on the other hand can be used instead of server commands, and are part of some other operations like code-actions. The difference to server commands is that they usually have more round-trips.

For example nvim-jdtls defines a java.action.generateToStringPrompt command (using vim.lsp.commands, which is documented), and advertises it in the settings of the initiailize request. The workflow is then like this:

• User triggers code action
• Server responds with available code actions, including an additional "Generate toString" action
• User selects "Generate toString", this triggers the client command
• Client command sends custom request to server (java/checkToStringStatus), server responds with the properties that can be included in a toString implementation.
• Client shows a prompt to the user to select the properties to use
• Client sends another request to the server (java/generateToString), using the properties the user selected
• Server responds with a WorkspaceEdit, that the client applies

This is all completely custom

---

Also note that what lspconfig does (or did?) with commands was to provide sugar for neovim user commands definitions. That's not really specific to the LSP module at all.

[justinmk]

Here is how coc nvim exposes all the commands that the different extensions register

@asmodeus812 but how are those commands discovered? Where is that info coming from?

Using rust-analyzer experimental/externalDocs as an example: is there any way to list all of its experimental/xx methods?

This isn't a client command. It's just a extra LSP method. There is no discovery mechanism for these. Just the documentation of the language server - or in many cases not even that, but reverse engineering of vscode extensions.

That isn't part of the "capabilities" response? It's strange that servers can't list their custom methods.

The main problems here are:

• You need to know the arguments required, and logic to create them.
• You need to know the response type
• You need custom logic to process the response.

Understood. But even just listing the names, and point to the upstream docs, would be very helpful, and avoids support requests. Some UX tweaks may be enough.

nvim-lspconfig has a CI job that pulls package.json files to get various bits of info. But custom methods aren't declared in package.json files, they are just loosely documented as you mentioned.

[asmodeus812]

Here is how coc nvim exposes all the commands that the different extensions register.

@justinmk they are registered through/using the coc-lsp client's api, by each extension, i think that was obvious from my reply. This is also what vs code does, the commands are registered by the extension, but the api to interact/add/register/use those commands is provided by the vs code lsp client as a library itself.

e.g. https://github.com/neoclide/coc-java/blob/43921b1eceb450a25d62545f6f206fd827235752/src/sourceAction.ts#L12

[clason]

This is also what vs code does, the commands are registered.

We're not VS Code, though, on purpose; neither do we plan on replacing coc.nvim. We implement the LSP specification and nothing but the specification. Anything beyond that is intentionally left for server-specific plugins (which correspond to VS Code extensions) built on top of the base LSP API (as well as other methods that compose nicely, which this may or may not be relevant to).

[asmodeus812]

@clason in that case if nvim only adheres to the spec and only the spec then this issue can be closed as resolved / not planned.

[clason]

Not quite; the last part about "other methods that compose nicely" may still be actionable here. (We do want to add general API methods that makes it easier to write such custom plugins -- given a positive, individual, cost-benefit analysis, of course.)

[mfussenegger]

That isn't part of the "capabilities" response? It's strange that servers can't list their custom methods.

No, the capabilities only contain the capabilities defined in the specification and there is also no OpenAPI style dynamic introspection of methods.

We could suggest to add that, but even if it were added, due to the BWC nature of the protocol, it would remain optional.

Understood. But even just listing the names, and point to the upstream docs, would be very helpful, and avoids support requests. Some UX tweaks may be enough.

We could add a lsp-offspec-extensions doc section, that lists the two common scenarios for plugin devs, including examples and some prose on how to best implement them:

• Using off-spec methods
• Creating custom client commands