New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Language Server Implementation (LSP) #5522

Open
tjdevries opened this Issue Oct 22, 2016 · 46 comments

Comments

Projects
None yet
@tjdevries
Contributor

tjdevries commented Oct 22, 2016

@justinmk, I have been looking into trying to write some sort of shim/plugin/client (still not sure what the name should be) to provide (at least some amount of) integration between the language server protocol and Neovim.

I have read your discussion in Gitter a bit ago and it got me interested in trying to implement some part of it. However, before I got working on implementing it, I wanted to try and figure out if I was understanding the idea of the integration correctly.

I made this diagram to explain some of what I was thinking. Obviously there are a lot more scenarios to cover, but this is what I was thinking.

basic

Also, do you know what language this client should be written in? I was thinking one of the remote plugins. It appears that Microsoft is working with this (https://github.com/Microsoft/vscode-languageserver-node) for a lot of their stuff, and we might be able to use it as well. Do you know which of the neovim node libraries would work well with this?

Thanks.

@KillTheMule

This comment has been minimized.

Contributor

KillTheMule commented Oct 22, 2016

Awesome, this things seems to be getting traction: https://internals.rust-lang.org/t/introducing-rust-language-server-source-release/4209.

@tjdevries

This comment has been minimized.

Contributor

tjdevries commented Oct 22, 2016

Yeah, it looks like there are several implementations coming along quite nicely (https://github.com/Microsoft/language-server-protocol/wiki/Protocol-Implementations). I think this would be a pretty killer feature for neovim.

@justinmk

This comment has been minimized.

Member

justinmk commented Oct 22, 2016

The node implementation would be a good start. A lua implementation could be shipped with Neovim later.

@tjdevries

This comment has been minimized.

Contributor

tjdevries commented Oct 23, 2016

Here's where I'll be working on this: https://github.com/tjdevries/nvim-langserver-shim

I will try and make it so we can swap out the remote plugin later (so get a lua implementation when a SDK for the language protocol is available).

@justinmk justinmk added this to the todo milestone Oct 23, 2016

@euclio

This comment has been minimized.

Contributor

euclio commented Oct 24, 2016

cc #4982

@tbelaire

This comment has been minimized.

Contributor

tbelaire commented Oct 25, 2016

Just to note, I had started on a python client with the intent of then writing vim and neovim, and making sure I can support the Rust LSP.

@purpleP

This comment has been minimized.

purpleP commented Oct 30, 2016

Not sure that this is the right piece for it, but I have this idea for a while.

If there would be a thing that would take a language specification (BNF etc) and make a parser for that language (should be totally possible and there is YACC).

The parser can generate AST and it should be possible to create a DSL for linting rules, so that not only linter designers could write them but users too. Which would solve a problem where pylint (python linter program) would complain in wrong places and won't where it should have been.

This tool should have integration with editors similar to what Microsoft proposes.

This would give smarter completions that take types (possibly inferred for dynamically typed languages) into account.

Next thing is refactorings. Because refactoring is just a function from one AST to another it should be possible to create fairly simple DSL or service for that also. So language specific refactorings could be added easily.

This thing could also remove the need of ctags as it could store all information about various program components (like classes, variable names etc) in some database like sqlite, mongo etc) and where ctags gives you ability only to jump to tag that is for the word under cursor this thing could integrate with Unite plugin and show things like all classes or all variables or all strings etc (more complex queries).

@tjdevries

This comment has been minimized.

Contributor

tjdevries commented Oct 30, 2016

If I'm understanding correctly, that's pretty much a language server as defined by microsoft's protocol.

@justinmk

This comment has been minimized.

Member

justinmk commented Oct 30, 2016

If there would be a thing that would take a language specification (BNF etc) and make a parser for that language (should be totally possible and there is YACC).

Check out the scintillua project for a library of PEG grammars for various languages. I'd really like to see this used for syntax highlighting in Nvim.

Because refactoring is just a function from one AST to another it should be possible to create fairly simple DSL or service for that also.

Refactoring is not AST => AST. Useful refactoring depends heavily on semantic analysis (type system) or more sophisticated inspection.

This thing could also remove the need of ctags as it could store all information about various program components

Microsoft Language Server Protocol abstracts those things into a common interface (this could be the DSL you mentioned) that can be used by any front end. AFAIK it is not a one-size-fits-all solution for storing, indexing, querying.

@purpleP

This comment has been minimized.

purpleP commented Oct 31, 2016

@justinmk As I've said type checker will be also implemented in my proposed solution. And ANY code change is AST -> AST and so is refactoring. As to scintilla it only have lexer, right? Whereas I'm talking about parser.

@purpleP

This comment has been minimized.

purpleP commented Oct 31, 2016

@tjdevries I've read briefly about Language Server. It differs from my idea in two ways. First I'm talking about implementation and protocol, whereas MLS is only a protocol. Second, in my solution protocol differs a bit. For example MLS doesn't define any means of querying the code base. Only jump to definitions. But I could be wrong about it.

One of the reasons I'm talking about this is that my proposed solution could be compatible at least partially with MLS, so if neovim would support that it would be good.

@prabirshrestha

This comment has been minimized.

prabirshrestha commented Nov 6, 2016

I would also be very interested in how this goes.

Given that my work machine is windows and neovim hasn't been working well there 😞 I managed to write LSP client in pure vimscript that works asynchronously in both vim8 and neovim.

Currently it just the client part.

https://github.com/prabirshrestha/vim-lsp

@tjdevries

This comment has been minimized.

Contributor

tjdevries commented Nov 6, 2016

Oh this is nice. I will take a look at this later today. Maybe we can
combine our work somewhere!

On Sun, Nov 6, 2016, 4:22 AM Prabir Shrestha notifications@github.com
wrote:

I would also be very interested in how this goes.

Given that my work machine is windows and neovim hasn't been working well
there 😞 I managed to write LSP client in pure vimscript that works
asynchronously in both vim8 and neovim.

Currently it just the client part.

https://github.com/prabirshrestha/vim-lsp


You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
#5522 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AEQo0-Zwd_lQ8-KnCa6hZrCVfQZpSxWbks5q7Zw-gaJpZM4Kd6cE
.

@prabirshrestha

This comment has been minimized.

prabirshrestha commented Nov 7, 2016

I'm all for working together.

@tjdevries

This comment has been minimized.

Contributor

tjdevries commented Nov 17, 2016

I'll try and post a screen cast of some features that I've gotten working, but I think I've got:

all set to go (at least in a very basic form). So it's working quite well :D

I'm excited about the results.

@tjdevries

This comment has been minimized.

Contributor

tjdevries commented Nov 17, 2016

Big shoutout to @prabirshrestha for his work on vim-lsp which I modified a bit and included in the shim.

@mhartington

This comment has been minimized.

mhartington commented Nov 17, 2016

Very excited to see your progress @tjdevries. I've recreated something similar for typescript but using the python api

https://github.com/mhartington/deoplete-typescript/blob/master/rplugin/python3/nvim-typescript/__init__.py

@prabirshrestha

This comment has been minimized.

prabirshrestha commented Nov 17, 2016

Glad to see this coming out very well. @tjdevries awesome job.

@mhartington unfortunately typescript (tsserver) does not implement the language server protocol :( Microsoft/TypeScript#11274 There does seem to be an effort by sourcegraph team https://github.com/sourcegraph/javascript-typescript-langserver

It will be interesting to see these LSP implementation working inside vim. https://github.com/Microsoft/language-server-protocol/wiki/Protocol-Implementations

@tjdevries

This comment has been minimized.

Contributor

tjdevries commented Nov 17, 2016

I have confirmed it working on at least the sourcegraph go langserver and their python langserver. So that's pretty exciting.

Documentation will come out soon with some more ease of use type things :) I'll keep you all posted when that's ready.

@dumblob

This comment has been minimized.

dumblob commented Dec 5, 2016

How did you solve the issue with concurrent edition of the same buffer (e.g. split buffer - possibly among different clients)? The Microsoft Language Server protocol still does not support it (Microsoft/language-server-protocol#23 (comment) ).

@hauleth

This comment has been minimized.

hauleth commented Dec 16, 2016

@dumblob but where is the issue? Vim has things enabled mostly per buffer, not per window, so there should be no problem with anything when using LS.

@dumblob

This comment has been minimized.

dumblob commented Dec 16, 2016

@hauleth I'm sorry, but I don't understand what you mean. What are "things enabled per buffer"? How does it relate to e.g. split buffer edition among different clients?

@hauleth

This comment has been minimized.

hauleth commented Dec 16, 2016

If you meant different instances of nvim then it would be the same problem among all editors, so nothing nvim specific and IMHO not an issue as when you edit files in different places simultaneously, then there always be problems. On the other hand, if you were talking about different vim windows then this is not an issue as LS would be operating on buffers not windows and windows shares the same buffer.

Am I missing something?

@prabirshrestha

This comment has been minimized.

prabirshrestha commented Dec 29, 2016

Something like file change notification would be useful to have in vim and neovim to create better integration for language server. Started a discussion in vim_dev at https://groups.google.com/forum/#!topic/vim_dev/oubHkRbdQW0. Some more discussion about this at #1380 including a WIP PR at #1791

@prabirshrestha

This comment has been minimized.

prabirshrestha commented Dec 29, 2016

In case anyone is curious to see demos :)

Async goto defintion in typescript using vim-lsp. More info at Quramy/tsuquyomi#57 (comment) This works in both vim and neovim.

lsp-ts-gotodef

async completion in vim (Fork of completor.vim with python removed implemented in pure vim script. Currently doesn't work yet in neovim as I'm waiting for neovim to support lambdas. More info at tjdevries/nvim-langserver-shim#11) Wished async completion was just part of core like completfunc and omnifunc

async_completion

Currently simulating async completion with timers.

function! s:emoji_completor(args, done)
  call timer_start(1000, {timer->a:done([{'word': ':smile:'}, { 'word': ':+1:'}])})
  " for sync directly call a:done()
  " call a:done([{'word': 'smile:'}, { 'word': '+1:'}])
endfunction

call completor#providers#register({
      \ 'name': 'emoji',
      \ 'filetypes': ['javascript'],
      \ 'completor': function('s:emoji_completor'),
      \ 'trigger': completor#providers#trigger_from_chars([':'])
      \ })

2017 might finally be the year of async vim and vim as an IDE :)

@Shougo

This comment has been minimized.

Contributor

Shougo commented Jan 1, 2017

@prabirshrestha Why don't you use deoplete?

@prabirshrestha

This comment has been minimized.

prabirshrestha commented Jan 30, 2017

Landed up writing my own async completion in pure vim script that works both on vim8 and neovim which depends only on timers. It is a fork of https://github.com/roxma/nvim-complete-manager. Huge props goes to roxma for his work.

Here is the fork: https://github.com/prabirshrestha/asyncomplete.vim. Beside removing python dependencies and supporting vim 8 I also added some features in this fork. Here is the demo in both vim8 and neovim on windows. timers is the only requirement.
Buffer completion and Ultisnips completion works synchronously.
Go completion works asynchronously via async.vim which is used normalize the job apis in both vim 8 and neovim so there is no if has('nvim') checks at all in the completion source plugin.

Pretty happy with the progress as you can see that there is no lag in the UI. Since it is pure async the initial gocode completion is a bit slow but as the server warms up the completions starts to be very fast.

vim-asyncomplete

neovim-asyncomplete

Currently this is an experimental plugin as I have not locked down the apis yet. In long term I would like to also add the python features I removed from the original. The why section is a good read. https://github.com/roxma/nvim-complete-manager#why. The plan would be to allows the users to register custom core providers and provide vim and python implementation by default. (Core provider is the one that handles fuzzy matches. I want this to also be extensible so rather the writing all the algorithms in core, I would expect external people to write better ones and let the community decide which they favor). Vim users will be limited to vimscript for fuzzy matches including blocking UI while matching and sorting is taking place during autocompletion, while neovim users can enjoy the benefits of remote plugins and custom fuzzy matchers in python and more responsive UI. Ideally one could also write the code provider as an executable instead so even vim users can enjoy the speed. Once everyone moves to lsp protocol having multiple core providers might not be a problem since most of them would had been handled by the language server instead.
I plan to do something very similar in vim-lsp so that even the users have option to use the core parser in vim script or take advantage of the remote plugin that neovim provides.

The goal is to have consistency between vim and neovim and then optimize for each separately without breaking.

roxma pushed a commit to roxma/nvim-completion-manager that referenced this issue Feb 7, 2017

@roxma

This comment has been minimized.

roxma commented Feb 7, 2017

Hi,

I'm planning on adding built-in language server support into nvim-completion-manager, at least for code completion, with python implementation.

But I have not reached a decent design. so I've just made a working one from scratch, nvim-cm-php-language-server, just for demonstration.

@prabirshrestha

This comment has been minimized.

prabirshrestha commented Feb 7, 2017

@roxma That is awesome. Glad to see more people on boarding to the language server protocol.
Looking at the source it doesn't seem like it is following strict protocol. I was actually very curious how writing to stdin for didOpen and didChange would affect the perf. Seems like most of node client-server uses node IPC to communicate.

@autozimu

This comment has been minimized.

autozimu commented Feb 12, 2017

I made a neovim plugin to support Language Server Protocol at

https://github.com/autozimu/LanguageClient-neovim

It still have rough edges, but right now is usable. (Well, at at least for Rust Language Server, I haven't got time to cover other server implementations yet.)

Thanks and happy hacking!

@roxma

This comment has been minimized.

roxma commented Mar 4, 2017

Since LanguageClient-neovim already has nice interface, I'm deprecating my previous nvim-cm-php-language-server

Here's a simple php language server package that could be easily setup to work with LanguageClient-neovim, and it is well integrated with nvim-completion-manager

output

@Shougo

This comment has been minimized.

Contributor

Shougo commented Jun 7, 2017

The pull request is here.

#6856

@prabirshrestha

This comment has been minimized.

prabirshrestha commented Jul 19, 2017

Started reworking on my vim-lsp plugin again in dev branch. (still lot of refactoring to do and finalizing the public apis). prabirshrestha/vim-lsp#11

Currently these are supported.

  • :LspGetWorkspaceSymbols
  • :LspGetDocumentSymbols
  • You can send custom request and notifications to the server. (Registering server notifications not supported yet)
call lsp#send_request('server_name', {
    \ 'method': 'initialize',
    \ 'params:' {....},
    \ 'on_notification': {server_name, response->lsp#log(server_name, response)},
    \ })

call lsp#send_notification('server_name', {
    \ 'method': 'initialize',
    \ 'params:' {....},
    \ })

Calling these methods will automatically flush the buffers (i.e. wait for server start, initialize, didOpen and didChange. I had to actually go far as writing an async helper method in vim script to coordinate this https://github.com/prabirshrestha/vim-lsp/blob/31743d44463cd62358503f83d4a87fd54b818cdd/autoload/lsp/utils/step.vim)

function! s:ensure_flush(buf, server_name, cb) abort
    call lsp#utils#step#start([
        \ {s->s:ensure_start(a:buf, a:server_name, s.callback)},
        \ {s->s:is_step_error(s) ? s:throw_step_error(s) : s:ensure_init(a:buf, a:server_name, s.callback)},
        \ {s->s:is_step_error(s) ? s:throw_step_error(s) : s:ensure_open(a:buf, a:server_name, s.callback)},
        \ {s->s:is_step_error(s) ? s:throw_step_error(s) : s:ensure_changed(a:buf, a:server_name, s.callback)},
        \ {s->a:cb(s.result[0])}
        \ ])
endfunction

Here is a sample .vimrc on how to use it.

Plug 'prabirshrestha/vim-lsp', { 'branch': 'dev' }
Plug 'prabirshrestha/asyncomplete.vim'    " optional only required for autocompletion
Plug 'prabirshrestha/asyncomplete-lsp.vim' " when server is initialized and it supports completion, asyncomplete source is automatically registered. no need to call asyncomplete#register_source in .vimrc

au User lsp_setup call lsp#register_server({
     \ 'name': 'rust',
     \ 'cmd': {server_info->['rustup', 'run', 'nightly', 'rls']},
     \ 'whitelist': ['rust'],
     \ })

au User lsp_setup call lsp#register_server({
    \ 'name': 'ts',
    \ 'cmd': {server_info->has('win64') ? ['cmd', '/c', 'javascript-typescript-stdio']  : ['sh', '-c', 'javascript-typescript-stdio']},
    \ 'whitelist': ['typescript'],
    \ })

Are there any plans for supporting remote lua plugins in neovim? I would like to support both vimscript and remote lua plugin specially since some of these parsing are very expensive. Another option I have been thinking is to have rpc support in async.vim so that it works in both vim8 and neovim.

@justinmk

This comment has been minimized.

Member

justinmk commented Jul 19, 2017

are there any plans for supporting remote lua plugins in neovim?

Yes.

@justinmk justinmk added the lsp label Aug 5, 2017

@HerringtonDarkholme

This comment has been minimized.

HerringtonDarkholme commented Aug 31, 2017

The most difficult problem I encounter in vim Langauge client is completion.

LSP's completion item is very rich and expressive, way beyond Vim's builtin completion mechanism.

Detail: https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md#textDocument_completion

For example, a server can return a completion item as snippet. In VSCode selecting this item wil trigger snippet expansion.

Item can also have TextEdit field, selecting such item will apply text edition. For example, changing characters before completion position.

Some servers, like Java LS, will even uses AdditionalTextEdit for things like import library.

I wonder how far we can support LSP using current completion in Vim.

@roxma

This comment has been minimized.

roxma commented Aug 31, 2017

For example, a server can return a completion item as snippet. In VSCode selecting this item wil trigger snippet expansion.

I believe the snippet support is possible with existing plugins: deoplete+neosnippet+LanguageClient-neovim
or nvim-completion-manager+neosnippet/ultisnips/snipmate+LanguageClient-neovim

But I haven't seen a language server with this capabilities yet (I've tested php-language-server, go language server, rls, python). Anyone point me a direction so that I can test it on my laptop?

@HerringtonDarkholme

This comment has been minimized.

HerringtonDarkholme commented Aug 31, 2017

Hi @roxma, thanks for reply! You can try https://github.com/vuejs/vetur/tree/master/server.

Example test code

<template>
  <div cl|></div>
</template>

The | indicates cursor position. VLS will return a snippet which expands to

<template>
  <div class="|"></div>
</template>

Note the position of cursor. It's inside of quotation.

Edit: update code demo

@BonaBeavis

This comment has been minimized.

BonaBeavis commented May 4, 2018

Hi, I am experimenting with LanguageClient-neovim and java and I stumbled on an issue. The java lsp responses with its own schema jdt://... for following a dependency definitions: autozimu/LanguageClient-neovim#392.
Is there a handler interface in neovim which could be extended to handle this issue?

@matheuslessarodrigues

This comment has been minimized.

matheuslessarodrigues commented Jun 8, 2018

do any of these LSP plugins work with csharp? couldn't figure it out on how to make it work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment