Discussion of Microsoft Language Server Implementation #5522

Open
tjdevries opened this Issue Oct 22, 2016 · 33 comments

Projects

None yet
@tjdevries
Contributor

@justinmk, I have been looking into trying to write some sort of shim/plugin/client (still not sure what the name should be) to provide (at least some amount of) integration between the language server protocol and Neovim.

I have read your discussion in Gitter a bit ago and it got me interested in trying to implement some part of it. However, before I got working on implementing it, I wanted to try and figure out if I was understanding the idea of the integration correctly.

I made this diagram to explain some of what I was thinking. Obviously there are a lot more scenarios to cover, but this is what I was thinking.

basic

Also, do you know what language this client should be written in? I was thinking one of the remote plugins. It appears that Microsoft is working with this (https://github.com/Microsoft/vscode-languageserver-node) for a lot of their stuff, and we might be able to use it as well. Do you know which of the neovim node libraries would work well with this?

Thanks.

@KillTheMule
Contributor

Awesome, this things seems to be getting traction: https://internals.rust-lang.org/t/introducing-rust-language-server-source-release/4209.

@tjdevries
Contributor

Yeah, it looks like there are several implementations coming along quite nicely (https://github.com/Microsoft/language-server-protocol/wiki/Protocol-Implementations). I think this would be a pretty killer feature for neovim.

@justinmk
Member

The node implementation would be a good start. A lua implementation could be shipped with Neovim later.

@tjdevries
Contributor

Here's where I'll be working on this: https://github.com/tjdevries/nvim-langserver-shim

I will try and make it so we can swap out the remote plugin later (so get a lua implementation when a SDK for the language protocol is available).

@justinmk justinmk added this to the todo milestone Oct 23, 2016
@euclio
euclio commented Oct 24, 2016

cc #4982

@tbelaire
Contributor

Just to note, I had started on a python client with the intent of then writing vim and neovim, and making sure I can support the Rust LSP.

@purpleP
purpleP commented Oct 30, 2016 edited

Not sure that this is the right piece for it, but I have this idea for a while.

If there would be a thing that would take a language specification (BNF etc) and make a parser for that language (should be totally possible and there is YACC).

The parser can generate AST and it should be possible to create a DSL for linting rules, so that not only linter designers could write them but users too. Which would solve a problem where pylint (python linter program) would complain in wrong places and won't where it should have been.

This tool should have integration with editors similar to what Microsoft proposes.

This would give smarter completions that take types (possibly inferred for dynamically typed languages) into account.

Next thing is refactorings. Because refactoring is just a function from one AST to another it should be possible to create fairly simple DSL or service for that also. So language specific refactorings could be added easily.

This thing could also remove the need of ctags as it could store all information about various program components (like classes, variable names etc) in some database like sqlite, mongo etc) and where ctags gives you ability only to jump to tag that is for the word under cursor this thing could integrate with Unite plugin and show things like all classes or all variables or all strings etc (more complex queries).

@tjdevries
Contributor

If I'm understanding correctly, that's pretty much a language server as defined by microsoft's protocol.

@justinmk
Member
justinmk commented Oct 30, 2016 edited

If there would be a thing that would take a language specification (BNF etc) and make a parser for that language (should be totally possible and there is YACC).

Check out the scintillua project for a library of PEG grammars for various languages. I'd really like to see this used for syntax highlighting in Nvim.

Because refactoring is just a function from one AST to another it should be possible to create fairly simple DSL or service for that also.

Refactoring is not AST => AST. Useful refactoring depends heavily on semantic analysis (type system) or more sophisticated inspection.

This thing could also remove the need of ctags as it could store all information about various program components

Microsoft Language Server Protocol abstracts those things into a common interface (this could be the DSL you mentioned) that can be used by any front end. AFAIK it is not a one-size-fits-all solution for storing, indexing, querying.

@purpleP
purpleP commented Oct 31, 2016

@justinmk As I've said type checker will be also implemented in my proposed solution. And ANY code change is AST -> AST and so is refactoring. As to scintilla it only have lexer, right? Whereas I'm talking about parser.

@purpleP
purpleP commented Oct 31, 2016 edited

@tjdevries I've read briefly about Language Server. It differs from my idea in two ways. First I'm talking about implementation and protocol, whereas MLS is only a protocol. Second, in my solution protocol differs a bit. For example MLS doesn't define any means of querying the code base. Only jump to definitions. But I could be wrong about it.

One of the reasons I'm talking about this is that my proposed solution could be compatible at least partially with MLS, so if neovim would support that it would be good.

@prabirshrestha

I would also be very interested in how this goes.

Given that my work machine is windows and neovim hasn't been working well there 😞 I managed to write LSP client in pure vimscript that works asynchronously in both vim8 and neovim.

Currently it just the client part.

https://github.com/prabirshrestha/vim-lsp

@tjdevries
Contributor

Oh this is nice. I will take a look at this later today. Maybe we can
combine our work somewhere!

On Sun, Nov 6, 2016, 4:22 AM Prabir Shrestha notifications@github.com
wrote:

I would also be very interested in how this goes.

Given that my work machine is windows and neovim hasn't been working well
there 😞 I managed to write LSP client in pure vimscript that works
asynchronously in both vim8 and neovim.

Currently it just the client part.

https://github.com/prabirshrestha/vim-lsp


You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
#5522 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AEQo0-Zwd_lQ8-KnCa6hZrCVfQZpSxWbks5q7Zw-gaJpZM4Kd6cE
.

@prabirshrestha

I'm all for working together.

@tjdevries
Contributor

I'll try and post a screen cast of some features that I've gotten working, but I think I've got:

all set to go (at least in a very basic form). So it's working quite well :D

I'm excited about the results.

@tjdevries
Contributor

Big shoutout to @prabirshrestha for his work on vim-lsp which I modified a bit and included in the shim.

@mhartington

Very excited to see your progress @tjdevries. I've recreated something similar for typescript but using the python api

https://github.com/mhartington/deoplete-typescript/blob/master/rplugin/python3/nvim-typescript/__init__.py

@prabirshrestha
prabirshrestha commented Nov 17, 2016 edited

Glad to see this coming out very well. @tjdevries awesome job.

@mhartington unfortunately typescript (tsserver) does not implement the language server protocol :( Microsoft/TypeScript#11274 There does seem to be an effort by sourcegraph team https://github.com/sourcegraph/javascript-typescript-langserver

It will be interesting to see these LSP implementation working inside vim. https://github.com/Microsoft/language-server-protocol/wiki/Protocol-Implementations

@tjdevries
Contributor

I have confirmed it working on at least the sourcegraph go langserver and their python langserver. So that's pretty exciting.

Documentation will come out soon with some more ease of use type things :) I'll keep you all posted when that's ready.

@dumblob
dumblob commented Dec 5, 2016

How did you solve the issue with concurrent edition of the same buffer (e.g. split buffer - possibly among different clients)? The Microsoft Language Server protocol still does not support it (Microsoft/language-server-protocol#23 (comment) ).

@hauleth
hauleth commented Dec 16, 2016

@dumblob but where is the issue? Vim has things enabled mostly per buffer, not per window, so there should be no problem with anything when using LS.

@dumblob
dumblob commented Dec 16, 2016

@hauleth I'm sorry, but I don't understand what you mean. What are "things enabled per buffer"? How does it relate to e.g. split buffer edition among different clients?

@hauleth
hauleth commented Dec 16, 2016

If you meant different instances of nvim then it would be the same problem among all editors, so nothing nvim specific and IMHO not an issue as when you edit files in different places simultaneously, then there always be problems. On the other hand, if you were talking about different vim windows then this is not an issue as LS would be operating on buffers not windows and windows shares the same buffer.

Am I missing something?

@prabirshrestha

Something like file change notification would be useful to have in vim and neovim to create better integration for language server. Started a discussion in vim_dev at https://groups.google.com/forum/#!topic/vim_dev/oubHkRbdQW0. Some more discussion about this at #1380 including a WIP PR at #1791

@prabirshrestha

In case anyone is curious to see demos :)

Async goto defintion in typescript using vim-lsp. More info at Quramy/tsuquyomi#57 (comment) This works in both vim and neovim.

lsp-ts-gotodef

async completion in vim (Fork of completor.vim with python removed implemented in pure vim script. Currently doesn't work yet in neovim as I'm waiting for neovim to support lambdas. More info at tjdevries/nvim-langserver-shim#11) Wished async completion was just part of core like completfunc and omnifunc

async_completion

Currently simulating async completion with timers.

function! s:emoji_completor(args, done)
  call timer_start(1000, {timer->a:done([{'word': ':smile:'}, { 'word': ':+1:'}])})
  " for sync directly call a:done()
  " call a:done([{'word': 'smile:'}, { 'word': '+1:'}])
endfunction

call completor#providers#register({
      \ 'name': 'emoji',
      \ 'filetypes': ['javascript'],
      \ 'completor': function('s:emoji_completor'),
      \ 'trigger': completor#providers#trigger_from_chars([':'])
      \ })

2017 might finally be the year of async vim and vim as an IDE :)

@Shougo
Contributor
Shougo commented Jan 1, 2017

@prabirshrestha Why don't you use deoplete?

@prabirshrestha

My work machine is windows and language is typescript and I can't get neovim with python working. Details at #5360 This means I'm back to using vim until then and deoplete doesn't work in vim 8 yet 😢

Also I don't think the current deoplete interface would satisfy my needs. I will file an issue in deoplete to discuss about the requirements. I need something like denite's __async_gather_candidates

Since I wanted to support vim too I posted my thoughts about async completion here tjdevries/nvim-langserver-shim#11 which describes the interface in bit more details.

@Shougo
Contributor
Shougo commented Jan 1, 2017

My work machine is windows and language is typescript and I can't get neovim with python working. Details at #5360 This means I'm back to using vim until then and deoplete doesn't work in vim 8 yet 😢

Hm. deoplete should support Vim8, but it is not easy.
I must implement neovim-python compatibility layer.

Also I don't think the current deoplete interface would satisfy my needs. I will file an issue in deoplete to discuss about the requirements. I need something like denite's __async_gather_candidates

Hm. But why __async_gather_candidates is needed?
deoplete is already asynchronous.

@prabirshrestha

I might be wrong on this but isn't gather_candidates supposed to return a list which is synchronous? Even though the gather_candidates is called asynchronously due to remote plugins in neovim the actual implementation should be synchronous on the python side.

Since gather_candidates is synchronous:

@prabirshrestha

Got a fork of go-vimlparser working as a an LSP server while I wait for the reply at haya14busa/go-vimlparser#26

Here is a demo of :FindDocumentSymbols using vim-lsp repo. Someone else should definitely pick up on vim language server.

langserver-vim

@Shougo
Contributor
Shougo commented Jan 5, 2017 edited

@prabirshrestha OK. deoplete should support the asynchronous feature.
It is not easy though...

@timeyyy
timeyyy commented Jan 8, 2017

The implementation of the protocol in lua could be simplified by using the haxe sd and generating lua. I've been playing with haxe and can reccommend.

@garyburd
garyburd commented Jan 8, 2017

@timeyyy See langserver.lua for a partial implementation of the protocol in Lua.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment