Question: Using neovim's remote plugin capability to talk to TSS asynchronously

[mhartington]

Neovim offers the ability to interact with node plugins via a native node background process. Curious if you've looked into it at all as a way to speed up completion? Currently vimproc can get a little slow depending on the project.

This is a similar method to how editors like Atom and VSCode talk to TSS.

[Quramy]

Hi @mhartington

It's so interesting, but I'm not familiar with neovim and it's functions.
Do you have some sample or demo which uses neovim and node bg process? Let me know.

I will not dispose vimproc because I support not only neovim but also classical vim.
However I add a faster function to communicate with tsserver for neovim if I can.
:)

[mhartington]

Maybe then a specific plugin for neovim then 😄

So general idea is that neovim has a lot more binding to various languages. Such as being able to write plugins in python3, node, or even go.

With either of those options, you could have TSS running asynchronously and provide the completion that way.

Now, my knowledge is limited, but I can try to throw together something simple.

For now, there is a simple CSSBeautify example in the node-host repo for neovim
https://github.com/neovim/node-host/tree/master/examples

Example of using a background daemon
https://github.com/nsf/gocode

[Quramy]

Thanks! I'll checkout them.

[ToucheSir]

See also https://github.com/neovim/neovim/wiki/Related-projects#plugins for projects using neovim's bindings.

[prabirshrestha]

Vim also supports jobs and channels but currently still in development so some of the features may not be implemented.

vim/vim@38a5563

[mhartington]

@Quramy just want to add to this a bit more. Seems that what could be the best way to approach completion is by writing a source for deoplete, similar to what is done for ternJS.

https://github.com/carlitux/deoplete-ternjs/blob/master/rplugin/python3/deoplete/sources/ternjs.py

I've started working on a similar source completion using your plugin as a base, the only problem is that if you call any vim-script in the python plugin, it breaks any async processes. Would you be up to collaborating seeing as you know a bit more about the TSSServer than I do 😄

[zbindenren]

+1

[mhartington]

If anyone would like to help out with this, I have

https://github.com/mhartington/deoplete-typescript

Which just calls tsuquyomi's completion method. If anyone would like to help out and write some python, I'll happily accept anything

[Quramy]

@mhartington
It's great!

[mhartington]

Would love to add this back to this repo and rewrite the completion portion to start the TSSServer via Python instead of vimscript. This will keep the completion portion of things nice and async for neovim, but then it could fall back the the vimscript implementation on regular vim. Would you be up for that?

[Quramy]

I feel that's a good idea.

I have no clue python coding but it's worth trying at all events.
I'm going to try it out. I'll add the following func:

• communicate between nvim and TSServer with nvim's native job-control(it's async)

TSServer has stdin and stdout interface, and nvim's job-control also can use stdin and stdout. So I think it's enough to start TSServer process with vimscript(and fallback with vimproc on regular vim) without python. Is this right? Why do you "start the TSSServer via Python instead of vimscript" ?

[prabirshrestha]

@mhartington you can still do async in regular vim if it has support for jobs and channels.

Here is how to listen to socket in regular vim.

And here is how to start a command asynchronously.

function! Handler(channel, message)
    echom a:message
endfunction

let job = job_start('cmd /c dir /s/b c:', { 'callback': 'Handler' })

Currently it is a bit buggy so I would suggest to try to only support vim with the latest patches at least for now.

I'm fine with python since it usually has better performance too (Shougo/unite.vim#1083 (comment)). And you could fallback to threads in old vim like how vim-plug works.

[Quramy]

@prabirshrestha

I see your comment in Shougo/unite.vim#1083 (comment).

And I like your idea:

Now someone needs to create a wrapper so that it uses native vim/neovim and fallbacks to python/ruby threads or use vimproc :)

Do you know some examples or implementations of the above wrapper system?

[prabirshrestha]

redismru.vim is the only one I have seen where there is an attempt to support both. But haven't tried it yet. It should be fairly easy but I'm not an expert on vim scripts.

[mhartington]

So the idea to use python is that its a bit faster, easier than viml, and the purpose of working with deoplete.
https://github.com/Shougo/deoplete.nvim/blob/master/doc/deoplete.txt#L507

Note: The sources must be created by Python3 language.
Note: If you call Vim functions in your source, it is not asynchronous.

At least form the completion/TSSserver portion, anything in vimscript will block. So if it could be written in python, it could be used in both neovim and regular vim.

Then if there was deoplete source, it would just import the completion lib, and send commands to that. This would allow people to create other completion sources, like for neocomplete, etc.

[Quramy]

@mhartington

At least form the completion/TSSserver portion, anything in vimscript will block. So if it could be written in python, it could be used in both neovim and regular vim.

Ok, I've got it.

I think my plugin's vimproc should be replaced with python(or jobs and channels or some async system).

It's a big change for tsuquyomi and it'll take me a while.

[ianks]

bump 😄

[prabirshrestha]

@Quramy Here is the async job control wrapper that works natively in neovim and vim https://github.com/prabirshrestha/async.vim

function! s:handler(job_id, data, event_type)
    echo a:job_id . ' ' . a:event_type
    echo a:data
endfunction

if has('win32') || has('win64')
    let argv = ['cmd', '/c', 'dir c:\ /b']
else
    let argv = ['bash', '-c', 'ls']
endif

let job = async#job#start(argv, {
    \ 'on_stdout': function('s:handler'),
    \ 'on_stderr': function('s:handler'),
    \ 'on_exit': function('s:handler'),
\ })

Currently it uses jobcontrol in neovim and vim. Next would be to fallback to python/ruby/vimproc if jobcontrol doesn't exist.

[faceleg]

See mhartington/nvim-typescript#2, if anyone with better python skills than me is interested in piling on, do so!

[HerringtonDarkholme]

https://github.com/prabirshrestha/async.vim

What about this? It normalizes behavior between vim and neovim. If @Quramy is willing to leverage asynchronous feature in vim then I will definitely recommend this.

[prabirshrestha]

vim-grepper recently supported async too. mhinz/vim-grepper#48

[mhartington]

Just a heads up, if anyone is using Neovim and wants async typescript completion, feel free to checkout

https://github.com/mhartington/deoplete-typescript

While this plugin is great in regular vim, neovim has been my daily editor for the past year or so. So I don't plan on switching back. Now both vim and neovim have typescript support.

[hpurmann]

Thank you, @mhartington! I also switched a year ago and haven't looked back. Async was my number one reason.

[prabirshrestha]

Unfortunately my work machine is windows and neovim support hasn't been so great there :(

Got a proof of concept async tsserver running at https://github.com/prabirshrestha/vim-ts. Currently only works in vim with jobs on windows. Should be very easy to update the code so it works for multiple environments.

Currently the only thing it supports to Goto Definition by :TsDefinition or Ctrl-].

[ianks]

wrt neovim, i've been experimenting with using the. ode-client and interacting with tsserverlibrary directly. the result seems much easier to maintain/iterate on.

[prabirshrestha]

@ianks would you mind sharing the link to repo if possible?

Ideally in the future I think we should just have language-server-protocol vim plugin that just calls lsp. https://github.com/Microsoft/language-server-protocol/wiki/Protocol-Implementations

[prabirshrestha]

I got async language server protocol working on vim 8 and neovim and it works on both windows and *nix environments. Here is a demo of go to definition in typescript in bash on windows. (Notice how the server as well as as goto def is all async)

The typescript language server I'm using is https://github.com/sourcegraph/javascript-typescript-langserver

Internally the it using vim-lsp which uses async.vim. The :GoToDef vim script can be found at prabirshrestha/vim-lsp#3 (comment)

vim-lsp and async.vim are more of utils functions, Ideally we would be using something like /nvim-langserver-shim. This means we would be getting langauage servers for free like css, json, php, golang, java and whatever exists from vscode or other IDEs for free.

I also got proof of concept async auto completion working in pure vim script https://gist.github.com/prabirshrestha/0b3b7b4e2112c662979bfeeec4e1edb7. I can't wait for ctrlp to support async api so I can implement find all references and other features.

2017 might finally be the year of async vim and vim as an IDE :)

[prabirshrestha]

The typescript language server protocol server still seems to be at a very early stage so landed up using https://github.com/runoshun/tscompletejob in vim8 for async instead.

Here is the demo of async completion in vim8 using https://github.com/prabirshrestha/asyncomplete.vim
The completion is async and the UI is very responsive.

Another interesting thing tscompletejob does is rather than using the tsserver directly it uses a wrapper so that converting tssserver completion responses to vim completion responses is done node in external process. This would make the ui more responsive. nvim-typescript wouldn't have this problem since it is already taking advantage of remote plugins in neovim.

[prabirshrestha]

Finally got some exciting good news to share 😃

I have implemented the Language Server Protocol for typescript that acts as a proxy for tsserver which can be found at https://github.com/prabirshrestha/typescript-language-server

If anyone wants to try it out here is the setup.

Make sure typescript-language-server and tsserver is in the PATH.

npm install -g typescript
npm install -g typescript-language-server

If you want to use in vim8 and neovim use the instructions at https://github.com/prabirshrestha/asyncomplete.vim#language-server-protocol-lsp. It is fully async in both vim8 and neovim. All is written in pure vimscript so no python required. But if you prefer nvim remote plugin you can try LanguageClient-neovim with the same server.

Make sure to use the dev branch of vim-lsp for now. It is still at a very early stage.

Plug 'prabirshrestha/asyncomplete.vim'
Plug 'prabirshrestha/async.vim'
Plug 'prabirshrestha/vim-lsp', { 'branch': 'dev' }
Plug 'prabirshrestha/asyncomplete-lsp.vim'

au User lsp_setup call lsp#register_server({
    \ 'name': 'typescript-language-server',
    \ 'cmd': {server_info->[&shell, &shellcmdflag, 'typescript-language-server', '--stdio']},
    \ 'whitelist': ['typescript'],
    \ })

You might also want to add root_uri: {server_info->s:find_nearest_tsconfig_directory()} which returns a string. If it returns empty the server is not initialized, if root_uri is not present it will use the getcwd() as the root directory.

Currently only completionProvider is supported by the typescript-language-server but it is easy to add other features. The language server code it self is very small and is written in typescript (No more excuse saying it is in vimscript or python and I can't contribute 😄 ). I have already implemented the hard part so feel free to send PRs that implement other features of the protocol. vim-lsp already implemented most of the LSP features. Currently it supports :LspDefinition, :LspDocumentSymbol, :LspHover, :LspReferences, :LspRename, :LspWorkspaceSymbol, :LspDocumentFormat, :LspDocumentRangeFormat. In the future I do plan to add ctrlp and fzf support instead of just vim's quickfixlist.

I do have lot of plans to improve this so if any one is interested in helping me out let me know. Any skills you have that is typescript, vimscript or lua would be appreciated.

[mightyiam]

Sounds lovely. A step towards a reasonable TypeScript vim experience?