New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RFC] Built-in LSP Support #6856
Conversation
Nice! I hope to improve jobs/channels in lua #6844 (comment) to the extent this can just use them and don't need to use libuv directly. |
@bfredl that sounds great! That would certainly ease some of the configuration for the client. I will try and watch out for that. |
runtime/autoload/lsp/request.vim
Outdated
function! s:get_client() abort | ||
if g:nvim_lsp_client == -1 | ||
lua << EOF | ||
local client = require('runtime.lua.lsp.client') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This dedented block looks ugly, do not use <<EOF
. Everything which does not fit into one line (lua require('lsp.client').get_client()
, or, better, just return luaeval("require('lsp.client').get_client()")
(move :if
to lua to not split the logic)) should be in a require
d function.
Also should be require('lsp.client')
, I do not know why #6789 still is not merged.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, I have removed many of them and rebased, so I can use the updates from #6789 . I will clean up the code as I go along. Still brainstorming a lot of ideas.
runtime/autoload/lsp/request.vim
Outdated
|
||
function! lsp#request#textdocument_references() | ||
lua << EOF | ||
print('TODO') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same.
runtime/lua/json.lua
Outdated
@@ -0,0 +1,383 @@ | |||
-- | |||
-- json.lua |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you just use vim.api.nvim_call_function('json_…'
? There already is a JSON parser in project, do not need to add a second one. If some features are missing they can be added.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't want to use the API to do it, since some of the json parsing might be done in a different thread I think. Unless it can work in another thread?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are using luv
to fork out different Neovim threads? I do not think this is a good idea, most of the code does not expect this and making sure that different shared resources (memory, file descriptors except for a small subset, etc) are not accessed through spawned threads is close to impossible. Generally: until there are Neovim own functions for creating threads or fork()
ing don’t do this in core plugins, it risks producing nasty heizenbugs.
As to the question, though not exactly API, but JSON decoder could be easily edited to work with lua natively: most of code is parsing there, just need to abstract away creation of values (I would personally go with creating jsondecode.c.h
macros-parametrized file for this purpose, should also be possible to do abstraction via struct
with function references). Encoder is a separate issue, the “easy” way is to not touch it at all, but create duplicate lua -> VimL conversion functions which are exactly like existing ones, but do not put allocated lists and dictionaries into M&S GC double-linked list. In both cases you will spend more time creating the tests then adjusting the code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Luv won't spawn threads, but it can run callbacks at times which are not considered vimL-safe, though as long as GC won't be invoked in such a time (potentially, rooting might be incomplete), json_decode shouldn't be a problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bfredl Container allocation functions are not reentrant because they alter a linked list in a number of steps. So it is not safe to call callbacks at random times even without GC run (how, BTW? I do not know how uv is going to perform async read at unsafe time without either using SIGALARM (should probably mess up with our timers), or a separate thread, or a separate process), unless you edit C code of the parser to work with lua containers.
Why not use existing jobs in any case, this is going to be safe for sure? Just have a lambda which immediately calls lua.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not random times, just times the event loop is invoked. As previously this could be done in a way that queues all vimL until later, such spots could potentially be vimL GC unsafe. Can container allocation invoke GC or does that only happen at states?
The plan is indeed to improve jobs so they can be used here.
runtime/plugin/lsp.vim
Outdated
|
||
" TODO: Automatically open file if not opened already | ||
function! LSP_References() abort | ||
lua << EOF |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lua lsp.client_references()
, indented:
- Do not use ugly
<<EOF
blocks. - All globals must be namespaced. This is not the only plugin using lua. I personally would prefer a single global per plugin or no globals at all (use
lua require("…").func()
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just like on #6789, I would say that it is vastly preferable if we put some of these finer points in the help docs, where people are likely to find them.
runtime/plugin/lsp.vim
Outdated
endfunction | ||
|
||
function! LSP_DidOpen() abort | ||
lua << EOF |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same.
runtime/plugin/lsp.vim
Outdated
" TODO(tjdevries): Make sure this works correctly | ||
" TODO(tjdevries): Figure out how to call a passed callback | ||
function! LSP_Request(request, params, callback) abort | ||
return luaeval('client_request(_A.request, _A.args)', {request: request, args: params}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same (globals).
runtime/plugin/lsp.vim
Outdated
@@ -0,0 +1,28 @@ | |||
" TODO(tjdevries): Move this to autoload? | |||
|
|||
execute('luafile ' . expand('<sfile>:h') . '/../lua/lsp/plugin.lua') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be something like lua lsp = require('lsp.plugin').setup()
. (Needs #6789 or you adding a path to package.path
.)
@@ -0,0 +1,38 @@ | |||
local helpers = require('test.functional.helpers')(after_each) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess this should be test/functional/plugin/lsp/…
: other runtime files are tested in plugin
unless these are runtime files used from core in a special fashion (like providers which have corresponding C code). Applies to all test files here.
runtime/lua/util.lua
Outdated
vim = vim or {} | ||
|
||
util.echom = function(message) | ||
vim.api.nvim_command('echom "' .. tostring(message) .. '"') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just remove this function and use print()
. And you have escaping issues here in any case.
runtime/lua/lsp/client.lua
Outdated
|
||
function client:_on_error(level, err_message) | ||
if type(level) ~= 'number' then | ||
util.echom('we seem to have a not number' .. util.tostring(level)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Message should start with a capital letter.
runtime/plugin/lsp.vim
Outdated
|
||
execute('luafile ' . expand('<sfile>:h') . '/../lua/lsp/plugin.lua') | ||
|
||
function! LSP_Start() abort |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tjdevries hi, I think maybe some people like autoload function more. for example
function! lsp#start() abort
and I have read most of vim's runtime file, the file in runtime/plugin/
do not definde func with capital letter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have changed them to autoload. Thanks :)
runtime/lua/lsp/client.lua
Outdated
@@ -0,0 +1,304 @@ | |||
local uv = require('luv') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is luv already included by default?
For me, inside of nvim:
:lua print(package.searchpath('luv', package.cpath))
/home/aktau/github/neovim/neovim/.deps/usr/lib/lua/5.1/luv.so
:lua print(package.cpath)
./?.so;/usr/local/lib/lua/5.1/?.so;/home/aktau/github/neovim/neovim/.deps/usr/lib/lua/5.1/?.so;/usr/local/lib/
lua/5.1/loadall.so
Or:
$ nvim -u NONE +'lua io.stderr:write(package.cpath)' +'lua os.exit()'
./?.so;/usr/local/lib/lua/5.1/?.so;/home/aktau/github/neovim/neovim/.deps/usr/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/loadall.so%
It appears my build folder is in my package.cpath
, this is the default package path from the LuaJIT compiled into nvim, but it can be overridden if the user sets their own path. This should be made much more robust. It is also not guaranteed that a user has the neovim .deps folder on their machine. We should attempt to not rely on optional libraries (especially those that need to be compiled, like luv).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm hoping to not have to use luv
soon. Once we get the jobs update (which I assume will happen before I finish this :) ) then I will use nvim's included job functionality, rather than a hacked together interpretation.
@ZyX-I Is there a preferred way to get rid of the luacheck errors about referencing "global vim" without declaring it? Should I just set |
Instead of callbacks why not publish RPC notifications? |
Then we need to be able to subscribe to them from vimL and Lua. Which we probably want anyway. |
@tjdevries Tell luacheck that there is global |
@justinmk I think RPC notifications would work as well. Originally I was thinking of callbacks because I thought GUIs/plugins/etc. might want to add their own callback or remove the default callback to extend behavior. But I think RPC would work well for that too. I never got around to implementing an async |
If the callbacks aren't needed then my suggestion is moot. My suggestion is only relevant under that assumption. |
What's the status of this? |
I'm waiting on a few PRs to get merged to ease development of this. I'm also working on a few different projects right now, so I haven't had a lot of time to work on this. I would recommend using one of the other plugins that provides the functionality for now. I'm still planning on finishing this though :) |
For those who are curious, could you name them, please? |
There's some work being planned for Primarily the |
Could we set up a BountySource for this to motivate @tjdevries get this through the finish line? I would be happy to donate some money to have native LSP support in neovim. |
How does it compare to https://github.com/autozimu/LanguageClient-neovim? |
Hi everyone 😄 Sorry for the long pause in dev, been working on some personal projects and practicing lua and lua within nvim. @blueyed It currently compares to the language client as not as many features and its broken 😄. The idea however is that if this ships with neovim, people will be able to contribute more effectively, since it will be an "official" implementation. Also, it won't require any dependencies (so you won't have to install python and set up neovim-python) as the Lua will all run with whatever neovim ships with. I have lots of other ideas, but I should really finish the basic implementation first :) I imagine there will still be other plugins around that integrate the core features built here to interact with other plugins. I can imagine sources for deoplete that use the builtin @jvican Not sure how that would work. If you're interested in using LSP in the mean-time, I would really recommend the implementation @blueyed linked. That way you could use most of the up-side of LSP and not have to spend money :) |
I'm currently using https://github.com/autozimu/LanguageClient-neovim with the RLS. If you're at a point where you'd want someone to test this, let me know :) |
@KillTheMule I'll let you know :D Thanks for the offer. |
88a04a0
to
a950255
Compare
runtime/lua/lsp/structures.lua
Outdated
@@ -0,0 +1,117 @@ | |||
local lsp_util = require('runtime.lua.lsp.util') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
drive-by comment: runtime.lua.lsp.util
looks too nested, possibly a sign of over-architecture. Hyper-categorization is bad enough, but sub-categories and sub-sub-categories are rarely needed.
I think we already have too many subdirectories in Nvim source. We should try to keep things relatively flat.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In fact the LSP support should be one big file, with truly common utilities in one separate file (call it foo.lua
, doesn't matter: again, one big file, we can nitpick over naming/placement later). "Common" means "can be used by any other Lua code, not just LSP".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The most important thing to avoid is shared state. The line count of a file doesn't mean anything as long as there is minimal or no shared global or pseudo-global state.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it looks too nested only because it's wrong: it should be require('lsp.util')
. That wouldn't seem too bad?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, it should just be lsp.util
, but I couldn't get the tests working at first without the runtime.lua
, so I'll fix that soon.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am looking to try and consolidate as much as possible :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@justinmk Should be fixed now. It's just lsp.util
or lsp.callbacks
, just like any other lua plugin.
Maybe neovim could only expose Lua API for LSP? Seeing how much effort is being put in user space to support LSP it would be unfair to discard it (and especially given that such effort has value both for vim and neovim users). That way neovim would provide backing API on which underlying user space implementations could rely when run on neovim. |
Somewhere in the comment cloud above, we discussed minimizing and, ideally, eliminating the VimL wrapper for this, so |
I understand, you were first, but if this gets built-in to Neovim, then I am afraid it should get a precedence, shouldn't it? |
@mcepl but as said above this will be trivially solved by renaming the file. Alternatively, as I might already have said in the hidden 170 comments, we should change the |
@bfredl certainly, and it is not a big deal. Actually, eliminating autoload seems to me like a good idea. |
It is extremely easy to add support for renaming of symbols (thank you!): function! LSPRename()
let s:newName = input('Enter new name: ', expand('<cword>'))
call lsp#request_async('textDocument/rename',
\ {'newName': s:newName})
endfunction
au FileType lua,sh,c,python
\ noremap <silent> <buffer> <F2> :call LSPRename()<CR> but there is a bug somewhere. Whenever I run this command, I get one more EOL in the end of the file. |
This comment has been minimized.
This comment has been minimized.
@tjdevries I've made nvim into a fake languageserver and used it for a test here. I'm planing to extend this a bit, seems to be a good way to test this. Generally, though, what's you're plan, do you think you'll find time to work on this? I offer my help, up to the point of bringing it over the finish line. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
@KillTheMule I have my last day of work at my current company on Friday and then have a week off after that. If I don't make any progress during that week, it'd be great if you could take this PR to the finish line for me. I'd be happy to hop on gitter more often if you need to bounce ideas around or try and figure out what I was doing during parts of the PR. Thanks for following up and offering your help. |
That's good to hear. Let me throw out some questions that occured to me during writing the integration test. Feel free to discard any of my thoughts, but I think I should at least mention them :) Talking on gitter would be fine, too :)
|
@tjdevries @KillTheMule ping? This LSP client is accused of being a cause of #10167. |
Hey, my latest info was that tj was working on a larger refactoring, but I haven't heard from him since. I'm pressed for time these days, so I did not push this. |
@KillTheMule @tjdevries Can I take over this PR? |
@h-michael Sure, open a new PR with your changes. |
Definitely. I'll keep track and try to help out, let me know if there's something particular you'd like me to do. In particular, I could extend my "nvim as a fake language server" approach for testing :) |
I open the PR #10222. And I would like to discuss where to set the fish line. :) |
Shouldn’t this PR be closed? The story is no longer here. |
So, here's the very beginnings of LSP support in neovim.
It can currently, start a server, say that it has opened the file and request references from the server. It loads the references using
setloclist
.Here's my vision (or at least a rough draft of it). It's too late for me right now to clean more of it up and I'm too excited not to finally at least put something as a WIP PR :)
Feedback welcome and appreciated.
Goals:
Registering callbacks
Should be something along the lines of
function lsp#request(request_name, arguments, use_default_callback, optional_callback)
as well as a lua function that has the same signature.
Where:
request_name
arguments
Position
, we would useline('.')
andcol('.')
use_default_callback
optional_callback
I imagine optional callbacks essentially allowing an even more extensive API, one that doesn't require Neovim explicit knowledge. So, to embed neovim in another editor, you could try and do lots of the information transfer with LSP callbacks.
From nvim
:call lsp#request('textDocument/references', [], v:false, custom_callback)
From remote
nvim.call_function('lsp#request', ['textDocument/references', [], false, custom_callback])
Creating default callbacks
function lsp#handle_response(request_name, response)
->
function lsp#response#textDocument.references(response)
->
function lsp#response#textDocument.hover(response)
Where:
response
:hover
callback to create a simple hover pane that other hosts/clients/GUIs would understand.