Implement LSP / Language Server

[chriseth]

The main task in this issue is probably identifying the steps towards full LSP support.

Features for first release (mainly related to file lookup):

• cwd should not matter for loading files / resolving paths
• loading a single file with only relative imports should work without any configuration
• it should be possible to configure root path and include paths
• internal names cannot have file:// prefix due to absolute / direct imports, so we need translation functions
• the server needs the ability to load files from disk itself (didOpen just means the client provides a "content override" to the server)

Features for subsequent releases (probably implemented in the client, but we should support it as much as possible):

• node_modules should be auto-detected as an include path as much as possible
• files synthesized by truffle should work (may need to invoke truffle to get it)
• we should standardize the way how frameworks configure the language server, so this does not have to be implemented in all clients

[christianparpart]

After reading somewhat into the official specification, I'd say this ticket first is not about fully implementing it but about having it started and got some initial support done.

Here are the steps I'd propose:

1. Implement a very simple solidity language server (I called it solsp - naming is a .... can be changed later ;-) ) that basically just reads messages, parses them, pretty prints them to the logging stream (so we get an idea what language clients are talking).
2. Figure out how we can get VS code talking to custom LSPs (I don't know this)
3. Decide which of these message need implementation; this is probably an incremental step.
4. Make sure this solsp works at least with VS Code as well as vim (coc-vim)
5. Have some documentation on how to set up VScode/vim for using this language server.

My proposal would be to start with the following features:

• compile error feedback (NB: we need to preserve the AST after compilation)
• semantic symbol highlighting
• tooltips (signature info, api documentation, ...)
• auto completion - this one may be a little trickier, let's see.

[christianparpart]

moved back to To Do as we agreed on holding it back due to other priorities for now (for 2-4 weeks)

[nassredean]

hey I was just curious if there are any updates on this or if this is an area that would be a good candidate for contribution?

[chriseth]

We have a prototype and it would be great if you could test it out! @christianparpart can you write a small text about how to use the LSP?

[elee1766]

i'd also be happy to test this out, let me know if that's possible

[nassredean]

We have a prototype and it would be great if you could test it out! @christianparpart can you write a small text about how to use the LSP?

Yea please do let me know I would love to test it!

[christianparpart]

Hey guys. Many thanks for your interest in this. I will be writing a guide on how to test and future work next Tuesday and share the link in here.

@nassredean, @elee1766
Please tell me what editors/IDEs you would like to use this with, so i can refer to them (I tested with VIM/NeoVim+coc.nvim, QtCreator, VScode).

[nassredean]

Hey guys. Many thanks for your interest in this. I will be writing a guide on how to test and future work next Tuesday and share the link in here.

@nassredean, @elee1766
Please tell me what editors/IDEs you would like to use this with, so i can refer to them (I tested with VIM/NeoVim+coc.nvim, QtCreator, VScode).

Amazing! Thank you for doing this!

I am using NVIM v0.5.0-dev+1191-gf89bfa686. nvim v0.5 supports lsp using https://github.com/neovim/nvim-lspconfig and then I use deoplete for completion.

Running on ubuntu 20.04.1 LTS

[christianparpart]

I am using NVIM v0.5.0-dev+1191-gf89bfa686. nvim v0.5 supports lsp using https://github.com/neovim/nvim-lspconfig and then I use deoplete for completion.

the feature set is still minimal, but surely I want to improve on that. Code completion will however have a lower priority as I first want to finish the lower hanging fruits (code completion actually means to properly deal with compilation errors and our the code base wasn't designed for that, but I am still confident that we can do that :) )

[christianparpart]

@nassredean @elee1766 @chriseth bare with me, it's a very long time ago I was writing textual stuff like that. I have put it into my own blog site for now to get it up ASAP, but i can move that over to our own solidity blog site at any time.

https://dev.to/christianparpart/builtin-solidity-language-server-267n

[nassredean]

@nassredean @elee1766 @chriseth bare with me, it's a very long time ago I was writing textual stuff like that. I have put it into my own blog site for now to get it up ASAP, but i can move that over to our own solidity blog site at any time.

https://dev.to/christianparpart/builtin-solidity-language-server-267n

amazing! I will try sometime this week as soon as I have a little more time. Thanks again!

[elee1766]

@christianparpart cool! ill try this today and let you know. if i don't reply today it means i died of frustration

edit: just wrote a contract with it. i haven't encountered any major issues so far, and it seems to be correct as far as I see.

[Tomburgs]

This looks exciting, very nice job @chriseth & @christianparpart!
Any chance of creating a Darwin build for us mac users?

[christianparpart]

This looks exciting, very nice job @chriseth & @christianparpart!
Any chance of creating a Darwin build for us mac users?

Hey @Tomburgs many thanks for your interest. Please give me some time before doing that. I want to tidy up a few things first, then I'll release a new VScode extension (if that is what you have been requesting for).

Also, please keep in mind that it is not officially supported nor feature-complete yet, see it more as an early beta with any feedback very welcome. :)

[elee1766]

Hey @christianparpart !

I've been using your LSP with coc.nvim for a month now! It's been pretty nice to me so far.

The biggest issue I'm having is support for the path mapping that truffle has for importing .sol files. I attempted to launch the LSP with the same flags as truffle, locating the correct folders in node_modules/@xyzzy, however never got that working, and haven't had the time to look through solc and find out why

Am I just configuring something incorrectly, or is this something that support would need to be added for?

[cameel]

@elee1766 Truffle has its own resolver for locating directories that you can import files from. solc and, by extension, the LSP does not have that and you have to directly specify the full import path. If you know the right directories though, you can get a similar effect using import remapping (see Layout of a Solidity Source File > Use in Actual Compilers).

For example, if you're running solc from your project directory and have my_contract.sol there that does import "@xyzzy/util.sol" while the library is physically located in node_modules/@xyzzy in that directory, you can make that import work using:

solc my_contract.sol @xyzzy=node_modules/@xyzzy

is this something that support would need to be added for?

Remappings should be enough but they have downsides: you have to remap every library explicitly and also they end up in contract metadata (because they were really meant for a different use case). Actually, I've been working recently on making the path-related behavior in Solidity less quirky and one of the ideas would be to let you specify a list of directories where importable libraries are located. So you could say that node_modules contains libraries and then you could just import from @xyzzy if it's inside. This would still not be as automatic as Truffle but should be enough for LSP.

[elee1766]

Actually, I've been working recently on making the path-related behavior in Solidity less quirky and one of the ideas would be to let you specify a list of directories where importable libraries are located. So you could say that node_modules contains libraries and then you could just import from @xyzzy if it's inside. This would still not be as automatic as Truffle but should be enough for LSP.

this sounds super exciting!!! woohoooo

i was trying to run solc --lsp with the @xcxyzyyz syntax, but it throws an angry message at me saying that input-file and lsp are mutually exclusive. seems like what ur planning will make that a non-issue

[hacker-DOM]

stupid question: will this be "backwards compatible" for previous Solidity versions?

[chriseth]

@hacker-DOM unfortunately not.

[psytron]

Will this handle symlinks?

[cameel]

I don't see why it wouldn't. The compiler follows symlinks and the LSP uses the same code for loading files.

[UltiRequiem]

Any Updates?

[christianparpart]

Any Updates?

Hey @UltiRequiem - I mean, I'm currently on "vacation" and back to work next Monday, we however have put this on hold (read: not rejected, just re-prioritized), this is why it's currently looking like it's not active. I though can't await resuming working on it, given enough resources :-)

p.s.: I think the goal was to have something in 2021, so we still have some months left :)

[edborsa]

How can I help with that?
Can someone tell me how to implement the lsp? Is is from the compiler? Does anyone have a tutorial?

[chriseth]

@EduardoBorsa thanks for your offer! The best way you can help currently is to try out our prototype. It should be explained here: https://dev.to/christianparpart/builtin-solidity-language-server-267n

[joshuavial]

Happy to be an early tester on mac if that's useful. Excited to see this ticket and would love to see it finished but understand there are lots of priorities.

[chriseth]

@joshuavial which IDE would you use together with the lsp?

[kazuki-hanai]

I'm very interested in this issue and would like to make a contribution to it. I usually use vim but there seems to be no LSP for Solidity in vim. I feel the inconvenience much, so I would like to improve this condition.

[gakonst]

@chriseth is anybody actively working on this? This is quite important and if not, it would be nice if we can find somebody to get it over the line. Is the issue funding? Documentation about Solidity internals? Would love to help in any way possible.

There seems to be another old project on this here as well.

[chriseth]

Thanks for offering your help! We plan to release the LSP by the end of the year. What we mostly need is testing in various different IDEs and feedback about it. It is being worked on at #11350