Clojure(Script) Language Server

[thheller]

I would like to start an open discussion about tool support for CLJ(S).

Motivation

I started writing a Language Server Protocol (LSP) implementation [1] [2] for CLJ(S). I have started to implement the basics and can now feed CLJS compiler warnings/errors back to an editor. This uses the textDocument/didOpen|didSave notifications sent by the editor. The server will attempt to compile the mentioned files and provide feedback (via publishDiagnostics) back to the editor. This allows the server to completely skip watching the filesystem as a bonus and lets the editor display warnings/errors without knowing anything about CLJ(S) at all.

Obviously this isn't very useful without editors that support this protocol. The above screenshot is taken in Visual Studio Code since it supports the LSP out of the box. It does not however support nREPL or any other REPL for that matter. Fundamentally the requirements of tools are represented better by a dedicated protocol anyways than just a REPL. Long discussions [3] about this have happened in the past. One point made by Rich Hickey stands out to me:

But, we now have nice editors/IDEs often running on the same machine, with the ability to use sockets. Multiplexing their needs with those of a human REPL consumer over the same connection is going to make things bad for one or both of them.

nREPL has become the de-facto standard for tools to communicate with the runtime, whether they actually multiplex everything over one connection or use multiple connections I don't know. At this point I'm strictly talking about tooling related things the LSP would cover. This includes code completions, code navigation, formatting, documentation lookups, etc. It does not cover a REPL and shouldn't since that has very different needs than tools have (as mentioned above).

nREPL vs JSON-RPC

Assuming for now that the editor will connect to a dedicated TCP port with a dedicated protocol. The protocol is about framing messages between the LSP and the editor. This must support request/response semantics as well as simple fire-and-forget notifications in both directions.

• LSP uses JSON-RPC as the transport encoding. Messages follow {:method string? :params any? :id num?} for requests and the same without :id for notifications. The other side must respond with {:id num-from-request} and a :result or :error entry to requests, notifications do not require a response.

• nREPL uses bencode as the transport encoding. The only mandatory key on a message is :op. I'm not entirely sure about the semantics after that. Some middleware may clone a new session and then add a session and/or id to every message, others just seem to send a single message. I do not know how the server would make a request to the client.

nREPL or JSON-RPC?

The question now is which protocol we (the Clojure community) should use. Most tools already support nREPL but not LSP. There is growing support for LSP [4] [5] and using this protocol enables talking to other languages as well.

Please note that this has absolutely nothing to do with a REPL, this is strictly about editors communicating with the server and which protocol to use.

Adding additional :op or :method messages can be achieved in both and I already want to add things like cljs/list-builds cljs/start-worker {:build-id :website :autobuild true} and more.

References

[1] https://github.com/Microsoft/language-server-protocol

[2] https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md

[3] https://groups.google.com/d/topic/clojure-dev/Dl3Stw5iRVA/discussion

[4] https://github.com/Microsoft/language-server-protocol/wiki/Protocol-Implementations#editors-ides-supporting-the-protocol

[5] http://langserver.org/

[thheller]

Pinging known tool authors. I hope to have an open discussion trying to find a "standard" everyone can benefit from. Feel free to ping anyone else you might think would be interested in this.

@cfleming @bbatsov @kotarak @cgrand

Sorry for the spam if you are not interested in this.

[bbatsov]

I understand the idea behind this, but that's something that has to be implemented by editors themselves - Emacs has to support this protocol, not CIDER for instance. I'm all for having standard tooling, but I'm not familiar with this protocol and have no idea how extensible and flexible it is.

[thheller]

@bbatsov there is a emacs client already. The protocol itself is easily extensible, how extensible and flexible the client you are using however is left to the client. I don't speak emacs but you could probably connect that to my server today.

Forget the link to the JSON-RPC spec. Which methods are defined is left to the implementation, so the Clojure implementation can easily add Clojure specific methods. This would not interfere with what the editor would want to do. The LSP standard defines common things that are more or less relevant to each language.

The LSP protocol has capability negotiation built in via (intialize, registerCapability), look for ServerCapabilities in the protocol.md. That part of the spec has some issues and could probably be improved.

The protocol is rather new so I would still consider it ALPHA despite the v3 version tag.

[fdserr]

Hi Thomas,
stumbled on this ticket (which made me aware of shadow-cljs, looks kewl!) and was wondering if you or anyone in the community is working on an LSP implementation for cli/cljs?
Best.

[thheller]

Nothing beyond the proof of concept work I did a couple months ago.

I still plan on continueing that work but didn't get to it yet. Not aware of anyone else working on it.

[pedrorgirardi]

Hi @thheller! What do I need to get your proof of concept working? And what is the next thing that you would like to see working?

[thheller]

At this point it would be better to start from scratch. Removed half of the code and moved some things around. The proof-of-concept wasn't anything I would build something real on, it was just about the proof that it can work.

I still have so many other things on my todo list and LSP is very very far down. :/

[thheller]

Closing this as it won't be part of shadow-cljs if I'm going to implement something related.