Language Server 2.0 #5419
Description
Language Server 1.0
The current version of the language server (LS) does not have an optimal performance. Back in the days it was built with the different requirements in mind, and can be implemented in a more optimal way.
Language Server 2.0
A new language server can be created with different requirements based on the experience of building the first version.
- Focus on performance. Performance should be the main requirement when designing a new LS.
- Switch to binary protocol. The text protocol is a legacy, and nothing prevents us to switching to more efficient binary format.
- Drop multi-user support. LS 1.0 can support multiple users but we've never used it this way. Removing this requirement should drastically simplify the logic. Experiments with y.js sharing has been made
- Switch to in-memory protocol between the LS and the Engine. In LS 1.0 we have to serialize-deserialize message every time it crosses the LS/Engine boundary. In-memory protocol will eliminate the overhead of serialization.
Use Virtual Threads
JDK21 is the first LTS version to provide support for virtual threads. Virtual threads are light to create and light to block. One can write sequential code and invoke it as effectively as reactive code. We should use one of the frameworks that provides support for virtual threads and simplify the current actor based system. Consider:
- Helidon Níma: https://www.youtube.com/watch?v=aYY04D4_OQA
- Micronaut: Virtual thread support micronaut-projects/micronaut-core#8180
- Quarkus: https://quarkus.io/guides/virtual-threads
Micronaut & Quarkus rely a lot on dependency injection. It may be impossible to use them without all the DI magic. Níma provides imperative API to build and handle the routes.
Distribute Language Server with the IDE
There is an effort to run language server as hosted Java. It'd be great if these two efforts emerged together.
Write Language Server in JavaScript or Enso
An alternative to above mentioned Java frameworks is to write language server in JavaScript and execute it by GraalVM. We cannot easily support all node.js modules, but we can support latest, greatest JavaScript specification and emulate essential node.js APIs (for example for handling websockets via WebSocketServer). Enso Engine is already running JavaScript (to process JSON), so running JavaScript for language server isn't going to add much overhead.
Alternatively the language server could be written in Enso itself. Using the same emulation of websockets we could handle the incoming and outgoing messages in plan Enso code. Using JavaScript or Enso comes with familiarity of the languages and also support the Distribute Language Server with the IDE goal - having it in JS or Enso will require less changes in IDE build system than writing the server in Java.