Flow diagram/documentation of the components

[yyoncho]

I am working on https://github.com/emacs-lsp/lsp-java which is Emacs LSP client for Eclipse JDT LS and I want to provide integration with sts. I am using vscode extension as a guide to figure out how to do the integration but I wonder whether there is some documentation/flow diagram on how each of the components communicates with each other which could save me some time reading and debugging the code.

[martinlippert]

Great to hear that you want to integrate the Spring Tools via the language server protocol into your Emacs LSP support. I think the setup for the basic Spring Boot language server that we have (in case you are looking for that and not the Cloud Foundry manifest file support, for example) is straight forward with regards to the overall language server itself.

The tricky part is indeed the integration with the Java Language Server, which is done via the general command mechanism of the LSP, but via an additional extension mechanism that the Java Language Server provides. As you can see here:

sts4/vscode-extensions/vscode-spring-boot/package.json

Lines 34 to 36 in 4cb796c

	"javaExtensions": [
	"./jars/jdt-ls-commons.jar",
	"./jars/jdt-ls-extension.jar"

We contribute two extensions to the Java Language Server. Once the Java LS finds those extensions, it loads them and they communicate via commands. In addition to that we also have a client-side-specific extension that adds specific command handling for the classpath lookup that you can find here: https://github.com/spring-projects/sts4/blob/master/vscode-extensions/commons-vscode/src/classpath.ts

Are you looking for something specific? Anything you are stuck with?

[yyoncho]

Thanks for the detailed explanation!
To be more specific I am interested in porting https://github.com/spring-projects/sts4/tree/master/vscode-extensions/vscode-spring-boot .
What bothers me the most is classpath.ts. Here it is my understanding of what is happening:

Both JDT and Spring Boot LS are running in parallel and the client (VSCode) is calling both at the same time(e. g. when doing document/onHover). The classpath.ts code is bridging classpath information from JDT to Spring Boot server.

[martinlippert]

Yes, that is correct. Maybe @kdvolder can provide more details on this, if needed.

[kdvolder]

@yyoncho It sounds like you got the gist of it.

We contribute an extension to the JDT language server via its extension mechanism. The implementation of that extension is here: https://github.com/spring-projects/sts4/tree/master/headless-services/jdt-ls-extension

The extension consists out of two osgi bundles / eclipse plugins, because the JDT language server is really a 'small slice of Eclipse' under the hood and its extension mechanics works by adding extra osgi bundles to its runtime.

Once the bundles have been built they are contributed to the JDT language server by adding some specific lines to our vscode package.json like this:

  "contributes": {
    "javaExtensions": [
      "./jars/jdt-ls-commons.jar",
      "./jars/jdt-ls-extension.jar"
    ],

See here: https://github.com/spring-projects/sts4/blob/master/vscode-extensions/vscode-spring-boot/package.json#L33

Don't know the exact details of how this works on the jdt side of things, but when you add these lines, then the redhat JDT LS will somehow find them and add them to its osgi runtime.

Inside our extension there are some xml bits (using Eclipse style extension points that are defined by the jdt language server). These are here: https://github.com/spring-projects/sts4/blob/master/headless-services/jdt-ls-extension/org.springframework.tooling.jdt.ls.extension/plugin.xml

THe jdt language server will read those bits and they are telling it to register commands sts.java.addClasspathListener and sts.java.removeClasspathListener (i.e. the operations that you found in classpath.ts.

These commands are registered by vscode language server client so that any vscode extension can call then using vscode command api. It is this command api that we use to be able to communicate with our jdt.ls.extension to listen for 'interesting' classpath changes inside the jdt language server process.

There is one more 'twist' to this story. It is not currently possible using the available standard LSP protocol for a language server to execute commands on the client. So to make that possible we had to add some custom protocol to allow our jdt.ls extension to execute vscode commands so we can invoke the 'callbacks' to send information whenever classpaths are changed. This bit was done via a PR on the redhat vscode-java code base, which they were kind enough to accept. This can be found here:

redhat-developer/vscode-java#476

I think that covers the whole mess of it, at least as far as I can remember it all. Let me know if you have more questions.

And yes, this is horribly complicated and we are not happy about that, but don't really know of a better way to make this all work.

I'm going to close this ticket now, but feel free to continue using it for asking questions if you want more help.

[yyoncho]

Thanks! On Emacs side, we will have to change our code to support more than one LSP server per file, after that, I will proceed with integration and eventually ping you to list the Emacs integration in your docs.

[yyoncho]

@kdvolder @martinlippert
I finally got to integrating STS4 to emacs and after some trial/error/debugging got it working. Can you advise on what is the best way to consume the STS4 java binaries? At this point, we are consuming JDT LS from https://download.eclipse.org/jdtls/snapshots/?d while https://github.com/Microsoft/java-debug is downloaded from maven central.

[martinlippert]

We don't really publish those bits officially to one of those channels, but you can grab the latest spring-boot-language-server, for example, from here:

https://dist.springsource.com/release/STS4/fatjars/spring-boot-language-server-1.2.0-201812201920.jar

Does that help?

[yyoncho]

@martinlippert it will work for us as long as we can find all versions of the artifact (or at least the latest one). Also, we will need the JDT plugin bundles.

[martinlippert]

@yyoncho Regarding the JDT extension JARs, we don't really publish them anywhere at the moment, as far as I know. You can, of course, find them inside of the vsix packages that we publish for VSCode, but I am not sure if that fits your needs.

@kdvolder Do you know a good way for them to consume those JAR files?

[yyoncho]

@martinlippert

You can, of course, find them inside of the vsix packages that we publish for VSCode, but I am not sure if that fits your needs.

If publishing the artifacts is an issue for you I will probably start consuming the vsix file or I can build the artifacts and attach them as a deliveries in lsp-java and update them when you do a release in STS.

[kdvolder]

Do you know a good way for them to consume those JAR files?

I think the best, most standard way would be that we somehow publish them to a maven repository. I'm sure this is possible somehow. It's just that, because we come from a different world of osgi/p2/eclipse we've never done this. So it might take us some time and releng effort to set that up, and at the moment there's nothing like that in our builds.

Other than that, I think extracting them from released .vsix file is probably the easiest path.

[d4ncer]

@yyoncho do you have a working example of integrating STS with lsp-java? I'd like to use it to work with Concourse files, and any help would be greatly appreciated :) Thanks in advance!

[yyoncho]

@d4ncer from what I can see Concourse is standalone language server so no integration with lsp-java is needed but only a configuration which starts this server (like what lsp-clients.el has) for the Concourse files.

[d4ncer]

Ah, I should have looked a little closer before posting; thanks @yyoncho! Will add a PR to lsp-mode for Concourse once I get a chance to play around with it.

[yyoncho]

@d4ncer no problem, for the record, I got most of the spring boot working, I had to do the installation and polishing.

[martinlippert]

@yyoncho great news, awesome to hear that. please share a demo screencast recording or something like that once you have a polished version, would love to see it in action and tweet about it... :-)

[yyoncho]

@martinlippert The initial support is now in - here it is random screenshot how it looks like:

https://github.com/emacs-lsp/lsp-java#spring-boot-support-experimental - here it is the documentation section - I have pretty much coppied the VScode readme and I have replaced the screenshots with the Emacs one.

[martinlippert]

this is super awesome to see... Congratulations!!!

One minor issue: the screenshots for "Quick-access for running apps" and "Navigating the source code" are interchanged.

[yyoncho]

@martinlippert thanks, will be fixed. I will create a screencast recording when I got some free time and I will post it in here.