including the JAR files cause our project to fail. Bad MANIFEST.MFs?

[fredg1]

We are working on a project built with ANT and I am trying to add a Language Server feature to it.

If I add the lsp4j source code of this repository (including the dependencies) to the project, everything is fine.
I can launch the project from my IDE (vscode), and building the project produces a .jar that can be ran properly.

However

The project has a "no 3rd party source code" policy (i.e. 3rd party stuff must be imported as a .jar) (which comes from, and I quote, "years of sheer pain and frustration" with them).

If I try to add lsp4j's .jar(s), I can still run the project just fine from vscode, but when I build, the resulting build fails to run (with java -jar <filename>) with the error

Error: Could not find or load main class <main class>
Caused by: java.lang.ClassNotFoundException: <main class>

The building process doesn't get interrupted, and looking at it, nothing goes wrong during it. Only the resulting .jar has a defect.
There's not even any reference to lsp4j yet. The only change is its inclusion in the libraries.

I tried to isolate the issue by going step-by-step

1. removed everything (to make sure I didn't inadvertently change anything) and built. The build runs.
2. added gson-2.8.6.jar to the libraries (the only dependency of lsp4j.jsonrpc) and built. The build runs.
3. added https://github.com/eclipse/lsp4j/tree/master/org.eclipse.lsp4j.jsonrpc/src/main/java (which has only a single dependency, gson-2.8.6.jar) to our project and built. The build runs.
4. Undid step 3.
5. used https://jar-download.com/artifacts/org.eclipse.lsp4j to download and add org.eclipse.lsp4j.jsonrpc-0.12.0.jar to our project and built. The build produces a .jar that does not run.
6. undid step 5.
7. used https://oss.sonatype.org/content/repositories/snapshots/org/eclipse/lsp4j/org.eclipse.lsp4j.jsonrpc/0.13.0-SNAPSHOT/ to download org.eclipse.lsp4j.jsonrpc-0.13.0-20210421.133210-5.jar, which is supposed to be the exact same files as step 3, and built. The build produces a .jar that does not run.

Someone with more knowledge of java than me guessed that the cause would lie in the jars' MANIFEST.MF

Can you take a look, please?

[jonahgraham]

Hi @fredg1 - I had a look at the manifest, and I don't know what in it could cause the problem. Can you create a minimal example and attach it, or upload to github and I can try that?

The ClassNotFoundException - when it is clearly there - normally means that some static initialisation or classloader failed while trying to create the class. Is there a nested exception that provides more detail? Have you tried to step through the code (attach to the process run externally from vscode)?

[fredg1]

test.zip
@jonahgraham here's a minimal example.
Just double-click on the ant.bat to create the .jar (requires http://ant.apache.org/ installed and on your $PATH)

The jar won't work, but go in src/jar, remove the org.eclipse.lsp4j.jsonrpc-0.13.0-20210421.133210-5.jar (then build again), and suddenly, it works! (then add it back and it breaks again)

Is there a nested exception that provides more detail?

Nope. The error is literally just that. Adding --stacktrace doesn't give anything more either.
This means that adding the jars somehow makes the native/local classes unreachable... (I tried using -cp to go to other classes. Nothing is found.)

Have you tried to step through the code (attach to the process run externally from vscode)?

tried java -agentlib:jdwp=transport=dt_socket -jar <path>. Nothing new (unsurprising, since based on the stack trace, this happens at top level)

[jonahgraham]

Thanks @fredg1 I can reproduce on Linux with your zip, but using ant directly and not using ant.bat. I will try and see what I can figure out here and report back.

[cdietrich]

@jonahgraham @fredg1 the problem is the signatures in the META-INF folder copied over.

jarsigner -verify -verbose test.jar

[jonahgraham]

+1 I had just come to the same conclusion - @fredg1 see https://stackoverflow.com/questions/999489/invalid-signature-file-when-attempting-to-run-a-jar for some suggestions, you can hopefully adapt them to ant - its too bad that java didn't just say that :-)

with output:

$ jarsigner -verify -verbose test.jar
jarsigner: java.lang.SecurityException: Invalid signature file digest for Manifest main attributes

This is not something to change in LSP4J so I am closing this issue.

[fredg1]

Well... sorry for disturbing you, and thanks for your time

[jonahgraham]

I'm glad that we were able to help. I hope your LSP feature integration goes well!

[fredg1]

I wasn't sure if this was going to get solved, so meanwhile, I was trying to compile and build from source myself. Issue is, Xtext wasn't really working, so instead, I was manually converting eve rything from Xtext to Java

It... uh... was taking quite some time, and I'm quite glad I won't have to go all the way 😨

[cdietrich]

what problems did you face with Xtend?
as we use active annotations doing it (convert to java) manually will be pita

[fredg1]

(Keep in mind, I know nothing about Xtext. All I know is from contextual clues from looking at the files)

I don't quite remember how it started, but if I recall correctly, there was some internal class that wasn't loading, which when looking it up, I saw that you had to upgrade your gradle version to fix it...

However, the version change meant that some features were now no longer supported. First, I had to transition from the osgi plugin to biz.aQute.bnd.builder, implementation and testImplementation instead of compile and test, respectively, and use maven-publish instead of maven.

Now, it's stuck at saying

- Type 'org.xtext.gradle.tasks.XtextGenerate' property 'bootstrapClasspath' has @input annotation used on property of type 'FileCollection'.
- Type 'org.xtext.gradle.tasks.XtextGenerate' property 'languages' is missing an input or output annotation.
- Type 'org.xtext.gradle.tasks.XtextGenerate' property 'sourceSetOutputs' is missing an input or output annotation.
- Type 'org.xtext.gradle.tasks.XtextGenerate' property 'sources' is missing an input or output annotation.

And I have no idea what it's even referring to

[fredg1]

by the way, how is it that this problem hasn't come up before? Looking at your link, it seems like this issue is caused by adding a signed jar (a jar with .RSA, .DSA and/or .SF files in META-INF/) as a dependency, and doesn't care about the building method.

But then, wouldn't there have been more instances of inexperienced people adding it as a dependency and getting this problem?

And for the future, should there be a note/warning about this somewhere in this repository?

[jonahgraham]

@fredg1 as a newcomer your perspective is very valuable. Can you write something and submit a PR in a location that you may have found the information?

[cdietrich]

the problem basically affects all signed jars and is not lsp4j specific.