Maven application--- No source code was seen and extracted

[bettyli037]

Hi all,

I am trying to generate codeql database for my Java application. But I got this error: no source code was seen and extracted. The building process is successful, however the codebase is not able to be generated.
The command i am using to generate the database is codeql database create codeql/dbs -l java -c 'mvn -f pom.xml clean install -DskipTests' --overwrite.

No src.zip was generated. Here is my db folder structure

root@67aa88c73a83:/codeql/dbs# ls
codeql-database.yml  log  trap	working

I found javac-output-.log that shows a lot of errors as this:

error: cannot access xxxxxxxxxxx
[2022-12-21 21:23:21] [javac-output-511] package xxxxxxxxxxxxx;
[2022-12-21 21:23:21] [javac-output-511] ^
[2022-12-21 21:23:21] [javac-output-511]   ZIP file can't be opened as a file system because entry "/." has a '.' or '..' element in its name

And as a result, a lot of symbols cannot be found. But obviously, the compiling process is successful, as I start from a clean repo without target folder. And after I run the above command, the target folder was formed and there are files exist there.

Please help me find out the problem.

Thank you so much!

Betty

[jketema]

Hi @bettyli037. Thanks you for your question. Without further details it's really difficult to tell what is going on. To start, Ccould you share the output produced by CodeQL?

[jketema]

Hi @bettyli037. Thanks for the additional details. Just to check: if you just run mvn -f pom.xml clean install -DskipTests on its own, does that work, or do you get the same errors?

[bettyli037]

Yes, that works. I found this error from the db logs generated by codeql. If I just run the mvn -f pom.xml clean install, it won’t produce any error. The building process is successful. Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Jeroen Ketema ***@***.***> Sent: Thursday, December 22, 2022 4:57:49 PM To: github/codeql ***@***.***> Cc: Jingyi Li ***@***.***>; Mention ***@***.***> Subject: Re: [github/codeql] Maven application--- No source code was seen and extracted (Discussion #11782) Hi @bettyli037<https://github.com/bettyli037>. Thanks for the additional details. Just to check: if you just run mvn -f pom.xml clean install -DskipTests on its own, does that work, or do you get the same errors? — Reply to this email directly, view it on GitHub<#11782 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUJ6LKVSRTYY5MRSJNW2OSTWOTFF3ANCNFSM6AAAAAATG6KZTQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[jketema]

Could you share the output codeql database create codeql/dbs -l java -c 'mvn -f pom.xml clean install -DskipTests' --overwrite produces on your terminal?

[bettyli037]

The output is this:

No source code was seen and extracted to /codeql/dbs.
This can occur if the specified build commands failed to compile or process any code.

• Confirm that there is some source code for the specified language in the project.
• For codebases written in Go, JavaScript, TypeScript, and Python, do not specify
  an explicit --command.
• For other languages, the --command must specify a "clean" build which compiles
  all the source code files without reusing existing build artefacts.

[jketema]

There will be quite a lot of output before this. That's what I'm interested in.

[smowton]

This error indicates that you're using a JAR file with a . or .. element in some path. For example, I can make such a jar using commands like

$ mkdir y
$ touch y/z.txt
$ jar cf test.jar y/.
$ jar tf test.jar 
META-INF/
META-INF/MANIFEST.MF
y/./
y/./z.txt

Then passing that jar on the classpath to javac will trigger the ZIP file can't be opened as a file system error you're seeing. Unfortunately it doesn't specify which jar file was problematic!

You have two options:

1. Build with a JDK new enough to have patch https://bugs.openjdk.org/browse/JDK-8286444 -- for example, 17.0.5 or 11.0.16. Then the error message will specify which jar is bad, and therefore which dependency you'll want to update, or if you're creating the jar locally, fix the jar creation step.
2. Build with a JDK old enough not to have patch https://bugs.openjdk.org/browse/JDK-8251329 -- for example, 17.0.1 or 11.0.13, or any version of JDK 8. These don't protect against jar files with . or .. in a filename at all, so the build will work (at the risk of course that the JAR really is malicious somehow, so it's better to use option 1 if practical).

[smowton]

Note by the way this should apply to any build, not just to building a CodeQL database. I would recommend starting by getting a build working under Actions independent of CodeQL, then bring those build steps over to your CodeQL run.

[jketema]

Note that I asked about a build without CodeQL above, and that seemed to have worked. Hence, me asking about more details about the build.

[smowton]

My guess is the test with just mvn -f pom.xml clean install -DskipTests without codeql involved may have been run in a different environment (local?) than CodeQL (actions?)

[bettyli037]

Okay. Thank you so much for your reply. I will try that way and let you know. Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Chris Smowton ***@***.***> Sent: Thursday, January 5, 2023 12:59:31 PM To: github/codeql ***@***.***> Cc: Jingyi Li ***@***.***>; Mention ***@***.***> Subject: Re: [github/codeql] Maven application--- No source code was seen and extracted (Discussion #11782) My guess is the test with just mvn -f pom.xml clean install -DskipTests without codeql involved may have been run in a different environment (local?) than CodeQL (actions?) — Reply to this email directly, view it on GitHub<#11782 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUJ6LKVTOOYMZVGLQKTKO5LWQ4DYHANCNFSM6AAAAAATG6KZTQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>