java.io.IOException: Wrong magic bytes of ... #497

BrainStone · 2017-11-09T01:59:22Z

Hi. When running spotbugs on my test classes through gradle I get the following error:

> Task :spotbugsTest
Scanning archives (33 / 33)
2 analysis passes to perform
Pass 1: Analyzing classes (96 / 96) - 100% complete
Pass 2: Analyzing classes (7 / 7) - 100% complete
Done with analysis
The following errors occurred during analysis:
  Cannot open codebase filesystem:C:\Users\Yannick\Project-Creative\Arrestia\GitLab-Telegram-Bot\build\classes\java\test\META-INF\org\apache\logging\log4j\core\config\plugins\Log4j2Plugins.dat
    java.io.IOException: Wrong magic bytes of 1 for zip file C:\Users\Yannick\Project-Creative\Arrestia\GitLab-Telegram-Bot\build\classes\java\test\META-INF\org\apache\logging\log4j\core\config\plugins\Log4j2Plugins.dat of 108 bytes
      At edu.umd.cs.findbugs.classfile.impl.ZipFileCodeBase.<init>(ZipFileCodeBase.java:87)
      At edu.umd.cs.findbugs.classfile.impl.ZipCodeBaseFactory.makeZipCodeBase(ZipCodeBaseFactory.java:39)
      At edu.umd.cs.findbugs.classfile.impl.ClassFactory.createFilesystemCodeBase(ClassFactory.java:123)
      At edu.umd.cs.findbugs.classfile.impl.FilesystemCodeBaseLocator.openCodeBase(FilesystemCodeBaseLocator.java:77)
      At edu.umd.cs.findbugs.classfile.impl.ClassPathBuilder.processWorkList(ClassPathBuilder.java:620)
      At edu.umd.cs.findbugs.classfile.impl.ClassPathBuilder.build(ClassPathBuilder.java:230)
      At edu.umd.cs.findbugs.FindBugs2.buildClassPath(FindBugs2.java:674)
      At edu.umd.cs.findbugs.FindBugs2.execute(FindBugs2.java:215)
      At com.github.spotbugs.internal.spotbugs.SpotBugsExecuter.runSpotbugs(SpotBugsExecuter.java:23)
      At sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      At sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      At sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      At java.lang.reflect.Method.invoke(Method.java:498)
      At org.gradle.process.internal.worker.request.WorkerAction.run(WorkerAction.java:100)
      At org.gradle.process.internal.worker.request.WorkerAction.runThenStop(WorkerAction.java:83)
      At sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      At sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      At sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      At java.lang.reflect.Method.invoke(Method.java:498)
      At org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
      At org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
      At org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:146)
      At org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:128)
      At org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
      At org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
      At org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
      At java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      At java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      At org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
      At java.lang.Thread.run(Thread.java:748)

The build then fails because of this.

I believe it is caused by the log4j2 plugin I have in my test code to capture log output.

A possible solution could be silently ignoring these errors if it is not a class file or not even reading non-class files.

findbugsproject/findbugs#157 seems to be related.

The text was updated successfully, but these errors were encountered:

KengoTODA · 2017-11-13T01:51:51Z

Hello, could you let us know which version of SpotBugs you're using?
If you aren't sure, please tell me the version of SpotBugs Gradle Plugin.

BrainStone · 2017-11-13T02:30:06Z

I’m using version 3.1.0 (manually set the version)

BrainStone · 2017-11-17T09:51:10Z

@KengoTODA, anyone?

iloveeclipse · 2017-11-17T09:53:07Z

Please remove Log4j2Plugins.dat from auxilarry class path. This is not a zip / jar file which need to be there.

BrainStone · 2017-11-17T10:01:30Z

The compilation process automatically generates this file.

Yet it might still be a good idea to not look for class files in the META-INF directory.

iloveeclipse · 2017-11-17T10:06:47Z

Yet it might still be a good idea to not look for class files in the META-INF directory.

We look into MANIFEST.MF and read libraries specified by the class path attribute. So can you please check which MANIFEST.MF provides the offending entry?

BrainStone · 2017-11-17T10:21:14Z

There is no such file. However during the compilation process log4j2 creates the dat file in the classes dir.

If it helps, I'll create a minimal test repo.

AdamRoberts · 2017-12-14T17:05:10Z

I am running into an identical issue for a different file type. We are using the Apache XMLBeans project which creates ".xsb" binary files in the classpath that is required for the library to function. When Spotbugs attempts to scan these files it logs a stacktrace and since there are a large number of these files it causes our build logs to be unmanageable.

There needs to be some type of mechanism for telling SpotBugs to not scan certain file types, or just have it only scan ".class" files.

iloveeclipse · 2017-12-18T13:55:25Z

Probably staying with ".class" files only would be the best way. Anyone wants to contribute a fix?

briehman · 2018-01-11T13:07:27Z

I tried to look into making a patch for this but wasn't sure of the best way to proceed. The method edu.umd.cs.findbugs.classfile.impl.ClassFactory#createFilesystemCodeBase assumes that any non-class file is a zip to be inspected. This is reasonable to be able to inspect .jar, .war, .ear, etc files and process their manifests.

I am not sure of a good solution to proceed. I considered both an allowed or rejected list of suffixes, but both suffer from the same problem of potentially being incomplete and unintentionally excluding files from processing.

I'd be happy to contribute if anybody has an idea on how to proceed.

iloveeclipse · 2018-01-11T13:25:24Z

@briehman : I guess you mean the last "if" block in the ClassFactory

} else {
       return ZipCodeBaseFactory.makeZipCodeBase(codeBaseLocator, file);
}

The code above is very optimistic :-)
But I also agree that a file suffixes list will always be incomplete.

I think in the last if block above we can afford to read few bytes from the beginning of the file to check if this has a valid zip header (see for example https://en.wikipedia.org/wiki/List_of_file_signatures). If not, I would log a warning and return some dummy "empty" IScannableCodeBase instance.

sewe · 2018-01-11T15:51:42Z

I think in the last if block above we can afford to read few bytes from the beginning of the file to check if this has a valid zip header (see for example https://en.wikipedia.org/wiki/List_of_file_signatures). If not, I would log a warning and return some dummy "empty" IScannableCodeBase instance.

That seems reasonable. I didn’t find a ready-made utility function, but java.util.zip.ZipConstants.LOCSIG seems to be the magic number you are after.

briehman · 2018-01-11T19:40:27Z

Performing the magic number check seems less resilient than letting the java.util.zip package handle the parsing. Taking a step back, it seems like the intention is to mimic how the JVM handles the classpath such that invalid classpath entries do not cause errors. Spotbugs would log the problem, likely as a warning, and interpret it as an "empty" IScannableCodeBase. Does that seem appropriate?

If so, it seems reasonable to assume in the final else block that it is a zip file and if that should fail for any reason, the entry is mapped to an empty IScannableCodeBase and logged appropriately. Would that be sufficient?

I'm coming at this without prior context so let me know if I'm missing something important regarding why this might behave differently.

BrainStone · 2018-01-11T20:25:34Z

It doesn't even have to be a warning. Because there can be plenty of non class files in the classpath and that would be spammy.

Cinderhaze · 2018-01-11T22:00:06Z

As a user who is seeing this problem, I would love the ability to simply disable the output when it encounters this problem, echoing what @briehman mentioned above about it being potentially spammy. It may simply be a problem with how our source sets are configured in gradle, but we have .sh files showing up in the section that is getting scanned, and causing the exception/output listed above.

In our case, simply being able to disable the error output, or potentially pass in a list of file types to omit would solve our problem. Also, trying to act the the JVM classpath and silently ignore any errors with files would fit our needs as well.

iloveeclipse · 2018-02-24T16:41:58Z

If I get it right, #529 fixes this one, closing.

iloveeclipse added this to the SpotBugs 3.1.2 milestone Jan 11, 2018

iloveeclipse added the bug label Jan 11, 2018

KengoTODA mentioned this issue Jan 22, 2018

497: Do not print exceptions for unsupported classpath files #529

Merged

2 tasks

iloveeclipse closed this as completed Feb 24, 2018

iloveeclipse mentioned this issue Feb 25, 2018

java.io.IOException: Wrong magic bytes of 4a4d4820 for zip file #557

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

java.io.IOException: Wrong magic bytes of ... #497

java.io.IOException: Wrong magic bytes of ... #497

BrainStone commented Nov 9, 2017 •

edited

Loading

KengoTODA commented Nov 13, 2017 •

edited

Loading

BrainStone commented Nov 13, 2017

BrainStone commented Nov 17, 2017

iloveeclipse commented Nov 17, 2017

BrainStone commented Nov 17, 2017 •

edited

Loading

iloveeclipse commented Nov 17, 2017

BrainStone commented Nov 17, 2017

AdamRoberts commented Dec 14, 2017

iloveeclipse commented Dec 18, 2017

briehman commented Jan 11, 2018

iloveeclipse commented Jan 11, 2018

sewe commented Jan 11, 2018

briehman commented Jan 11, 2018

BrainStone commented Jan 11, 2018

Cinderhaze commented Jan 11, 2018

iloveeclipse commented Feb 24, 2018

java.io.IOException: Wrong magic bytes of ... #497

java.io.IOException: Wrong magic bytes of ... #497

Comments

BrainStone commented Nov 9, 2017 • edited Loading

KengoTODA commented Nov 13, 2017 • edited Loading

BrainStone commented Nov 13, 2017

BrainStone commented Nov 17, 2017

iloveeclipse commented Nov 17, 2017

BrainStone commented Nov 17, 2017 • edited Loading

iloveeclipse commented Nov 17, 2017

BrainStone commented Nov 17, 2017

AdamRoberts commented Dec 14, 2017

iloveeclipse commented Dec 18, 2017

briehman commented Jan 11, 2018

iloveeclipse commented Jan 11, 2018

sewe commented Jan 11, 2018

briehman commented Jan 11, 2018

BrainStone commented Jan 11, 2018

Cinderhaze commented Jan 11, 2018

iloveeclipse commented Feb 24, 2018

BrainStone commented Nov 9, 2017 •

edited

Loading

KengoTODA commented Nov 13, 2017 •

edited

Loading

BrainStone commented Nov 17, 2017 •

edited

Loading