New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error while processing #1

Closed
floyd-fuh opened this Issue May 24, 2018 · 2 comments

Comments

2 participants
@floyd-fuh

floyd-fuh commented May 24, 2018

Hi there,

I was trying ahead of time instrumentation for Apache tika, but it threw a lot of errors at me:

$ java -jar java-afl-instrument.jar instrumented/ tika-app-1.18.jar
[...]
rror while processing tika-app-1.18.jar/com/google/common/collect/Synchronized.class: Type com/google/common/collect/Synchronized$SynchronizedRandomAccessList not present
Error while processing tika-app-1.18.jar/com/google/common/collect/Tables.class: Type com/google/common/collect/Table not present
Error while processing tika-app-1.18.jar/com/google/common/collect/TreeRangeSet.class: Type com/google/common/collect/TreeRangeSet$AsRanges not present
Error while processing tika-app-1.18.jar/com/google/common/eventbus/AnnotatedSubscriberFinder.class: Type com/google/common/eventbus/EventSubscriber not present
Error while processing tika-app-1.18.jar/com/google/common/io/BaseEncoding$StandardBaseEncoding.class: Type com/google/common/io/BaseEncoding$StandardBaseEncoding not present
Error while processing tika-app-1.18.jar/com/google/common/io/Closer.class: Type com/google/common/io/Closer$SuppressingSuppressor not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/CycleDetectingLockFactory$PotentialDeadlockException.class: Type com/google/common/util/concurrent/CycleDetectingLockFactory$ExampleStackTrace not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/CycleDetectingLockFactory$WithExplicitOrdering.class: Type com/google/common/util/concurrent/CycleDetectingLockFactory$CycleDetectingReentrantLock not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/CycleDetectingLockFactory.class: Type com/google/common/util/concurrent/CycleDetectingLockFactory$CycleDetectingReentrantLock not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/ListenerCallQueue.class: Type com/google/common/util/concurrent/ListenerCallQueue not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/MoreExecutors.class: Type com/google/common/util/concurrent/MoreExecutors$ScheduledListeningDecorator not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/Striped.class: Type com/google/common/util/concurrent/Striped$SmallLazyStriped not present
Error while processing tika-app-1.18.jar/org/apache/tika/xmp/XMPMetadata.class: Type com/adobe/xmp/XMPException not present
Error while processing tika-app-1.18.jar/org/apache/tika/xmp/convert/TikaToXMP.class: Type org/apache/tika/xmp/convert/ITikaToXMPConverter not present
Error while processing tika-app-1.18.jar/com/adobe/xmp/impl/XMPMetaImpl.class: Type com/adobe/xmp/XMPException not present
Error while processing tika-app-1.18.jar/com/adobe/xmp/impl/XMPNode.class: Type com/adobe/xmp/XMPException not present
Error while processing tika-app-1.18.jar/com/adobe/xmp/impl/XMPNormalizer.class: Type com/adobe/xmp/XMPException not present
Error while processing tika-app-1.18.jar/com/adobe/xmp/impl/XMPUtilsImpl.class: Type com/adobe/xmp/impl/XMPNode not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/BatchProcess.class: Type org/apache/tika/batch/IFileProcessorFutureResult not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/FileResourceConsumer.class: Type org/apache/tika/batch/FileResource not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/builders/BatchProcessBuilder.class: Type org/apache/tika/exception/TikaException not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/builders/CommandLineParserBuilder.class: Type org/apache/tika/exception/TikaException not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/fs/FSBatchProcessCLI.class: Type org/apache/tika/batch/fs/FSBatchProcessCLI not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/fs/FSOutputStreamFactory.class: Type org/apache/commons/compress/compressors/bzip2/BZip2CompressorOutputStream not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/fs/builders/FSCrawlerBuilder.class: Type org/apache/tika/batch/fs/FSListCrawler not present
Error while processing tika-app-1.18.jar/org/slf4j/impl/Log4jLoggerFactory.class: Type org/slf4j/impl/Log4jLoggerAdapter not present
Error while processing tika-app-1.18.jar/org/slf4j/impl/StaticLoggerBinder.class: Type org/apache/log4j/Level not present
Error while processing tika-app-1.18.jar/org/apache/log4j/chainsaw/Main.class: Type org/apache/log4j/chainsaw/LoadXMLAction not present
Error while processing tika-app-1.18.jar/org/apache/log4j/helpers/OptionConverter.class: Type org/apache/log4j/spi/Configurator not present
Error while processing tika-app-1.18.jar/org/apache/log4j/helpers/PatternParser.class: Type org/apache/log4j/helpers/PatternParser$LocationPatternConverter not present
Error while processing tika-app-1.18.jar/org/apache/log4j/jmx/AppenderDynamicMBean.class: Type org/apache/log4j/jmx/LayoutDynamicMBean not present
Error while processing tika-app-1.18.jar/org/apache/log4j/jmx/HierarchyDynamicMBean.class: Type org/apache/log4j/jmx/LoggerDynamicMBean not present
Error while processing tika-app-1.18.jar/org/apache/log4j/jmx/LayoutDynamicMBean.class: Type org/apache/log4j/Level not present
Error while processing tika-app-1.18.jar/org/apache/log4j/jmx/LoggerDynamicMBean.class: Type org/apache/log4j/jmx/AppenderDynamicMBean not present
Error while processing tika-app-1.18.jar/org/apache/log4j/lf5/viewer/LogBrokerMonitor.class: Type org/apache/log4j/lf5/viewer/LogFactor5ErrorDialog not present
Error while processing tika-app-1.18.jar/org/apache/log4j/lf5/viewer/configure/ConfigurationManager.class: Type org/apache/log4j/lf5/LogLevelFormatException not present
Error while processing tika-app-1.18.jar/org/apache/log4j/net/JMSAppender.class: Type javax/jms/JMSException not present
Error while processing tika-app-1.18.jar/org/apache/log4j/net/JMSSink.class: Type javax/jms/JMSException not present
Error while processing tika-app-1.18.jar/org/apache/log4j/net/SMTPAppender.class: Type javax/mail/MessagingException not present
[...]
Error while processing tika-app-1.18.jar/org/apache/commons/io/FileUtils.class: Type org/apache/commons/io/filefilter/IOFileFilter not present
Output files are available at /opt/java-afl/instrumented
Instrumented 0 .class files and 1 .jar files with 1120718 locations.

Are these fatal errors? Did the instrumentation work at all? Are these just warnings?

@Barro

This comment has been minimized.

Show comment
Hide comment
@Barro

Barro May 26, 2018

Owner

They are technically warnings. This is a limitation with the libraries that this tool depends on.

Owner

Barro commented May 26, 2018

They are technically warnings. This is a limitation with the libraries that this tool depends on.

@Barro Barro closed this May 26, 2018

@floyd-fuh

This comment has been minimized.

Show comment
Hide comment
@floyd-fuh

floyd-fuh May 29, 2018

Ok. Can we turn these into warnings then?

But I doubt they are warnings. Because afterwards when I try to fuzz, it won't find org/apache/tika/exception/TikaException or no instrumentation is detected:

$ ./java-afl-showmap -m 2000 -o out.txt -- java -cp instrumented/tika-app-1.18.jar:instrumented/ SilentDriver /opt/tika-corpus/206.tiff 
afl-showmap 2.52b by <lcamtuf@google.com>
[*] Executing '/usr/bin/java'...

-- Program output begins --
May 29, 2018 11:23:07 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: J2KImageReader not loaded. JPEG2000 files will not be processed.
See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io
for optional dependencies.

May 29, 2018 11:23:07 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: org.xerial's sqlite-jdbc is not loaded.
Please provide the jar on your classpath to parse sqlite files.
See tika-parsers/pom.xml for the correct version.
-- Program output ends --

[-] PROGRAM ABORT : No instrumentation detected
         Location : main(), afl-showmap.c:773

Can we fix this? Did you try running Tika with java-afl?

floyd-fuh commented May 29, 2018

Ok. Can we turn these into warnings then?

But I doubt they are warnings. Because afterwards when I try to fuzz, it won't find org/apache/tika/exception/TikaException or no instrumentation is detected:

$ ./java-afl-showmap -m 2000 -o out.txt -- java -cp instrumented/tika-app-1.18.jar:instrumented/ SilentDriver /opt/tika-corpus/206.tiff 
afl-showmap 2.52b by <lcamtuf@google.com>
[*] Executing '/usr/bin/java'...

-- Program output begins --
May 29, 2018 11:23:07 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: J2KImageReader not loaded. JPEG2000 files will not be processed.
See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io
for optional dependencies.

May 29, 2018 11:23:07 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: org.xerial's sqlite-jdbc is not loaded.
Please provide the jar on your classpath to parse sqlite files.
See tika-parsers/pom.xml for the correct version.
-- Program output ends --

[-] PROGRAM ABORT : No instrumentation detected
         Location : main(), afl-showmap.c:773

Can we fix this? Did you try running Tika with java-afl?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment