Error while processing

[floyd-fuh]

Hi there,

I was trying ahead of time instrumentation for Apache tika, but it threw a lot of errors at me:

$ java -jar java-afl-instrument.jar instrumented/ tika-app-1.18.jar
[...]
rror while processing tika-app-1.18.jar/com/google/common/collect/Synchronized.class: Type com/google/common/collect/Synchronized$SynchronizedRandomAccessList not present
Error while processing tika-app-1.18.jar/com/google/common/collect/Tables.class: Type com/google/common/collect/Table not present
Error while processing tika-app-1.18.jar/com/google/common/collect/TreeRangeSet.class: Type com/google/common/collect/TreeRangeSet$AsRanges not present
Error while processing tika-app-1.18.jar/com/google/common/eventbus/AnnotatedSubscriberFinder.class: Type com/google/common/eventbus/EventSubscriber not present
Error while processing tika-app-1.18.jar/com/google/common/io/BaseEncoding$StandardBaseEncoding.class: Type com/google/common/io/BaseEncoding$StandardBaseEncoding not present
Error while processing tika-app-1.18.jar/com/google/common/io/Closer.class: Type com/google/common/io/Closer$SuppressingSuppressor not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/CycleDetectingLockFactory$PotentialDeadlockException.class: Type com/google/common/util/concurrent/CycleDetectingLockFactory$ExampleStackTrace not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/CycleDetectingLockFactory$WithExplicitOrdering.class: Type com/google/common/util/concurrent/CycleDetectingLockFactory$CycleDetectingReentrantLock not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/CycleDetectingLockFactory.class: Type com/google/common/util/concurrent/CycleDetectingLockFactory$CycleDetectingReentrantLock not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/ListenerCallQueue.class: Type com/google/common/util/concurrent/ListenerCallQueue not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/MoreExecutors.class: Type com/google/common/util/concurrent/MoreExecutors$ScheduledListeningDecorator not present
Error while processing tika-app-1.18.jar/com/google/common/util/concurrent/Striped.class: Type com/google/common/util/concurrent/Striped$SmallLazyStriped not present
Error while processing tika-app-1.18.jar/org/apache/tika/xmp/XMPMetadata.class: Type com/adobe/xmp/XMPException not present
Error while processing tika-app-1.18.jar/org/apache/tika/xmp/convert/TikaToXMP.class: Type org/apache/tika/xmp/convert/ITikaToXMPConverter not present
Error while processing tika-app-1.18.jar/com/adobe/xmp/impl/XMPMetaImpl.class: Type com/adobe/xmp/XMPException not present
Error while processing tika-app-1.18.jar/com/adobe/xmp/impl/XMPNode.class: Type com/adobe/xmp/XMPException not present
Error while processing tika-app-1.18.jar/com/adobe/xmp/impl/XMPNormalizer.class: Type com/adobe/xmp/XMPException not present
Error while processing tika-app-1.18.jar/com/adobe/xmp/impl/XMPUtilsImpl.class: Type com/adobe/xmp/impl/XMPNode not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/BatchProcess.class: Type org/apache/tika/batch/IFileProcessorFutureResult not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/FileResourceConsumer.class: Type org/apache/tika/batch/FileResource not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/builders/BatchProcessBuilder.class: Type org/apache/tika/exception/TikaException not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/builders/CommandLineParserBuilder.class: Type org/apache/tika/exception/TikaException not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/fs/FSBatchProcessCLI.class: Type org/apache/tika/batch/fs/FSBatchProcessCLI not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/fs/FSOutputStreamFactory.class: Type org/apache/commons/compress/compressors/bzip2/BZip2CompressorOutputStream not present
Error while processing tika-app-1.18.jar/org/apache/tika/batch/fs/builders/FSCrawlerBuilder.class: Type org/apache/tika/batch/fs/FSListCrawler not present
Error while processing tika-app-1.18.jar/org/slf4j/impl/Log4jLoggerFactory.class: Type org/slf4j/impl/Log4jLoggerAdapter not present
Error while processing tika-app-1.18.jar/org/slf4j/impl/StaticLoggerBinder.class: Type org/apache/log4j/Level not present
Error while processing tika-app-1.18.jar/org/apache/log4j/chainsaw/Main.class: Type org/apache/log4j/chainsaw/LoadXMLAction not present
Error while processing tika-app-1.18.jar/org/apache/log4j/helpers/OptionConverter.class: Type org/apache/log4j/spi/Configurator not present
Error while processing tika-app-1.18.jar/org/apache/log4j/helpers/PatternParser.class: Type org/apache/log4j/helpers/PatternParser$LocationPatternConverter not present
Error while processing tika-app-1.18.jar/org/apache/log4j/jmx/AppenderDynamicMBean.class: Type org/apache/log4j/jmx/LayoutDynamicMBean not present
Error while processing tika-app-1.18.jar/org/apache/log4j/jmx/HierarchyDynamicMBean.class: Type org/apache/log4j/jmx/LoggerDynamicMBean not present
Error while processing tika-app-1.18.jar/org/apache/log4j/jmx/LayoutDynamicMBean.class: Type org/apache/log4j/Level not present
Error while processing tika-app-1.18.jar/org/apache/log4j/jmx/LoggerDynamicMBean.class: Type org/apache/log4j/jmx/AppenderDynamicMBean not present
Error while processing tika-app-1.18.jar/org/apache/log4j/lf5/viewer/LogBrokerMonitor.class: Type org/apache/log4j/lf5/viewer/LogFactor5ErrorDialog not present
Error while processing tika-app-1.18.jar/org/apache/log4j/lf5/viewer/configure/ConfigurationManager.class: Type org/apache/log4j/lf5/LogLevelFormatException not present
Error while processing tika-app-1.18.jar/org/apache/log4j/net/JMSAppender.class: Type javax/jms/JMSException not present
Error while processing tika-app-1.18.jar/org/apache/log4j/net/JMSSink.class: Type javax/jms/JMSException not present
Error while processing tika-app-1.18.jar/org/apache/log4j/net/SMTPAppender.class: Type javax/mail/MessagingException not present
[...]
Error while processing tika-app-1.18.jar/org/apache/commons/io/FileUtils.class: Type org/apache/commons/io/filefilter/IOFileFilter not present
Output files are available at /opt/java-afl/instrumented
Instrumented 0 .class files and 1 .jar files with 1120718 locations.

Are these fatal errors? Did the instrumentation work at all? Are these just warnings?

[Barro]

They are technically warnings. This is a limitation with the libraries that this tool depends on.

[Barro]

They are technically warnings. This is a limitation with the libraries that this tool depends on.

[floyd-fuh]

Ok. Can we turn these into warnings then?

But I doubt they are warnings. Because afterwards when I try to fuzz, it won't find org/apache/tika/exception/TikaException or no instrumentation is detected:

$ ./java-afl-showmap -m 2000 -o out.txt -- java -cp instrumented/tika-app-1.18.jar:instrumented/ SilentDriver /opt/tika-corpus/206.tiff 
afl-showmap 2.52b by <lcamtuf@google.com>
[*] Executing '/usr/bin/java'...

-- Program output begins --
May 29, 2018 11:23:07 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: J2KImageReader not loaded. JPEG2000 files will not be processed.
See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io
for optional dependencies.

May 29, 2018 11:23:07 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: org.xerial's sqlite-jdbc is not loaded.
Please provide the jar on your classpath to parse sqlite files.
See tika-parsers/pom.xml for the correct version.
-- Program output ends --

[-] PROGRAM ABORT : No instrumentation detected
         Location : main(), afl-showmap.c:773

Can we fix this? Did you try running Tika with java-afl?

[floyd-fuh]

Ok. Can we turn these into warnings then?

But I doubt they are warnings. Because afterwards when I try to fuzz, it won't find org/apache/tika/exception/TikaException or no instrumentation is detected:

$ ./java-afl-showmap -m 2000 -o out.txt -- java -cp instrumented/tika-app-1.18.jar:instrumented/ SilentDriver /opt/tika-corpus/206.tiff 
afl-showmap 2.52b by <lcamtuf@google.com>
[*] Executing '/usr/bin/java'...

-- Program output begins --
May 29, 2018 11:23:07 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: J2KImageReader not loaded. JPEG2000 files will not be processed.
See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io
for optional dependencies.

May 29, 2018 11:23:07 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: org.xerial's sqlite-jdbc is not loaded.
Please provide the jar on your classpath to parse sqlite files.
See tika-parsers/pom.xml for the correct version.
-- Program output ends --

[-] PROGRAM ABORT : No instrumentation detected
         Location : main(), afl-showmap.c:773

Can we fix this? Did you try running Tika with java-afl?