SnappyLoader.loadNativeLibrary may fail due to bugs in URLClassLoader, we should not rely on getResourceAsStream

[jizhilong]

The following IOException happens occasionally in our environment.

java.io.IOException: Stream closed
	at java.util.zip.InflaterInputStream.ensureOpen(InflaterInputStream.java:67)
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:142)
	at java.io.FilterInputStream.read(FilterInputStream.java:133)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
	at org.xerial.snappy.SnappyLoader.contentsEquals(SnappyLoader.java:209)
	at org.xerial.snappy.SnappyLoader.extractLibraryFile(SnappyLoader.java:278)
	at org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:355)
	at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:176)
	at org.xerial.snappy.SnappyLoader.loadSnappyApi(SnappyLoader.java:154)
	at org.xerial.snappy.Snappy.<clinit>(Snappy.java:47)
	at org.xerial.snappy.SnappyInputStream.hasNextChunk(SnappyInputStream.java:435)
	at org.xerial.snappy.SnappyInputStream.read(SnappyInputStream.java:466)
	at java.io.DataInputStream.readByte(DataInputStream.java:265)
	at org.apache.kafka.common.utils.ByteUtils.readVarint(ByteUtils.java:168)
	at org.apache.kafka.common.record.DefaultRecord.readFrom(DefaultRecord.java:292)
	at org.apache.kafka.common.record.DefaultRecordBatch$1.readNext(DefaultRecordBatch.java:264)
	at org.apache.kafka.common.record.DefaultRecordBatch$RecordIterator.next(DefaultRecordBatch.java:563)
	at org.apache.kafka.common.record.DefaultRecordBatch$RecordIterator.next(DefaultRecordBatch.java:532)
	at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.nextFetchedRecord(Fetcher.java:1060)
	at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.fetchRecords(Fetcher.java:1095)
	at org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.access$1200(Fetcher.java:949)
	at org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:570)
	at org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:531)
	at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1170)
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1103)
	at io.opentracing.contrib.kafka.TracingKafkaConsumer.poll(TracingKafkaConsumer.java:103)
	at com.leyantech.chaos.kafka.AbstractKafkaConsumerClient.lambda$poll$11(AbstractKafkaConsumerClient.java:512)
	at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
	at com.github.rholder.retry.Retryer.call(Retryer.java:160)
	at com.leyantech.chaos.kafka.AbstractKafkaConsumerClient.poll(AbstractKafkaConsumerClient.java:510)
	at com.leyantech.chaos.kafka.AbstractKafkaConsumerClient.lambda$start$3(AbstractKafkaConsumerClient.java:277)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)

According to the googled result, the exception may be caused by an open JDK BUG JDK-8205976, and JDK-8205976 is also causing issues in Android Gradle Plugin: https://issuetracker.google.com/issues/137929327?pli=1

So I propose to replace SnappyLoader.class.getResourceAsStream calls in SnappyLoader.java with the following way:

private static InputStream getResourceInputStream(String resourcePath) throws IOException {
    URL url = SnappyLoader.class.getResource(resourcePath);
    URLConnection connection = url.openConnection();
    if (connection instanceof JarURLConnection) {
      JarURLConnection jarConnection = (JarURLConnection) connection;
      jarConnection.setUseCaches(false);  // workaround for https://bugs.openjdk.org/browse/JDK-8205976
      return jarConnection.getInputStream();
    } else {
      return connection.getInputStream();
    }
  }

[xerial]

The fix is merged and will be available in the next version snappy-java 1.1.10.0