Any plans to support version 63?

[MatthewSteeples]

Related #131

Steps to reproduce

1. Try opening https://storage.googleapis.com/dx-nhs-production/DATOPIAN_DATA/EPD_202103.zip (warning: file is over 550mb) using ZipFile and iterating through ZipEntry records

Expected behavior

File loads as per usual

Actual behavior

ICSharpCode.SharpZipLib.Zip.ZipException: 'Version required to extract this entry not supported (63)' thrown when trying to read the first ZipEntry

Version of SharpZipLib

1.3.2

Obtained from (only keep the relevant lines)

• Package installed using NuGet

The place where I've sourced this file from usually publishes them as version 45 but for some reason this month it was version 63. I've extracted and re-compressed it locally using 7zip so I can still process it using SharpZipLib, and I've contacted the provider asking if this change is a permanent one or whether they can continue publishing using v45 but I thought it was worth raising here too.

Sorry I can't provide a smaller file, but I was unable to find the option in 7zip to instruct it to create a v63 file

[piksel]

6.3 is LZMA according to APPNOTE. There have been questions about this earlier, but no ongoing work afaik.

I have experimented a little with adding support for "raw" decompression where the data is treated as if it was uncompressed, that way you can plugin in your own decompression code. That way you could use something like https://github.com/weltkante/managed-lzma to just handle the stream decompression.

[piksel]

Actually... It's not actually using LZMA at all, it just has Version needed to extract set too high...
https://pub.p1k.se/sharpziplib/archivediag/EPD_202103.zip.html

The uncompressed size is 6339300445 byte(s) (6GiB 😨), which might be a problem though...

[MatthewSteeples]

@piksel The LZMA thing confused me as well (when I was trying to re-create a smaller example).

As for the size, handles it no problem (main reason I'm using SharpZipLib is because it supports streaming the data through rather than having to load it all into memory, so a single pass on the CSV contents is possible just reading it "line by line"). The code I've got is working fine once I've re-compressed the zip file (using 7zip). I'm hoping it's just a blip and they'll be back to 45 next month.

This is what 7zip shows me from (top to bottom)

1. Most recent zip file, unmodified
2. Most recent zip file, recompressed to v45
3. Last month's zip file. unmodified

2. and 3) both process successfully through SharpZipLib

[piksel]

As this wasn't actually about supporting 6.3/LZMA I am closing this.