NotSupportedException for 7zip Archive using Ultra Compression

[nightblade9]

I zipped up a file using 7zip (file manager / GUI). I tried with both LZMA and LZMA2, using ultra compression. I can't seem to figure out how to extract it; I always get a NotSupportedException.

Here are the two snippets I tried with the archiving API, which supposedly works?

                // 1) Archive API? not supported exception.
                var archive = ArchiveFactory.Open(archivePath);
                archive.WriteToDirectory(outPath, options);

                // 2) Archive API: not supported exception
                var archive = ArchiveFactory.Open(archivePath);
                foreach (var entry in archive.Entries.Where(e => !e.IsDirectory))
                {
                    entry.WriteToDirectory(outPath, options);
                }

I think they fail with the same exception. Here's the stack trace for the second one:

Exception has occurred: CLR/System.NotSupportedException
An unhandled exception of type 'System.NotSupportedException' occurred in SharpCompress.dll: 'Specified method is not supported.'
   at SharpCompress.Compressors.LZMA.DecoderRegistry.CreateDecoderStream(CMethodId id, Stream[] inStreams, Byte[] info, IPasswordProvider pass, Int64 limit)
   at SharpCompress.Compressors.LZMA.DecoderStreamHelper.CreateDecoderStream(Stream inStream, Int64 startPos, Int64[] packSizes, CFolder folderInfo, IPasswordProvider pass)
   at SharpCompress.Common.SevenZip.ArchiveDatabase.GetFolderStream(Stream stream, CFolder folder, IPasswordProvider pw)
   at SharpCompress.Common.SevenZip.SevenZipFilePart.GetCompressedStream()
   at SharpCompress.Archives.SevenZip.SevenZipArchiveEntry.OpenEntryStream()
   at SharpCompress.Archives.IArchiveEntryExtensions.WriteTo(IArchiveEntry archiveEntry, Stream streamToWriteTo)
   at SharpCompress.Archives.IArchiveEntryExtensions.<>c__DisplayClass2_0.<WriteToFile>b__0(String x, FileMode fm)
   at SharpCompress.Common.ExtractionMethods.WriteEntryToFile(IEntry entry, String destinationFileName, ExtractionOptions options, Action`2 openAndWrite)
   at SharpCompress.Archives.IArchiveEntryExtensions.WriteToFile(IArchiveEntry entry, String destinationFileName, ExtractionOptions options)
   at SharpCompress.Common.ExtractionMethods.WriteEntryToDirectory(IEntry entry, String destinationDirectory, ExtractionOptions options, Action`2 write)
   at SharpCompress.Archives.IArchiveEntryExtensions.WriteToDirectory(IArchiveEntry entry, String destinationDirectory, ExtractionOptions options)

I'm sure it must be something I'm doing wrong (perhaps in my 7zip archive creation?) but I can't figure out what or how to make this work.

[adamhathcock]

It's likely hitting the default case in the below switch statement that means it's an implemented method of compression

switch (id._id)
            {
                case K_COPY:
                    if (info != null)
                    {
                        throw new NotSupportedException();
                    }
                    return inStreams.Single();
                case K_LZMA:
                case K_LZMA2:
                    return new LzmaStream(info, inStreams.Single(), -1, limit);
                case CMethodId.K_AES_ID:
                    return new AesDecoderStream(inStreams.Single(), info, pass, limit);
                case K_BCJ:
                    return new BCJFilter(false, inStreams.Single());
                case K_BCJ2:
                    return new Bcj2DecoderStream(inStreams, info, limit);
                case K_B_ZIP2:
                    return new BZip2Stream(inStreams.Single(), CompressionMode.Decompress, true);
                case K_PPMD:
                    return new PpmdStream(new PpmdProperties(info), inStreams.Single(), false);
                case K_DEFLATE:
                    return new DeflateStream(inStreams.Single(), CompressionMode.Decompress);
                default:
                    throw new NotSupportedException();
            }

[nightblade9]

Is there something wrong with my archive that it's not triggering the K_LZMA or K_LZMA2 cases?

Is there a work-around or some other steps I can take to get this working?

Do you want me to attach the archive so you can debug it further?

[adamhathcock]

I can tell you I'm not going to get any time to debug this. You can debug it yourself and see what the ID is.

Like I said, I'm guessing it's something newish that isn't implemented. Might be LZMA with different settings but no idea.

[nightblade9]

Ok, no worries.

I got this working with a different NuGet package.

[adamhathcock]

If you've found a solution that doesn't use the native library, let me know. I'll leave this issue open as a reminder there's a new compression method in 7Zip

[nightblade9]

I switched to SevenZipExtractor, which describes itself as "C# wrapper for 7z.dll (x86 and x64 included)."

I don't think it's a new compression method, but I didn't go beyond the 7-zip GUI, which suggested that I'm using "LZMA" or "LZMA2" - both of which throw the same exception.

[Erior]

Tried to recreate this but failed, 7zip file worked in my case, any hints on how to trigger or if you have a test file somewhere @nightblade9 ?

[nightblade9]

Tried to recreate this but failed, 7zip file worked in my case, any hints on how to trigger or if you have a test file somewhere @nightblade9 ?

As I mentioned, I switched to SevenZipExtractor. The repro case for the zip file wasn't anything special, just a regular MP3 file. My initial comment in this issue mentions this, you can try that:

I zipped up a file using 7zip (file manager / GUI). I tried with both LZMA and LZMA2, using ultra compression. I can't seem to figure out how to extract it; I always get a NotSupportedException.

This is referring to the 7-zip GUI file manager (7zFM.exe on Windows), hope that helps.

[Erior]

Ah, music files, that would probably then be the Delta filter https://www.reddit.com/r/touhou/comments/44thjk/use_7zips_delta_filter_to_decrease_the_size_of/ as in "const UInt32 k_Delta = 3;"

…

On Tue, Feb 14, 2023 at 11:52 PM TraceEntertains ***@***.***> wrote: Something I find very odd is that I had practically the same exception, specifically the part that all of the files up to a WAV file extracted fine, which I find odd because you mentioned that you got stuck on an MP3 file. Maybe there is something making it to where audio files can't be extracted? — Reply to this email directly, view it on GitHub <#526 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOI7CLCOLREJGUDZWSZCGXDWXQED3ANCNFSM4O3M6ZAQ> . You are receiving this because you commented.Message ID: ***@***.***>

-- There is nothing that cannot be solved through sufficient application of brute force and ignorance.

[TraceEntertains]

Ah, music files, that would probably then be the Delta filter https://www.reddit.com/r/touhou/comments/44thjk/use_7zips_delta_filter_to_decrease_the_size_of/ as in "const UInt32 k_Delta = 3;"
…
On Tue, Feb 14, 2023 at 11:52 PM TraceEntertains @.> wrote: Something I find very odd is that I had practically the same exception, specifically the part that all of the files up to a WAV file extracted fine, which I find odd because you mentioned that you got stuck on an MP3 file. Maybe there is something making it to where audio files can't be extracted? — Reply to this email directly, view it on GitHub <#526 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOI7CLCOLREJGUDZWSZCGXDWXQED3ANCNFSM4O3M6ZAQ . You are receiving this because you commented.Message ID: @.>
-- There is nothing that cannot be solved through sufficient application of brute force and ignorance.

I managed to make sufficient progress on implementing the delta filter after many times asking in the C# discord server, using ChatGPT, et cetera. I am currently stuck, as the buffer size passed to it when it gets read is 128kb, but the output size is less than that, and it seems to be causing problems (garbage data, preventing extraction of further files, etc).

[TraceEntertains]

Currently my error is that after one file gets finished, a 0-byte decompress (happens a lot with delta decoding, atleast with my code), actually has a size of "count (of the file before the 0-byte) - 131072" (that file usually having around 80-90k bytes, so not 128kb, and ending up underflowing when trying to initialize a negative sized array)

If you want to help, please reach out to me on discord: TraceEntertains#5092

I really want to see Delta decompression on this get added, as it is one of the final roadblocks (that I can tell) before the vast majority of 7z files can be decompressed successfully.

[Erior]

@adamhathcock , Should we move the K_xxx defines into CMethodId.cs instead of Registry.cs ?
I see some are in both.

[Erior]

@TraceEntertains , I would like to know if you have a test file that fails with the pull request I created. the current test contains the general setup of files used in the test suite and that passed.

[TraceEntertains]

@TraceEntertains , I would like to know if you have a test file that fails with the pull request I created. the current test contains the general setup of files used in the test suite and that passed.

Both my delta test file and the original archive it was based on extracted seemingly perfectly, and the music files were the quickest to extract out of almost all other files that I could tell.

[adamhathcock]

Thanks Erior. I've been away for personal reasons.