Unpacking zip, encoding corrupts filenames

[tkisme]

It's happening to names of files and folders within zip archives (scrambled when looking or unzipping them), and to the content in some Text files, other files seems to fare better even though their names are gibberish.

[tkisme]

https://github.com/kjerk/sjisunzip

[tkisme]

no offence, Entropy do it well.

[aonez]

thanks for the tip, will look at it

[aonez]

@tkisme can you share a file to test?

[maz-1]

Maybe use p7zip-natspec or unzip-iconv for encoding autodetection?
https://aur.archlinux.org/packages/p7zip-natspec/
https://aur.archlinux.org/packages/unzip-iconv

[aonez]

Just deleted one of you comments because it contained a file with adult images.

For the battle-net issue, here is the right issue that already contains a test file: #2

For the zip encoding, here an all ages test file, found here.

Just note that the bundled macOS compression utility also fails to extract it with the correct encoding.

[aonez]

Just for test I've opened Unicode test.zip file, that you can find inside the test file posted before and Entropy did not extracted the files with the Chinese encoding, omitted them. Also the adult file you've posted before got unusable with Entropy, no images nor text file when extracted.

[aonez]

Here another test file. This one is encoded in Greek DOS (737 codepage). This one was found here.

[aonez]

Some useful info here: https://sourceforge.net/p/p7zip/discussion/383044/thread/3d213124

No quick fix though.

[tkisme]

maybe add iconv is enough?Can't wait to test.

[tkisme]

from The Unarchiver
maybe show a window to select encoding?

[maz-1]

Official fix could take some time, but as I have pasted above, there is already third-party patches to add encoding autodetection support to unzip and p7zip :-)
Anyway, the default 'Archive Utility.app' seems to handle different encodings of zip files well, so you can use the bultin app for zip files.

[tkisme]

I think keka is the replacement of the default 'Archive Utility.app'.Use different application to handle different archives is just annoying.
So I think this is the necessary function.

[maz-1]

I can't get the point. Keka is behaving similar to built-in archive utility (double click and decompress), I did not see anything annoying.
Anyway, I tried to build latest p7zip with natspec patch and replaced Keka.app/Contents/Resources/keka7z, still no luck. The patch needs to be fixed for p7zip 16.02.

[aonez]

@maz-1 , what natspec library you've used to compile p7zip? Can you share those files?

[maz-1]

Here is the natspec patch for p7zip 15.14.1
https://aur.archlinux.org/cgit/aur.git/tree/natspec.patch?h=p7zip-natspec
line 31 doesnt apply on p7zip 16.02, need a fix.
Here is the natspec library
https://sourceforge.net/projects/natspec/

@tkisme
Whatever. Why not stop arguring and start finding a solution to the problem?

[aonez]

@maz-1 I've already downloaded both the patch and the library files, and fixed the patch. But I'm unable to link p7zip to the natspec library. How you did it?

In case you need it, here the patch: natspec_p7zip1602.patch.zip

[maz-1]

I made a homebrew formula to build natspec:
https://gist.github.com/maz-1/2c797fa2ccf5aa815017676bbb884a73

Download natspec.rb and run "brew install natspec.rb" should install the library.
And then patch p7zip with this(Modified a little to link to natspec):
https://gist.github.com/maz-1/f8e0bce8516e11337a758314c07ca423

But the encoding detection is still borken (15.14.1 has the same problem):

/Users/ling/Downloads/p7zip_16.02/bin/7z  x /Users/ling/Downloads/test/Unicode\ test.zip

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=utf8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x64)

Scanning the drive for archives:
1 file, 8458 bytes (9 KiB)

Extracting archive: /Users/ling/Downloads/test/Unicode test.zip
Broken encoding: '(null)' (to) or 'CP936' (from) or UCS2. May be you forget setlocale in main or gconv-modules is missed?
: Invalid argument
…
Broken encoding: '(null)' (to) or 'CP936' (from) or UCS2. May be you forget setlocale in main or gconv-modules is missed?
: Invalid argument
--
Path = /Users/ling/Downloads/test/Unicode test.zip
Type = zip
Physical Size = 8458

  0%Broken encoding: '(null)' (to) or 'CP936' (from) or UCS2. May be you forget setlocale in main or gconv-modules is missed?
: Invalid argument
…
Broken encoding: '(null)' (to) or 'CP936' (from) or UCS2. May be you forget setlocale in main or gconv-modules is missed?
: No such file or directory
Everything is Ok

Folders: 7
Files: 30
Size:       132
Compressed: 8458

[aonez]

Another file that gets broken, Russian content shared by Колобок via mail: IIS.zip

[InfinityMe]

IIS.zip was probably created by the built-in Windows archiver, I now checked, unpacked and re-packed the files, then unpacked, the encoding went bad.
WinRar on Windows at the same time works with such an archive correctly.

[aonez]

Should check https://github.com/ethereon/p7zip-hybrid encoding changes

[aonez]

This one should be fixed in 1.1.6!!!

[c0494133d4]

I am doing a research whether I could replace 'The Unarchiver' with this great App.

The only concern seems to be zip file name encoding. IMHO, auto encoding detection does not work 100%, so manual selection should be provided as a last resort.

With no offence, is it implemented?

[aonez]

It was implemented in 1.1.6 @ffffwh. Inspired in The Unarchiver way, so will be very (maybe too much) familiar. You can test the feature with the example file found in the comment #12 (comment).