Consider switching from 7za to unarchiver #795
Comments
fasterthanlime
added
ui / ux
|
Some research: it's a mercurial repository (cries), the engine is contained in the enigmatically-named for scheme in lsar unar; do xcodebuild -configuration Release -workspace XADMaster.xcodeproj/project.xcworkspace -scheme $scheme -derivedDataPath build; doneSimply running Contrary to 7-zip, unarchiver provides two separate binaries for listing and extracting, both of which are around 5MB (debug), 2.5MB (release), 1.8MB (release stripped), and a .zip archive containing the both of them is 1.8MB (for comparison, the current OSX 7za binary we distribute is 969KB). That seems acceptable considering the added features.
All of these (afaict) are built-ins on OSX, so shouldn't be an issue. We might need to start depending on libbz2 and libz on Linux, but will be able to drop the p7zip dependency. The generated unar is able to properly extract Momodora 1's rar file: https://rdein.itch.io/momodora-i (also, it runs under Wine! but that's another ticket..) lsar has a detailed output mode with the This is non-trivial to parse, but possible (and most importantly, it seems the output format is constant). unar accepts Thus, with a combination of lsar and unar, we can get 'extraction progress / ETA information'. |
|
The next step is to set up a repository under |
|
So, I'm a bit of a CI fanboy ~~~these days~~~ this year, but considering that unarchiver requires GNUStep to build on Windows (and probably Linux), that's a lot of deps to impose on our CI build machines. Besides, I don't expect the source code of unarchiver to change that often - I'll just include a cookbook in the repository, but not enable CI for it. update installed the full GNUStep development suite, windows build is in progress, the tension is palpable:
update 2: windows build was a success, but the resulting .exe depends on a bunch of .dlls, will see if I can come up with a static build of some sort. |
|
Apparently shipping libstdc++'s dll is enough. Can someone confirm that they can run both (Warning: the archive contains 3 files that aren't into a specific folder, so it'll dump Download here: 📦 unarchiver.zip |
|
@fasterthanlime Doesn't work for me. Both executables need |
|
@quyse thanks! what tool did you use to find that out? I usually use ldd but it refused to run on those executables for some reason, and Dependency Walker is hit and miss. |
|
@fasterthanlime I just run the executables, and Windows tells me everything in three consecutive message boxes :) Also I use Dependency Walker, sometimes it cannot find Windows system libs for no reason, but other than that it's usually helpful. |
|
Hmmm maybe installing GNUStep modified my PATH environment variable.. that wouldn't be good. (and would explain why on my windows it does find |
Sure enough, it did... Oh boy, 38mb's worth of DLLs later, here's a new archive |
|
I get no missing DLL errors but: |
Still doesn't work, both need |
|
@quyse |
Yeah, 38mb was unstripped, I stripped them. I'll add those other three. But theunarchiver is increasingly looking like it's not really an option :( Although there are ways to reduce ICU data size to a few kbs (currently at 15MB). |
|
I'm pretty sure I've statically linked libstdc++ on mingw before too, although it seems to only be 957 KB, so I'm not sure it's worth it. |
|
With libffi, libintl, libxml2: 📦 unarchiver.zip |
|
Seems to run fine! |
|
@fasterthanlime Yep, finally works, successfully extracted Momodora rar with |
|
Phew! Now let's try building a smaller icu4c. Without libicudata, the archive size is down to 7MB which, I suppose, is reasonable to be able to extract anything and everything reliably? |
@fasterthanlime Sorry, what was the problem with official |
|
@quyse well, the license restriction was my only objection to shipping a version of 7z that has unrar support. The fewer licenses I have to worry about, the better I feel! Since theunarchiver is LGPL all around, it's relatively easy to deal with. So it's more of an idealistic/politics decision than a technical one. |
|
Quick status update: even though theunarchiver's code explicitly references ICU4C types and functions, it doesn't look like it's used at all in the Windows codepath. Instead, it's the default GNUStep distribution that uses it. I'm currently building a stripped-down version of gnustep-base to try and trim down the eventual size even further. update 1: for example, libxml2 isn't actually necessary since theunarchiver only uses NSXMLParser and that doesn't use libxml2. update 2: neither is libgnutls! Steps so far:
In theunarchiver sources:
|
|
If you folks want to help while I'm dealing with GNUStep's build system, preparing .zip and .rar archives with unicode file characters would help (french éàâç, cyrillic, japanese/chinese characters, etc.). I want to make sure that disabling libicu altogether doesn't prevent theunarchiver from extracting those (the documentation isn't entirely clear on that). |
|
Oooh why do I see that only now, this is so good
One more reason to spend time getting theunarchiver working! |
|
Update: successfully built unar.exe and lsar.exe with my custom-built GNUstep runtime! Extracts Momodora.rar just fine. unstripped exe size is about 7MB, similar to the others, I'll start DLL hunting now. |
|
Update 2: still had the other GNUstep distribution in my path. Not so fast! |
https://drive.google.com/folderview?id=0BwTCpCPC0rtuZjM0cGJNdTcxUXM&usp=sharing |
|
Wow okay when actually built against my runtime, -g0 and -Os, the binaries are 3.1MB, let's see what they depend on. |
|
All .exe + dll required are now 12MB uncompressed, Here's a 4.5MB archive that should hopefully work for you: (edit: the .exes are above are not signed, let me know if you want me to sign them first) |
|
(Note: when testing unarchiver-lite.zip, extract it into an empty directory, not over the other one, otherwise there'll be too many DLLs) |
|
@fasterthanlime |
|
Ah, those make sense! They're small though (36K & 92K) |
|
Works nicely (with the two dlls copied over from the previous build), tested the rar archive with the unicode characters from @quyse. Good job! |
Seem to work for me as well with all three archives (zip, 7z and rar) |
You folks are stealing my job! 😸 Here's the (hopefully final) 32-bit build, with all DLLs, stripped and signed by me: Working on a 64-bit build now |
|
Other helpful research would be:
|
|
Here's a 64-bit build, stripped, signed, and should have all DLLs: |
|
@fasterthanlime Both 32-bit and 64-bit final versions work for me for all test archives. |
|
Symlinks are getting skipped though. Got the following output: I've added |
On windows, that's normal — it would either have to create junctions (which can only point to absolute paths) or require superuser privilege |
|
First linux build, for 32-bit, stripped, statically linked with a bunch of stuff (including minimal libicu), dynamically linked with libffi, libgnustep-base and libobjc, shipped along with it and with an RPATH of Give that baby a try! You'll still need a 32-bit linux distro, the barer the better: (Not signed because signing binaries on Linux is not a thing apparently) |
Checked on Arch Linux 64-bit (with additionally installed 32-bit system libs), works! |
|
@quyse woo! even with non-ascii characters? |
@fasterthanlime yes! |
|
And here's the 64-bit build, same build workflow: |
|
64-bit build works for me too, tested with all archives 👍 |
|
Just uploaded all builds as an ibrew repository to https://dl.itch.ovh/ Example install: This is a new format of ibrew package (previously there was naked executable and 7z), and will also require having its subdirectory added to Here's the complete ibrew repository structure: |
|
This is going really really well! Adding .tar.gz formulas was dead simple (and doesn't require any v8 bindings), I even cleaned up ibrew's code a bit while I was at it. Adding Things I'm happy to see go:
|
maybe finally bringing rar support without silly license restrictions?
The Unarchiver is an OSX app that can extract various archive formats (see the Full list).
The underlying compression engine is open-source and cross-platform and has a command-line front-end. It's LGPL'd (including the unrar code), and available on bitbucket at: https://bitbucket.org/WAHa_06x36/theunarchiver/src
The text was updated successfully, but these errors were encountered: