Repository navigation
[BUG] Keka doesn’t extract tarballs with XZ compression correctly. #712
Description
Activity
By the way, if I have a file named Firefox Beta (84.0).txz, Keka extracts it to Firefox Beta (84 folder.
This isn’t that much of a problem (but you can see it in the screenshot) so I'll just leave it FYI.
@VaslD what
tarversion are you using? Just tried to reproduce with the macOS bundled version and I was unable to reproduce this in Big Sur or Catalina. I'm seeing in the log (thanks so much for sharing it) thatbsdtarextraction fails sop7zipis used producing an invalid app bundle. My guts say is not yourtarversion, probably the same bundled one as in my tests, but you Affinity bundle that has some metadata different that the one I've just downloaded from their web.Since the bundle is so big, maybe you can just create a txz with a file of the bundle and share it with me, for example:
tar -cJf ~/Desktop/Test.txz -C /Applications Affinity Publisher.app/Contents/Frameworks/libxmp.dylibWhat I've did:
- Copy Firefox and Affinity Publisher to Applications folder
- TXZ with
tar -cJf <TXZ_FILE> -C <PATH> <APP_BUNDLE> - Extract with Keka -> No problemo
aONe-Mini:~ aone$ which tar /usr/bin/tar aONe-Mini:~ aone$ tar --version bsdtar 3.3.2 - libarchive 3.3.2 zlib/1.2.11 liblzma/5.0.5 bz2lib/1.0.6
Firefox Beta (84.0).txz, Keka extracts it to Firefox Beta (84 folder
Reproduced this issue, only when using "Always extract in a subfolder" or with the context menu item "Extract to Firefox (84.0)". Fixing this one right away!
I think my last issue with Keka was also when compressing or uncompressing Affinity apps
Indeed! 😂
I have the exact
bsdtar 3.3.2 - libarchive 3.3.2 zlib/1.2.11 liblzma/5.0.5 bz2lib/1.0.6(from macOS 11.0.1) on both machines.But I can no longer reproduce the issue with just a .dylib, or modified Affinity Publisher.app less than 100 MB. I haven’t tried re-tarring the 3 GB package. I’ll get back to you when I have more info.
@VaslD also for me it's no problem downloading a 3GB (or more) package, so in case is no problem for you just send me a link to info@keka.io.
Since I can't reproduce this anymore I'll close it until next time.
@VaslD I've finally reproduced it (thanks to a mail from Jérôme from https://foxtrot-search.com) and fixed it in this build. The fix will come eventually in v1.2.7 so hopefully you'll never find this issue again.
It was also reported in #741, it's almost the same issue, and the feedback with the build was positive :)
Awesome! May I know what went wrong (in some detail)? It's been bugging me constantly, especially that it doesn't happen every time.
1 remaining item
Sorry I have to reopen this. The latest GitHub version of Keka (1.2.12) on macOS 11.2.3 (20D91) still has the same issue.
XZ (
.tar.xz) archives created by Keka cannot be unzipped (or rather, unxzzed 😃) correctly by Keka. Just test it compressing Keka.app with XZ Slow profile and all additional options turned off. Keka was granted Full Mac and External Volumes permissions.Logs attached:
In the first session, Keka failed to extract its own application bundle from XZ archive. But in the second session, Keka succeeded in compressing and decompressing XZ of another open-source utility App Cleaner.
I also tried excluding resource forks, leaving the app running or killing it before compressing, using different output folders. None worked.
No problem for me on 10.15.7. Can you share with me that
Keka.tar.xzfile?Probably won't fix it, but at least I should try. This build also applies the UTF-8 variable during compression. So let me know if compressing with that version fixes the issue:
I'm mostly sure it won't. I'll have to deeply debug this one again, then.
- Appreciated. I’ll run it on the machine that exhibit the issue and get back to you hopefully tomorrow. If you need any additional debugging info or to run additional tests, let me know. Best, Yi…On Apr 13, 2021, at 22:50, aONe ***@***.***> wrote: Probably won't fix it, but at least I should try. This build also applies the UTF-8 variable during compression. So let me know if compressing with that version fixes the issue: Keka-1.2.13r4522.7z I'm mostly sure it won't. I'll have to deeply debug this one again, then. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.Reacted by aONe
- Also, would it be possible to then fallback to system/BSD `tar` if extraction result was unexpected? E.g. checksum failure, recursively extracting archive from archive, extraction produced none-sense or internal metadata like PAXHeader, to name a few. Since system `tar` is pretty much always available and stable, it would be a better/prioritized choice then falling back to 7-zip.…On Apr 13, 2021, at 22:54, Yi Ding ***@***.***> wrote: Appreciated. I’ll run it on the machine that exhibit the issue and get back to you hopefully tomorrow. If you need any additional debugging info or to run additional tests, let me know. Best, Yi >> On Apr 13, 2021, at 22:50, aONe ***@***.***> wrote: >> > > Probably won't fix it, but at least I should try. This build also applies the UTF-8 variable during compression. So let me know if compressing with that version fixes the issue: > > Keka-1.2.13r4522.7z > > I'm mostly sure it won't. I'll have to deeply debug this one again, then. > > — > You are receiving this because you modified the open/close state. > Reply to this email directly, view it on GitHub, or unsubscribe.
Same error with system one (running through Keka). The problem here is sandboxing...
Turns out this was indeed a sandbox issue, that package contains quarantine data and sandboxed
bsdtaris unable to set that quarantine back usingfsetxattr.Since all sandboxed applications (like Keka) have a quarantine applied for any newly created files (macOS system applies it), that quarantine would be replaced anyway so I'm skipping that warning.
This build should extract the package properly: Keka-v1.2.13-r4540.7z
If you have 2 minutes try to apply a color flag to
Keka.app, compress it (with Exclude Mac resource forks DISABLED) and try to extract it with this build to see if the color flag is applied (should be).Thanks again for all the feedback @VaslD!
Apologies for not checking the other build today as promised. Mac should flag my Mail.app to remind me to check my flagged emails.
So, the new build worked flawlessly. Tested using another 100 MB+ app bundle to be sure, both color attribute and permissions needed to execute the app were set as expected.
Thank you for keeping Keka live and awesome. Closing this as resolved.
Reacted by aONe
Configuration
Describe the bug
Our company’s managed account had no access to Disk Utility and DMG functionalities so I wrote a bash (probably zsh syntax) script to compress each application in
/Applicationsusingtarcommand. So later I could selectively untar individual apps using Keka.For most apps this worked flawlessly. But for Affinity Publisher, Keka and macOS Archive Utility exhibited entirely different behaviors.
(I’ve been doing it this way for years. Interestingly, I think my last issue with Keka was also when compressing or uncompressing Affinity apps.)
All archives were created with
tar -c -J(which means create an archive with XZ compression). Their extensions were set to.txz.I did 4 runs for science:
tar -x -J(which means uncompress an XZ archive) in Terminal.On first 2 runs, Keka seemed to process an archive twice: First extracting a
*.taror a file with the same filename but no filename extension; then processing the file in the first step to extract a folder named PaxHeader with the original app, which macOS deemed cannot be opened because of permission issues.On last 2 runs, both
tarcommand and AU extracted the original app with no extra steps/files. The app can be opened and passed GateKeeper.I did not do a hash check for individual files in the application bundle.
To Reproduce
Not sure this is 100% reproducible though.
Create a tarball with XZ compression for any application.
Uncompress new archive using Keka.
Uncompress the same archive using Archive Utility.
Uncompress with
tar:Compare results.
Expected behavior
Keka should recreate the original app from archive with correct permissions, and should swallow any "header" (without actually extracting them).
Screenshots
This is what happened to Affinity Publisher and Firefox. All tarballs were created equal.
Additional context
Keka log for extracting Firefox (correctly) and Affinity Publisher (incorrectly).
Keka_2020-12-02-200230755.log