Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REQ] Add support for lrz / lrztar #549

Closed
zenspider opened this issue Apr 23, 2020 · 26 comments
Closed

[REQ] Add support for lrz / lrztar #549

zenspider opened this issue Apr 23, 2020 · 26 comments
Assignees
Milestone

Comments

@zenspider
Copy link

@zenspider zenspider commented Apr 23, 2020

I've been testing lrz against some of my larger archives and even with default settings (eg not -9 etc) it's beating them out. Also it uses ALL the available CPU so it seems to do it at a very nice clip as well. Would you be open to adding this compression format?

@aonez aonez added this to the Look at milestone Apr 24, 2020
@aonez
Copy link
Owner

@aonez aonez commented Apr 24, 2020

Thanks @zenspider! It's already in the list #84. Did some tests yesterday with the stable version 0.631 and found no improvements over 7Z, that also uses LZMA. Only using zpaq -z got some better results but not consistent and the time spent was crazy. Can you show some of your results?

If I see fit for it would be easy to support it in Keka :)

@aonez aonez removed this from the Look at milestone Apr 26, 2020
@aonez aonez added this to the 1.2.0 milestone Apr 26, 2020
@aonez
Copy link
Owner

@aonez aonez commented Apr 26, 2020

Had some time and went ahead with this one. The progress bar is a little useless sometimes due to the steps used in LRZIP.

I'm using LZMA compression but ZPAQ on the slowest setting.

https://github.com/aonez/Keka/releases/download/dev-test-builds/Keka-LRZIP.7z

@aonez aonez mentioned this issue Apr 27, 2020
17 tasks
@zenspider
Copy link
Author

@zenspider zenspider commented Apr 27, 2020

Awesome. Thank you... If it's the same algs as 7Z then maybe I should just switch over to that. I'm good either way as long as I get those compression ratios.

@aonez
Copy link
Owner

@aonez aonez commented Apr 28, 2020

I'm adding it because ZPAQ, that sometimes has better ratio than LZMA! It is very, very slow but might be worth in some cases.

@akrabu
Copy link

@akrabu akrabu commented May 2, 2020

Had some time and went ahead with this one. The progress bar is a little useless sometimes due to the steps used in LRZIP.

I'm using LZMA compression but ZPAQ on the slowest setting.

https://github.com/aonez/Keka/releases/download/dev-test-builds/Keka-LRZIP.7z

So I tried using this version to extract an XZ archive containing an IMG file to test compression with, but after extraction the file just disappears haha. I tested it twice. It puts original in the trash, and the extracted file disappears. Just a heads up!

@aonez
Copy link
Owner

@aonez aonez commented May 3, 2020

@akrabu can you reproduce this while in a Keka debug session? Late 1.2.0 builds use a new extraction step, maybe it's failing for you. Also use a fallback folder (Desktop by default), check it's not going there the extraction.

Screen Shot 2020-05-03 at 11 41 54

@akrabu
Copy link

@akrabu akrabu commented May 7, 2020

@aonez Actually the file was on my desktop. And, if I try to extract the Keka-LRZIP.7z itself, with the version from within Keka-LRZIP.7z, it does the same thing.

I then gave the new Keka full disk access. Still doesn't work. :(

Here's the log file.

Here's what my File Access tab looks like:

Screen Shot 2020-05-07 at 9 37 43 AM

@aonez
Copy link
Owner

@aonez aonez commented May 7, 2020

Thanks for the time @akrabu! So seems to be the new extraction step:

2020-05-07 09:42:00.154 The extraction at /Users/akrabu/Desktop/Keka-LRZIP.kextraction/Operation is empty

Is ok to you if I send you a build with more logging for this issue? May I ask what macOS version are you using?

@akrabu
Copy link

@akrabu akrabu commented May 7, 2020

@aonez
Copy link
Owner

@aonez aonez commented Jun 10, 2020

Released in the latest dev build: v1.2.0-dev.3901

@aonez
Copy link
Owner

@aonez aonez commented Jun 10, 2020

Sorry @akrabu forgot about the build! Can you test the last dev build?

@akrabu
Copy link

@akrabu akrabu commented Jun 11, 2020

@aonez Everything seems to get stuck after 5 seconds of extraction...

Screen Shot 2020-06-10 at 6 30 09 PM

Screen Shot 2020-06-10 at 6 32 33 PM

@aonez
Copy link
Owner

@aonez aonez commented Jun 11, 2020

@akrabu can you check on the Console.app (filtered by Keka) if there are any more logs while you do an operation?

@akrabu
Copy link

@akrabu akrabu commented Jul 21, 2020

@aonez Hey, sorry, I missed this message! But, v1.2.0-beta.1 appears to have fixed the problem. The latest betas weren't working for me at all and I started to use the main version. This one finally extracts though! Though it doesn't always show progress (like no window at all)? But at least it works haha

@aonez
Copy link
Owner

@aonez aonez commented Jul 21, 2020

Glad to hear that @akrabu! 👏🏼

Though it doesn't always show progress (like no window at all)?

By design in 1.2.0 Keka waits 0.5 seconds to show the tasks window. If the operation lasts less than that, you won't see the task window. Is that the case? Or is this happening with longer tasks?

@akrabu
Copy link

@akrabu akrabu commented Jul 21, 2020

@akrabu
Copy link

@akrabu akrabu commented Jul 24, 2020

@aonez Spoke too soon - I'm extracting a 16GB multi-part RAR file and it's definitely working, and I see the pie-chart progress in the Finder, but there's no progress window anywhere. Just a heads up!

In addition it stalled in the end again, and just sat there. I was able to "show contents" and copy the files out, plus verify integrity using the included MD5 sums, but Keka beta is apparently still not always extracting properly on my machine. :(

@aonez
Copy link
Owner

@aonez aonez commented Jul 24, 2020

@akrabu what machine are you using? Any special configuration that I can try to reproduce? Working on an external disk/format?

Maybe you can take a look at the Console.app (filtered with Keka) to see if there's any message regarding the tasks window or the stall end...

@aonez
Copy link
Owner

@aonez aonez commented Jul 24, 2020

@akrabu reproduced the no show tasks window. Going to fix that!

@akrabu
Copy link

@akrabu akrabu commented Jul 25, 2020

@aonez Looks like the extraction bug has popped back up.

Screen Shot 2020-07-25 at 11 14 01 AM

Screen Shot 2020-07-25 at 11 14 20 AM

2020-07-25 11:10:29.843 Dev mode enabled
2020-07-25 11:10:29.844 READER: Dev reader mode enabled
2020-07-25 11:10:29.845 NOTIFICATION: Dev notifications mode enabled
2020-07-25 11:10:29.845 Running on OS X/macOS version 10.15.6 (Build 19G73)
2020-07-25 11:10:29.846 Running sandboxed Keka (WEB) version 1.2.0-beta.1 (3988)
2020-07-25 11:10:29.847 Used localization: en
2020-07-25 11:10:40.681 DEV: Checking tasks queue
2020-07-25 11:10:40.688 DEV: Starting queued task
2020-07-25 11:10:40.691 DEV: Path set as unavailable: /Users/akrabu/Downloads/test.kextraction
2020-07-25 11:10:40.700 DEV: Format detected: ZIP
2020-07-25 11:10:40.701 DEV: Binary used: kekaunar
2020-07-25 11:10:40.703 DEV: Arguments: (
"-q",
"-r",
"-K",
"-nq",
"-o",
"/Users/akrabu/Downloads/test.kextraction/Operation",
"/Users/akrabu/Downloads/test.zip"
)
2020-07-25 11:10:49.345 DEV: Checking ended task
2020-07-25 11:10:49.346 DEV: Skipping quarantine...

Is there any way to toggle showing the extraction window? I have to queue up a second task to be able to cancel the first, and it makes the log messy haha. But basically it just sits there at this point. No errors or anything. So then I queue up a second extraction, cancel it, plus the first, in order to make it stop.

@aonez
Copy link
Owner

@aonez aonez commented Jul 26, 2020

I'll push the task window fix in a few

@aonez
Copy link
Owner

@aonez aonez commented Jul 26, 2020

@akrabu the task window bug is fixed https://github.com/aonez/Keka/releases/tag/v1.2.0-beta.2, can you check there the logs?

@akrabu
Copy link

@akrabu akrabu commented Jul 28, 2020

@aonez I'm testing with an 834MB zip file, and I do get the window now! I like the orange color instead of blue, too haha. However, it still stops at the very end. It isn't frozen, it just doesn't finish. I still have full-disk access enabled. I can actually show the contents of the .kextraction folder and check the hashes and everything is intact. If I cancel, it deletes everything though. So currently I have to copy it out of the folder then cancel as a workaround.

Screen Shot 2020-07-28 at 11 51 13 AM

2020-07-28 11:50:51.370 Dev mode enabled
2020-07-28 11:50:51.372 Running on OS X/macOS version 10.15.6 (Build 19G73)
2020-07-28 11:50:51.374 Running sandboxed Keka (WEB) version 1.2.0-beta.2 (4008)
2020-07-28 11:50:51.378 Used localization: en
2020-07-28 11:50:51.910 DEV: Adding 1 files
2020-07-28 11:50:51.911 DEV: Waiting for more input: 0.250000
2020-07-28 11:50:51.913 DEV: Requesting access to all bookmarks
2020-07-28 11:50:51.915 DEV: Bookmarks updated: 2
2020-07-28 11:50:52.287 DEV: Checking tasks queue
2020-07-28 11:50:52.299 DEV: Starting queued task
2020-07-28 11:50:52.302 DEV: Path set as unavailable: /Users/akrabu/Downloads/test.kextraction
2020-07-28 11:50:52.314 DEV: Format detected: ZIP
2020-07-28 11:50:52.316 DEV: Binary used: kekaunar
2020-07-28 11:50:52.318 DEV: Arguments: (
"-q",
"-r",
"-K",
"-nq",
"-o",
"/Users/akrabu/Downloads/test.kextraction/Operation",
"/Users/akrabu/Downloads/test.zip"
)
2020-07-28 11:50:52.736 DEV: Showing delayed tasks
2020-07-28 11:51:06.585 DEV: Checking ended task
2020-07-28 11:51:06.586 DEV: Skipping quarantine...
2020-07-28 11:51:24.283 Dev mode enabled
2020-07-28 11:51:24.284 READER: Dev reader mode enabled
2020-07-28 11:51:24.285 NOTIFICATION: Dev notifications mode enabled
2020-07-28 11:51:24.287 Running on OS X/macOS version 10.15.6 (Build 19G73)
2020-07-28 11:51:24.288 Running sandboxed Keka (WEB) version 1.2.0-beta.2 (4008)
2020-07-28 11:51:24.289 Used localization: en
2020-07-28 11:51:39.089 DEV: Checking ended task
2020-07-28 11:51:39.090 DEV: Ignoring queue check, already quitting
2020-07-28 11:51:39.091 DEV: Cancelled
2020-07-28 11:51:39.093 DEV: (
"Format detected: ZIP",
"Binary used: kekaunar",
"Arguments: (\n "-q",\n "-r",\n "-K",\n "-nq",\n "-o",\n "/Users/akrabu/Downloads/test.kextraction/Operation",\n "/Users/akrabu/Downloads/test.zip"\n)",
Cancelled
)
2020-07-28 11:51:39.108 DEV: Path set as available: /Users/akrabu/Downloads/test.kextraction
2020-07-28 11:51:39.109 DEV: Operation cancelled
2020-07-28 11:51:39.111 DEV: Stopping access to all bookmarks

@aonez
Copy link
Owner

@aonez aonez commented Jul 29, 2020

Finally reproduced the issue! There's a bug that if you have "Inherit quarantine from downloaded files" disabled, the task does never finish (but the extraction does).

Fixing it right away! Thanks a lot @akrabu!

You can enable that option meanwhile, I'll push this fix in the third beta in a few days.

Fixed in this build: https://github.com/aonez/Keka/releases/download/dev-test-builds/Keka-v1.2.0-dev.r4019.7z

You can simply replace your current beta with this one :)

@akrabu
Copy link

@akrabu akrabu commented Aug 7, 2020

@aonez Yay that fixed it! Never would have thought of that option as being the culprit haha. Thanks!

@aonez
Copy link
Owner

@aonez aonez commented Aug 10, 2020

Nice @akrabu! 😁

@aonez aonez closed this as completed Aug 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants