New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7z vs zip compression 8th gen i9 Intel processors #345

Open
matmal opened this Issue Dec 18, 2018 · 29 comments

Comments

3 participants
@matmal
Copy link

matmal commented Dec 18, 2018

Hi,

I do not know if this is a problem with 7z algorithm or something else, but I have noticed that the processor is not used in 100% of it's frequency. I was running a simultaneous compression of 12 files and the frequency was around 2.2GHz (7zip - ultra compression) and on the contrary with ZIP (Ultra compression) I was reaching around 4.2GHz. What is wrong, does anybody know?

@issuelabeler issuelabeler bot added rar zip labels Dec 18, 2018

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 18, 2018

@matmal just did a quick test with an 8th gen processor and a good turbo speed of 4,3 was achieved with either 7Z or ZIP for one operation compressing a 60MB folder. Maybe your CPU was suffering from thermal throttling?

Anyway I see 2.2GHz very low even for thermal throttling though. What is the base CPU clock? Did a test of 12x10+GB operations, and even with thermal throttling the clock was between 3,9-4,1.

@aonez aonez added p7zip more info needed and removed rar zip labels Dec 18, 2018

@aonez aonez self-assigned this Dec 18, 2018

@aonez aonez added this to the Look at milestone Dec 18, 2018

@matmal

This comment has been minimized.

Copy link

matmal commented Dec 18, 2018

Hi, it's Macbook Pro w/i9 and quite frankly ZIP's are perfect no effect of thermal throttling. The problem is 7z, somehow it does not work. Look at the screenshots.
7zip
zip

@no-response no-response bot removed the more info needed label Dec 18, 2018

@matmal

This comment has been minimized.

Copy link

matmal commented Dec 18, 2018

I want to add that this also happens on my old 2013 i7 MBP, so I guess that there must be something not okay.

@gingerbeardman

This comment has been minimized.

Copy link
Contributor

gingerbeardman commented Dec 18, 2018

Does it happen with other apps that use p7zip?

@matmal

This comment has been minimized.

Copy link

matmal commented Dec 18, 2018

I tried also with Entropy and it looks like same thing happens. Cannot reach base clock speed at least.

@gingerbeardman

This comment has been minimized.

Copy link
Contributor

gingerbeardman commented Dec 18, 2018

Then I think this a problem with the p7zip binary? @aonez where is best to report this?

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 18, 2018

With 8 operations (mostly stable at 4,00):

screenshot 2018-12-18 at 14 46 06

With 18 operations, 10 active (mostly stable at 3,90):

screenshot 2018-12-18 at 14 50 44

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 18, 2018

@gingerbeardman the think is that I can't reproduce this one. @matmal is possible you test this one in a freshly installed Mojave?

@gingerbeardman

This comment has been minimized.

Copy link
Contributor

gingerbeardman commented Dec 18, 2018

OK, I see. Interesting.

(I only have an i7 so I cannot contribute here)

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 18, 2018

@gingerbeardman mine is also i7 (finally).

@gingerbeardman

This comment has been minimized.

Copy link
Contributor

gingerbeardman commented Dec 18, 2018

So the problem is only with i9?

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 18, 2018

He reported that also with an i7 #345 (comment). Both 7Z and ZIP are created with p7zip, this is what bothers me most.

@gingerbeardman

This comment has been minimized.

Copy link
Contributor

gingerbeardman commented Dec 18, 2018

What is the command line or args used by Keka for both?

-stm?
-mmt?

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 18, 2018

@gingerbeardman, for 7Z the default is:

  • -t7z
  • -ms=on

For ZIP:

  • -tzip

@matmal since the only difference as far as p7zip arguments is the -ms=on, if you disable the "Solid archive" (Utwórz archiwum ciągłe) option in 7Z, there's any difference?

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 18, 2018

@gingerbeardman there's no limitation in the CPU threads done by Keka, no -stm or -mmt is set. But even if they were set, the core frequency shouldn't be affected. I can execute an operation in just one thread and the Intel Power Gadget reports a 4,30 frequency.

@matmal

This comment has been minimized.

Copy link

matmal commented Dec 18, 2018

Okay, I want to say that all screenshots have been made on a freshly installed Mojave (latest version). I have forgotten to say that my old MBP with i7 is also running Mojave (probably minor version 1). I also tested this in Boot Camp and same result, cannot reach the base clock speed. Maybe it would be better to test it via Terminal with original 7z library, any clues?

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 18, 2018

also tested this in Boot Camp

You mean using 7-Zip? And same results?

Maybe it would be better to test it via Terminal with original 7z library

You can try this for example installing p7zip from brew, but I think you'll get the same numbers.

all screenshots have been made on a freshly installed Mojave

I asked for that because of the installed apps from your screenshot, like LittleSnith or Bartender. But if it's pretty vanilla, it probably won't make any difference. Sadly I only have a machine able to run the Intel gadget.

It happens with other file types also? And with a single operation?

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 18, 2018

Then I think this a problem with the p7zip binary? @aonez where is best to report this?

Best place to report a p7zip issue, if also 7-Zip is affected:

https://sourceforge.net/p/sevenzip/discussion/45797
https://sourceforge.net/p/sevenzip/bugs

If 7-Zip is not affected, then the p7zip bugs site:

https://sourceforge.net/p/p7zip/bugs

@matmal

This comment has been minimized.

Copy link

matmal commented Dec 18, 2018

Hi, my recent findings are as this: 7z, xz, Plzip, bzip2, gzip and tar work the same way. The crazy thing is that when compression method is changed to lowest (not storing), the CPU clock speed is okay, but if you set compression method to a little bit higher then things change drastically.

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 19, 2018

@matmal this is even crazier. All tarballs use bsdtar with external compressors like lzip, pbzip2 and pigz. I still think there's some kind of throttling. Can you try with a pre-Mojave OS? High Sierra or similar?

@aonez aonez added the works for me label Dec 19, 2018

@aonez aonez added performance and removed p7zip labels Dec 19, 2018

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 19, 2018

Maybe this is happening also with other processes, like for example an encoding using HandBrake or an exportation of iMovie or Final Cut?

@matmal

This comment has been minimized.

Copy link

matmal commented Dec 19, 2018

Dear @aonez this is a Vega 20 version with i9, this it will not work on earlier versions of the OS. I decided to test it once again in Boot Camp and the CPU was working okay, maintaining around 3.0 GHz with 6 files (proper boosting with one file), but this topic does not concern Keka. My assumption is that the error is system dependent or hardware dependant - you know Apple does magic to the H/W. If anyone has i9 MBP even earlier, please do my test and report. Maybe then I will know where this problem should belong (7zip or Apple). PS. I do not think the problem is thermal throttling related as everything was perfect with ZIP compression, however other algorithms are midiocre.

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 20, 2018

Hopefully someone else with a similar HW can share some more feedback.

it will not work on earlier versions of the OS

You should try Parallels Desktop Lite, despite its unfortunate name.

then I will know where this problem should belong (7zip or Apple)

Note that if it affected the same on 7z, xz, lzip, bzip2 and gzip then 7-Zip is not to blame.

@matmal

This comment has been minimized.

Copy link

matmal commented Dec 20, 2018

The problem witjlh VMs is that it stimulates processors, VMs do not get a direct access to the CPU it is being run in an emulator of some sort. I must correct myself as I mafe6a terrible mistake by saying that on my MBP11,3 with i7 results are the same, they are not and this MBP work under macOS 10.14.2. So this leaves me with either 7z library or Apple messing with config of the system. I posted this issue in Apple Bug report and in 7-zip as instructed above. Time will tell.

@gingerbeardman

This comment has been minimized.

Copy link
Contributor

gingerbeardman commented Dec 21, 2018

With Intel VM on Intel CPU there is basically pass-through of the CPU. So do try a VM.

Next, it could be the way the p7zip was compiled? It may have been optimised towards the older CPU configurations (but current at the time of compilation). Are there any different builds of p7zip?

Let's not rule out Apple though! 🙄

@matmal

This comment has been minimized.

Copy link

matmal commented Dec 21, 2018

It looks like Keka is responsible for 7zip implementation on macOS, and maybe they should check compatibility with latest Intel processors.

@aonez

This comment has been minimized.

Copy link
Owner

aonez commented Dec 21, 2018

@matmal, you've said that the same occurs with gzip, bzip2, lzip and xz, so if this is true, then you can discard this as a p7zip (or 7-Zip) issue.

You could build p7zip on your i9 machine, if you'd like so. The easiest way I can think is using brew. Just install it forcing to compile with this command:

brew install --build-from-source p7zip

If you don't have brew yet, install it with (you can see that in their homepage):

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

@aonez aonez changed the title 7z vs zip compression 8th gen Intel processors 7z vs zip compression 8th gen i9 Intel processors Dec 21, 2018

@matmal

This comment has been minimized.

Copy link

matmal commented Dec 21, 2018

Thank you @aonez, will try your idea.

@stale

This comment has been minimized.

Copy link

stale bot commented Jan 14, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jan 14, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment