Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a method to set the number of cpu thread for 7z compressing? #33

Closed
LARVM opened this issue Nov 7, 2019 · 8 comments
Closed
Assignees
Milestone

Comments

@LARVM
Copy link

LARVM commented Nov 7, 2019

No description provided.

@rikyoz rikyoz self-assigned this Nov 9, 2019
@rikyoz rikyoz added this to the v4.0 milestone Nov 9, 2019
@rikyoz rikyoz added this to To do in bit7z v4.0-beta Nov 9, 2019
@rikyoz
Copy link
Owner

rikyoz commented Nov 9, 2019

Unfortunately bit7z does not currently support this feature.
However, I expect to be able to include it in the next stable version (v4.0) which is in the works.

@LARVM
Copy link
Author

LARVM commented Nov 11, 2019

  1. I adjust the BufferUpdateCallback class to support many file stream compressing,which refines the class of FileUpdateCallback. Curently it work, but I want to know, is there any problem? And would you add this kind of method?
  2. A fast compress method is our product urgent need, so I will also try to implement muti-threaded compressing. Would you remind me some ideas to carry out this work?
    I know that there is a property of "mtf", which can set 7-zip opening muti-thread, but it is not useful when i add it in method of BitArchiveCreator::setArchiveProperties.

@rikyoz
Copy link
Owner

rikyoz commented Nov 11, 2019

  1. I adjust the BufferUpdateCallback class to support many file stream compressing,which refines the class of FileUpdateCallback.

The BufferUpdateCallback class is meant to be used when the input is a single buffer and you want it to be compressed to a filesystem archive file, another buffer, or a standard stream.
I'm sorry, but I probably don't understand what you mean by "support many file stream compressing". Multiple buffers compressing?

Curently it work, but I want to know, is there any problem?

There is no problem, feel free to open a pull request and I'll review it asap.

And would you add this kind of method?

I still don't quite catch what functionality you're trying to add, so I'll be able tell you only when I will.

  1. A fast compress method is our product urgent need, so I will also try to implement muti-threaded compressing. Would you remind me some ideas to carry out this work?
    I know that there is a property of "mtf", which can set 7-zip opening muti-thread, but it is not useful when i add it in method of BitArchiveCreator::setArchiveProperties.

As far as I know, the property "mtf" is useful for setting the multithreading mode only for the filters of the 7z format.
For enabling the complete multithreading mode the correct property should be "mt" (or at least this is what I understand from the command line documentation: https://sevenzip.osdn.jp/chm/cmdline/switches/method.htm).
I didn't test this, but it is the way I would try in implementing this.
Let me know if you are successful and feel free to open a pull request, if you want!

@LARVM
Copy link
Author

LARVM commented Nov 24, 2019

I found the reason why 7z couldn't multithread when I compressed multiple files. Just need to remove the setting of SOLID_ARCHIVE.
image

@rikyoz
Copy link
Owner

rikyoz commented Nov 26, 2019

I found the reason why 7z couldn't multithread when I compressed multiple files. Just need to remove the setting of SOLID_ARCHIVE.
image

Interesting: by default, the "solid archive" mode is set to false; however, if I understand correctly, 7-zip cannot exploit the multi-threading all the same, right?
Maybe explicitly specifying the "s" property disables it, or the property must be set to true to make multiple threads to work, I don't know. Anyway, I cannot remove those lines of code for obvious reasons (they are needed to support compression to solid archives).
Possibly, in the future, that "if" will check if multi-threading is requested or not, and enable/disable solid compression accordingly.

@levicki
Copy link

levicki commented Mar 23, 2020

Possibly, in the future, that "if" will check if multi-threading is requested or not, and enable/disable solid compression accordingly.

Please don't do that.

It should be the bit7z library user's responsibility to know what 7-Zip options work together.

If I wanted to make a solid archive, but I also specified multi-threading, I'd like to get the same behavior from the library that I would get from 7-Zip File Manager.

In my opinion, it is bad to create API whose behavior differs from the UI.

@rikyoz
Copy link
Owner

rikyoz commented Mar 25, 2020

Please don't do that.

It should be the bit7z library user's responsibility to know what 7-Zip options work together.

If I wanted to make a solid archive, but I also specified multi-threading, I'd like to get the same behavior from the library that I would get from 7-Zip File Manager.

In my opinion, it is bad to create API whose behavior differs from the UI.

Undoubtedly you've raised some very good points.
However, the problem here, as far as I understand it, is that whether the solid archive option is set to false or true, in all cases multi-threading is unavailable! This issue may be caused by the fact that the option for solid compression is always set explicitly, rather than only when the user set it to true (a change that I did some time ago).
At least, this is the only reason I can think of for the need of removing those lines, instead of simply call BitArchiveCreator::setSolidMode(true).
I haven't had a chance yet to test if this is the problem, anyway I will study the default behavior of 7-zip and try to provide a reasonable API for both multi-threading and solid compression.

@rikyoz rikyoz moved this from To do to In progress in bit7z v4.0-beta May 5, 2020
@rikyoz rikyoz moved this from In progress to Done in bit7z v4.0-beta May 6, 2020
@rikyoz
Copy link
Owner

rikyoz commented Oct 1, 2023

Implemented in v4.0.0.

@rikyoz rikyoz closed this as completed Oct 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

No branches or pull requests

3 participants