Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it possible that i can steam the volume rather than 2 step? As I research 7zip.exe doing it using steam which will not produce extra combination zip file. #90

Closed
1 task done
jf3096 opened this issue Jun 16, 2022 · 4 comments
Assignees
Projects
Milestone

Comments

@jf3096
Copy link

jf3096 commented Jun 16, 2022

bit7z version

3.1.x

7-zip version

v21.07

7-zip DLL used

7z.dll

MSVC version

2017

Architecture

x86_64

Which version of Windows are you using?

Windows 10

Bug description

here is my code:

void extractVolumeExample()
{
	Bit7zLibrary lib{ L"C:/Users/admin/source/repos/bit7z-demo/libs/bit7z/dll/7z.dll" };
	BitExtractor extractor{ lib, BitFormat::Split };

	// step 1: after extract, it combine all the volumes zip file into one
	extractor.extract(L"D:/test/volumes/Anaconda3-2022.05-Windows-x86_64.7z.001", L"D:/test/extract-to");
	// step 2: then what i need to do is to unzip it.
	extractFileExample(L"D:/test/extract-to/Anaconda3-2022.05-Windows-x86_64.7z", L"D:/test/extract-to/");

	// Question: is it possible that i can steam the volume rather than 2 step? As I research 7zip.exe doing it using steam which will not produce extra combination zip file
	std::cout << "Done";
}

Is it possible that i can steam the volume rather than 2 step? As I research 7zip.exe doing it using steam which will not produce extra combination zip file.

Steps to reproduce

No response

Expected behavior

No response

Relevant compilation output

No response

Code of Conduct

@jf3096 jf3096 changed the title [Bug]: Is it possible that i can steam the volume rather than 2 step? As I research 7zip.exe doing it using steam which will not produce extra combination zip file. Jun 16, 2022
@rikyoz
Copy link
Owner

rikyoz commented Jun 16, 2022

Hi!

Is it possible that i can steam the volume rather than 2 step?

There's no direct way to achieve this in bit7z, but you might be able to accomplish it by using a custom class implementing std::iostream and using a memory buffer.
This class would implement both std::ostream and std::istream; hence you would be able to use both:

In this way, you might be able to avoid extracting twice to the filesystem.
However, I never tested this approach, so I'm not entirely sure it would work in practice.

Alternatively, if the input archive is not too big, you might extract all of it to a memory buffer and then extract this latter using BitMemExtractor::extract. However, since the input archive is split, it's probably too big for this last approach.

As I research 7zip.exe doing it using steam which will not produce extra combination zip file.

I don't know how 7-zip works in this specific case; however, in other situations, it often uses temporary files, so it might be that it actually implements a two-step extraction to the filesystem under the hood.

@jf3096
Copy link
Author

jf3096 commented Jun 17, 2022

I don't know how 7-zip works in this specific case; however, in other situations, it often uses temporary files
Thank you for your reply. I did some test and found out:

  1. No temp file generated. As I did try to look for it in window temp folder and anywhere I can think of
  2. Since 7zip.exe gui allow to pause during unzipping. It first generated the unzip file then it fill it up with more data (I open the unzipped file and monitor it using winhex.exe). It feels like streaming.

@rikyoz
Copy link
Owner

rikyoz commented Jun 17, 2022

I don't know how 7-zip works in this specific case; however, in other situations, it often uses temporary files
Thank you for your reply. I did some test and found out:

1. No temp file generated. As I did try to look for it in window temp folder and anywhere I can think of

2. Since 7zip.exe gui allow to pause during unzipping. It first generated the unzip file then it fill it up with more data (I open the unzipped file and monitor it using winhex.exe). It feels like streaming.

Ok, interesting! I'll need to investigate further in the 7-zip source code, hoping there's an API it provides for directly extracting split archives. Unfortunately, the 7-zip code is poorly documented.

@rikyoz rikyoz added this to the v4.0 milestone Dec 29, 2022
@rikyoz rikyoz added this to Done in bit7z v4.0 Dec 29, 2022
rikyoz added a commit that referenced this issue Dec 30, 2022
Add support for extracting multi-volume archives without creating the intermediate whole archive. Closes issue #90.
@rikyoz
Copy link
Owner

rikyoz commented Oct 1, 2023

Implemented in v3.2.0 and v4.0.0.

@rikyoz rikyoz closed this as completed Oct 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
bit7z v4.0
  
Done
Development

No branches or pull requests

2 participants