Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

p7zip error 2 when compressing pvm file #412

Open
zeff013 opened this issue May 22, 2019 · 7 comments

Comments

Projects
None yet
3 participants
@zeff013
Copy link

commented May 22, 2019

In the event, when Keka is compressing pvm file (Parallels Desktop virtual machine ~20G size) and there is not enough free space on macOS Mojave encrypted apfs volume, but more than enough purgeable space - purgeable space won't be freed-up automatically and Keka will fail with fatal error 2 (no free space available).

This will not happen, when compressing non-pvm files. I have tried this with centos-everything.iso, which is over 10G. Keka compressed it successfully and macOS freed up purgeable space automatically.

Not sure whose fault it is:

Parallel's, for creating pvm in some erroneous form of collection of folders and files, which leads to Keka not being able to compress it correctly.

Keka's, for not being able send correct command to macOS to free up purgeable space, when compressing large amount of files.

macOS's, for not being able to timely understand that purgeable space need to be cleaned up.

@zeff013

This comment has been minimized.

Copy link
Author

commented May 22, 2019

I did another test - coping pvm files from another volume to macOS Mojave encrypted apfs volume, which has not enough free space, but has enough purgeable space - everything went smoothly.

From this I tend to draw conclusion, that Keka is not handling compression of large pvm files well enough and might fail on other large collections of files and folders, when it needs to free-up purgeable space on ssd in order to store resulting archive successfully.

@gingerbeardman

This comment has been minimized.

Copy link
Contributor

commented May 22, 2019

I'm wondering how you can let macOS know that you need X space ahead of time? And how much would X be in this instance? What proportion of 20GB?

@zeff013

This comment has been minimized.

Copy link
Author

commented May 23, 2019

For example just tell macOS to free up the same amount of purgeable space, which is occupied by original files, you are going to compress.
I'm always using "store" compression for pvm files, so it should be very easy.
I understand that it is hard to predict the target size on normal compression settings, and I think there is no point to be very precise in freeing up space.

This is what was happening, when macOS it self was coping files: it was always freeing up space with some margin.
I had 7GB of free space and 280GB of purgeable space.
I have copied several pvm files, which were 20GB, 20GB and 40GB in size.
At first macOS freed up just 50GB of purgeable space for the first two files.
But when I started to copy 40GB file, it freed up 220GB of purgeable space.

@aonez

This comment has been minimized.

Copy link
Owner

commented May 24, 2019

Thanks for the feedback @zeff013! This seems to me like an OS thing. The cause might be similar to #362, where the reason seems to be that macOS denies access to the source file during an operation and that operation fails, rather than a space issue. Would be interesting to replicate the issue and see if always fails at the same time.

I don't use the purgeable feature at the moment.

@aonez aonez added this to the Look at milestone May 24, 2019

@aonez aonez self-assigned this May 24, 2019

@gingerbeardman

This comment has been minimized.

Copy link
Contributor

commented May 24, 2019

@aonez have you turned it off? Or do you use a macOS version from before it was introduced.

I use High Sierra and I'd like to turn it off.

@aonez

This comment has been minimized.

Copy link
Owner

commented May 24, 2019

@gingerbeardman I understand this purgeable thing appears only if you enable "Optimize Mac Storage" in iCloud settings, which I don't use. I like my files to be local if no sync is needed.

Screen Shot 2019-05-24 at 11 25 28

@zeff013

This comment has been minimized.

Copy link
Author

commented May 24, 2019

Yes, I confirm issue replication as "always" and "each time at the same time".

When I was trying to compress the same pvm file, operation was failing at the same moment in time each and every attempt.
I noticed that Keka was writing compressed data in 8GB chunks to disk and when it was time to write another chunk, but there was not enough free space (only space, that needs purging) - operation was failing with the mentioned code.

What is probably also worth mentioning - when Keka was trying to write another chunk and there was not enough free space, the already created archive file was disappearing and only after a minute or so of keep trying, Keka was throwing the error and failing.

I have "Optimize Mac Storage" enabled, but the thing is that "purgeable" appears for any file deleted from home folder.
In theory this might be connected with TimeMachine.
On practice, when I delete any file in my home folder - it goes to purgeable space immediately each and every time.
In theory, TimeMachine still might do a backup of deleted file, if I will not request macOS to purge it and make space available for something else.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.