p7zip error 2 when compressing pvm file

[zeff013]

In the event, when Keka is compressing pvm file (Parallels Desktop virtual machine ~20G size) and there is not enough free space on macOS Mojave encrypted apfs volume, but more than enough purgeable space - purgeable space won't be freed-up automatically and Keka will fail with fatal error 2 (no free space available).

This will not happen, when compressing non-pvm files. I have tried this with centos-everything.iso, which is over 10G. Keka compressed it successfully and macOS freed up purgeable space automatically.

Not sure whose fault it is:

Parallel's, for creating pvm in some erroneous form of collection of folders and files, which leads to Keka not being able to compress it correctly.

Keka's, for not being able send correct command to macOS to free up purgeable space, when compressing large amount of files.

macOS's, for not being able to timely understand that purgeable space need to be cleaned up.

[zeff013]

I did another test - coping pvm files from another volume to macOS Mojave encrypted apfs volume, which has not enough free space, but has enough purgeable space - everything went smoothly.

From this I tend to draw conclusion, that Keka is not handling compression of large pvm files well enough and might fail on other large collections of files and folders, when it needs to free-up purgeable space on ssd in order to store resulting archive successfully.

[gingerbeardman]

I'm wondering how you can let macOS know that you need X space ahead of time? And how much would X be in this instance? What proportion of 20GB?

[zeff013]

For example just tell macOS to free up the same amount of purgeable space, which is occupied by original files, you are going to compress.
I'm always using "store" compression for pvm files, so it should be very easy.
I understand that it is hard to predict the target size on normal compression settings, and I think there is no point to be very precise in freeing up space.

This is what was happening, when macOS it self was coping files: it was always freeing up space with some margin.
I had 7GB of free space and 280GB of purgeable space.
I have copied several pvm files, which were 20GB, 20GB and 40GB in size.
At first macOS freed up just 50GB of purgeable space for the first two files.
But when I started to copy 40GB file, it freed up 220GB of purgeable space.

[aonez]

Thanks for the feedback @zeff013! This seems to me like an OS thing. The cause might be similar to #362, where the reason seems to be that macOS denies access to the source file during an operation and that operation fails, rather than a space issue. Would be interesting to replicate the issue and see if always fails at the same time.

I don't use the purgeable feature at the moment.

[gingerbeardman]

@aonez have you turned it off? Or do you use a macOS version from before it was introduced.

I use High Sierra and I'd like to turn it off.

[aonez]

@gingerbeardman I understand this purgeable thing appears only if you enable "Optimize Mac Storage" in iCloud settings, which I don't use. I like my files to be local if no sync is needed.

[zeff013]

Yes, I confirm issue replication as "always" and "each time at the same time".

When I was trying to compress the same pvm file, operation was failing at the same moment in time each and every attempt.
I noticed that Keka was writing compressed data in 8GB chunks to disk and when it was time to write another chunk, but there was not enough free space (only space, that needs purging) - operation was failing with the mentioned code.

What is probably also worth mentioning - when Keka was trying to write another chunk and there was not enough free space, the already created archive file was disappearing and only after a minute or so of keep trying, Keka was throwing the error and failing.

I have "Optimize Mac Storage" enabled, but the thing is that "purgeable" appears for any file deleted from home folder.
In theory this might be connected with TimeMachine.
On practice, when I delete any file in my home folder - it goes to purgeable space immediately each and every time.
In theory, TimeMachine still might do a backup of deleted file, if I will not request macOS to purge it and make space available for something else.