Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should zstd check archive consistency before overwriting files? #4006

Open
Mitmischer opened this issue Mar 31, 2024 · 1 comment
Open

Should zstd check archive consistency before overwriting files? #4006

Mitmischer opened this issue Mar 31, 2024 · 1 comment

Comments

@Mitmischer
Copy link

Is your feature request related to a problem? Please describe.
I stumbled upon this when reporting #4005.
The scenario is that corrupt archives can naturally emerge from disks that fill up during compression.
zstd will overwrite files (after prompting the user).
If the decompressed file is corrupt, the user will lose the overwritten file and he will not have obtained any data from the corrupt archive. This could result in potential data loss if the user is inattentive.
Note that while the user is to blame for explicitly allowing overwriting, zstd could mitigate this very easily.

Describe the solution you'd like
When overwriting files, zstd should check the archive for integrity first (or decompress to a temporary file and move into the existing file when decompression has succeeded).
Along the same lines, decompressing to a temporary would also be beneficial in the scenario where the disk fills up during compression. Then, we could just savely delete the (worthless) temporary without having harmed any other data.

Describe alternatives you've considered
zstd could offer to save the decompressed file under a different name. That's how some folder synchronizing software deals with conflicts.

Additional context
None.

@gcflymoto
Copy link

I don't agree with all these extra mechanics imposed on Zstd. The user can handle all these mechanics for Zstd and the scenarios they want to mitigate. If you need /tmp compressed/decompressed files handle that yourself. If you need to test the compressed file first, handle that yourself. I definitely do not want a third temporary file taking up all that extra disk space during compression/decompression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants