Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --preserve-compression option #763

Closed
wants to merge 1 commit into from

Conversation

tub5ta
Copy link

@tub5ta tub5ta commented Aug 22, 2022

This option allows the compression type to be preserved from the source host to the destination. Useful where there are different compression types used in various datasets that are being replicated as part of a recursive tree.

This option allows the compression type to be preserved from the source host to the destination. Useful where there are different compression types used in various datasets that are being replicated as part of a recursive tree.
@dodexahedron
Copy link
Contributor

I'd suggest a more generic options flag, which gets passed to the -o parameter of zfs send, so that any number of options desired can be set. Since syncoid is often used for snapshot replication in a backup type scenario, it seems logical that the destination may often be at a higher compression setting than the source.

It's difficult to find a definitive answer in documentation for the behavior, but my limited testing has shown that the compression of the source is maintained on the receiver, regardless of the compression setting on the receiver. The way I tested was with source and destination with zstd-4 and zstd-14 respectively. When a send/receive (without -c flag) is performed, the compression ratio and physical data size on the destination matches the source. When I use zfs receive -o compression=zstd-14, the ratio is higher and the data smaller. This tracks with how sends are supposed to send the properties of the source snapshot, needing a -o to override them at the destination.

Allowing a generic send/receive options flag would allow for any mix to be applied, along with whatever other options one may want to set (different hash algorithms, etc).

Thoughts?

@phreaker0
Copy link
Collaborator

@dodexahedron this is already implemented, see

sanoid/syncoid

Line 1961 in ef601eb

--recvoptions=OPTIONS Use advanced options for zfs receive (the arguments are filtered as needed), e.g. syncoid --recvoptions="ux recordsize o compression=lz4" sets zfs receive -u -x recordsize -o compression=lz4 ...

@phreaker0
Copy link
Collaborator

@tub5ta seems you missed the actual code

@tub5ta
Copy link
Author

tub5ta commented Mar 19, 2023

Reviewing that section, it appears I have and I can revert my changes in my fork. I was looking for something simple like other 'preserve' flags when I was reviewing the code and missed this bit you have highlighted.

You can close this. If I hit a bug using the existing feature, I'll open a new PR. Thanks.

@phreaker0
Copy link
Collaborator

k, thx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants