Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explore backups for fixed LVM, non-CoW storage #31

Closed
Tracked by #117
tasket opened this issue Apr 13, 2019 · 4 comments
Closed
Tracked by #117

Explore backups for fixed LVM, non-CoW storage #31

tasket opened this issue Apr 13, 2019 · 4 comments
Labels
enhancement New feature or request
Milestone

Comments

@tasket
Copy link
Owner

tasket commented Apr 13, 2019

Many systems are installed with fixed (non-thin) lvm volumes and documentation suggests thin snapshots can be created for fixed volumes. If so, it may be possible to use the thin lvm tools to track deltas for fixed volumes the same way as in an all-thin environment.

The challenges for implementation are:

  • Determining actual thin-tools compatibility with these hybrid snapshots.
  • Tracking the fixed nature of configured volumes during archive ops, if any special steps are needed.
  • Advising users on a straightforward way to configure their fixed-lvm system with a suitable thin pool.

Barring the above, it may be helpful to reference documentation showing the best practices for converting fixed-lvm systems over to thin pools, btrfs, etc.

@tasket
Copy link
Owner Author

tasket commented May 16, 2021

After researching hybrid snapshots, there is no way to target a non-thin vol + thin snapshot consistently since the non-thin volume's state cannot be known from its metadata. The combination could easily be corrupted and Wyng would have no way to detect or compensate for it.

So I'm changing this issue to a general 'Support volumes in non-snapshot storage' theme. Such volumes would have to be added to the archive with a special option flagging them as non-CoW, and they would essentially trigger a --remap on each incremental send. IOW, this volume type could be supported if Wyng treated it like a traditional incremental backup, comparing all the volume data against hashes for each run.

Users should be consistently reminded of this special mode to prevent confusion about slower backup times, etc.

@tasket tasket changed the title Explore backups for fixed LVM + thin snapshots Explore backups for fixed LVM, non-CoW storage May 16, 2021
@tasket tasket added this to the v0.4 milestone May 16, 2021
@tasket tasket modified the milestones: v0.4, far future Aug 31, 2021
@tasket tasket added the enhancement New feature or request label Jun 15, 2023
@tasket tasket modified the milestones: far future, v0.8 Jun 15, 2023
@tasket
Copy link
Owner Author

tasket commented Jun 15, 2023

Some support for this can be added via an option like --import-from to enable reading volumes that sit outside of a supported snapshot storage system. This would complement the existing --save-to option.

@tasket tasket mentioned this issue Jun 15, 2023
12 tasks
tasket added a commit that referenced this issue Aug 11, 2023
@tasket
Copy link
Owner Author

tasket commented Aug 11, 2023

@tlaurion Implemented option --import-other-from which is to be used like this:

wyng send --import-other-from=volname:|:path/to/file

The char sequence ':|:' is used as a delimiter between the archive volume name and the source file path.

This requires testing.

tasket added a commit that referenced this issue Aug 11, 2023
tasket added a commit that referenced this issue Aug 12, 2023
receive: Fix exception with zero-sized volumes
@tasket tasket closed this as completed Aug 14, 2023
@tlaurion
Copy link
Contributor

@tlaurion Implemented option --import-other-from which is to be used like this:

wyng send --import-other-from=volname:|:path/to/file

The char sequence ':|:' is used as a delimiter between the archive volume name and the source file path.

This requires testing.

@tasket i tried to test this today through wyng-util-qubes -w import-other-from=boot:|:/dev/sda1 without success. Not sure what I'm missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants