Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Attempt to better handle incomplete transfers #13

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Attempt to better handle incomplete transfers #13

wants to merge 2 commits into from

Commits on Nov 2, 2019

  1. dry-run: Add ability to really execute some "safe" commands 

    dry-run doesn't work currently, as some operations need real output from
real commands. In some cases, it's always safe to actually execute the
command (e.g. `ls`), so add a "dry_safe" parameter to check_call which
can allow these commands to execute even in dry-run mode.
    @usedbytes
    usedbytes committed Nov 2, 2019
    Configuration menu
    Copy the full SHA
    0244b04 View commit details
    Browse the repository at this point in the history

  2. WIP: handle-incompletes: Attempt to handle incomplete transfers 

    If a btrfs send | receive is interrupted, then an incomplete snapshot
can be left on the remote side. This causes two problems:

1) It cannot be used as a parent for any future receives
2) A subsequent snapbtrex clean operation will think that the incomplete
   snapshot is the most recent valid one, and may delete the most recent
   actually valid parent - leading to a situation where no valid parent is
   available on the sender and receiver for incremental backups.

Fixing this is somewhat tricky, as there's no sure-fire way to identify
the incomplete snapshots.

However, _if_ a snapshot is "received", then it will have a
received_uuid, and if it's incomplete, this will be '-'.

This commit attempts to alleviate the problem in a few ways:

 - When starting a transfer, the name of the snapshot is stored into a
   ".snapbtrex_incomplete" file on the remote side. This is used as an
   indicator that a snapshot with no received_uuid really is incomplete.

 - When cleaning a directory, if a ".snapbtrex_incomplete" file is
   present and contains the name of a snapshot, then abort the clean
   operation. Manual intervention will be required, but incremental
   sends won't be inadvertently broken.

 - When sending snaps, when looking for parents, if the parent doesn't have
   a valid received_uuid then try two things:
   - If .snapbtrex_incomplete is present, and contains the name of that
     snapshot, we can be sure that it previously failed to complete, and
     delete the broken snapshot. It will be re-transferred.
   - If .snapbtrex_incomplete is not present or doesn't contain that
     snapshot, then simply ignore the "invalid" parent. This should
     allow us to transfer the snapshot with an older parent.

This isn't perfect, but it should prevent failures more often than
before.

All of the functionality is behind a "--handle-incomplete" option, to
preserve the old behaviour and not break anyone's current workflow.
    @usedbytes
    usedbytes committed Nov 2, 2019
    Configuration menu
    Copy the full SHA
    66d7481 View commit details
    Browse the repository at this point in the history