-
Notifications
You must be signed in to change notification settings - Fork 3
ParcelRecovery
So you're running an ISR parcel on your laptop and your host didn't shut down properly when the battery ran out. Or your desktop machine hit a power failure. Whatever the reason, your host crashed while a parcel was running, and now isr resume says you can't check in your parcel. Worse, you can't even resume it: the guest BIOS says your parcel doesn't have an operating system anymore.
If your host crashes while your parcel is writing to its virtual disk, you can end up with some disk chunks that have been updated on disk but not in the keyring. Since the keyring stores the chunks' encryption keys, there's no way to read those damaged chunks. If the keyring itself was being written when you crashed, it might also be damaged, giving you damaged metadata for chunks you weren't even using.
The next time you try to use the parcel, the ISR client will notice that the parcel crashed and will check it for damage. If it finds any, the client will prevent you from checking in your parcel until you discard the damaged data. This keeps you from permanently storing the damage back to the server. You can still resume the parcel, though you won't be able to read damaged chunks. Unfortunately, it's likely that chunk 0 is one of them. This chunk usually contains filesystem metadata and is frequently written, making it a primary candidate for corruption. If chunk 0 is unreadable, you can't boot your parcel or mount its filesystem.
If you just want to throw away the damage and get on with your life, say isr discard <parcel>. That will roll back your parcel to its state as of the last checkin or sync and make everything consistent again. Of course, it will also throw away any changes you haven't checked in, including that report you were working on. Luckily you can probably get some of this data out of the parcel. Here's what you can do:
- Run
isr checkp --splice <parcel>. This will attempt to fix up your parcel so that you can successfully read every chunk. It does this by overwriting a damaged chunk with an older version of the chunk from the server. Needless to say, this makes a mess of your filesystem, but stale metadata turns out to be more useful than totally unreadable metadata. If the keyring itself is damaged,--splicewill attempt to fix it; the resulting keyring is not to be trusted, but it might be good enough to let you get your data out. - Don't even try to boot your parcel. Booting requires a lot of I/O to your virtual disk, and since your virtual disk is inconsistent, it's unlikely that the whole boot process will work correctly.
- Run
isr resume -m shell <parcel>. This will make the virtual disk available in the host without launching a VMM. The client will tell you where to find your disk, which will be something like/dev/openisra. - Mount your parcel's filesystem read-only in your host. This will involve something like
sudo mount -o ro /dev/openisra1 /mnt. - Copy your data into the host's filesystem (e.g.
/home). You only need to recover files you've modified since your lastcheckinorsync, since everything else is safely stored on the server. You may see I/O errors from the host kernel, or discover that not all of your data can be read. Recover what you can. - Unmount the parcel's filesystem. This will involve something like
sudo umount /mnt. - Exit the shell launched by
isr resume. This will suspend the parcel. - Use
isr discard <parcel>to discard the damaged parcel state. - Resume your parcel and copy the recovered data back inside (e.g. with
sftp).
You're done! You can now continue using your parcel as normal.