RPC endpoint for syncing filesystem

[v6ak]

I'd appreciate a synchronous endpoint for flushing all FS changes. Currently, I can run qvm-run -p vm_name sync (from dom0). It will work on Linux. But its behavior on other systems is generally not defined (it will most likely be an undefined command).

Why? Because of syncing a running VMs: v6ak/qubes-incremental-backup-poc#46

[jpouellet]

I cannot possibly imagine any way to implement this in a race-free way (with fs state flushed & return from RPC (& block-level snapshot taking affect) being atomic).

It seems to me that the only way to really guarantee consistency would be vfs-layer snapshot capability and to do syncing from that at that layer, or to be happy with backups only between vm shutdowns.

This is quite unfortunate, but I do not see any way around it other than crossing your fingers and hoping that journaling has reasonable outcome.

[jpouellet]

I cannot possibly imagine any way to implement this in a race-free way (with fs state flushed & return from RPC (& block-level snapshot taking affect) being atomic).

It seems to me that the only way to really guarantee consistency would be vfs-layer snapshot capability and to do syncing from that at that layer, or to be happy with backups only between vm shutdowns.

This is quite unfortunate, but I do not see any way around it other than crossing your fingers and hoping that journaling has reasonable outcome.

[jpouellet]

There is also the question of application state (consider a database that has a file open and locked). From OS/kernel view, all pending things are synced. In reality (with respect to data user cares about) that is not the case.

Consistency is hard :(

Personally, I'm totally fine with backups requiring rebooting. For the uptime of typical Qubes use case (laptop/workstation) IMHO this is reasonable.

[jpouellet]

There is also the question of application state (consider a database that has a file open and locked). From OS/kernel view, all pending things are synced. In reality (with respect to data user cares about) that is not the case.

Consistency is hard :(

Personally, I'm totally fine with backups requiring rebooting. For the uptime of typical Qubes use case (laptop/workstation) IMHO this is reasonable.

[jpouellet]

Don't let my ramblings be discouraging though. I'm just happy someone is continuing the effort to improve backups :)

[jpouellet]

Don't let my ramblings be discouraging though. I'm just happy someone is continuing the effort to improve backups :)

[v6ak]

I know this has inherently some limitations, but it can be still useful: Let's suppose that the VM is able to recover from a sudden power failure. This is not uncommon, both filesystems and applications are usually designed for that. Then cloning the image in an atomic way gets some consistent state. This is true even without sync. The sync is a best-effort way to create as recent backup as possible. I know the app-level limitations (they are essentially the same as what you lose on sudden power failure…), but it does not mean that syncing is totally pointless.

[v6ak]

I know this has inherently some limitations, but it can be still useful: Let's suppose that the VM is able to recover from a sudden power failure. This is not uncommon, both filesystems and applications are usually designed for that. Then cloning the image in an atomic way gets some consistent state. This is true even without sync. The sync is a best-effort way to create as recent backup as possible. I know the app-level limitations (they are essentially the same as what you lose on sudden power failure…), but it does not mean that syncing is totally pointless.

[jpouellet]

[...] but it does not mean that syncing is totally pointless

Fair enough

[jpouellet]

[...] but it does not mean that syncing is totally pointless

Fair enough