whiteout file type?

[ThomasWaldmann]

Accidentally stumbled over a file type borg does not support yet: a whiteout (used by overlay fs)

Infos:

• "mknod filename w" on macOS
• https://www.kernel.org/doc/html/latest/filesystems/overlayfs.html#whiteouts-and-opaque-directories
• https://lwn.net/Articles/312641/ there is also some info about white-out
• https://www.man7.org/linux/man-pages/man2/rename.2.html
• https://docs.python.org/3/library/stat.html#stat.S_ISWHT

Linux: has a hack avoiding a new file type, using a char dev 0:0 for file whiteouts and an xattr for directory whiteouts
macOS: has whiteouts
FreeBSD: has support for whiteouts, separate file type DT_WHT
OpenBSD: no whiteouts
NetBSD: ?

Is there a need for backing up / restoring whiteouts?

• for a mounted overlay fs, it should be backed up as visible to the user (no whiteouts visible)
• can one access the layers individually, so that WHT files would become visible (on BSD)?

[ThomasWaldmann]

We could add a branch checking for stat.S_ISWHT inside borg create (that stat method exists on all recent pythons and I checked win32, cygwin, linux: all actually have stat.S_ISWHT).

Without that branch, borg will just emit a warning about Unknown file type: <path>.

Options:

• keep code as is (emit warning about unknown file type)
• silently skip the file (like borg does for sockets / solaris doors / solaris event ports) - if we are sure backing up such files makes no sense.
• archive it (and also add support to restore it) - in case this makes sense / there is a use case for this.

Currently, keeping this as is (warning) is the best option until someone using this type of files adds more insights here about the desired behaviour to this ticket.

[enkore]

Whiteouts are mostly used these days in container layers (which may use overlayfs, depending on the container runtime), so they're not uncommon per se, but you'd normally not include container storages in backups (which can have unpacked layers in them). Container image files, which some people might want to back up, are just normal files and pack each layer in a tar file, including the respective whiteouts. If you look at these backing directories through the lens of an overlayfs, whiteouts are processed by overlayfs, and not visible to the user.

I doubt there's much use to this.