mount: allow / in archive names

[enkore]

Only seems to be forbidden because it would make the borg mount code a bit more complex. Parsing is not hard; extra assertion would probably be posixpath.normpath(archive) == archive (to rule ../../ & friends out) and it does not start with a /.

Allowing these gives a nicer, more intuitive separation of archives, e.g. {hostname}/{date...} is much neater with / as a separator than -_#@ (whichever) for mounting it, but also because / is considered a separator in general.

Btw., currently . and .. are valid archive names; while these don't crash mount, they're inaccessible there.

[ThomasWaldmann]

Yes, one could do some nice tricks with that.

But it would also make some stuff harder:

Currently, if you mount a repo, you can expect 1 layer of archive names as directories and the archive contents directly inside these dirs.

With that change, you'ld have to first get a list of archive names from borg list repo to get the archive names which are now also the path prefixes needed so you know where the archive content starts below the repo mountpoint.

[enkore]

Currently, if you mount a repo, you can expect 1 layer of archive names as directories and the archive contents directly inside these dirs.

Well the point of using / is to explicitly not have just that one layer...

[ThomasWaldmann]

nitpick: there is only 1 "layer" in borg list output though...

[enkore]

History: Slashes were forbidden in Borg 0.25 due to jborg/attic#180 ("Creating archive with a slash breaks mount option"), previously they were allowed. As noted in #2278 the fix was actually not complete, because the path-like archive names "." and ".." remained allowed and broke it as well (though who does that?). For everything except borg-mount the archive name is just an opaque handle (with a big "don't care" stamped on it).

[ThomasWaldmann]

I am a bit torn about this:

• nice hack to allow structuring your archive names
• otoh, it might trigger structure accidentally where it never was intended
• also, it is giving structure/meaning to the archive name, which is also a major change and might be a troublemaker in case we want to get rid of archive names some day and move to hostname/datasetname/tags/comments.

Just as an example, people could have archive names like:

• "backup done on 02/28/2017 before the migration"
• "backup done before mb/tw changed foo"
• "/I had no idea this gets meaningful/"
• "backup of //share/my/path and //othershare/his/path"

[enkore]

All of these were invalid until know, though (and the //smb examples would still be).

If one doesn't know about mount, then the mental model is likely archive name ~ file name, so slash and other fancy characters would be considered illegal. Which is btw. another thing to discuss, it probably makes sense to restrict the available characters more, forbidding e.g. <>?|\, since these are not compatible with some operating systems.

also, it is giving structure/meaning to the archive name, which is also a major change and might be a troublemaker in case we want to get rid of archive names some day and move to hostname/datasetname/tags/comments.

I don't think that's likely (at least within 1.x). We might add tags, and we already have comments, but neither would or could replace archive names.

Archive names are almost always structured, just using different characters, and obviously do have a meaning. I don't understand what you mean here.

IMHO this only adds an option to nicely structure things, but no one has to use it. Accidentally "stumbling" into it is unlikely in my opinion.

[ThomasWaldmann]

Well, we already have archive IDs (the hash), so these could be used instead of the name to refer to archives. Not very readable, not great for recreate (ID changes, content is logically same). Also we have the date/time separately. IIRC, restic does not have archive names.

We have one problem already with the archive names, which is prune. Although archive names are not interpreted at all other places (yet), prune expects at least some prefix structure if you backup multiple different data sets into one repo. With dataset name separately, that would be cleaner.

[enkore]

Hashes don't work for humans.

Tags and timestamps (without names) only work for a very narrow set of applications. In fact, I'm strictly against removing names.

Adding tags was previously discussed: #846

However, this is all off topic to the discussion at hand IMHO, which is at it's core about allowing users to transfer the structure they already decided on to the filesystem-space offered by mount.

[ThomasWaldmann]

Git users aren't human. :D

[ThomasWaldmann]

Tags or different attributes would work in a equivalent way to names. It is just about not mixing things into the archive name (as now), but keeping them separate. To uniquely identify an archive, you'ld either give its ID or enough tags / attribs, so that the matching set size is 1. To identify bigger sets, one would give less tags / attribs, like for prune.

BTW, if we implement this, it would need some rather strong recommendation (with explanation) in the docs, that archive names with "/" will now trigger non-flat fuse archive directories and that having a reasonable hierarchy in the names will matter for that. Suggest fqdn/datasetname/datetime.

[textshell]

This looks to me like this would cause more confusion than help. For machines and humans alike.
Borg mount is a major feature and esp. having archive contents mix with starting points of other archives seems and not having a simple way to detect where an archive root is seems confusing.

[enkore]

textshell and I discussed this a bit, and I noticed that I didn't really explain well what the intended use (from my PoV) is. Borg is one of very few programs that offer properly implemented, strong checksumming, efficient {encryption, deduplication, compression}, rich FS meta-data preservation and fast mounts. [1]

This makes Borg well-suited not only for pure backup use ("I have a bunch of hosts and datasets and a script makes Borg Archives from those every $time_interval"), but also for "structured archival". Now of course you could use this to give backups more structure when browsed in a mount, undoubtedly. Structured archival is somewhat similar to "just" having files on a FS and textshell pointed out as well that this use is more like "borgfs" (as a concept, not the binary name). However, I see the read-only nature of a Borg mount as an advantage, because modifications to existing archived pictures, movies and documents (or similar) should be the exception, not the rule, which is what Borg is built on. A rw mount would not only be unfeasibly complex to implement, but also make damaging archives by accident much easier; we probably all agree that we don't want to bother with rw-borgfs. The nature of using special Borg commands to manipulate archives thus has the notion of more permanent actions, similar to burning a DVD or carving symbols into rock.

This use is a bit different from the more frequent application as a regular backup tool, but IMHO does not only work very well, esp. with this change, but also shares so much, on a technical level, that it doesn't make sense to have a separate project just for that. The fact is that Borg is largely unopinionated about how one structures the contents of a repository, what one stores in a repository or how to retain data in a repository, in other words, that Borg is mostly a deduplicating archiver and not a backup-specific tool. I think besides making Borg easier to use for some ("valid") use cases, the unopinionated approach is a strong argument here. We don't really have a good, technical or maintenance-based reason why this should be explicitly prohibited.

[1]

btrfs is unsuitable, because it uses weak checksums and is only supported on Linux.
ZFS runs on more platforms, but is cumbersome to use on Linux.
Traditional FS encryption is poorly suited for archival, since it cannot use authentication and therefore almost exclusively uses modes that aim to have harder malleability properties, so they are far inferior for data recovery.
Other dedupearchivers exist, but I haven't looked at them in enough detail to be sure that their crypto and other implementation details are solid enough.

[ThomasWaldmann]

how about a practical example of that use case? could maybe also go into docs, if we implement it.

[enkore]

You'd make up some level where you want to routinely add stuff. For example, for a picture collection that is normally already sorted into film rolls (darktable parlance, LR users just have folders) one might use Pictures/<Year>/<Filmroll> for archive names.