data modeling is not portable across backends

[seliopou]

The Irmin_mem backend allows the user to store values at any point within key hierarchy, while the Irmin_git.FS backend does not. Rather than trying to describe the problem further in prose, consider the following pseudo-ish code:

Store.update (v "fst one") ["fst"] "ok" >>= fun () ->
Store.update (v "snd one") ["fst"; "snd"] "maybe?"

If Store is created from the Irmin_mem.Make functor, then both operations will succeed. On the other hand, if Store is created from the Irmin_git.FS functor, the second operation will fail with a Unix.Unix_error(Unix.ENOTDIR, "chdir", "/path/to/repo") exception.

The behavior I expected was for data modeling that I perform and verify to be correct using one backend will be just as correct when I switch to another backend. The behavior I observed was that this is not the case, at least for the in-memory and git backends.

I'm working with irmin 0.9.4.

[samoht]

This is indeed a bug due to changing back and forth the behaviour between releases. I am now convinced that the proper behaviour is to disallow contents in the nodes, e.g. to look like a normal filesystem. The Irmin_mem backend needs to be fixed.

Thanks for the report!

[samoht]

btw, 0.9.5 should be a bit better regarding the error message of the Git backend.

[samoht]

The new behaviour is now much stricter:

A node cannot be at the same time a filename and a directory anymore.
Irmin.update always overwrite the node kind:

• if /fst is a file, then Irmin.update /fst/snd foo will make fst become a dirname.
• if fst is a dirname, then Irmin.update /fst foo will make fst become a filename. All /fst/** will be automatically deleted.

This also needs a patch in ocaml-git to fix non-bare Git repositories.

[seliopou]

One other possibility that crossed my mind last week was to allow any key to be used as a "file" and as a "directory". So, for example, you have the following operations

Irmin.update "/fst" "ok"
Irmin.update "/fst/snd" "maybe"

the filesystem this would produce is

/
|-- fst
    |-- __value__
    `-- snd
        `-- __value__

While this makes for a slightly messier filesystem, it provides much more flexible API. I suppose one drawback of this approach is that the semantics of delete aren't completely obvious.

[samoht]

This was indeed exactly like that at one point, but it causes all kinds of problems. I'm still unsure about the correct solution for this, but at least now it is consistant.

[seliopou]

👍 Feel free to close when the PR lands.