FUSE interface design

[Kubuxu]

As I am getting finally to split the fuse out of the main go-ipfs daemon here are my thoughts about the design and layout.

I think that using one mount instead of 2 (and possibly 3) is beneficial and that is why I propose following layout:

• /mfs/ - mutable file system root
  • /mfs/ipfs/ - ipfs root
  • /mfs/ipns/ - ipns root
    • /mfs/ipns/local - symlink for local name which is mutable
  • /mfs/.pins/ - pined - only direct and roots of recursive - maybe symlinks that could be named (and act as labels - connected with Build-in notes for pins. kubo#2544) - top level is mutable but only allows symlinks to /mfs/ipfs/Qm...AAA or /ipfs/Qm...AAA
  • /mfs/.local/ - locally available blocks
  • /mfs/... - user files and everything else
• /ipfs/ -> /mfs/ipfs/ - symlink
• /ipns/ -> /mfs/ipns - symlink

Only problem I see is that we won't be able to handle cp command so it doesn't perform byte by byte copy. It would be possible if FUSE was allowing to implement reflink syscall and the default cp was compiled with --reflink=auto instead of --relink=never by default. So in short, the reflink way won't happen. If anyone has any ideas how we can implement zero cost cp command using FUSE, please share.

@whyrusleeping what do you think about general direction?

[Kubuxu]

Other thing we might want to think is making FUSE a global daemon and then allowing each user to see FUSE IPFS subsystem from its' daemon perspective.

It is simple to detect uid of process that is accessing it will be harder to map uid to IPFS API access, unless we create sort of global location for it.

[whyrusleeping]

@Kubuxu lots of good thoughts here. I think we should take this step by step, where:

• step one would be to refactor mfs to use an interface that may be satisfied by either core or an api client
• step two will be to write a fuse binary that can display an mfs filesystem
  • (can be done independent of step one)
• step three get a first implementation of an external fuse binary showing the files api contents of a running ipfs node
• step four expand that binary to do more cool and interesting things like you describe above.

As far as the actual mounts you propose, i like it for the most part, except for the comingling of the files api namespace with the rest of the mounts. That bit might get tricky (what if i want to make a file in the files api at /ipfs ?)

file cloning can be worried about later, its an optimization and most of the time people are used to using a separate tool for doing these optimized clones. In our case this would be as simple as ipfs files cp ....

[Kubuxu]

Of course I don't want to implement it all at once. I will be making small steps closer and closer to final aim.

About merging the mounts, isn't it already like that?

ipfs files cp /ipfs/Qm... /bar

Wouldn't directory named /ipfs/ under the files API be inaccessible by files API?

EDIT: after some experiments, some commands can access it some do not.

[whyrusleeping]

@Kubuxu touche

[Kubuxu]

I also think that we should somehow promote culture of using Files API as it was system root. You never create files in system root, don't do that in Files API either.

[jbenet]

@Kubuxu i will get back to this later, bit it is a HARD CONSTRAINT that ipfs be able to mount at /ipfs and ipns at /ipns. Please read the rationale littered through the old issues as to why we dont just do the easy thing of bundling it all under some other subpath.

[jbenet]

I understand the symlink idea, but that doesn't cover all cases. And symlinks are not enough, synlinks break a lot of things so it MUST BE POSSIBLE (even if not default) to mount directly.

this is another possible route:

ipfs mount <ipfs-path> <local-fs-path>
ipfs mount /ipfs /ipfs
ipfs mount /ipns /ipns

With these you can do what you propose, and in addition someone can do any of:

ipfs mount /ipfs/<hash>/a/b/c  ~/foo/bar/c
ipfs mount /ipfs ~/ipfs
ipfs mount /ipns/<hash>/myfiles ~/sync

[Kubuxu]

I understand that we want to provide /ipfs and /ipns at system root.
I didn't think about symlinks breaking things but we can use bind mounts to bind /iipfs to /mfs/ipfs.

The reason behind it is also that number protocols provided in ipfs path will only grow, inside our world it isn't bad but looking at system root it isn't really desirable.

We soon will have /ipld then /iprs after it and then possibly even more protocols in the system root.

The option to do things you said is in essence mount --bind ... which is also available for all users on Linux based system (I don't know about darvin and for sure not on Windows).

Bind mounts are indistinguishable from normal mounts so I think it would be enough.

I also have an idea for later to allow ipfs mount to be system global daemon which will allow multiple users use same mount paths but using separate, per user, go-ipfs (or maybe js-ipfs) daemons, it would require sort of registry of APIs but I think it is possible. Especially if I get to making API on UNIX based sockets (as then access control is both safe and trivial).

[Kubuxu]

I was wrong about the bind mount, it can't be run by normal user. Still with right design it should be trivial to create multiple mount points referencing same underlying logic.

[vorburger]

ipfs/roadmap#90