FUSE mount Mutable File System (MFS)

[vorburger]

It could be neat if ipfs mount would FUSE the Mutable File System (MFS), e.g. on /ipmfs(in addition to /ipfs & /ipns), as an alternative to having to use the ipfs files CLI.

E.g. https://github.com/piedar/js-ipfs-mount or https://github.com/jfmherokiller/ipfs-mfs-fuse appear to have dabbled in this space in the past from what I could tell from a bit of quick research. But I couldn't find any existing issue specifically about this goal in core, so hope you don't mind that I'm adding this to the roadmap here, for high-level tracking. I'm sure it needs to be further broken down into finer grained requirements; I'll tag a number of issues that seem related which I've found while searching.

This could subsequently perhaps be a foundation e.g. for an Kubernetes Container Storage Interface (CSI) implementation backed by IPFS.

I thought about this while dreaming on https://github.com/vorburger/LearningLinux/blob/develop/docs/roadmap/readme.md

[welcome]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[bam80]

ipfs/kubo#5504

[wallabra]

Any progress on this? I imagine it depends on the related problem of writable mounting?

[ruifortes]

let me join the anxiously waiting masses :-)

[wallabra]

I love this, it's like, a bunch of people sitting at the cinema, all waiting anxiously, even though there is no movie scheduled for the day :D

[2minwia]

excite

[thomaseth2]

excited

[wallabra]

excit

[ShadowJonathan]

The ipfs files command becomes cumbersome after a while, please bump this up in priority.

[djdv]

For what it's worth, I had this implemented a few years ago (another demo).

Originally part of go-ipfs itself, it was later requested that I drop that effort and turn it into a daemon plugin instead.
Ultimately, neither were merged into the project.
But people have requested that I continue the effort by developing a standalone utility / IPFS client.

I can try to port this feature to the standalone utility, but I expect the performance will be bad compared to when things were done in/by the node directly.
And there may be synchronization issues due to the remote APIs being limited in contrast to having direct node access.

However, the way I have structured the libraries should make it possible to adapt them into a daemon plugin once again, so the same functionality could be used remotely in the standalone binary or directly by the node itself if the plugin is loaded and a command is added to call it (the latter implies a slightly modified fork of go-ipfs which does just that, unless the plugins API is extended to support adding custom commands).

If people are still interested, I can work on it and post back here when the standalone variant is functional. And consider adapting it into a daemon plugin at a later time.
However, it might make more sense for people to wait for a first-party/official stable implementation instead.

[djdv]

Follow up to #90 (comment)

Sorry for the delay on this, I should have mentioned in the last post that I wasn't going to have time to work on it immediately.
I probably should have since this only took an hour to get a working prototype.

That said, I have the work in progress PR up for my mount utility that mounts an IPFS node's Files API in read and write mode.
It's up here djdv/go-filesystem-utils#37 with a video showing how to build and run it.
For now, users will need the same dependencies as cgofuse
Meaning WinFSP for Windows, and some libfuse implementation for everyone else. 9P2000.L support is planned which has native clients in systems like NT, Linux, and more.

Be aware that it likely contains bugs

If you want to try it, make sure you have backups or that you don't care about the data you're testing with.
I use Windows so that's what sees the most testing. It's likely that something on Linux, macOS, or the BSDs is borked. If you find issues, please report them.

-- Below is extra follow-up commentary (ignorable) --

---

I can try to port this feature to the standalone utility, but I expect the performance will be bad compared to when things were done in/by the node directly.

It's not too terrible, but it's certainly not as fast as the old implementation. And without changes to the IPFS APIs, I don't think we can really improve it that much. The old implementation relied on a custom MFS library I made, that seemed to get less caught up on locking, and added features like symlink file support.
Without a way to replace the root node, and with a hard dependence on the HTTP RPC API, we're kind of limited.

The IPFS daemon plugin API would give us those capabilities though. And I'd have to look into how flexible it lets us get in terms of being able to add custom commands or even replace subcommands like mount when loaded.
I'm not super interested in trying that route though unless there's demand for it. Until people have hard problems with the remote API version, I'm not gonna consider it anytime soon.

---

However, it might make more sense for people to wait for a first-party/official stable implementation instead.

From what I've heard there isn't likely going to be an official implementation nor any real fixes/improvements to the official mount implementation.
I think it's really unfortunate for a project with FS in the name to neglect mounting the file system.
I feel as though it's something a lot of users expect and hold to be important.
And hope this changes in the future.
:^/

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[pirate]

Bump

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[pirate]

Bump

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[pirate]

Bump

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[bam80]

Bump

[2color]

As a community-driven project, IPFS does not have a single roadmap. As of 2023, there are multiple companies and teams building IPFS implementations.

To encourage both innovation and interoperability, the IPFS community focuses on specs and standards and uses the IPIP process as a Lightweight Improvement Process for IPFS Specifications.

Therefore, I'm closing this issue to avoid setting any unrealistic expectations. If someone wishes to pick this up, they're more than welcome to and we encourage to join one of the various working groups.