[2020 Theme Proposal] IPFS in Rust

[Gozala]

Theme description

Rust is a high performance language with on garbage collection & type system that provides memory safety guarantees that no other system language offers that make it a great fit for performance critical software and constrained environments like embedded devices. Furthermore, Rust has best support for WASM and working groups collaborate tightly to ensure that uncompromising results.

Core needs & gaps

At the moment there are two primary implementations of IPFS in Go and JS & it is not uncommon to incur incompatibilities or features that take longer to roll out while both implementations catch up. Implementation effort is duplicated & testing is also complicated as everything needs to be verified for interoperability or assumed to have no issues at risk of having those.

As IPFS becomes more mainstream integrating with system software like browsers becomes more relevant and both Go & JS implementations seem like an inadequate solutions for browser vendors to consider. Rust implementation could provide a viable option. Some vendors are already shipping some rust and others can easily embed it as compiled system library without runtime overhead.

In short rust can make IPFS applicable to wider range of use cases & at the same time reduce implementation / maintenance / quality control efforts.

Why focus this year

The longer we wait the more difficult task it becomes due to growing code base and consumer base, not to mention the longer we wait the slimmer the chance of migration become.

Additionally there seems to be an effort to revamp APIs and specs (e.g. new stuff coming in IPLD, JS code base moving to async await, etc...) Rust type system provides a great way to document interface to much greater detail (than go) and ensure that it's true to implementation. Having well defined & documented IPFS (that never gets out of sync with implementation) would really help adoption.

WASM system interface is in active development and so is rust async runtime which is to suggest now might be an only opportunity to inform & influence design of both. Later would imply hoping for the best and living with a consequences.

Milestones & rough roadmap

• IPLD in Rust
• IPLD rust replaces IPLD in go through a system library
• IPLD rust / wasm replaces JS IPLD
• Rust IPFS
• WASM IPFS
• WASM IPFS effectively replacing JS IPFS, or rather JS IPFS becomes a thin wrapper of WASM IPFS
• Rust IPFS replacing other component of Go IPFS
• Go IPFS is just a Rust IPFS wrapper for backwards compatibility

Desired / expected impact

How will we measure success? What will working on this problem statement unlock for future years?

• 📉 JS / Go duplication effort reduced.
• 📊 Target runtimes Rust implementation unlocks.
• 📉 System resource usage in comparison to existing implementations.

[mikeal]

ping @vmx who is leading the Rust-IPLD efforts.

[atenjin]

interesting.
we are a blockchain team Polka(https://github.com/PolkaX) in China. In fact we are interesting in ipfs and developing our own rust-ipfs: https://github.com/PolkaX/rust-ipfs.
maybe this could be another version for "Rust IPFS"

[vmx]

@atenjin thanks for mention that! I wasn't aware of that effort. I'll take a look at what you already have.

I currently try to get all people interested in a Rust IPFS to work together on a solution so that we can save time with collaboration instead of having many double efforts.

For IPLD related things, the development happens at this repo: https://github.com/ipfs-rust/rust-ipld.

And there is also the Multiformats org, which contains a lot of the lower level building blocks. I currently try to get all forks/changes merged into a single source of truth and also improving the APIs (like e.g. multiformats/rust-multihash#45).

[Ericson2314]

https://github.com/libp2p/rust-libp2p is the most developed IPFS-related thing in Rust today. I'm sure all you commenters know that, but just dropping this here for anyone that's just browsing.

[vmx]

For those following this issue, a lot of development is currently going on here: https://github.com/ipfs-rust/rust-ipfs/

[NatoBoram]

Was Go->wasm attempted?

https://github.com/golang/go/wiki/WebAssembly

[Stebalien]

Yes. There's even a demo PR: ipfs/kubo#5694.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.