Feature Request: [Re]add IPFS support

[tswast]

Beaker works great, though I wish there was more p2p web content out there it could access.

It seems that IPFS has more web content than Dat right now. For example, there is a distributed version of Wikipedia, especially relevant after Turkey blocked access to Wikipedia.

Also, neocities.org has made good progress in getting their content hosted on IPFS.

Regarding the URL spec (#455), I agree that Dat's makes more sense for browsers. I wonder if the browser could use ipfs://hash, ipns://key-hash, or ipns://domain.name style URLs instead and convert them before passing them on to the IPFS daemon?

The version logs are nice, too, though it sounds like that could actually be a problem in the case of neocities content. In the linked bug, Kyle says

Some of our sites have tens of millions of versions.

That's probably too much for a browser to store. It's nice to have a complete list of versions, but maybe it shouldn't be forced.

[pfrazee]

We'll consider it. There are a lot of IPFS fans, and we'd like to support them. What we're trying to balance is, the cost of supporting a second protocol when the two protocols are so similar.

The problem with IPFS's NURIs is, to support conversion, we either need to fork Electron or inject a script into every web page. Both options are brittle; Electron/Chromium bakes in a lot of assumptions around the "standard url" format. (@zeke I'd be interested to hear your thoughts on that.) We'd be more willing to do it if IPFS fully sunsets NURIs, so that the conversion-code could be temporary.

RE the issue of long histories, Dat will have the ability to partially replicate the history log, to improve performance.

[interfect]

https://github.com/ipfs/js-ipfs is coming along; a new IPFS implementation might want to use that rather than relying on an external daemon.

Also, I'm having trouble understanding why an IPFS implementation would have to be so intrusive. What Electron assumptions in particular need to be worked around?

I think you would (and it looks like the old implementation did) use a structure like fs:/ipfs/QmU1GSqu4w29Pt7EEM57Lhte8Lce6e7kuhRHo6rSNb2UaC as specced out in ipfs/kubo#1678 (comment). Anything that says href="/ipfs/whatever" in an HTML document stored in IPFS should get interpreted as a relative link. Everything that's deployed right now is set up to be accessed from a gateway and uses relative paths already.

What would the injected script or Chromium fork need to do?

[pfrazee]

Hey @interfect, nothing's really changed on this front. IPFS is good tech, but so is Dat. We're not against integrating IPFS perhaps down the road, but for now we think it'd be a distraction from other more beneficial work that we could be doing, like improving site-creation flows. Adding a second protocol just creates the opportunity for subtle differences and new challenges.

About the URLs: The non-standard URL form doesn't have an origin definition. That causes anything related to the Single Origin Policy to fail, and creates issues for tasks like redirects. Consider: any code that's designed to parse URLs will need to be updated to understand this new scheme. For what? I personally feel like it's a needless change to a standard that didn't need changing. If we ever support IPFS, it will be with the ipfs://{hash}/{path...} form, and we'll need to create code to convert away from the NURI form wherever it's used.

[olizilla]

Good news! IPFS is moving to base32 by default for CIDs https://github.com/ipfs/ipfs/issues/337

This is so they can be used as the authority component of URLs, in the form you suggested ipfs://{hash}/{path...}, and in subdomains for http-to-ipfs gateway requests, so they can offer sensible isolation in the context of the same-origin policy.

[pfrazee]

@olizilla glad to hear it!

[zeke]

@olizilla I'm not versed in the terminology here. Does this update imply there's hope for IPFS support in Beaker?

[pfrazee]

@zeke it solves a major blocker to having IPFS, but we're not going to put IPFS on the roadmap in the near-term.

[zeke]

@pfrazee. Gotcha. Thanks for the clarification.

[pfrazee]

Hey, sorry all. We're not looking to add any new tech at this point, though we'll keep discussing it