Hosting Web Apps on IPFS #405
Comments
cwaring
added
dif/hard
|
I love the idea of pursuing the goal that helps the most real-world developers host sites on IPFS without having to alter the rest of their stack, including in the very common pattern of client-side routing. Quick clarifying question on this aspect of Option 1, and apologies in advance because I know my vocabulary isn't up to asking it properly:
Is this about... (A) letting people choose between accessing the site on "THE gateway" - the thing we run ourselves that is sometimes down - versus "THEIR gateway node" - which I assume is the equivalent of a local/offline cache, only available if they've visited and pinned the site before? (B) giving people the option to view the site on the "normal internet" as a fallback if "the gateway" (access to IPFS) is down? Kudos in advance to whoever can parse my question. :) |
use base32 cid v1 flavour and use dweb.link as the preview domain. see: ipfs-inactive/docs#405 License: MIT Signed-off-by: Oli Evans <oli@tableflip.io>
|
how do you like these apples ipfs-shipyard/ipfs-dns-deploy#7 |
|
@olizilla I like these apples very much, they are delicious |
|
@terichadbourne Yes! it's about always loading the "normal internet" (aka http) version first, and informing the user that an ipfs flavour version of the site is available if they would rather fetch it that way. I assume it would also mean "fallback to ipfs" if http is unavailable. |
|
Starting with IPFS Companion v2.9.0.872 user will load DNSLink website in "normal" way by default (over original HTTP server), but still get context actions including new one "View on Gateway" for opening via Gateway (public or local) in a new tab. If To borrow illustration from ipfs/ipfs-companion#826 (comment): |
|
I’m not sure I’ve fully processed option #1 - but my understanding is the suggestion means the vuepress site will still be broken when loaded over ipfs, but to effectively turn off loading over ipfs without explicit user interaction for most people browsing with companion/desktop so they’re less likely to experience this issue in our docs site. Is that an accurate summary? In that model, what would happen if I tried to reload the site from my local gateway (via the button you describe), cohost it using ipfs-cohost, share it with someone else by ipns or ipfs cid, or otherwise fetch the site over the ipfs network? Is there a different answer if I’m using an embedded js-ipfs node vs ipfs-desktop? More broadly, I don’t think we want to default ipfs-powered sites to use http on first load. Can you explain more why you think that is necessary (@lidel)? Of note, I don’t think a performance reason should apply here - this feedback loop is actually extremely critical for users of companion (frequently developers dogfooding) to understand how long it takes to load from ipfs and we don’t want to remove that dogfooding option from our network use. The solution to that problem is making ipfs more performant, not relying on http for first load. No one is going to regularly use a drop down menu in an extension, even if we make some sort of alert that”this site is ipfs-powered!” - clicking that button would just reload the site slowly, making ipfs look even worse in comparison - so I don’t see that as a mollifying replacement. Most concerningly, this transfers a large number of requests from the actual ipfs network (right now resolved with js-ipfs in companion or go-ipfs in ipfs-desktop) to the ipfs http gateway - which is the opposite direction than we want to be heading and significantly reducing the frequency with which users would interface directly with the ipfs network.
Edited: let's have a meeting on Monday CET to discuss the pros/cons of this change before surfacing to @jbenet (IPFS BDFL) for approval/ack. |
|
I've wanted this for a long time ipfs/ipfs-companion#710 |
|
I too look forward to hearing a little more about the order of operations for Option 1. In my experience with standard Offline First patterns and PWAs, I've found that the best user experience is to load first from local cache if you have it (fastest possible load time and avoiding getting stuck in LieFi state but possibly outdated information), then immediately try to pull an updated version from the network. What counts as "the network" is the challenge here. 😂 (And I'm super excited to see the goal of making our docs accessible offline via a PWA! 🎉) I'd also love clarity on what the Option 1 experience would be like if I visited the new docs site having never installed IPFS or IPFS Companion. We've pushed a lot of content that would normally be on a marketing site into the docs site, so I don't think we should assume that the only folks referencing it are IPFS aficionados. Just a few thoughts from the vantage of someone who's a self-appointed advocate for newer users, among whom I see "(How) can I host my web site / web app on IPFS?" as one of the most frequently occurring questions... By pure numbers, there are far more traditional web developers out there than there are DWeb/IPFS enthusiasts who already build on IPFS and have all of our tools installed, and I happen to be on the "more familiar with traditional web dev, new to IPFS" end of that spectrum. The audience of traditional but DWeb-curious web devs -- including those rallying around Offline First and PWAs -- is a huge potential market for adoption in my opinion, and one we should keep in mind as we make decisions. What would they do if faced with our current decision-making process, and how can we best serve them moving forward? If a traditional web dev were already familiar with tools that met her needs in web app development (VuePress for a documentation site, tools that easily support client-side routing, traditional PWA stacks, etc.), and she bumped into a poor fit between IPFS and those trusted tools, which would win out... her curiosity to try out hosting on IPFS or her reliance on trusted tools of the trade? Recognizing the use cases for which IPFS isn't currently the ideal solution, and responding by making IPFS compatible with a range of existing tools to resolve that (but only when we consider that use case to be a key one for us) is key to increasing adoption and improving our user experience, and I really appreciate the careful thought being put into long-term solutions that will meet the needs of diverse users. ❤️ My concern with creating a plugin for Vuepress (Option 2) or solving things upstream in Vuepress (Option 3) is that both approaches solve a compatibility issue within a very limited scope (as I understand, in a way that won't also apply to the many other stacks with this client-side routing challenge), leaving the problems of hosting web apps on IPFS unaddressed overall. I would definitely lean towards finding solutions within IPFS itself where can have a greater and broader long-term impact. Unless we expect future users to individually create plugins that make IPFS compatible with specific widely-used tools for building web apps, the process of doing so ourselves strikes me as a bit contrary to the dogfooding concept of putting ourselves in our users' shoes. ¯_(ツ)_/¯ As a newer IPFS user from the traditional web dev community, I'm really excited about the possibilities offered by Option 1 as I understand them, including:
Thanks to all of the less-new users who are taking the time to dig into all of these options so thoughtfully. You rock! 🎉 |
|
I think the proposed changes to companion call for a design review meeting. What we decide and spec out from that review should also get input from @jbenet to ensure we have longer-term roadmap alignment. Let's keep surfacing details and constraints on this thread, and synthesize into prioritized decision making criteria to refine during that meeting. I propose @lidel as ultimate decision-owner responsible for synthesizing input from key parties and making the final call. @lidel - can you propose a deadline by which we can surface and synthesize feedback & make a call on this decision? I'll be in CET next week - can you schedule a design review call for ~Monday/Tuesday to unpack this more? Thanks!! |
|
(I'll try to be brief 🙈 ) TL;DRI believe the underlying idea behind Option 1 is to double down on our efforts to move towards subdomain gateways and Origin-based security to make it possible to use existing web tools like vuepress as-is, and stop patching other people's software, or requiring people to use custom build configuration when publishing on IPFS (hugo, gatsby, vuepress.. what will be next? we could be spending this energy elsewhere 🚀). On changes in ipfs-companion v2.9.0.872
A side effect of ipfs-companion change helps with docs site, but it was not the main motivation, only a signal to do the change sooner than later. Context: The change means ipfs-companion should stop redirecting DNSLink websites until switch to CIDv1 is done and we have native support for subdomain gateway in go-ipfs that works with Note: even without immediate redirect we could decide to prefetch data to IPFS node in the background, making it available in offline contexts.
I believe the change proposed in v2.9.0.872 will make performance eyeballing bit easier: user will see the difference between public gateway load and one produced by For constant dogfooding, the old behavior can be restored via Preferences, but this is now a conscious decision that the user is okay with lifting Origin isolation guarantees from all DNSLink websites. I believe this temporary change is an overall net gain for Companion users (not only improves security of the default state, but gives us some breathing room until DHT is improved). On vuepress loaded from local gateway
IIRC the idea for Option 1 is to run a small JS snippet to detect when page is loaded from Caveat: we don't expose subdomain gateway on Design Review Call (Tuesday 4pm CET)
@momack2 ack, I created a meeting in IPFS Community calendar on Tuesday 4PM CET to give everyone time to provide feedback in this thread and via other means before we synthesize during the call. On my end the decision blocks releasing Companion to the Stable channel, but there are no critical fixes in it so it can wait: publishing to Chrome Web Store takes about a week anyway. Would be great to resolve it before Friday 13th tho 💀 🙈 |
|
Great, let’s make Thursday our official decision deadline then to allow a release on Friday. SGTY? |
|
Thanks so much, @lidel, for all this context. I’ve unfortunately got an immovable dentist appointment during that time, but not sure how much value I have to add to the meeting that can’t already be provided by @cwaring, @johnnymatthews and @terichadbourne. Much appreciated! |
|
PS - just confirming Tues, not Thurs, as that’s what I currently see on the calendar. |
I want to emphasize that this particular bit is not optional for the project as a whole. Bypassing the same-origin policy is a serious security concern, and IPFS HTTP gateways are configured with this problem out of the box. We've long known this, and as the various linked issues this thread show, we intend to fix it. The CID-in-subdomain part of option 1 is required for our HTTP compatibility story, docs website decisions aside. This is an opportunity to accelerate getting there.
This feels like the metrics tail wagging the vision dog (wow, apologies for that abomination of a sentence!). We should prioritize the design and implementation of a safe and functional IPFS in order to grow long term adoption over short term numbers. IIRC also we've got ~25k users of Companion, based on the install numbers from Chrome/Firefox extension stores? That seems pretty small compared to overall gateway traffic figures. And of those Companion users, do we know how many have Desktop installed as well? |
|
I know there is a lot of people here are doing some deep thinking in the best solution to this challenge and I understand that there are new ideas to this old problem. I do want to share some notes from the dinner convo I had with @cwaring yesterday. I believe we yielded some great litmus tests that can aid to make our final decision. Take the following just as things to consider :)
|
jessicaschilling
added
topic/design-front-end
|
Design decision has been made, and will be issue-ized and worked shortly. |
|
I'll post meeting notes and summary of next steps on Companion end later today/tomorrow morning. |
|
Thank you all for contributing to this conversation in text form and during the call ❤️ Meeting notes can be found at ipfs/team-mgmt#1081 Below is my understanding of decisions made. DecisionsIn a spirit of disagree and commit, our path to move forward is:
In the future, we will try to eat smaller slices of the decision cake 🍰 |
|
Thanks for the great preparation, summary, and ownership @lidel!! Appreciate all your great work and preparation making sure we came into a deeply technical and interwoven topic with the needed context and background! 👍❤️🎉 |
This change restores DNSLink redirect to local gateway as the default setting: https://docs.ipfs.io → http://127.0.0.1:8080/ipns/docs.ipfs.io/ During the Design Review Call for we've decided to keep the default user experience of loading DNSLink websites from local gateway by default. Ref. ipfs-inactive/docs#405 (comment)
|
Closing this issue, since work is being executed in a variety of other locations. See #271 for work on hosting the docs site on IPFS. |
Thanks to @hugomrdias and @lidel for chatting through all various scenarios yesterday, below are a few thoughts and next steps balanced against time/effort and overall impact.
The first (and perhaps most important) thing to note is that this problem is not isolated to VuePress, and is currently causing many common web applications with client-side routing and absolute paths to appear broken when loaded via a gateway URL, such as
http://gateway.ipfs.io/ipfs/<cidv0>.In order to make IPFS a first-class citizen for hosting web apps, the consensus is that we should move to subdomain model as soon as possible (e.g.
https://<cidv1b32>.ipfs.dweb.link).This provides origin isolation for app security and session storage, while also fixing client-side routing and asset issues that people are facing with some of the most popular webpack application bundlers that currently rely on absolute paths (a few examples are Next, Gatsby, create-react-app, Nuxt, and VuePress).
Additionally, this change will provide familiarity to new users attempting to host these type of web apps on IPFS — thus lowering the barrier to adoption and increasing our reach.
The subdomain pattern is a familiar one for developers. Most hosting services use a
https://<subdomain>.<service>.<ext>pattern, and with the recent efforts/refactors across our projects we are very close to making this a reality for IPFS too. To that end, we believe we should invest our effort towards that direction instead of patching frameworks on a case-by-case basis.With that said, here is a summary of our current options:
Option 1: Make IPFS Companion "redirect to gateway" opt-in and default to dweb links
[certainty: high, effort: medium, impact: high]
This option ensures that IPFS Companion will not automatically push users to load a resource via their local gateway node by default, leaving people with a broken experience when an app fails to load correctly. @lidel also suggests that we should move to an opt-in model, so that users have the option to explore sites via their local gateway instead of forcing this as default behaviour. Ultimately, an in-browser proxy would allow us to use
cidv1urls locally alongside dweb links; however, this won't be the case until further work has landed in the Chrome/Firefox vendor space.For this to work successfully, the following tasks must be completed:
cidv1b32by default with a dweb preview link.cidv1b32dweb link.This does mean that a VuePress site will not run locally until we have a subdomain gateway in
go-ipfsthat we can use as a proxy inipfs-desktop. However, we believe this is an acceptable tradeoff in the meantime, as this is a niche use case — particularly considering that the main app will provide PWA functionality with offline caching via service workers (enhancing the experience on all platforms, including mobile).Related issues/efforts:
Option 2: Create a plugin to rewrite VuePress compiled output
[certainty: medium, effort: medium, impact: low]
As sketched out in this gist, we could build a post-process plugin that rewrites all the absolute paths to relative ones, making an app work regardless of path depth. However, this could have a number of unknown and potentially experience-breaking side effects such as interfering with interactivity, routing, failure of offline caching via PWA, and uncertainty around how plugins may respond.
This is approach would be similar to gatsby-plugin-ipfs and may be acceptable for simple sites — but post-processing compiled assets with regex is a fragile and messy solution. We believe this should only be done as an absolute last resort or a stopgap option.
Option 3: Fix the problem upstream in VuePress
[certainty: low, effort: high, impact: medium]
We could apply development effort to fix this issue upstream in VuePress. Such work would help resolve additional absolute-path-related issues for other users; however, it's strongly unlikely that we could solve the key issues, patch affected plugin dependencies, and land a merged PR in a timely fashion without unnecessarily delaying the launch of IPFS docs and losing valuable beta user testing time.
The effort here is complex, involves consensus with multiple third parties, and only adds value to a niche use-case. If absolutely necessary, we could investigate pursuing this in the new year, but our recommendation is to avoid postponing beta launch by pursuing one of the other options in the meantime.
Related issues:
Option 4: Use GitHub Pages, Netlify or similar (stopgap)
[certainty: max, effort: zero, impact: 🤷♂️]
We are setup this way at the moment and it has enabled us to focus on delivering end user functionality in the docs beta until there was a solution in place to fully migrate to
cidv1b32.Note that Filecoin and IPLD docs are currently configured like this too.
Summary
We are keen to pursue option 1 for maximum overall impact, however I'd like to hear your thoughts. 👂✨
ref: #271
The text was updated successfully, but these errors were encountered: