Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: preload visited DNSLink URLs to local node #827

Merged
merged 1 commit into from
Dec 10, 2019

Conversation

lidel
Copy link
Member

@lidel lidel commented Dec 9, 2019

For the context see #826 and discussion in ipfs-inactive/docs#405
cc #710

With #826 the data of visited DNSLink websites was no longer getting into the cache of local IPFS node by default.

This PR fixes the data distribution problem with a background task queue that ensures every visited DNSLink asset is preloaded to the cache of local IPFS node, contributing to the swarm. If original HTTP server is down, IPFS Companion will recover (#640) in a new tab.

This preload of DNSLink data is enabled by default, but can be disabled with a toggle on Preferences screen:

2019-12-09--16-05-19

@lidel lidel changed the title feat: preload DNSLink websites in background feat: preload visited DNSLink URLs to local node Dec 9, 2019
With
#826
data was no longer getting into the cache of local node.
This adds background queue that ensures every visited DNSLink asset
is preloaded to the cache of local IPFS node.
@lidel lidel force-pushed the feat/background-preload-dnslink branch from 2b7c145 to 9344b6f Compare December 9, 2019 16:09
@lidel lidel added this to In progress in Resilience & Offline via automation Dec 9, 2019
@lidel lidel marked this pull request as ready for review December 9, 2019 17:21
@cwaring
Copy link
Member

cwaring commented Dec 10, 2019

This looks great, a couple of questions:

  1. Will companion only cache visited pages or in the case of a root CID does it recursively cache the entire website? (considering bandwidth and space required with the latter option)
  2. What happens if a PWA loads from the local browser cache and displays the website while HTTP is down, will companion still attempt to recover from IPFS in a new tab that scenario?

@lidel
Copy link
Member Author

lidel commented Dec 10, 2019

Ad 1: only visited pages (requested URLs)
Ad 2: If it loads from cache, it will work fine. Companion recovers by opening URL in a new tab only if the main request (from location bar) ends with network error or HTTP error.

@cwaring
Copy link
Member

cwaring commented Dec 10, 2019

With both those scenarios that makes this implementation ideal 👍

@lidel
Copy link
Member Author

lidel commented Dec 10, 2019

Cool, let's merge this.
This feature will be useful to users who decide to disable DNSLink redirect via Preferences.
(I will flip the default setting in separate PR, keeping this one on topic).

@lidel lidel merged commit 6c37c6a into master Dec 10, 2019
Resilience & Offline automation moved this from In progress to Done Dec 10, 2019
@lidel lidel deleted the feat/background-preload-dnslink branch December 10, 2019 17:50
@lidel lidel added this to the v2.10 milestone Dec 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

None yet

2 participants