Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support DNSLink resolution #13609

Closed
bbondy opened this issue Jan 15, 2021 · 23 comments · Fixed by brave/brave-core#8068
Closed

Support DNSLink resolution #13609

bbondy opened this issue Jan 15, 2021 · 23 comments · Fixed by brave/brave-core#8068
Assignees

Comments

@bbondy
Copy link
Member

bbondy commented Jan 15, 2021

All of this works only when IPFS Resolve Method is not Disabled and not Ask.

  • DNSLink lookup works only for pages with 5xx response codes without a check for x-ipfs-path header and according to the IPFS Resolve Method settings above

How DNSLink lookup works:

  • it looks for a DNS TXT record of next format dnslink=[value] for _dnslink.[hostname] domain. It should meet current dns preferences and use DoH and Secure DNS according to the settings. DNS requests should be cached and work without sending second requests to DNS servers for same hostname.
  • If the lookup is successful and the dnslink is present it shows the Open in IPFS button in location bar, the button navigates to ipns://[hostname]

For pages with other response codes we check for x-ipfs-path header in response and according to the IPFS Resolve Method settings we show the Open in IPFS button in location bar without lookups, the button navigates to ipns://[hostname]

We should not show the botton for IPFS Gateway urls.

Screen Shot 2021-01-15 at 1 12 19 PM

Example TXT record:
dnslink=/ipfs/<CID for your content here>

@lidel
Copy link

lidel commented Jan 21, 2021

Sweet! This will be fantastic onboarding experience and I feel we should do this as soon as we can.
(cc @autonome @jessicaschilling @Gozala)

tip: if you find it too expensive to run additional DNX TXT lookup for every visited website, you can optimize things by:

  • doing the lookup only when x-ipfs-path HTTP header is present
    (so only if site announces that it's also available on IPFS)
  • or if page returned code different from OK 200 or failed due to network/DNS (eg. invalid A record)
    (this will make it possible for people to recover from broken/DDoSs HTTP servers etc)

@bertrandfalguiere
Copy link

bertrandfalguiere commented Jan 21, 2021

I think the DDoS use-case is very important.
I can see the headline in tech news after the next inevitable DNS DoS: "How your website could have been up when the Internet was down last Friday Step one: put a DNSLink to your website, etc...."

In parallel, onboarding of website should be made as easy as possible (but i'm a bit off-topic here).

@jessicaschilling
Copy link

Love it! Small nit: would Open with IPFS or Open using IPFS be more accurate?

@bbondy
Copy link
Member Author

bbondy commented Jan 21, 2021

@jessicaschilling good call

@Gozala
Copy link

Gozala commented Jan 26, 2021

Would be nice if opt-in in here would be persisted in some way, so that next navigation would go straight to ipfs:// as opposed to prompting user once again. Otherwise I fear user needs to be highly motivated to reload already loaded page.

@bbondy bbondy added the priority/P3 The next thing for us to work on. It'll ride the trains. label Jan 26, 2021
@bbondy
Copy link
Member Author

bbondy commented Jan 26, 2021

@Gozala see here:
#13611

@spylogsster spylogsster self-assigned this Feb 19, 2021
@spylogsster
Copy link

spylogsster commented Feb 22, 2021

as decided in slack chat

  • we have to check original domain + _dnslink.[domain] prefix for TXT records, for example,
dig +short TXT _dnslink.en.wikipedia-on-ipfs.org
"dnslink=/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco"

dig +short TXT brantly.eth.link
"dnslink=/ipfs/QmeXGvLAsLwuq4uDLqsH3XECWZYiJrAszRvkDuyR91ERQX"
  • we should not do this lookup in private windows and tor windows

@diracdeltas
Copy link
Member

@bbondy does this only show up for users who already have Redirect IPFS resources to the configured IPFS gateway enabled?

@bbondy
Copy link
Member Author

bbondy commented Feb 25, 2021

@bbondy does this only show up for users who already have Redirect IPFS resources to the configured IPFS gateway enabled?

It would show up any time the x-ipfs-path header is present in a response body of the main frame.

@diracdeltas
Copy link
Member

It would show up any time the x-ipfs-path header is present in a response body of the main frame.

i don't think this should be running for users who've already set IPFS to disabled. suggested that in the code review and it was added here: https://github.com/brave/brave-core/pull/8068/files#diff-b2b99579204c3baf1314de0455a345809f2fee17c392a9a8b612dd839e99be9dR147

for users whose setting is 'ask' (the default), i confirmed it shows the interstitial page when 'open in IPFS' is clicked. 👍

@bbondy
Copy link
Member Author

bbondy commented Feb 26, 2021

I agree on the disabled state, good catch. Thanks!

@diracdeltas
Copy link
Member

  • or if page returned code different from OK 200 or failed due to network/DNS (eg. invalid A record)
    (this will make it possible for people to recover from broken/DDoSs HTTP servers etc)

it looks like the current implementation will initiate 2 dnslink lookups for every status code from 500-504 (https://github.com/brave/brave-core/pull/8068/files#diff-b2b99579204c3baf1314de0455a345809f2fee17c392a9a8b612dd839e99be9dR182-R185)

given the extra scrutiny on brave network requests lately, perhaps we should ask the user for consent to do these lookups if they aren't already opted into IPFS? for instance show a prompt similar to the one for the wayback machine.

@bbondy
Copy link
Member Author

bbondy commented Feb 26, 2021

@diracdeltas @spylogsster @lidel how about so we can make progress here we limit to only when an IPFS selection is made away from Ask and not Disable. I.e. only for Gateway and Local options. We can discuss what to do with Ask in a different issue.

@lidel
Copy link

lidel commented Feb 26, 2021

@bbondy sgtm.

  • Let's unblock this for users who gave clear consent to enable IPFS ("Gateway" or "Local")
  • As for "Ask" behavior, ok to add it later, I like the idea suggested by @diracdeltas that it could be a similar/same consent UI as for 404 errors and existing Wayback Machine integration.

@diracdeltas
Copy link
Member

  • Let's unblock this for users who gave clear consent to enable IPFS ("Gateway" or "Local")
  • As for "Ask" behavior, ok to add it later, I like the idea suggested by @diracdeltas that it could be a similar/same consent UI as for 404 errors and existing Wayback Machine integration.

sgtm!

@spylogsster
Copy link

spylogsster commented Mar 3, 2021

as discussed in slack with @lidel we should ignore dnslink value and the button should navigate to ipns://[domain], for example:
in the case when https://en.wikipedia-on-ipfs.org has TXT record dnslink=/ipfs/QmeXGvLAsLwuq4uDLqsH3XECWZYiJrAszRvkDuyR91ERQX we show in address bar ipns://en.wikipedia-on-ipfs.org and the button navigates to ipns://en.wikipedia-on-ipfs.org

@spylogsster
Copy link

Updated description to meet new conditions after some discussions in slack

@stephendonner stephendonner added the QA/In-Progress Indicates that QA is currently in progress for that particular issue label Mar 10, 2021
@stephendonner
Copy link

Verified PASSED using nightly on macOS build

Brave 1.23.31 Chromium: 89.0.4389.86 (Official Build) nightly (x86_64)
Revision 0c2dac31bc21e806a791641316e99920f84ea52c-refs/branch-heads/4389@{#1472}
OS macOS Version 11.2.3 (Build 20D91)

Verified:

  • brantly.eth.link and en.wikipedia-on-ipfs.org load tabs with Open in IPFS on the right-hand side of the URL bar
screenshotsScreen Shot 2021-03-10 at 2 45 08 PM
  • when Automatically redirect to IPFS pages via DNSLink when possible is enabled, loading brantly.eth.link and en.wikipedia-on-ipfs.org loads ipns://en.wikipedia-on-ipfs.org/wiki/ and ipns://brantly.eth.link/#/ respectively
screenshots Screen Shot 2021-03-10 at 2 26 45 PM
  • Learn more links to https://docs.ipfs.io/concepts/dnslink/
  • brantly.eth.link and en.wikipedia-on-ipfs.org load tabs via New Private Window and New Private Window with Tor, but do not offer to Open in IPFS, nor do they; they only load their HTTPS links, https://brantly.eth.link/#/ and https://en.wikipedia-on-ipfs.org/wiki/, respectively, with Automatically redirect to IPFS pages via DNSLink when possible enabled
screenshots

Preffed off:
Screen Shot 2021-03-10 at 2 26 11 PM

Preffed on:
Screen Shot 2021-03-10 at 2 49 53 PM

I also verified that when both: 1) Method to resolve IPFS resources is Disabled and 2) Automatically redirect to IPFS pages via DNSLink when possible is also preffed off, we don't auto-resolve or offer to load IPFS resources, and the URLs are loaded over their original HTTPS counterparts

screenshots Screen Shot 2021-03-10 at 3 02 49 PM Screen Shot 2021-03-10 at 3 03 13 PM

@stephendonner stephendonner added QA Pass-macOS and removed QA/In-Progress Indicates that QA is currently in progress for that particular issue labels Mar 10, 2021
@maxihatop
Copy link

maxihatop commented Apr 28, 2021

I think, your current algorithm:

DNSLink lookup works only for pages with 5xx response codes

is not good, because of:

  1. If I would like to use IPFS for deploy file, this is assume, I do not use web-server.
  2. If I must use "fake web-server", always returns error 5xx, then this will drop decentralization of IPFS nature: whenever server blocked, there no way to send to Brave status code 501, and activate DNSLink. As result, blocked centralized server will block access to decentralized IPFS content.
  3. More worst scenario - when server is seized, and start returns fake answers with code 200. In this case, Bravo will automatically switch to answers from that server, and ignore IPFS-files. Client unable to aware of such "upgrade", and as result, security of such approach reduced to security of centralized server.

I think, more nice and correct idea - to use IP-address 0.0.0.0 (INADDR_ANY) as a flag of "there is no server, use dnslink".
Just for example:

  1. User create DNS-record "example.com", contains A-record as "A=0.0.0.0"
  2. User creates subdomain "_dnslink.example.com", contains regular dnslink record, as same as in using now.
  3. When browser receives URL, it anyway calls gethostbyname(). If it receives "A=0.0.0.0", it instantly start lookup DNSLink.
  4. As result, this is quick and efficient algorithm for seamless using DNSLink in the browser.

As an alternative: instead of A=0.0.0.0 we can use 127.0.0.127. This address included into loopback netmask /8, and actually don't used widely. We can reuse it as a flag for "No site, search DNSLink"

@lidel
Copy link

lidel commented Apr 28, 2021

There is no need for special convention around A record: if there is no HTTP server to return error 500, then there is a network error of some sort, and that should also recover via DNSLink if TXT record is present.

IPFS Companion explicitly listens for specific error events and triggers DNSLink lookup for recovery:

  'net::ERR_NAME_NOT_RESOLVED', // dns failure
  'net::ERR_CONNECTION_TIMED_OUT', // eg. http server is offline
  'net::ERR_INTERNET_DISCONNECTED' // no network

Brave could do the same (if it does not already).

Sidenote about threat model: if block occurs at DNS level, all bets are off, because the same entity that censored A record can remove TXT one. If you want to protect yourself from that level of threats, you should not use DNS, need to use cryptographic addresses like IPNS.

@Jeff-Bouchard
Copy link

There is no need for special convention around A record: if there is no HTTP server to return error 500, then there is a network error of some sort, and that should also recover via DNSLink if TXT record is present.

IPFS Companion explicitly listens for specific error events and triggers DNSLink lookup for recovery:

  'net::ERR_NAME_NOT_RESOLVED', // dns failure
  'net::ERR_CONNECTION_TIMED_OUT', // eg. http server is offline
  'net::ERR_INTERNET_DISCONNECTED' // no network

Brave could do the same (if it does not already).

Sidenote about threat model: if block occurs at DNS level, all bets are off, because the same entity that censored A record can remove TXT one. If you want to protect yourself from that level of threats, you should not use DNS, need to use cryptographic addresses like IPNS.

or use a Real decentralized DNS Server and there's only one at the moment that's RFC-1035 compliant: EmerDNS. The rest are glorified bookmark managers. no offense intended.

@kay-is
Copy link

kay-is commented Jun 18, 2022

I tried this after creating a DNSLink domain for a page, and it didn't work.

If I activate "Automatically redirect to IPFS pages via DNSLink when possible," it would lead to a "could not resolve name" error.

If I deactivated it, everything would work as normal, but then Brave had basically the same behavior as all other browsers, which makes its IPFS features kinda moot.

@maxihatop
Copy link

maxihatop commented Jun 19, 2022

I assume, soemthing wrong on Brave side. Or, you do not use OpenNIC (or another emerDNS) resolver. I tested it with Mozilla IPFS plugin, everything works perfect, see the screenshot: http://olegh.ftp.sh/emercoin/ipfs-emerdns.png
And this is DNS record in the emerDNS blockchain: https://explorer.emercoin.com/nvs/dns/ipfs-docs.emc/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.