New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Custom Protocols in WebExtension #164

Open
lidel opened this Issue Nov 3, 2016 · 13 comments

Comments

Projects
5 participants
@lidel
Copy link
Collaborator

lidel commented Nov 3, 2016

(living summary: updated @ 2018-09)

This issue tracks browser extension support for various protocol schemes and URIs according to four stages of the upgrade path for path addressing and IPFS Addressing in Web Browsers memo, namely:

URLs:

ipfs://{cidv1b32}http://127.0.0.1:8080/ipfs/{cidv1b32}
ipns://{hash}http://127.0.0.1:8080/ipns/{hash}

URI:

dweb:/ip[f|n]s/{hash}http://127.0.0.1:8080/ip[f|n]s/{hash}

This issue also tracks (currently nonexistent) ways to address/workaround how Origin is calculated (Problem #2)

WebExtension APIs

Universal API

No such thing yet (but see WIP work in comments)

  • BUT! We use the most insane workaround in #164 (comment). It works in both Firefox (Desktop-only – #348) and Chrome and is the best thing we can do as of late 2017. It was merged with #283.

Firefox

Chrome / Chromium

Brave

Muon-based (deprecated in 2018)

  • browser.protocol.registerStringProtocol available to trusted extensions
  • missing api for Buffer/Stream-based protocols (#312 (comment))

Chromium-based

  • Basically same challenges as Chromium right now

Related discussions

@lidel lidel added this to the v2.0.0 milestone Nov 3, 2016

@lidel lidel changed the title Support Custom Protocols Support Custom Protocols in WebExtension Feb 6, 2017

@lidel lidel moved this from Blocked to In Progress in Rewrite as WebExtension Feb 9, 2017

@lidel lidel moved this from In Progress to Blocked in Rewrite as WebExtension Feb 9, 2017

@lidel

This comment has been minimized.

Copy link
Collaborator Author

lidel commented Feb 25, 2017

Support for simplified redirect-based protocol handler (simple redirect, no origin support) landed in Firefox 54:

This a a thin UX on top of Navigator.registerProtocolHandler():

Note that this is just a redirect:

This means the address in location bar changes (Origin barrier is broken) and there are different security context for public and local gateways (different cookies, etc).

And yes, this is Firefox-only.

@lidel

This comment has been minimized.

Copy link
Collaborator Author

lidel commented Mar 17, 2017

It seems that this new API has the same naming limitation as Chrome, firefox-54.0a2 says:

Reading manifest: Error processing protocol_handlers.0.protocol: Value must either: be one of ["bitcoin", "geo", "im", "irc", "ircs", "magnet", "mailto", "mms", "news", "nntp", "sip", "sms", "smsto", "ssh", "tel", "urn", "webcal", "wtai", "xmpp"], or match the pattern /^(ext|web)+[a-z0-9.+-]+$/

This means we can't support ipfs://hash anymore and are forced to use web+ipfs://hash 👎

@Kubuxu

This comment has been minimized.

Copy link

Kubuxu commented Mar 18, 2017

When we get official registration of the schema we should be able to request the exemption to be added.

lidel added a commit that referenced this issue Mar 18, 2017

feat(protocol): support web+foo protocols
- implements simplified handlers described in #164

@lidel lidel moved this from Blocked to In Progress in Rewrite as WebExtension Mar 18, 2017

@lidel lidel moved this from In Progress to Blocked in Rewrite as WebExtension Jul 22, 2017

@lidel lidel referenced this issue Aug 15, 2017

Closed

Publish to Chrome Web Store #268

7 of 7 tasks complete
@lidel

This comment has been minimized.

Copy link
Collaborator Author

lidel commented Sep 10, 2017

I've just tested a ridiculous PoC that restores support for ipfs:// dweb: etc in WebExtension:

  • We know that:
    • permission <all_urls> fires onBeforeRequest for every URL
    • when URL starting with an unknown protocol is entered in location bar, a browser url-escapes entire address and converts it to a search query, e.g:
      ipfs://QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnRhttps://www.google.com/search?q=ipfs%3A%2F%2FQmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR&ie=utf-8&oe=utf-8&client=firefox-b-ab
  • So what happens if you:
    • detect requests with URL containing url-encoded :/ ( %3A%2F)
    • find ones that match a query starting with one of custom protocols: /=(ipfs|ipns|dweb)%3A%2F(%2F[^&]+)/
    • extract IPFS path and check if passes IsIpfs.path test
      • if so, replace request to search engine with request to resource at public gateway 'https://ipfs.io/' + extractedIpfsPath
      • if not, return original request

Well.. it just works 🚀 🙃 (not sure if I am proud or ashamed, probably both)

Expect PR in near future. My plan is to polish and ship this workaround as the default behavior, so that there is no functional regression when users update from v1.6.0 to v2.x.x.

Of course there will be a preference that disables this: to pass review at Mozilla and in case someone really wants to search for ipfs://QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR

Update 2017-12-27: this hack does not work on Android – see #348

lidel added a commit that referenced this issue Sep 17, 2017

feat: Poor Man's Protocol Handlers
This is an implementation of ridiculous idea from:
#164 (comment)

I am really sorry.

lidel added a commit that referenced this issue Sep 27, 2017

feat: Poor Man's Protocol Handlers
This is an implementation of ridiculous idea from:
#164 (comment)

I am really sorry.

@lidel lidel added the discussion label Sep 27, 2017

@lidel

This comment has been minimized.

Copy link
Collaborator Author

lidel commented Nov 3, 2017

@dag Thanks! Mind moving it to ipfs/specs#152? It is a better place for this discussion, more people are watching that repo. This issue is only about implementing current consensus in the browser extension :)

(If you are interested in tl;dr with rationale behind use of URL for ipfs:// and ipns:// and URI for dweb:, see "Four stages of the upgrade path for path addressing")

@lidel

This comment has been minimized.

Copy link
Collaborator Author

lidel commented Jan 6, 2018

As a result of a thread at dev-addons Andre opened a Bugzilla ticket to Whitelist custom protocol handlers for work on decentralization technologies with WebExtensions.
If accepted, Firefox users will no longer have to rely on hack from #164 (comment), but will get UX from #164 (comment) without web+ prefix.

A small step, it would be still just a redirect, not a real protocol handler, but worth mentioning here, as it would solve #348 :)

Of course redirect-based handler this does not solve Problem #2, we need New, Native, Programmable Protocol Handler API for WebExtensions for that.

@lidel lidel referenced this issue Jan 8, 2018

Closed

Expose IPFS API as window.ipfs #330

7 of 7 tasks complete
@soapdog

This comment has been minimized.

Copy link

soapdog commented Jan 9, 2018

@lidel it worked! The patch landed on nightly (I think, I am new to this process). Check out https://hg.mozilla.org/mozilla-central/rev/c2cb8a06bcf1

(PS: I never used IPFS but I thought that since I was going through the process of doing a patch to whitelist scuttlebutt, I should include ipfs and dat as well... glad it worked)

Also I was checking a bit from this thread, and (please correct if I am wrong) apparently you're intercepting a protocol and doing a web redirect. Is that right? If so, then I believe you might be able to use the WebRequest API from WebExtensions to intercept the request and fiddle with the origin handlers you wanted, it might help there.

@lidel

This comment has been minimized.

Copy link
Collaborator Author

lidel commented Jan 9, 2018

@soapdog that is really cool, thank you!! 🎉 ❤️
We'll switch to non-prefixed handler as soon it lands in regular Firefox release. 👌

Yes, we are intercepting and redirecting. The problem is that we want to control Origin in the browser itself, to isolate DApps from each other. AFAIK modifying Origin header in requests will only spoof Origin for remote server, while things like JS on a page will still use standard Origin for cookies and storage. Proper security context for IPFS won't work without webextension support for synthetic origins/protocols.

Update 2018-01-15: Bugzilla ticket is marked as resolved/fixed in Firefox 59.

@mitra42

This comment has been minimized.

Copy link

mitra42 commented Jan 18, 2018

I was one of the authors on some drafts of some of those specs (URN and URI) so I can give you some of the thinking that went behind decisions.

Double slashes were in there to distinguish them from a single leading slash. i.e /foo means go to the same protocol, same host, and ask for /foo. While //xx.yy.zz meant to go the xx.yy.zz host first and ask for /foo. That was a feature of all known protocols at that time and as pointed out doesn't mean anything in a host-less world.

urn: was in there for two reasons (instead of isbn)
a) because the part before the : implied a protocol, what you use to talk to the resolver, in practice it told you what library to hand the query off to.
c) there is no isbn protocol, so it shouldn't really be before the first :, at the time there was a good likelyhood of there being a protocol for resolving URNs.
c) because its a hard place for extensibility. Netscape added urn: around 1996 on my request to allow us to experiment, but there was no way they were going to add one protocol for each of isbn, and each of the other naming authorities, especially without any library/protocol to hand it off to.

lidel added a commit that referenced this issue Jan 22, 2018

lidel added a commit that referenced this issue Feb 5, 2018

lidel added a commit that referenced this issue Feb 5, 2018

lidel added a commit that referenced this issue Feb 5, 2018

lidel added a commit that referenced this issue Feb 5, 2018

lidel added a commit that referenced this issue Feb 12, 2018

lidel added a commit that referenced this issue Mar 2, 2018

lidel added a commit that referenced this issue Mar 14, 2018

@lidel lidel added the libdweb label Aug 15, 2018

@lidel lidel added this to the libdweb demos milestone Aug 17, 2018

@lidel lidel removed the libdweb label Aug 17, 2018

@lidel

This comment has been minimized.

Copy link
Collaborator Author

lidel commented Sep 9, 2018

FYSA some recent developments (Q3 2018):

  • Experimental Protocol Handler API for WebExtensions is being designed as a part of mozilla/libdweb:

    The Protocol API allows you to handle custom protocols from your Firefox extension. This is different from the existing WebExtensions protocol handler API in that it does not register a website for handling corresponding URLs but rather allows your WebExtension to implement the handler.
    More: https://github.com/mozilla/libdweb/#protocol-api

    • Still experimental, requires a special build, but the goal is for this to land in regular builds of Firefox Nightly at some point in the future

    • We are working on a PoC handler that loads data over raw IPFS and keeps ipfs:// in address bar. It can be found in libdweb branch. More info in PR #533 and /libdweb/docs/libdweb.md

  • Safelisting DWeb Protocols | arewedistributedyet/arewedistributedyet#23

    • tl;dr The goal is to be able to register it in all browsers without web+ prefix.
      (Chrome 67 requires web+ prefix for non-safelisted protocols)
    • Firefox already allows non-prefixed version, Chrome published intent-to-implement.
@lidel

This comment has been minimized.

Copy link
Collaborator Author

lidel commented Sep 12, 2018

If DWeb protocols get safelisted, we need manifest.json/protocol_handlers feature parity to solve UX issues mentioned in arewedistributedyet/arewedistributedyet#23 (comment)

Filled a Chrome bug: Extensions API should implement manifest.json/protocol_handlers.

@lidel lidel added the backlog label Sep 22, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment