[Review] Prototype for new protocol detection method

[zone117x]

I think we should prioritize the auth problems due to the protocol handler. I'm requesting several reviewers so we can validate or invalidate this new technique.

This technique uses a redirect "ping" of sorts to detect the protocol handler in a way not dependent on browser quirks and user-agent detection. Compared to the existing method, this implementation errs on the side of caution..

• False-positive detections are treated as unacceptable. Web app completely broken when user cannot login because we incorrectly assumed a protocol handler.
• False-negatives should be rare, but are acceptable. The web app may be redirect to https auth when the protocol handler worked. These do not break the auth flow but rather slightly compromise the UX.

How it works:

1. Web app tab A performs a protocol redirect for the propose of a ping (rather than auth). Example: blockstack://echo?redirectURL=https://testapp.com/echoReply?requestID=12345
2. Web app tab A starts a 1000ms timer waiting for the echo reply. Side note: a couple browser APIs related to DOM window focus & visibility are leveraged to detect probable protocol handler capability, and increase the timeout to 2-4 seconds.
3. If the local auth app is installed, it is launched and immediately opens the ping-echo url for the web app that initiated it. As in, it opens web app tab B using the query param echoReply=[requestID].
4. When tab B is loaded it immediately leverages cross-window communication to tell tab A that the protocol handler was launched and is working. In most cases, this all occurs within the 1-4 second timeout, and the originating tab knows not to redirect itself to the https auth.
5. Tab B then redirects back to the localhost/native auth flow.
6. Tab A either:
   1. Acknowledges tab B's echo reply which indicates the protocol handler is working, and does not redirect to https web auth.
   2. OR it does not get the echo reply within the timeout period (1000ms) and it redirects the page to the https web auth.

Testing out the prototype:

• Demo site for testing: https://zone117x.github.io/blockstack-protocol-ping-demo/public/ (NOTE: requires using with the change to the Blockstack Browser app, see below)
• Build of Blockstack Browser prototype for testing: Support for new protocol detection technique stacks-archive/blockstack-browser#1774 (ctrl+click the blockstack tray-icon and enable development mode, pull the branch, run npm run dev

This approach requires a minor change to the auth browser (Blockstack Browser), see stacks-archive/blockstack-browser@3cfb204

Improvements and fixes:

• Should work on all browsers and unlikely to be broken by any given browser.
• Prevents the browser prompt from Safari and others about unknown protocol when auth-browser is not installed by using an iframe technique rather than window.location to launch the protocol uri.
• Fixes the bug where the Blockstack Browser app is uninstalled but the browser / OS still has a protocol handler registered. See Failed to open uri while redirectAndSignIn #571 (comment)
• Lessens the severity of the bug where using your non-default web browser causes the default browser to be opened and signed in. With this approach the default browser will still be opened, but the original browser will notice the failure and redirect to https auth rather than do nothing and prevent auth.
• In theory this should work on mobile as well. No more browser user-agent detection code.

Downsides:

• Requires a minor wait/timeout before redirecting to the http/web auth.
• The window redirection dance is a bit odd.

Good & bad:

If the native app is installed and opens, but something prevents the ping reply to the web app from occurring within the timeout (like a slow computer), then the original tab will end up with a false-negative (the protocol handler is not detected), and will redirect to the https auth.

I think this could also be considered good behavior that indicates the resilience of this approach, because in the situation of some unknown failure condition, the user auth flow will still be able to progress through the https auth, as opposed to the web app doing nothing upon login-click.

And in the case of this false-positive (where the redirect to https auth even though the protocol handler did actually launch), that original tab was useless anyway with the existing approach which leaves that web app open and unauthenticated.

Backwards compatibility / upgrade path

This approach does not require a blockstack.js major semver bump or any code usage changes from app devs. It should also not break auth flow when blockstack.js is updated and the auth-browser is outdated or vice versa. It does, however, require both an updated blockstack.js and auth-browser for users experience the improvements offered.

Reference issues:

• https://forum.blockstack.org/t/proposed-solutions-for-desktop-safari-firefox-sign-in/6713
• Redirect to browser.blockstack.org not working anymore in Safari and Firefox #558
• Failed to open uri while redirectAndSignIn #571
• Trouble detecting blockstack on Brave browser #488
• Improve custom protocol detector to work better on safari & edge #298
• Ensure user is routed back to browser instead of system default during authentication stacks-archive/blockstack-browser#1759
• Check to make sure transaction value is bigger than dust stacks-archive/blockstack-browser#172
• Unsupported protocol handler - Opera Browser stacks-archive/blockstack-browser#1689
• Cannot Open Blockstack Apps on Chromebook stacks-archive/blockstack-browser#1669
• Can't detect blockstack on Brave Browser stacks-archive/blockstack-browser#1120
• macOS: app doesn't redirect when started by custom protocol handler click stacks-archive/blockstack-browser#1102
• Ensure that user can authenticate in non-default web browser when local Blockstack browser installed stacks-archive/blockstack-browser#314

[CLAassistant]

All committers have signed the CLA.

[yknl]

This is a great solution to the protocol handler problems we've been experiencing, but I think there are a few UX issues that we should resolve before moving forward.

When I tested this out on Safari, the open protocol handler prompt appeared, but if I don't click on "Allow" immediately, the app begins to load browser.blockstack.org in the background. I think we need a longer timeout.

Regarding implementing Larry's idea #2, it doesn't improve UX for 90% of the use cases. Because once a user is logged into an app, they're unlikely to have to log in again for a while. We would only need to trigger the auth flow for new apps that the user hasn't logged into before and which we don't have a preference for.

I think one potential enhancement would be to show a pop up indicating that the user is being redirected, and if it didn't work, they can click a button to go to the web app browser. However this means that we have to bundle UI into blockstack.js.

[hstove]

I think one potential enhancement would be to show a pop up indicating that the user is being redirected, and if it didn't work, they can click a button to go to the web app browser. However this means that we have to bundle UI into blockstack.js.

Matt discussed this in the forum, but we could inject an iframe with the UI. It'd be very little javascript required. Then we could also update it without apps having to update blockstack.js.

[zone117x]

EDIT: The UX issue discussed below has been solved in this PR.

---

@yknl thanks for looking this over.

I think there are a few UX issues that we should resolve before moving forward ... When I tested this out on Safari, the open protocol handler prompt appeared, but if I don't click on "Allow" immediately, the app begins to load browser.blockstack.org in the background.

I just discovered this issue but worse. Safari creates some background page noise (minor inconvenience but not a deal breaker), however, all chromium-based browsers will auto-close the prompt if the page is redirected, which is definitely a deal-breaker. I outlined two solutions to this in the forum post I just made https://forum.blockstack.org/t/proposed-solutions-for-desktop-safari-firefox-sign-in/6713/9

I think one potential enhancement would be to show a pop up indicating that the user is being redirected, and if it didn't work, they can click a button to go to the web app browser. However this means that we have to bundle UI into blockstack.js.

This is my favorite approach at the moment, which I described in the forum post - this new detection method combined the with the choice modal.

I think the iframe for the https login modal (that @hstove just mentioned) is even better UX, but it comes at the cost of some minor security issues like user url src validation and click-jacking. However, at the moment I don't think we are doing anything on our https auth endpoint to prevent these types of attacks anyway, like x-frame-options, Content-Security-Policy frame-ancestors or iframe-busting.

@yknl @hstove Do you think we can move forward with either of these fixes (choice modal or iframe)? If so I'll update the prototype for additional review. If we are not concerned about the potential iframe security issues then that would be quick and easy to implement.

[zone117x]

Update: figured out a way to leverage a couple js dom APIs that behave consistently and reliably across browsers to fix the browser prompt auto-dismissal bug with this new protocol detection method. So far extensive testing on macOS with Chrome, Firefox, Safari, Opera, and Brave show it working really well. Opera shows an error prompt but works fine after dismissing. Working on testing Edge.

These methods are a continuation of the graceful-fallback / probability based approach - where all the techniques / APIs are currently working well in all tested browsers, but if something breaks from some future browser update or edge case, then the https auth redirect will always still be triggered.

[hstove]

One behavior on Brave right now is that when I use the demo site, it redirects to localhost:8888, but in the original tab, it redirects to browser.blockstack.org. Is that expected behavior? I have no prompt for the protocol handle because it's set to always redirect.

[zone117x]

One behavior on Brave right now is that when I use the demo site, it redirects to localhost:8888, but in the original tab, it redirects to browser.blockstack.org. Is that expected behavior? I have no prompt for the protocol handle because it's set to always redirect.

Sounds like the auth-browser branch isn't being ran on your machine?

[hstove]

Sounds like the auth-browser branch isn't being ran on your machine?

Oh yeah, whoops!

[zone117x]

Completed testing with Edge browser. This new method works in Edge but has a minor UX imperfection - when the native app is not installed, the browser will ask the user if they want to "look for an app in Microsoft app store". The web app does detect that a protocol handler is not installed, and does redirect them to the https auth. The user simply has to dismiss the prompt.
When the native app is installed, it works without issue. This seems like an acceptable trade off.

[kantai]

This worked for me pretty seamlessly in Chromium on Linux. I tested it by changing my protocol handler in Linux:

$ cat ~/.local/share/applications/blockstack-protocol-handler
#!/bin/bash
AUTH=$(echo "$1" | sed s#/##g | sed s/blockstack:// | sed 's/ //g')
xdg-open "https://deploy-preview-1774--reporter-beaver-73821.netlify.com/auth?authRequest=${AUTH}"

Which is the netlify preview of the browser built with echo response features. For other Linux testers, this could be the fastest way to test it out.

[kantai]

I just want to make sure I understand the old-version/new-version behavior here:

1. If you're using this version of blockstack.js with an old version of the native browser, the app will (1) open an auth handler in the native browser and (2) also open browser.blockstack.org ?

2. If you're using normal versions of blockstack.js with a new version of the browser, the app will function exactly as it did before this protocol version update.

It seems like case (1) there is new enough behavior that we shouldn't call it backwards-compatible.

[zone117x]

@kantai Correct on both those situations.

In situation 1, the native browser will be opened, and then a short time (a few seconds usually) later, the original tab will redirect to the hosted browser. Previous behavior would have left the original tab open on the unauthenticated web app and required being refreshed, or more likely closed by the user.

I'm calling it backward-compatible because it is not a breaking change, it just changes the UX from one type of confusing behavior to another type. Although, arguably, the new behavior may be more confusing.

[kantai]

I'm happy to approve this -- we should add this new behavior to the CHANGELOG.md file though:

If you're using this version of blockstack.js with an old version of the native browser, the app will (1) open an auth handler in the native browser and (2) also open browser.blockstack.org ?

So that we're aware whenever we merge this that some behavior will be changing.