page action issues/improvements

[grahamperrin]

Spun off from #37 (comment) test 5:

http://thegearcalculator.appspot.com

Pre-release 0.10.0b4 with Firefox,

grahamperrin@momh167-gjp4-8570p:~ % date ; uname -v
Tue 19 Nov 2019 07:29:26 GMT
FreeBSD 13.0-CURRENT #36 r354616: Tue Nov 12 01:28:03 GMT 2019     root@momh167-gjp4-8570p:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG 
grahamperrin@momh167-gjp4-8570p:~ % pkg query '%o %v %R' firefox waterfox
www/firefox 70.0.1_3,1 FreeBSD
www/waterfox 2019.10.c.1 unknown-repository
grahamperrin@momh167-gjp4-8570p:~ % 

Sometimes the page action disappears (appears greyed out beneath the ⋯ menu). When this occurs after whitelisting, it's necessary to use the preferences for the extension if the listing is to be removed.

Parts of these two screen recordings might suggest transient improvement after disabling enhanced tracking protection, but there's a sense of randomness; I have not figured out how to make things consistently reproducible.

• https://put.re/player/77kHP2wV.mp4 2019-11-19 07:18.mp4
• https://put.re/player/Hgr6rcg3.mp4 2019-11-19 07:24.mp4
• two screen recordings.zip

Refreshing the profile is not a workaround.

Please note, I have not tested release 0.9.4 with the site with Firefox.

(I hope that the Waterfox Classic (Firefox 56.⋯)-oriented enhancements in 0.10.0b4 are not detrimental to Firefox Quantum use cases.)

[claustromaniac]

There are two instances when the page action is meant to be visible:

1. When HTTPZ redirects an HTTP request to HTTPS
2. When it does not redirect an HTTP request to HTTPS, but the site is whitelisted.

Both situations share the following conditions. They are meant to happen only when:

• The browser is about to make a main-frame request (requests for resources triggered by the page do not apply, like images, scripts, and so on)
• ...over HTTP (there is an exception to this one, explained below)
• ...to a site that is not already on the ignore list or whitelist, is not a local address (like localhost or 127.0.0.1), and is not an onion site (Tor)
• ...and it is in neither memory nor fastback caches (because the browser does not trigger requests when loading content from RAM, so it does not fire the webRequest events the extension is listening to) No request = nothing to do

Once a request is upgraded to HTTPS, the extension also shows the icon for all subsequent requests to that site over HTTPS (even if it does not need to redirect them because they are already HTTPS requests) that happen in the first 100 to 300 seconds (this varies purposely) after the redirection takes place.

---

As far as I can see, the extension is working as intended regarding this, but this is not the first time someone brought this to my attention (in #13 for example). It's understandable that the behavior looks erratic to someone who does not know exactly what is going on behind the scenes. A toolbar icon would likely prevent such confusion. I may eventually add one, but that is very low priority to be honest.

[grahamperrin]

Thanks. Still getting my head around things such as these.

UX feels just a bit off when (for example):

• addition to the whitelist is followed by (greyed out page action) inability to remove from the whitelist
• there's an invitation to add, to the whitelist, a host that is already whitelisted …

[claustromaniac]

• there's an invitation to add, to the whitelist, a host that is already whitelisted …

You get that in FF70?

Can you give me an example of a site where you get that?

Also provide steps to reproduce, please.

[claustromaniac]

addition to the whitelist is followed by (greyed out page action) inability to remove from the whitelist

That's actually very easy to reproduce, and is expected behavior, because whitelisting means ignoring. If you whitelist a site that supports only HTTPS (no HTTP), whitelisting is useless (you'll end up connecting over HTTPS anyway, and HTTPZ will have nothing to do with that = no reason to display the icon)

[grahamperrin]

• there's an invitation to add, to the whitelist, a host that is already whitelisted …

http://meyerweb.com/eric/thoughts/2017/03/07/welcome-to-the-grid

STR I can't recall exactly, sorry, but I probably manually (re)visited the https URL some time (not immediately) after whitelisting the site. There's the heavily extended profile in that screenshot, I should probably aim to make it it's reproducible with a clean minimalist profile (HTTPZ alone enabled). Is this PEBKAM again? :-)

grahamperrin@momh167-gjp4-8570p:~ % date ; uname -v
Thu 21 Nov 2019 04:39:11 GMT
FreeBSD 13.0-CURRENT #36 r354616: Tue Nov 12 01:28:03 GMT 2019     root@momh167-gjp4-8570p:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG 
grahamperrin@momh167-gjp4-8570p:~ % pkg query '%o %v %R' firefox
www/firefox 70.0.1_3,1 FreeBSD
grahamperrin@momh167-gjp4-8570p:~ % 

[claustromaniac]

Should work as intended now. Thanks for the feedback 👍

[grahamperrin]

Without reopening …

0.10.2 plus flip-flopping, I pushed things to a point where, after removal from the whitelist, it was difficult to regain the page action (for re-addition to the list). This successive flip-flopping is probably insanely pushy so I should not expect the behaviour to be easily reproducible. A screen recording:

• https://put.re/player/C2E6MwZS.2.mp4
• 2019-11-22 03:04 HTTPZ 0.10.2.zip

Worth noting but IMHO entirely negligible.

grahamperrin@momh167-gjp4-8570p:~ % pkg query '%o %v %R' firefox
www/firefox 70.0.1_3,1 FreeBSD
grahamperrin@momh167-gjp4-8570p:~ % 

Thoughts

Firefox on FreeBSD is Tier-3. If behaviours are not reproducible on either of the higher tiers then it's possible that I'm exposing a Tier-3 issue … possible but (in my experience) unlikely.

---

Defocusing from this issue … sometimes where there's a sense of randomness with an extension, I look to this:

• Mozilla bug 1378459 - (webextensions-startup) [meta] Allow some addon functionality to load prior to any content loading

FWIW I haven't sensed 1378459 in any of my tests in recent days.

[claustromaniac]

I'm pretty sure that's actually because the browser is fetching the previously rendered page from the fastback cache, which does not trigger network requests (so HTTPZ does not have to do anything). Sorry, I totally forgot about it in this comment. Go to about:config and set browser.sessionhistory.max_total_viewers to 0 (in addition to disabling the cache, as you did), then try again.

If the page action still seems to be displayed at random, see if there are any errors in the console related to popup.js, runtime.js, or webRequest.js. Otherwise the "culprit" of the extension's apparent misbehavior (emphasis in apparent because it is actually the intended behavior) is that Firefox was loading the page from the memory and fastback caches.

In that case, I don't consider that a problem, because the extension is showing the page action when it redirects requests, and not showing it when it doesn't have to do anything according to the criteria above (which is the point of using a page action instead of a toolbar button). It is not unexpected behavior to me, but it certainly is to some users (such as yourself), so I might do something about it anyway if I come up with something nice code-wise.

[grahamperrin]

0.11.0b test results – Firefox 70.0.1 (64-bit)

Three browser defaults:

• browser.cache.disk.enable true
• browser.cache.memory.enable true
• browser.sessionhistory.max_total_viewers -1

– and HTTPZ in its default automatic mode.

http://meyerweb.com/eric/thoughts/2017/03/07/welcome-to-the-grid

Page action behaviours all good.

http://thegearcalculator.appspot.com/

Page action behaviours all good.

http://vpri.org/

The page action invites removal from the whitelist when the whitelist is empty.

Switch from automatic to manual, forget ignored sites, save, http://vpri.org/ leads to the HTTPZ error page (underlying error: SSL_ERROR_BAD_CERT_DOMAIN)

[claustromaniac]

The page action invites removal from the whitelist when the whitelist is empty.

That was an oversight. 😅 Should work in 0.11.0b2

[grahamperrin]

Fix confirmed, thanks, there's no longer a page action at http://vpri.org.

[claustromaniac]

Thank you. You have been of great help.

I ended up rolling back those changes and replaced them with a different approach that is simpler and more effective, because I noticed another issue with the initial approach.

I'll try to make another pre-release when I can.

[grahamperrin]

Likewise, thank you,

You have been of great help.

– your responses are superb.

another issue with the initial approach.

(Was that, the HTTPZ error page sometimes not appearing when expected with manual fallback?)

[claustromaniac]

(Was that, the HTTPZ error page sometimes not appearing when expected with manual fallback?)

No, it had to do with the page action, but I also fixed other minor issues while refactoring the code.