compatibility with Waterfox - strict_min_version 57.0 and 60.0

[grahamperrin]

d7d8856#diff-051a8a7bcae8db1e4cee8eb09b52e619L5

b3c55c9#diff-051a8a7bcae8db1e4cee8eb09b52e619L5

Loosely speaking: should users of Waterfox Classic e.g. 56.2.14 expect some features of 0.6.0 and greater to be simply non-functional?

https://addons.mozilla.org/addon/httpz/versions/

tl;dr Waterfox 56.0 was based on Firefox 56.0.2 and to be clear, I'm not suggesting that (#32) Waterfox Classic should be another supported browser.

I'm just curious about functionality. Background: https://www.reddit.com/comments/dahnh0/-/f1te63v/

TIA

[claustromaniac]

A few of the times that I increased the strict_min_version was because it sometimes takes Mozilla longer to implement APIs in Firefox for Android (e.g. some API in FF 67 for desktops may be available only from 68 onwards on mobile), and AMO wouldn't have let me proceed with the release if I hadn't (because HTTPZ supports both desktop and mobile). In other words: the actual minimum Firefox version required for HTTPZ to work correctly on desktop does not match the value of strict_min_version (it is an older one).

Still, I'm not saying that HTTPZ should therefore work perfectly in Waterfox. I don't know that because I lack the time and motivation to support forks (so I haven't looked at that, and I won't).

[claustromaniac]

It should now be possible to use HTTPZ in Firefox 56.0 (desktop) and its forks without having to jump through hoops. Thanks for bringing this to my attention.

[claustromaniac]

I narrowed down the cause and (hopefully) fixed it in 0.10.0b3 0.10.0b4

It was the optional loadReplace flag in tabs.update(). I made it so it is never used if not supported by the browser.

[claustromaniac]

Thanks for reporting, and for the help!

Any objection? Understood (from the pre-release notes)

What I mean when I say that FF56 is still not officially supported is mostly that I won't test the extension on FF56 before each release to make sure it works well. Also, it is partial compatibility because some functionality is lost (for example, without the loadReplace flag in FF56, sometimes duplicate entries are left in the browser's navigation history).

In other words: I just don't want to make promises regarding this.

You're free to share your experiences as a user though. 👍

[grahamperrin]

Done: https://redd.it/dzl75h

[grahamperrin]

Incidentally,

loadReplace

AFAICT support for this was gained with Waterfox 56.2.0, BrowserWorks/Waterfox@bad1c19

[claustromaniac]

This is the original commit for FF57.

Before then, loadReplace did not exist. I don't know which versions of Waterfox are meant to support it, but that should not matter, because HTTPZ now uses loadReplace whenever available (it determines its availability on the fly)

[grahamperrin]

I don't know which versions of Waterfox are meant to support it, but that should not matter,

Understood, thanks.

From the range of tags, I suspect that 56.2.0 was the first release to support loadReplace … on the other hand, there's the announcement linked from (unofficial) https://github.com/grahamperrin/Waterfox/wiki/Archive-of-change-logs,-announcements-and-downloads#5620 (… Tested a new way to keep Waterfox components up to date …)

---

@MrAlex94 please, is the 56.2.0 tag on commits such as BrowserWorks/Waterfox@bad1c19 a sure sign that the commits truly applied to release 56.2.0?

TIA

[claustromaniac]

0.11.0b5 should fix the checkboxes' visibility, but I'm pretty sure they won't look the same as in FF71.

http://www.youronlinechoices.com/uk/your-ad-choices opened in a new tab from this comment does not; the result is http

I can't reproduce that. Do you get any errors in the console?

[claustromaniac]

Actually, the fact that you can't see the checkboxes now is rather strange, because the same checkboxes were visible in 0.9.4... There's something I'm missing.

Edit: I figured it out. I have a few options. I just have to assess their pros and cons before doing anything. I'll get back to this later.

[claustromaniac]

Please try 0.11.0b6 when you can.

[grahamperrin]

Confirmed, 0.11.0b6 fixes the about:addons UI. Thanks!

[grahamperrin]

Hi, I seem to no longer get good results for part of the test set at #37 (comment).

Recalling your #37 (comment) I kept quiet about most such results whilst beta testing towards 0.11.0.

https://addons.mozilla.org/addon/httpz/versions/0.11.0/updateinfo/ and https://addons.mozilla.org/addon/httpz/versions/ noted with thanks,

… Also, fixed a number of minor compatibility issues with older versions of Firefox …

– there's no longer a yellow alert re: incompatibility for 0.11.0 added to Waterfox Classic 2019.10.

I am, however, puzzled by this:

httpz/src/manifest.json

Lines 2 to 7 in 4dfe2cb

	"applications": {
	"gecko": {
	"id": "httpz@cm.org",
	"strict_min_version": "60.0"
	}
	},

… or maybe I should say, I'm puzzled by Waterfox Classic not showing a yellow alert for an extension with this strict_min_version

[grahamperrin]

Does it help to compare with these sections of code? From within https://robwu.nl/crxviewer/?crx=https%3A%2F%2Faddons.mozilla.org%2Ffirefox%2Fdownloads%2Ffile%2F3460706%2Fmalwarebytes_browser_guard-2.1.4-fx.xpi (for Malwarebytes Browser Guard, which "Works with firefox 57.0 and later"):

    "browser_specific_settings": {
        "gecko": {
            "strict_min_version": "57.0"
        }
    }

    "browser_specific_settings": {
        "gecko": {
            "id": "org_malwarebytes_browserguard@malwarebytes.org",
            "strict_min_version": "57.0"
        }
    }

– for this version of this extension, I do get the yellow alert at about:addons

[claustromaniac]

Hi, I seem to no longer get good results for part of the test set at #37 (comment).

I just tested that list of sites in FF56. Here are my results:

Number	Site	Error	Redirect	STS	Page Action	Can exclude	Problem?
1	http://addonconverter.fotokraina.com/	No	No	No	Yes	Yes	No
2	http://cosmicshambles.com/	No	No	Yes	Yes	Yes*	Yes
3	http://meyerweb.com/eric/thoughts/2017/03/07/welcome-to-the-grid/	No	No	No	Yes	Yes	No
4	http://spaceweather.com/	No	Yes	No	Yes	Yes*	Yes
5	http://thegearcalculator.appspot.com/	No	No	No	Yes	Yes	No
6	http://www.keithfullertonwhitman.com/bio	No	No	No	Yes	Yes	No
7	http://www.macattorney.com/upos.html	Yes	No	No	No	No	No
8	http://www.xorosho.com/xoroshaya_muzika/52710-sven-grunberg-hingus-1981-ambient.html	Yes	No	No	No	No	No

* can exclude even after adding the site to the list of exclusions

All tests passed, but 2 and 4 made me notice a current limitation of HTTPZ. It is OK to show the page action after upgrading requests to HTTPS-only sites, but HTTPZ should not allow users to add those to the list of exclusions, because they do not support HTTP anyway. To avoid complicating things, I think I'm going to prevent HTTPZ from displaying the page action in those cases.

Edit: actually, I already had a check here for the server-initiated redirection scenario, but it does not cover STS, and it apparently does not work as intended either:

httpz/src/bg/webRequest.js

Lines 139 to 149 in f74fb71

	webReq.onBeforeRedirect.addListener(d => {
	/* prevent showing the page action when something (anything) redirects a whitelisted or
	ignored request from http to https
	IMPORTANT: the documentation for this event says it is fired only on server-initiated
	redirections, but it is wrong. Even this very extension fires this when upgrading */
	const target = new URL(d.redirectUrl);
	if (
	target.protocol === 'https:' &&
	( isWhitelisted(target.hostname) || isIgnored(target.hostname) )
	) processed.delete(target.hostname);
	}, filter);

Edit 2: I figured out the issue. This snippet works as intended, but not in FF56, because the page action is not automatically hidden in that version. So, I just have to make the extension hide it.

there's no longer a yellow alert re: incompatibility for 0.11.0 added to Waterfox Classic 2019.10.

I don't even know what is the yellow alert in question. Care to elaborate?

[grahamperrin]

Hi, compare redirect results for 1, 2, 3, 5 and 6 above with results in Firefox 71.

IIRC for 56-based browsers, the desired redirects were more likely to succeed with an earlier release or beta of the extension.

HTH, sorry to throw this spanner in the works

[claustromaniac]

I suppose you misunderstood what I meant by Redirect in my table. Redirect refers to server-initiated redirections from HTTPS to HTTP HTTP to HTTPS. Only the 4th site in the list does this; it does not support HTTP (only HTTPS).

With the current version of HTTPZ I get exactly the same results in FF71 as in FF56.

Your mileage may vary. Let me know if that is the case.

[claustromaniac]

For completeness' sake, I will add that Error in that table refers to the server's response when HTTPZ attempts to upgrade requests (only 7 and 8 resulted in errors), and STS stands for Strict Transport Security. Servers that implement STS do not support HTTP, only HTTPS (in this case, only the second item in the list implements STS).

[grahamperrin]

Your mileage may vary.

Ha! It did a short while ago, but not now.

Now I see page action icons for all five, even with my very heavily extended profile. Consistent, at a glance, with what's seen in Firefox 71.0.

Previously I saw none and TBH I can't recall looking to the left (to tell whether there was redirection from HTTP to HTTPS). Let's ignore this earlier result, I was being lazy/non-methodical at the time.

Thanks again!

Re: the yellow alert, I might follow up at the weekend.

Postscript

Three frames from the screen recording of the lazy test session that probably caused me to imagine a problem:

(click)

There was simply a long wait – twenty-eight seconds – between appearance of Done (in the status bar) and appearance of the page action. I didn't notice the page action until after a view of the recording.

Not a bug. A casualty of my choice of extensions, some of which require single-process mode.

[claustromaniac]

Now I see page action icons for all five, even with my very heavily extended profile. Consistent, at a glance, with what's seen in Firefox 71.0.

click to show

Let's ignore this earlier result, I was being lazy/non-methodical at the time.

Nah, it was very helpful. Thanks to that I ended up discovering more defects than I care to admit.

Thank you again for testing and reporting everything.

[grahamperrin]

Looking great.

From #37 (comment) (now hidden, resolved):

http://www.youronlinechoices.com/uk/your-ad-choices opened in a new tab from this comment does not (present a page action); the result is http

That's no longer true. Thanks for the fix.

[grahamperrin]

Bonus: the screen recording at thegearcalculator.appspot.com - Firefox - Malwarebytes Forums

Interesting interactions of the site with:

• pervasive tracking protection enabled (not a default) in Waterfox Classic 2019.10
• malware protection in Malwarebytes Browser Guard 2.1.4.

Feel free to describe it as not a bonus, if my choice of test site exposed you to malware 🤯

[claustromaniac]

Feel free to describe it as not a bonus, if my choice of test site exposed you to malware 🤯

Thanks for the info. And don't worry (about me) because, whether that's malware or not, I was not exposed, because I did something not ideal from a methodology standpoint: I tested with uBlock Origin ON (and blocked JavaScript).

In the future, unless you're facing problems with a specific site and you suspect HTTPZ to be the culprit, you can use http://example.com/ or http://example.org/ for testing.

Other sites I often use for testing are: http://http.badssl.com/ (redirects from HTTPS to HTTP), and http://error.net/ (errors out over HTTPS).