Define Client Support strategy

[rgaudin]

Following #1010 ; as kiwix-serve is meant to be used on non-evolving environments, I think we should have a clear support strategy. It's kinda bad that such an essential piece of our tool-chain has no clear target and no policy/process in terms of client support.

There are many ways to proceed of course:

• We need to decide on some target. browserslist can help
• We could transpile our code using Babel or swc but that's intrusive and makes testing harder
• We could lint our code using eslint-plugin-compat

IMO, the most practical thing from where we are now is to add eslint-plugin-compat to our CI (and developers' setup)

[Jaifroid]

I strongly support this. Even quite recent browsers fail with Kiwix Serve because of the use of advanced JS without consideration for browser support. An example is the use of replaceAll which immediately cuts out any Chromium browser < 85, yet it's really easy to avoid. Another is the use of optional catch binding -- this caused us a headache when incorporating library.kiwix.org as a download library in KJS:

We (belatedly) transitioned from JQuery to transpiling with Babel over at KJS several months ago, and we don't find it intrusive at all. Testing can still be done on untranspiled code (and we work a lot that way), and then you just do something like npm run build when you want to test the transpiled code.

[kelson42]

I propose to move forward on this with Browserlist >1%, not dead.

I understand that:

• This is a limited support
• This is not focused on an offline audience which has a lot of out-of-support Browsers

But it is done like this because:

• This is a list of browsers we guaranty, not the browsers we only care
• We should IMHO, first, focus on something we could guaranty.
• This does not avoid - to the contrary - to discuss case by case - Kiwix Server JS related code to better support browsers out of this list
• This is a baseline, we could then improve based on findings/use cases if we can provide more

[Jaifroid]

I think the problem with >1%, not dead is first of all that it provides less than 80% worldwide coverage, especially in China, and second, we would not be officially supporting anything other than the very latest stable versions of some major browsers (such as Firefox -- only supports Firefox 124+!!!). I realize in practice we would be supporting compatible browsers below that, but if we don't test on such browsers (easy to do in an automated way via BrowserStack), then we'll never know which browsers we actually support and incompatibilities will creep in. In my opinion, the official support policy should be somewhat aligned with what we actually want to support.

A more representative list might be >0.3%, not dead, though it's still not perfect, because with a tiny bit of effort the Kiwix Serve library could easily support Chromium >= 80, Firefox >= 70 and Safari >= 14. You could even say that those three are your target, and rely on the fact that a lot of other stuff we don't care about that much would still work with this level of compatibility. Just changing the code for the two mentioned problems (replaceAll EDIT: Fixed, and optional catch binding) would pretty much get there (these were the two blockers we found when testing, though there are likely a few others).

If you want to do automated testing of the library.kiwix.org UI via BrowserStack's API (I do this for KJS browser extension with Selenium WebDriver), then you have to declare the specific browser versions to run each test on. In practice, that's your supported browser list. Tests would need to include downloading a small ZIM, because one of the blockers mentioned is only encountered when you actually try to download. See kiwix/kiwix-js#1174 EDIT: fixed in #1051.

EDIT: BrowserStack only has macOS, iOS, Android and Windows (XP-11) as testing platforms for browsers (these are invoked from a Linux runner on GitHub). It doesn't provide a testing platform for browsers on Linux, but I'm not aware of any major browser version that works differently on Linux than on the other platforms. At KJS, we do automated testing of recent Linux builds of Firefox and Chrome directly on GitHub's Linux VMs, which provides some assurance there.

[kelson42]

A more representative list might be >0.3%, not dead

@Jaifroid This list has a few browsers I don't really know and don't know how to clearly test automatically. Do you?

[Jaifroid]

@Jaifroid This list has a few browsers I don't really know and don't know how to clearly test automatically. Do you?

I checked the funny Android ones, and they're all using Chromium, so testing could be sort-of covered by a test on Chrome on Android. However, as I mentioned, IMHO it might be simpler (and easier to test) if we didn't try to use a moving target, and simply decided to support, officially, Chromium >= 80, Firefox >= 70 and Safari >= 14 (those are just suggestions based on my support experience in KJS and on what I believe the main blockers are currently that were revealed in KJS testing and would need fixing to support these versions).

Incidentally, library.kiwix.org works on a recent Samsung Internet (I tested it manually, though it can be tested on BrowserStack automatically).

[Jaifroid]

Ooh, I forgot we already fixed #1051, so that should make it even easier to hit suggested browsers.

[benoit74]

From my perspective, before defining the details, we need to define the objective. What do we want to do?

From my PoV, we want to support as many mainstream browsers as possible AND as many versions of these browsers as possible.

As possible could be rephrased "as reasonable given our constraints".

And this is true for kiwix-serve, but this is also true for scrapers (even if it will be even harder to enforce compliance for some of them like zimit). We should hence move this issue to openzim/overview from my perspective.

If we agree on that, it means that we have to define:

• what are the mainstream browsers we want to support?
• for each mainstream browsers selected, we have to define which version we support?

In term of mainstream browsers, I think it is mandatory to cover desktops, phones and tablets.

In term of versions, I think it is mandatory to express it in terms of "oldest supported version" for simplicity, and to make it clear that it doesn't necessarily implies that all browsers versions from this oldest supported version till now will work, since there are obviously some bugs in all browsers in the wild. We will inevitably encounter some buggy browser versions and we don't expect to make much effort to support them.

This means that we do not really care about the most recent browser versions (especially since what is reported on browserlist stats are "online" browsers reporting telemetry, so again not our core public).

This is kinda problematic since devs are usually working on these most recent versions. It means that we need a solution for developers and testers to work on these "ancient" versions so that we do not have wait for user feedback to identify issues.

The list of mainstream browsers and minimum support versions can then be reviewed once in a while when we realize that the list of browsers is not accurate anymore or that the minimum supported versions is causing too much trouble.

[Jaifroid]

@benoit74 I agree. It seems that we're currently in a quite a good place, because after #1051, some quite old browsers are already supported. I tested library.kiwix.org on Safari 14 on macOS (Big Sur) and iOS (an old BrowserStack iPad) yesterday during final testing for a PR on KJS, and it works fine for everything important. The only thing that appeared not to work was getting a magnet: link, but there was no error in console, so I think it's just that the particular BrowserStack machines had no torrenting software installed, and so had idea what to do with the link and silently blocked it. That would need verifying.

Incidentally, a lot of inline JS is used in https://library.kiwix.org, which is blocked in a local Chrome Browser Extension. I'll open a separate issue for that. (We fall back to https://download.kiwix.org in KJS in such cases.)

[kelson42]

@Jaifroid @benoit74 From your discussion it is not clear what is the alternative to Browserlist >1%, not dead you have in mind. I want to remember you that if we are not able to automating the test of browsers, then they can not be supported because we will have no way to secure the support.

For the moment, I'm ready to:

• Write Browserlist >1%, not dead
• Update We need to test kiwix-serve UI (with frontend js code) #815 with the information

[Jaifroid]

The version you suggest, @kelson42, provides only 81.8% worldwide coverage and less than 80% in China, United States, Germany, Canada, Australia.

An automated alternative I would suggest is: Firefox >= 70, Chrome >= 80, Edge >= 80, ChromeAndroid >= 80, Safari >= 14, iOS >= 14 .

Advantages:

• Provides 89.5% global coverage, though it's still less than 80% in China and Nigeria. China could be covered >80% by adding UCAndroid >= 15;
• Target is fixed rather than moving as happens with percentage-based rules;
• Aligns with the support strategy of Kiwix JS.

Of course, we'd need to be sure Kiwix Serve can in fact be used in these browsers currently (with the polyfills).

[kelson42]

@Jaifroid Is that testable with Browseratack? This is not a Christmas wishlist, it has to be doable in a reasonable effort.

[veloman-yunkan]

@Jaifroid Is that testable with Browseratack? This is not a Christmas wishlist, it has to be doable in a reasonable effort.

Browseratack reads hilariously ambiguous 😃

[Jaifroid]

@Jaifroid Is that testable with Browseratack? This is not a Christmas wishlist, it has to be doable in a reasonable effort.

Yes, all of those have BrowserStack versions. I don't see anything Christmasy about that list...!

[kelson42]

Then we adopt Firefox >= 70, Chrome >= 80, Edge >= 80, ChromeAndroid >= 80, Safari >= 14, iOS >= 14 .