Maintenance: consider migration from karma-runner to jasmine-browser-runner.

[jayaddison]

Describe the bug

The Karma browser-based test framework that we use to run tests for the Sphinx JavaScript code that runs in the client browser is deprecated.

We may want to consider migration. The karma project's deprecation notice suggests jasmine-browser-runner as an alternative.

How to Reproduce

N/A

Environment Information

N/A

Sphinx extensions

N/A

Additional context

No response

[picnixz]

Oh, so it's deprecated. Err... yes we should migrate the framework.

@chrisjsewell Maybe we shouldn't add the tip for karma then?

[chrisjsewell]

Maybe we shouldn't add the tip for karma then?

Well that's still the tip for getting it to work right now 😄

My question in general would be, if we need to migrate, should we take the opportunity to move to the "best", "most well used" framework available, and what is that?

Selenium, Playwright or Cyress come to mind.
Does anyone have any experience of them?
(FYI for sphinx-needs we use cypress)

[picnixz]

should we take the opportunity to move to the "best", "most well used" framework available

I would say, the one that features what we would need and possibly what we could need. Something that does not need 10000 of dependencies and something that can be used with the most common browsers (Firefox, Chrome, and Safari, I'd say).

Personally, I only wrote code in JS using Angular (and it was actually typescript) and it was using karma... So perhaps we can use the same framework as what Angular or React would use behind the scenes. It could increase the community. I actually don't know how popular Selenium, PW or Cyress are compared to each others and which one is the best and why so I'll leave it to people who would contribute to JS tests.

[jayaddison]

Lightweight, simple, available, and working would be my vote (so that's approximately: stay with karma for now, and begin investigating their recommended migration path of jasmine-browser-runner).

The fact that we don't use frameworks or have heavyweight dependencies is useful for compatibility, adoptability by downstream users, auditability, performance and various other things.

[chrisjsewell]

I would definitely look at https://www.cypress.io/ as well though, because it was pretty lightweight and easy to integrate into sphinx-needs testing

[jayaddison]

Ok, will do :) (and will compare to sphinx-needs)

[chrisjsewell]

Ok, will do :) (and will compare to sphinx-needs)

https://github.com/useblocks/sphinx-needs/blob/2c4541b1245c97bdcf15da25e758a58c8091f09f/tests/conftest.py#L134

[electric-coder]

(...) I actually don't know how popular Selenium, PW or Cyress are compared to each other (...)

My rule of thumb for popularity (i.e. adoption by size of user base) is checking how many SO Qs there are for each tag, together with GH users (e.g. playwright indicates 72.7k whereas cypress indicates 1.3 million)... So at time of writing:

1. 57,128 questions selenium-webdriver
2. 9,841 questions cypress
3. 2,868 questions playwright

I'd tend to avoid playwright because it's Microsoft so it's not much inline with Sphinx's open-source philosophy.

[AA-Turner]

The JavaScript tests are really very simple so I'd be wary of a complex or complicated testing solution.