Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When running tests locally, NVDA depends on the system language to be English #13362

Closed
MarcoZehe opened this issue Feb 18, 2022 · 2 comments · Fixed by #14148
Closed

When running tests locally, NVDA depends on the system language to be English #13362

MarcoZehe opened this issue Feb 18, 2022 · 2 comments · Fixed by #14148
Milestone
2023.1

Comments

@MarcoZehe
Copy link
Contributor

Steps to reproduce:

  1. On a system that is not set to English as the language, but has a language supported by NVDA, build NVDA from source.
  2. Run tests, at least the system tests.

Actual behavior:

Almost all tests fail because the NVDA test harness expects English language output everywhere, including the Chrome UI. Local tests cannot be run and fail almost 100%.

Expected behavior:

NVDA should force itself and the apps it uses to run in English if it needs that, even on non-English systems.

System configuration

NVDA installed/portable/running from source:

Running from source.

NVDA version:

Any current master that supports running tests.

Windows version:

Any supported.

Name and version of other software in use when reproducing the issue:

Chrome browser for system robot Chrome tests.

Other information about your system:

Other questions

Does the issue still occur after restarting your computer?

Yes.

Have you tried any other versions of NVDA? If so, please report their behaviors.

No.

If NVDA add-ons are disabled, is your problem still occurring?

Yes.

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

N/A.
@CyrilleB79
Copy link
Collaborator

Expected behavior:
NVDA should force itself and the apps it uses to run in English if it needs that, even on non-English systems.

While I agree that being able to run system tests on a non English environment would be appreciable, I think it is not realistic:

  • forcing NVDA to run in English is probably doable with the new --lang command flag
  • Forcing the app to run in English may add extra steps and it should then be guaranteed that the initial language of the app is restored after the test even if test fails: this begin to be very difficult.
  • I think changing Windows UI in English requires to restart the computer or the session; if this is confirmed, I do not see how automated tests can do it.

Maybe an alternative Expected behavior could be to force NVDA to run in English and tag the tests that depends on an external English UI input.
Then add a command to run only the subset of tests that do not depend on English UI. This would allow to run a subset of the tests on a non-english system. Better than nothing.

@lukaszgo1
Copy link
Contributor

Most of the failures are caused by NVDA language and therefore can be temporarily fixed by forcing it to English in the config files used for system tests. After I've done so all tests passes for me on Polish version of Windows, however I have Chrome in English which may make a difference here. It seems there is a desire to make tests work independently from NVDA's language i.e. in #12710 it has been requested to make tests work with NVDA language set tot French. This is certainly doable and I intent to look into this at some point thought cannot promise a specific ETA.

@CyrilleB79 CyrilleB79 mentioned this issue Sep 14, 2022
Merged
9 tasks
@nvaccessAuto nvaccessAuto added this to the 2023.1 milestone Sep 26, 2022
seanbudd added a commit that referenced this issue Sep 26, 2022
@CyrilleB79 @seanbudd
Make system tests pass when run locally on non-English systems (#14148)
7948008 
Closes #13362

 Summary of the issue:
The majority of system tests fail when run on non-English systems. On my system (French), failures are due to the following reasons:
- Default NVDA's UI language is user's default Windows language, thus the messages of NVDA are reported in this language.
- If not specified, the language of an HTML page is set by the browser itself which may not be English (system language?)
- According to the locale keyboard layout, someemulated  keypress may fail. E.g. "NVDA+\" cannot be emulated with French keyboard layout since \ (backslash) can only be accessed with modifiers (AltGr).

In #13362, it was also stated that Chrome's language may make some tests fail. That's not the case on my French system. But we may imagine that the keyboard shortcuts could vary from one language to another, e.g. "Alt+d" to focus the address field.

 Description of user facing changes
System tests now pass on French system. It should pass on any system regardless of its language.

 Description of development approach

1. NVDA interface language is forced to English in the various config files.
2. HTML sample content forced to English (via lang HTML attribute)
3. Removed the unneeded NVDA+\ custom gesture which cannot be executed without modifier on some keyboard layout (e.g. French). Actually, there is no need for custom gesture since `NVDA+d` is now a native gesture of NVDA for the same script.
   Even if not used anymore for now, I have not removed the possibility to use a custom gesture file in case an unassigned script has to be tested in the future. Should it be the case, care should be taken to assign a gesture that can be executed on any keyboard layout, i.e. use a letter rather than punctuation as the main key name.
E. Chrome UI language is forced to English via command line parameter (see List of Chromium Command Line Switches « Peter Beverloo: https://peter.sh/experiments/chromium-command-line-switches/)

Note: For the Chrome command line --lang parameter to be taken into account, it should be ensured that no previous Chrome window is open when the tests are run.

---

* Make system tests pass when run locally on non-English systems.

For this, the following changes have been made:
- NVDA interface language forced to English (in config files)
- Chrome UI language forced to English (via command line parameter)
- HTML sample content forced to English (via lang HTML attribute)
- Removed the unneeded NVDA+\ custom gesture which cannot be executed without modifier on some keyboard layout (e.g. French)

Note: For the Chrome command line --lang parameter to be taken into account, it should be ensured that no previous Chrome window is open when the tests are run.

* Bump CI

* Remove unused .ini file.

* update changes

Co-authored-by: Sean Budd <sean@nvaccess.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
2023.1
Development

Successfully merging a pull request may close this issue.

Make system tests pass when run locally on non-English systems CyrilleB79/nvda
4 participants
@MarcoZehe @lukaszgo1 @nvaccessAuto @CyrilleB79