Skip to content

System tests: run Chrome in kiosk mode to stop treeGrid test failing#12023

Merged
michaelDCurran merged 3 commits intomasterfrom
fixChromeSystemTests
Feb 3, 2021
Merged

System tests: run Chrome in kiosk mode to stop treeGrid test failing#12023
michaelDCurran merged 3 commits intomasterfrom
fixChromeSystemTests

Conversation

@michaelDCurran
Copy link
Copy Markdown
Member

@michaelDCurran michaelDCurran commented Feb 1, 2021

Link to issue number:

None.

Summary of the issue:

Around 50% of the time, NVDA builds (including prs) fail on the ARIA treegrid system test. This seems to be because the test is needlessly pressing f6 when already on the document, most likely because of a delay in focus events caused by Chrome loading the content of an iframe (the treeGrid system test embeds a standard ARIA accessibility test in our normal test page using an iFrame).

Description of how this pull request fixes the issue:

The test framework now always starts Chrome in kiosk mode (meaning that the address bar and other parts of the chrome are hidden, leaving only the document content visible). The test framework no longer has to worry about pressing f6 a number of times to get to the document, rather it only needs to wait until NVDA lands on the "Before test" marker itself.

Testing performed:

  • Produced a try build twice with no system test failures.
  • PR has been built multiple times now with no system test failures.

Known issues with pull request:

Before this pr, the system test never failed on my machine, but only on appveyor. This change so far seems to run fine on both appveyor and my machine, but only time will tell with appveyor. We have tried to fix this in the past. Let's hope this one works.

Change log entry:

None.

… to press f6 1 or more times to move from the Chrome to the document.
@AppVeyorBot
Copy link
Copy Markdown

See test results for failed build of commit 8b11f5c329

Copy link
Copy Markdown
Contributor

@feerrenrut feerrenrut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm worried that this might just be fixing the symptom rather than the cause. It would be good to have a solid understanding of why the test doesn't detect the correct focus location. I wonder if we could more reliably replicate the failure in a local VM.

I think it is important that we resolve these false failures now, but in the longer run this might just be hiding the problem. When / if we get to testing window chrome in browsers while need to face this.

@michaelDCurran michaelDCurran merged commit 65e183b into master Feb 3, 2021
@michaelDCurran michaelDCurran deleted the fixChromeSystemTests branch February 3, 2021 01:55
@nvaccessAuto nvaccessAuto added this to the 2021.1 milestone Feb 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants