System tests: run Chrome in kiosk mode to stop treeGrid test failing#12023
Merged
michaelDCurran merged 3 commits intomasterfrom Feb 3, 2021
Merged
System tests: run Chrome in kiosk mode to stop treeGrid test failing#12023michaelDCurran merged 3 commits intomasterfrom
michaelDCurran merged 3 commits intomasterfrom
Conversation
… to press f6 1 or more times to move from the Chrome to the document.
See test results for failed build of commit 8b11f5c329 |
LeonarddeR
approved these changes
Feb 2, 2021
feerrenrut
approved these changes
Feb 3, 2021
Contributor
feerrenrut
left a comment
There was a problem hiding this comment.
I'm worried that this might just be fixing the symptom rather than the cause. It would be good to have a solid understanding of why the test doesn't detect the correct focus location. I wonder if we could more reliably replicate the failure in a local VM.
I think it is important that we resolve these false failures now, but in the longer run this might just be hiding the problem. When / if we get to testing window chrome in browsers while need to face this.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to issue number:
None.
Summary of the issue:
Around 50% of the time, NVDA builds (including prs) fail on the ARIA treegrid system test. This seems to be because the test is needlessly pressing f6 when already on the document, most likely because of a delay in focus events caused by Chrome loading the content of an iframe (the treeGrid system test embeds a standard ARIA accessibility test in our normal test page using an iFrame).
Description of how this pull request fixes the issue:
The test framework now always starts Chrome in kiosk mode (meaning that the address bar and other parts of the chrome are hidden, leaving only the document content visible). The test framework no longer has to worry about pressing f6 a number of times to get to the document, rather it only needs to wait until NVDA lands on the "Before test" marker itself.
Testing performed:
Known issues with pull request:
Before this pr, the system test never failed on my machine, but only on appveyor. This change so far seems to run fine on both appveyor and my machine, but only time will tell with appveyor. We have tried to fix this in the past. Let's hope this one works.
Change log entry:
None.