Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky cypress test "Spotlight > should be able to add and remove filters via keyboard" #26053

Closed
andybalaam opened this issue Aug 25, 2023 · 9 comments · Fixed by matrix-org/matrix-react-sdk#11590
Closed

Flaky cypress test "Spotlight > should be able to add and remove filters via keyboard" #26053

andybalaam opened this issue Aug 25, 2023 · 9 comments · Fixed by matrix-org/matrix-react-sdk#11590
Labels
A-Developer-Experience O-Uncommon Most users are unlikely to come across this or unexpected workflow S-Minor Impairs non-critical functionality or suitable workarounds exist T-Defect Z-Flaky-Test A test is raising false alarms

Comments

@andybalaam
Copy link
Contributor

I got this failure: https://cloud.cypress.io/projects/ppvnzg/runs/17651/test-results/e1150166-7a80-4155-9428-76e9ec118440 when I was trying to merge a PR for an unrelated change.

Timed out retrying after 10000ms: expected '<li#mx_SpotlightDialog_button_explorePublicRooms.mx_AccessibleButton.mx_SpotlightDialog_explorePublicRooms.mx_SpotlightDialog_option>' to have attribute 'aria-selected' with the value 'true', but the value was 'false'

cypress/e2e/spotlight/spotlight.spec.ts at line154
@andybalaam andybalaam added Z-Flaky-Test A test is raising false alarms S-Minor Impairs non-critical functionality or suitable workarounds exist A-Developer-Experience O-Uncommon Most users are unlikely to come across this or unexpected workflow T-Defect labels Aug 25, 2023
@andybalaam andybalaam mentioned this issue Aug 25, 2023
Merged
@richvdh richvdh mentioned this issue Aug 29, 2023
Closed
@andybalaam
Copy link
Contributor Author

https://github.com/matrix-org/matrix-react-sdk/actions/runs/6069472488/job/16463907578

@richvdh richvdh changed the title Flaky test "Spotlight > should be able to add and remove filters via keyboard" Flaky cypress test "Spotlight > should be able to add and remove filters via keyboard" Sep 8, 2023
@andybalaam andybalaam mentioned this issue Sep 8, 2023
Merged
@andybalaam
Copy link
Contributor Author

Spent some time trying to debug this. I ran it ~10 times locally and couldn't get it to happen, even with --browser=chromium

@andybalaam
Copy link
Contributor Author

Ah, after another few tries it did happen.

@andybalaam
Copy link
Contributor Author

The failure was on the SECOND time it asserted that aria-selected==true

@andybalaam
Copy link
Contributor Author

I added a longer timeout and left this running over and over in the hope that I can get it to fail and stick in the failing state.

@andybalaam
Copy link
Contributor Author

andybalaam commented Sep 8, 2023 •
edited

The "Explore Public Spaces" option is selected and we are expecting "Explore Public Rooms" to be.

@andybalaam
Copy link
Contributor Author

image

andybalaam added a commit to matrix-org/matrix-react-sdk that referenced this issue Sep 8, 2023
@andybalaam
Add waits in Spotlight Cypress tests, hoping this unflakes them
878f94b 
Attempt to fix:

* element-hq/element-web#26053
* element-hq/element-web#26140
* element-hq/element-web#26139
* element-hq/element-web#26138
@andybalaam andybalaam mentioned this issue Sep 8, 2023
Merged
github-merge-queue bot pushed a commit to matrix-org/matrix-react-sdk that referenced this issue Sep 8, 2023
@andybalaam
Add waits in Spotlight Cypress tests, hoping this unflakes them (#11590)
f1f8c6b 
Attempt to fix:

* element-hq/element-web#26053
* element-hq/element-web#26140
* element-hq/element-web#26139
* element-hq/element-web#26138
@richvdh richvdh reopened this Sep 29, 2023
@richvdh
Copy link
Member

richvdh commented Sep 29, 2023 •
edited

https://github.com/matrix-org/matrix-js-sdk/actions/runs/6349567009/job/17247955477#step:8:15197

@richvdh richvdh mentioned this issue Sep 29, 2023
Merged
@richvdh
Copy link
Member

richvdh commented Jan 19, 2024

since this hasn't been reported for several months, let's assume for now that it is fixed.

@richvdh richvdh closed this as completed Jan 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-Developer-Experience O-Uncommon Most users are unlikely to come across this or unexpected workflow S-Minor Impairs non-critical functionality or suitable workarounds exist T-Defect Z-Flaky-Test A test is raising false alarms
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add waits in Spotlight Cypress tests, hoping this unflakes them matrix-org/matrix-react-sdk
2 participants
@andybalaam @richvdh