Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(tests): Fix cypress flakiness on CI by slowing down test setup #523

Merged
merged 19 commits into from
Dec 26, 2021

Conversation

adrienjoly
Copy link
Member

@adrienjoly adrienjoly commented Dec 26, 2021

Problem

Despite several attempts to improve the situation (see PRs #501, #504, #505, #507, #508, #510, #513, #520, #521, #522), Cypress tests are still flaky: they timeout for random reasons.

Hypothesis

This timeout could be caused by the fact that:

  • some XHR requests (e.g. to /api/notif to fetch notifications, and Deezer) may never respond => onLoad is never triggered
  • when a new test starts, we call /testing/reset to refresh the DB, which can cause pending DB queries from never resolving

See #520 (comment).

Proposed solution

  • wait for Openwhyd server to be up and reset before running tests
  • intercept and mock flaky XHR requests
  • fail faster when test steps are hanging for too long
  • reduce the amount of resources to load: fonts

@adrienjoly adrienjoly self-assigned this Dec 26, 2021
@cypress
Copy link

cypress bot commented Dec 26, 2021



Test summary

25 0 1 0


Run details

Project Openwhyd
Status Passed
Commit 22ea5b3 ℹ️
Started Dec 26, 2021 5:18 PM
Ended Dec 26, 2021 5:21 PM
Duration 03:13 💡
OS Linux Ubuntu - 20.04
Browser Chromium 96

View run in Cypress Dashboard ➡️


This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

@sonarcloud
Copy link

sonarcloud bot commented Dec 26, 2021

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@adrienjoly adrienjoly marked this pull request as ready for review December 26, 2021 17:21
@adrienjoly adrienjoly merged commit b9c8707 into master Dec 26, 2021
@adrienjoly adrienjoly deleted the ci/fix-cypress-flaky-by-slowing-down-test-setup branch December 26, 2021 17:45
adrienjoly added a commit that referenced this pull request Dec 29, 2021
Follow up of PR #523.

Cf "Try to avoid requiring a 3rd party server", from [Best Practices | Cypress Documentation](https://docs.cypress.io/guides/references/best-practices#Visiting-external-sites).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant