Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(tests): Track visual regressions #413

Merged
merged 9 commits into from
Dec 14, 2020
Merged

Conversation

adrienjoly
Copy link
Member

@adrienjoly adrienjoly commented Nov 29, 2020

Refs:

--> https://eyes.applitools.com/app

@adrienjoly adrienjoly marked this pull request as draft November 29, 2020 11:17
@adrienjoly adrienjoly self-assigned this Nov 29, 2020
@adrienjoly adrienjoly marked this pull request as ready for review November 29, 2020 13:55
@adrienjoly
Copy link
Member Author

adrienjoly commented Nov 29, 2020

Problems I'm facing with the solution proposed in this PR:

  1. I did not find a way to navigate back to the homepage, from the /login page, in Cypress (cf https://github.com/openwhyd/openwhyd/pull/413/files#diff-3893d76fb76135516a86c6fda5b739fc53f17633ea2fdd6861442f445ef9157bR45)
  2. Applitools Eyes sometimes takes a snapshot too early, before the new content was injected into the body of the page (e.g. https://eyes.applitools.com/app/test-results/00000251795643670265/00000251795643670138/steps/4?accountId=zf2VcF6gtk2wqY43MNPslw~~&mode=step-editor)

=> About the second problem, here's what I posted on Twitter:

Is it me, or there is no Visual Regression testing plugin for Cypress that retries/wait until the screen matches the expected/baseline snapshot?
This makes it hard to implement visual testing on http://openwhyd.org, because pages are rendered programmatically when navigating
I.e. on http://openwhyd.org, whenever the user clicks on a link, some client-side JavaScript displays a loading animation, fetches the content from the server, injects it into the and then hides the loading animation.
So far, the only solution I found is to ask Cypress to check for the presence of some text on the page, after navigating... Visage avec des sueurs froides I find this solution too verbose and fragile.
=> @applitools, do you have any suggestion?
The test suite in question: https://github.com/openwhyd/openwhyd/pull/413/files#diff-3893d76fb76135516a86c6fda5b739fc53f17633ea2fdd6861442f445ef9157bR28

EDIT, for reference: Cypress automatically re-evaluates / retries assertions until they are true (cf https://docs.cypress.io/guides/references/best-practices.html#Unnecessary-wait-for-cy-get) => it would be awesome if Applitools could apply that same philosophy.

@adrienjoly adrienjoly merged commit 7c48ba0 into master Dec 14, 2020
@adrienjoly adrienjoly deleted the fix/track-visual-regressions branch December 14, 2020 18:42
adrienjoly pushed a commit that referenced this pull request Dec 14, 2020
## [1.44.7](v1.44.6...v1.44.7) (2020-12-14)

### Bug Fixes

* **tests:** Track visual regressions ([#413](#413)) ([7c48ba0](7c48ba0))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant