Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cypress Visual Regression Test Adjustments #492

Closed
wants to merge 28 commits into from

Conversation

kylemh
Copy link
Member

@kylemh kylemh commented May 13, 2019

Description of changes

  • Move visual regression tests to their own folder
  • Create a cypress snapshot creating utility to ensure more consistent screenshots (and to do full page of mobile and desktop)
  • Use arrow funcs in all describe
  • Add a recurrently appearing file to .gitignore

On Hold

We're going to add Docker to the equation so that testing headlessly is the same between CI and local environments. That should remove the variability between fullPage screenshots I'm seeing. On top of that, this PR would lead to much larger build times, so I'm going to put it on hold for #449 as well.

See comments below for more context.

@vercel
Copy link

vercel bot commented May 13, 2019

This pull request is automatically deployed with Now.
To access deployments, click Details below or on the icon next to each push.

Latest deployment for this branch: https://operation-code-git-make-cypress-snaps-consistent.operation-code.now.sh

@kylemh kylemh changed the title Make cypress snaps consistent Cypress Visual Regression Test Adjustments May 13, 2019
@kylemh
Copy link
Member Author

kylemh commented May 13, 2019

🤖 This is a bot 🤖

🎉 Deployed Storybook preview! 🎉

Click the link at the bottom of this comment to see it live.

Built with commit a298d6a

https://deploy-preview-492--operation-code-storybook.netlify.com

@kylemh
Copy link
Member Author

kylemh commented May 13, 2019

Screen Shot 2019-05-12 at 10 16 51 PM

So, we could bump variance up to 3% to account for login and join, but /press and /code_schools variance need to be tackled within the spec. Their rendering inconsistencies make sense as they both rely upon API requests to render properly.

Code Schools, with our API GET request to the code_schools endpoint
Press, with its consumption of multiple YouTube videos.

I'm gonna focus on duct-taping this and fix the issues later, as I should really be focusing on finishing registration + profile editing.


Lastly, I couldn’t quite get mobile + desktop visual regression tests to work in conjunction. It seemed as though the viewport change is asynchronous… I wonder if it’d be better instead to create two functions (1 for desktop and 1 for mobile) and put them in two different test names :thinking_face:

@vercel vercel bot temporarily deployed to staging May 13, 2019 18:09 Inactive
@kylemh
Copy link
Member Author

kylemh commented May 13, 2019

Working on resolving the build failures in jaredpalmer/cypress-image-snapshot#82

@kylemh kylemh self-assigned this May 13, 2019
@kylemh kylemh added the Status: On Hold Not quite blocked, but also not allowed to move forward label May 13, 2019
@kylemh
Copy link
Member Author

kylemh commented May 27, 2019

In the end, no matter what I did, I encountered either

cypress-io/cypress#2175

OR

an issue where my screenshot would be 1px different between builds.

Linking helpful issues for posterity:
cypress-io/cypress#3324
cypress-io/cypress#2348
cypress-io/cypress#2175
cypress-io/cypress#2102
cypress-io/cypress#495

@kylemh kylemh closed this May 27, 2019
@kylemh kylemh deleted the make-cypress-snaps-consistent branch May 27, 2019 02:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: On Hold Not quite blocked, but also not allowed to move forward
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant