Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Regression Testing #147

Merged
merged 71 commits into from
Jul 18, 2019
Merged

Improve Regression Testing #147

merged 71 commits into from
Jul 18, 2019

Conversation

PTaylour
Copy link
Contributor

@PTaylour PTaylour commented Mar 13, 2019

Motivation

  • getting occasional non-deterministic fails with current setup
  • currently not straight forward to add new tests, need to create HTML each time.
  • no visible feedback on why tests fail (only percentage diff)
  • inaccurate screenshot timing (can't guarantee we're always taking pictures of the same frame)

This PR

  • adds cypress to run the end to end tests
  • moves the video context test code into the tests themselves
  • uses a snapshot library that provides visual diffs on fail
  • adds a test harness for ensuring frame accurate snapshots

Progress

Known issues

@PTaylour PTaylour self-assigned this Apr 2, 2019
@PTaylour PTaylour added this to In progress (currently being worked on) in VideoContext: Work In Progress via automation Apr 2, 2019
@PTaylour
Copy link
Contributor Author

@richski This is now done. BIG PR.

@PTaylour PTaylour marked this pull request as ready for review July 16, 2019 13:39
@richski richski self-requested a review July 17, 2019 08:32
Copy link
Contributor

@richski richski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made a few minor non-code (README & comments) formatting tweaks - d7d443e.

Looks good!

@PTaylour PTaylour merged commit d68a3e6 into develop Jul 18, 2019
VideoContext: Work In Progress automation moved this from In progress (currently being worked on) to Done Jul 18, 2019
@PTaylour PTaylour deleted the update-regression-testing branch July 18, 2019 14:05
@richski richski mentioned this pull request Nov 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants