Adding Visual Testing: Cypress + Percy by lee00678 · Pull Request #419 · f5/unovis

lee00678 · 2024-07-24T22:18:43Z

I'm setting the trigger for visual testing to happen when merge to main for now. I think we could also have Percy runs on a weekly basis just in case.

Installed Cypress and Percy for visual testing, built into our CI pipeline, triggers on merge to main
Added a random seed generator in our dev environment so the test can be consistent.

rokotyan

This is great! Can you please also provide some steps how to set it up locally?

package.json

rokotyan · 2024-07-24T23:51:30Z

packages/dev/cypress/support/commands.ts

@@ -0,0 +1,37 @@
+/// <reference types="cypress" />


Do we need this file at all?

It turns out we do need this file. Without it Cypress can't resolve /commands. See here

packages/dev/src/utils/data.ts

packages/dev/cypress/fixtures/example.json

packages/dev/cypress/e2e/unovis.cy.ts

rokotyan · 2024-07-24T23:56:47Z

packages/dev/cypress/e2e/unovis.cy.ts

+
+  it('Basic Annotation', () => {
+    cy.contains('Basic Annotations').click()
+    cy.wait(3000)


Is there a specific reason to wait 3000 ms?

Is there a specific reason to wait 3000 ms?

Some page has animations. This way we wait till everything is fully rendered before taking the screenshot.

Should we add duration as a query string argument to the pages? I think this way you can speed up the testing significantly.

Should we add duration as a query string argument to the pages? I think this way you can speed up the testing significantly.

What do you mean? Add something like ?duration=3000 to the url?

More like ?duration=0 to render charts without animation

More like ?duration=0 to render charts without animation

That's a good idea.

More like ?duration=0 to render charts without animation

See a POC here

packages/dev/cypress.config.ts

rokotyan · 2024-07-24T23:59:27Z

packages/dev/cypress/e2e/unovis.cy.ts

+    cy.percySnapshot('Stacked Bar with Labels', { scope: '.exampleViewer' })
+  })
+})
+


We can try generating this file automatically based on the example files. Technically they have the required information, like the description text.

We can try generating this file automatically based on the example files. Technically they have the required information, like the description text.

I suppose we could, but some page require additional settings. For example the API Endpoints Tree requires a bigger minHeight, we'd have to manually set things like that for specific test cases. Do you have a workaround for cases like this?

You can have a map of additional settings in the testing file to apply on the go if needed. I can explain more if you want, just let me know.

Do you mean to have an array or load a separate file which contains

testingArray = [ {name: 'Single Container', duration:3000, additionalParams: {width: [1300]}}, {name: 'Sankey', duration:3000}, ... ]

And iterate through the array inside the unovis.cy.ts?

Take a look at unovis/packages/dev/src/examples/index.ts, I was thinking about something similar to generate this test file automatically. You'll need to "compile" it using Webpack but I think it will be a reasonable trade-off.

So the script will:

Batch import all the index.tsx files in the examples folder;

You'll be able to get the required information from the imports (titles, ...);

You can define additional testing configuration in the unovis.cy.ts file and match the corresponding examples by name or path. Or you can define it in the index.tsx files of the examples themsevles.

@rokotyan I will give it a try. Is this PR getting too big? Should we have a separate PR for this and the duration params?

Up to you. I would do it here, but I don't see any problem with merging this PR first and having another one

lee00678 · 2024-07-25T16:54:24Z

This is great! Can you please also provide some steps how to set it up locally?

Running this locally is a little tricky. You'd have to have a Percy account and a token to be able to do that. See this. It is possible to have a free personal account, which is what I did when I was testing this out. You need to create a project and it will generate a PERCY_TOKEN. You will then set it as part of your local env.

lee00678 · 2024-08-06T19:22:22Z

packages/dev/src/examples/auxiliary/annotations/basic-annotations/index.tsx

@rokotyan @reb-dev Updating this as a proof of concept to see this is the right approach. I will update the rest of the files if this looks good.

Nice! I think I would try doing it in packages/dev/src/components/ExampleViewer/index.tsx and passing as a prop, or defining a dedicated React hook that can be imported to avoid code duplication.

lee00678 · 2024-08-07T20:23:10Z

packages/dev/src/components/ExampleViewer/index.tsx

@rokotyan @reb-dev updated by passing duration as props.

@reb-dev the latest push should fully implement the duration props for all dev files. I put the urls in a separate file, we can fully implement the autogen for testing urls in a separate PR.

sumitkumar25 · 2024-08-15T22:40:21Z

@lee00678 dont enable percy run on merge to master. it will eat up the screenshot quota.

i would recommend a weekly run.
We should check if we can have a percy run before a new release version is published.

Core | Test: Add random seed generator for dev data

…ercy integration via browserstack

…erge to main

lee00678 · 2024-08-15T23:01:32Z

@lee00678 dont enable percy run on merge to master. it will eat up the screenshot quota.

i would recommend a weekly run.

We should check if we can have a percy run before a new release version is published.

It has a weekly run. Removed the trigger for merge to main.

lee00678 requested a review from reb-dev July 24, 2024 22:18

rokotyan reviewed Jul 25, 2024

View reviewed changes

lee00678 force-pushed the qian/feature/cypress branch from 82eeb5e to 9d567db Compare July 25, 2024 18:56

reb-dev previously approved these changes Jul 26, 2024

View reviewed changes

lee00678 force-pushed the qian/feature/cypress branch from 9d567db to 86f9ed0 Compare August 1, 2024 17:28

lee00678 dismissed reb-dev’s stale review via 9de7f3f August 1, 2024 18:11

lee00678 force-pushed the qian/feature/cypress branch 2 times, most recently from 6f2198b to 7780999 Compare August 1, 2024 22:49

lee00678 marked this pull request as draft August 2, 2024 17:23

lee00678 marked this pull request as ready for review August 2, 2024 19:06

lee00678 commented Aug 6, 2024

View reviewed changes

lee00678 force-pushed the qian/feature/cypress branch from 0c8331e to 2ead8ea Compare August 7, 2024 20:22

lee00678 commented Aug 7, 2024

View reviewed changes

lee00678 force-pushed the qian/feature/cypress branch from 29a2648 to feddac8 Compare August 12, 2024 22:23

reb-dev previously approved these changes Aug 15, 2024

View reviewed changes

lee00678 added 13 commits August 15, 2024 16:00

Core | Test: Install cypress

690db32

Core | Testing: Setup Percy, set minHeight

f7bf5b0

Core | Test: Add PERCY_TOKEN

f4e00ac

Core | Test: Add random seed generator for dev data

Core | Test: Update percy token to stellar

e0b51e4

Core | Test: Add percy.json, update gitignore

351d841

Core | Test: Update rng for testing data

0382df4

Core | Test: Update Actions to trigger test when merge to main, add p…

d2492ae

…ercy integration via browserstack

Core | Test: Fully Integrated with browserstack

8363744

Core | Test: Add Cron to run Percy on schedule

ec636e0

Core | Test: Finalize cron to run on every monday

0f81956

Core | Test: Pass in duration params to improve testing time

ecd7c97

Core | Test: Import urls for Cypress Tests

e9441fa

Core | Test: Only run percy on a weekly basis, remove percy CI from m…

fd7304c

…erge to main

lee00678 dismissed reb-dev’s stale review via fd7304c August 15, 2024 23:00

lee00678 force-pushed the qian/feature/cypress branch from feddac8 to fd7304c Compare August 15, 2024 23:00

reb-dev approved these changes Aug 19, 2024

View reviewed changes

lee00678 merged commit d20f8d9 into main Aug 19, 2024

Conversation

lee00678 commented Jul 24, 2024

Uh oh!

rokotyan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lee00678 commented Jul 25, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sumitkumar25 commented Aug 15, 2024

Uh oh!

lee00678 commented Aug 15, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants