You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Any test that intentionally fails in a local environment is always passed when it is run as a GitHub action. I checked the screenshot of the cypress dashboard to see what happens when I run the GitHub action.
The first image is a baseline screenshot.
The second image is a snapshot of the ui intended to fail. An extra button has been added.
As I assume, the test should not pass if the snapshot taken in the second GitHub action test differs from the baseline in the first GitHub action test after the baseline is generated by the first GitHub action test.
In fact, local tests do not pass the above tests. The settings are as follows.
However, if you try to test with GitHub action, all of them succeed.
(Run Starting)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 11.1.0 │
│ Browser: Chrome 107 (headless) │
│ Node Version: v16.13.0 (/home/runner/runners/2.299.1/externals/node16/bin/node) │
│ Specs: 1 found (cameraTranslateInDirection.cy.ts) │
│ Searched: cypress/e2e/**/*.cy.{js,jsx,ts,tsx} │
│ Params: Tag: false, Group: false, Parallel: false │
│ Run URL: https://cloud.cypress.io/projects/9iq3e5/runs/3 │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: cameraTranslateInDirection.cy.ts (1 of 1)
Estimated: 9 seconds
translateInDirection
✓ translate to N (2162ms)
✓ translate to NE (687ms)
✓ translate to E (655ms)
✓ translate to SE (621ms)
✓ translate to S (636ms)
✓ translate to SW (641ms)
✓ translate to W (706ms)
✓ translate to NW (725ms)
✓ translate to backward (976ms)
✓ translate to forward (1121ms)
10 passing (9s)
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 10 │
│ Passing: 10 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 10 │
│ Video: true │
│ Duration: 8 seconds │
│ Estimated: 9 seconds │
│ Spec Ran: cameraTranslateInDirection.cy.ts │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
The reasons I expect are as follows
The threshold setting for the github action exists independently, and the default threshold value is more generous than the threshold value I set in the project(cypress.config).
The test always passes because the baseline of the test to be attempted with the github action is initialized every time. ⇒ If so, how do you generate baseline and where is it stored?
I hope the results of the local test and the test run by GitHub action are the same. I'd really appreciate your help
Any test that intentionally fails in a local environment is always passed when it is run as a GitHub action. I checked the screenshot of the cypress dashboard to see what happens when I run the GitHub action.
The first image is a baseline screenshot.
The second image is a snapshot of the ui intended to fail. An extra button has been added.
As I assume, the test should not pass if the snapshot taken in the second GitHub action test differs from the baseline in the first GitHub action test after the baseline is generated by the first GitHub action test.
In fact, local tests do not pass the above tests. The settings are as follows.
However, if you try to test with GitHub action, all of them succeed.
The reasons I expect are as follows
I hope the results of the local test and the test run by GitHub action are the same. I'd really appreciate your help
The text was updated successfully, but these errors were encountered: