Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rerun failed e2e tests with enabled recording #1940

Closed
Efefefef opened this issue Apr 17, 2019 · 6 comments

Comments

Projects
None yet
3 participants
@Efefefef
Copy link
Contributor

commented Apr 17, 2019

Description of the problem

Since Cypress introduced new Dashboard Plans we need to reduce the number of test recordings to try to fit the cheapest plan possible

Suggested solution

  1. Run all test files without recording
  2. Save the log
  3. Grep all test files names preceded with ✖ ✖ search.spec.js from the results table
  4. If all is passing notify Github with green and without URL for dashboard run
  5. If there are failed tests - run tests again adding --record and --spec with a list of grepped test files
    https://docs.cypress.io/guides/guides/command-line.html#cypress-run-spec-lt-spec-gt
  6. Notify Github with success or failure with the link as it works now

image

@Efefefef Efefefef added the type: test label Apr 17, 2019

@Efefefef

This comment has been minimized.

Copy link
Contributor Author

commented Apr 17, 2019

For the test we need to change the CYPRESS_RECORD_KEY env. Because our current 'organisation' ran out of test recording limit for April. I will provide you in PM

@slaweet

This comment has been minimized.

Copy link
Member

commented Apr 17, 2019

I like the idea, @Efefefef.

If we go for it, we should keep an eye on how many tests are in each file so that we don't rerun too many tests unnecessary. Most are under 10, which is IMO good. Bigger ones (e.g. wallet.spec.js) can maybe split to smaller.

@fchavant

This comment has been minimized.

Copy link
Member

commented Apr 17, 2019

There is no such secret stored in Jenkins and I cannot find any usage of CYPRESS_RECORD_KEY in the Jenkinsfile either.

@slaweet

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

@fchavant maybe CYPRESS_RECORD_KEY not secret, but just regular env variable. It must be there, otherwise, we wouldn't have any recordings in our cypress dashboard organization.

@fchavant

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

@slaweet @Efefefef how can I get cypress to run only certain tests?
Passing files as parameter (e.g. npm run cypress:run -- --record search.spec.js) does not seem to work.

@Efefefef

This comment has been minimized.

Copy link
Contributor Author

commented Apr 18, 2019

@fchavant Pass it with --spec
npm run cypress:run -- --spec "test/cypress/katze.spec.js,test/cypress/hund.spec.js"

Efefefef added a commit that referenced this issue May 15, 2019

Merge pull request #1941 from LiskHQ/1940-new-cypress-setup
Rerun failed e2e tests with recording - #1940

@Efefefef Efefefef closed this May 15, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.