Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#12059] Separate stable and unstable E2E tests #12515

Merged
merged 6 commits into from
Jul 27, 2023

Conversation

domoberzin
Copy link
Contributor

@domoberzin domoberzin commented Jul 14, 2023

Fixes #12059

Outline of Solution

Moves tests that frequently require multiple test runs into a separate Github job within the same E2E testing workflow.

Created a new command within build.gradle to run only tests specified in src/e2e/resources/testng-unstable-e2e.xml
Run stable tests: ./gradleW e2eTests
Run unstable tests: ./gradleW e2eTests -Punstable

Copy link
Member

@wkurniawan07 wkurniawan07 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes from this PR doesn't solve the problem. It is merely separating to a different step, which means if all 5 attempts failed, everything (including the stable tests) needs to be started from the beginning.

@domoberzin domoberzin marked this pull request as draft July 16, 2023 04:32
@domoberzin domoberzin marked this pull request as ready for review July 16, 2023 06:39
.github/workflows/e2e.yml Outdated Show resolved Hide resolved
build.gradle Outdated Show resolved Hide resolved
@nusoss-bot
Copy link

Folks, This PR seems to be stalling (no activities for the past 7 days). 🐌 😢
Hope someone can get it to move forward again soon...

@weiquu
Copy link
Contributor

weiquu commented Jul 27, 2023

@wkurniawan07 / any other contributors free to review this PR? I'm not familiar with GitHub workflows...

Also, should the "E2E-testing (firefox)" check still be there? Not sure if by updating the workflow it should be removed:
Screenshot 2023-07-27 at 4 47 11 PM

@wkurniawan07 wkurniawan07 added s.ToMerge The PR is approved by all reviewers including final reviewer; ready for merging c.DevOps Process-related or build-related improvement and addition labels Jul 27, 2023
@weiquu weiquu merged commit 5e58bfa into TEAMMATES:master Jul 27, 2023
9 checks passed
@samuelfangjw samuelfangjw added this to the V8.29.0 milestone Aug 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c.DevOps Process-related or build-related improvement and addition s.ToMerge The PR is approved by all reviewers including final reviewer; ready for merging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Separate stable and unstable E2E tests for CI run
5 participants