Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CLD-5704 Migrate daily master/cloud tests #27393

Merged
merged 7 commits into from
Jun 25, 2024
Merged

CLD-5704 Migrate daily master/cloud tests #27393

merged 7 commits into from
Jun 25, 2024

Conversation

mvitale1989
Copy link
Member

@mvitale1989 mvitale1989 commented Jun 18, 2024 •
edited
Loading

Summary

This PR adds support for daily master/cloud E2E tests in GHA.

  • The REPORT_TYPE input will decide how the test will behave.
  • In case of daily tests (MASTER, MASTER_UNSTABLE, CLOUD and CLOUD_UNSTABLE), the mattermostdevelopment/mattermost-enterprise-edition:master image is automatically picked, no matter the given commit_sha or PR_NUMBER you specified at triggering time
  • Zephyr reporting is implemented

The triggering of this job will be done by argo-events.

Ticket Link

https://mattermost.atlassian.net/browse/CLD-5704

Release Note

NONE

@mm-cloud-bot mm-cloud-bot added the release-note-none Denotes a PR that doesn't merit a release note. label Jun 18, 2024
@mvitale1989 mvitale1989 changed the title Migrate daily master/cloud tests CLD-5704 Migrate daily master/cloud tests Jun 18, 2024
@github-actions GitHub Actions
Copy link

E2E test run is starting for commit 66ae7fd6a84f69a29ba3013b4f82024be1cdae77.
You can check its progress by either:

@github-actions GitHub Actions
Copy link

E2E test run is starting for commit 77dc4351143e361f357c07d971f0352132bd6cdb.
You can check its progress by either:

@github-actions GitHub Actions
Copy link

E2E test run is starting for commit 96bd28b7511e1f5f41ab3ab2a275e5fecc40471b.
You can check its progress by either:

@mvitale1989
Copy link
Member Author

☝️ the run above was triggered for the CLOUD_UNSTABLE test, and produced the following message in the channel: https://community.mattermost.com/core/pl/7pn9rcgk1iygirz5zqx8ntnwmw

@github-actions GitHub Actions
Copy link

E2E test run is starting for commit 38126f6de47dd50160bcf3043584e3c0b1c30329.
You can check its progress by either:

@mvitale1989
Copy link
Member Author

mvitale1989 commented Jun 19, 2024 •
edited
Loading

☝️ the run above was triggered for the CLOUD test, and it uses the latest master image available. It produced the following message: https://community.mattermost.com/core/pl/7ayi563gqprsjdnjhzjsmkx69c

@mvitale1989 mvitale1989 marked this pull request as ready for review June 19, 2024 16:48
@saturninoabril
Copy link
Member

/e2e-test

@unified-ci-app
Copy link
Contributor

E2E test triggered successfully for PR #27393. The corresponding commit's status check will be available shortly.

@github-actions GitHub Actions
Copy link

E2E test run is starting for commit 38126f6de47dd50160bcf3043584e3c0b1c30329.
You can check its progress by either:

saturninoabril
saturninoabril approved these changes Jun 20, 2024
View reviewed changes
Copy link
Member

@saturninoabril saturninoabril left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it, great work @mvitale1989!

e2e-tests/.ci/report.publish.sh Outdated Show resolved Hide resolved
.github/workflows/e2e-fulltests-ci.yml Show resolved Hide resolved
saturninoabril
saturninoabril reviewed Jun 20, 2024
View reviewed changes
.github/workflows/e2e-fulltests-ci.yml
TEST_FILTER_VARIABLE="TEST_FILTER_DAILY_${SERVER@U}"
BUILD_ID_SUFFIX="${IS_TEST_UNSTABLE:+unstable-}daily-${SERVER}-ent"
echo "status_check_context=E2E Tests/test-${BUILD_ID_SUFFIX}${MM_ENV:+/$MM_ENV_HASH}" >> $GITHUB_OUTPUT
echo "workers_number=10" >> $GITHUB_OUTPUT # Daily tests are not time critical, and it's more efficient to run on fewer workers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍
(Originally, all 20 workers except for unstable test but sounds good for daily tests.)

@github-actions GitHub Actions
Copy link

E2E test run is starting for commit e8e74d2c11c0ba2d06664a9781cb0dae330bb766.
You can check its progress by either:

@github-actions GitHub Actions
Copy link

E2E test run is starting for commit e8e74d2c11c0ba2d06664a9781cb0dae330bb766.
You can check its progress by either:

@mvitale1989
Copy link
Member Author

/e2e-test

@unified-ci-app
Copy link
Contributor

E2E test triggered successfully for PR #27393. The corresponding commit's status check will be available shortly.

@github-actions GitHub Actions
Copy link

E2E test run is starting for commit e8e74d2c11c0ba2d06664a9781cb0dae330bb766.
You can check its progress by either:

toninis
toninis approved these changes Jun 25, 2024
View reviewed changes
Copy link
Contributor

@toninis toninis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love to see this integrated with spinwicks on demand 🚀

e2e-tests/cypress/save_report.js
@@ -24,7 +24,9 @@
* For sending hooks to Mattermost channels
* - FULL_REPORT, WEBHOOK_URL and DIAGNOSTIC_WEBHOOK_URL
* Test type
* - TYPE=[type], e.g. "MASTER", "PR", "RELEASE", "CLOUD"
* - TYPE=[type]; valid values: "PR", "RELEASE", "MASTER", "MASTER_UNSTABLE", "CLOUD", "CLOUD_UNSTABLE", "NONE"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not need cloud anymore correct ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This variable just contains the type of test that was run, rather than the type of server binary (as you say onprem and cloud are now the same binary). As of now there are test cases that apply only when a cloud license is used.

@mvitale1989
Copy link
Member Author

I would love to see this integrated with spinwicks on demand 🚀

Thanks for reviewing ❤️ as of now the tests depend on accessing the dependencies directly (e.g. we need to check if some migrations are complete), but it's worth checking if we can adapt them to run against spinwicks. I'll create a ticket to explore this

@mvitale1989 mvitale1989 merged commit 4f68dbb into master Jun 25, 2024
89 checks passed
@mvitale1989 mvitale1989 deleted the cld-5704-migrate-scheduled-e2e branch June 25, 2024 13:46
@amyblais amyblais added Changelog/Not Needed Does not require a changelog entry Docs/Not Needed Does not require documentation labels Jun 25, 2024
@mvitale1989
Copy link
Member Author

/cherry-pick release-9.10

@mattermost-build
Copy link
Contributor

Cherry pick is scheduled.

mattermost-build pushed a commit that referenced this pull request Jul 9, 2024
@mvitale1989 @mattermost-build
CLD-5704 Migrate daily master/cloud tests (#27393)
3694a76 
* [skip ci] Support Cloud/daily tests and Zephyr integration

* [skip ci] Fix workflow file

* [skip ci] Fix typo in workflow input name

* Fix cloud variable passing

* [skip ci] Fix typo

* Utilize master branch image for daily tests

* Apply Saturn's suggestion, fixes and improvements

(cherry picked from commit 4f68dbb)
@mattermost-build mattermost-build mentioned this pull request Jul 9, 2024
Merged
@mattermost-build mattermost-build added the CherryPick/Done Successfully cherry-picked to the quality or patch release tracked in the milestone label Jul 9, 2024
mvitale1989 added a commit that referenced this pull request Jul 9, 2024
@mattermost-build @mvitale1989
CLD-5704 Migrate daily master/cloud tests (#27393) (#27606)
2297732 
* [skip ci] Support Cloud/daily tests and Zephyr integration

* [skip ci] Fix workflow file

* [skip ci] Fix typo in workflow input name

* Fix cloud variable passing

* [skip ci] Fix typo

* Utilize master branch image for daily tests

* Apply Saturn's suggestion, fixes and improvements

(cherry picked from commit 4f68dbb)

Co-authored-by: Mario Vitale <mvitale1989@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@saturninoabril saturninoabril saturninoabril approved these changes

@toninis toninis toninis approved these changes

@phoinixgrr phoinixgrr Awaiting requested review from phoinixgrr

Assignees
No one assigned
Labels
Changelog/Not Needed Does not require a changelog entry CherryPick/Done Successfully cherry-picked to the quality or patch release tracked in the milestone Docs/Not Needed Does not require documentation release-note-none Denotes a PR that doesn't merit a release note.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@mvitale1989 @saturninoabril @mattermost-build @toninis @amyblais @mm-cloud-bot