Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TeamCity: update nightly workflow to use static branch #18447

Merged
merged 8 commits into from
Jun 21, 2024

Conversation

BBBmau
Copy link
Contributor

@BBBmau BBBmau commented Jun 14, 2024

Continuation of #18241

An issue with TeamCity unable to perform updates on branchFilter spec resulted in the date itself not matching the nightly test branch created.

This is resolved with using a static branch filter (+:UTC-*) within teamcity configs. To make this work we'll create a branch such as UTC-2024-06-24 for use by TeamCity to run the nightly test. Once a considerable period of time has passed, a cronjob will trigger that renames the UTC-2024-06-24 -> UTC-nightly-tests-2024-06-24. The renaming is just for Github and allows us to only focus on one branch which must match both +:UTC-* and -:UTC-nightly-tests-*

The update to the TeamCity branchFilter spec can be seen here: GoogleCloudPlatform/magic-modules#10785

Example run of the new workflows:
teamcity-nightly-workflow.yaml

teamcity-nightly-sweeper.yaml

@BBBmau BBBmau requested a review from SarahFrench June 14, 2024 19:10
Copy link

** WARNING **: This repository is generated by https://github.com/GoogleCloudPlatform/magic-modules. Any changes made directly to this repository will likely be overwritten. If you have further questions, please feel free to ping your reviewer or, internal employees, reach out to one of the engineers. Thank you!

@github-actions github-actions bot requested a review from c2thorn June 14, 2024 19:10
Copy link
Member

@SarahFrench SarahFrench left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments!

Have you have this PR's GHAs labelling your fork of the TPG repo and seeing if your TeamCity project is picking up the static named branch ok?

.github/workflows/teamcity-nightly-workflow.yaml Outdated Show resolved Hide resolved
.github/workflows/teamcity-nightly-sweeper.yaml Outdated Show resolved Hide resolved
@BBBmau
Copy link
Contributor Author

BBBmau commented Jun 17, 2024

@SarahFrench From teamcity config change PR:

new linked PR allows static branch filtering: we now only build if we detect a branch that matches both rules: +:refs/heads/UTC-* -:refs/heads/UTC-nightly-*

image

Once a period of time has passed (12 hours) the newly used branch for nightly tests will be renamed to match -:refs/heads/UTC-nightly-*. Allowing a new branch to be created which will be the only branch matching both rules. (As seen in the image above)

An explanation of the new gha workflow for this can be seen: hashicorp/terraform-provider-google#18447

Link to the TeamCity builds can be seen: https://hashicorp.teamcity.com/project/TerraformProviders_MauricioSTestProjectSameCommitNightlyTestsWork_GOOGLE_NIGHTLYTESTS?branch=refs%2Fheads%2FUTC-2024-06-14&mode=builds

had to update the cronjob due to it being the same time as when the teamcity builds are triggered. This caused teamcity to filter before the job was completed. Fixed now by setting the cronjob to run at 3AM UTC.

Copy link
Member

@SarahFrench SarahFrench left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, but given that there are 2 workflows now you'll also need to add a new workflow in the google-beta repo that re-uses the teamcity-nightly-sweeper.yaml workflow.

Next step lets make sure your TeamCity test project is able to pull in the new branches each night.

@BBBmau
Copy link
Contributor Author

BBBmau commented Jun 18, 2024

Approving, but given that there are 2 workflows now you'll also need to add a new workflow in the google-beta repo that re-uses the teamcity-nightly-sweeper.yaml workflow.

Next step lets make sure your TeamCity test project is able to pull in the new branches each night.

We got a successful branch creation and build from last night, we can see it here: https://hashicorp.teamcity.com/buildConfiguration/TerraformProviders_MauricioSTestProjectSameCommitNightlyTestsWork_GOOGLE_NIGHTLYTESTS_GOOGLE_PACKAGE_ACCESSAPPROVAL?branch=&mode=branches#all-projects

I noticed that the sweeper didn't delete old branches, this was due to the filter not being updated. This PR already has the updated changes.

I'll make a PR for beta repo but after one more cronjob trigger we can merge this. 👍🏼

BBBmau and others added 2 commits June 21, 2024 12:43
Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>
@BBBmau
Copy link
Contributor Author

BBBmau commented Jun 21, 2024

merging after seeing successful branch creation and branch sweeping with new workflow method. Other changes that were made were done in teamcity config files.

@BBBmau BBBmau merged commit e432037 into hashicorp:main Jun 21, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants