Improve Actions hygiene #306
Conversation
👋 hello there! I'm a fellow Googler who works on projects that leverage GitHub Actions for CI/CD. Recently I noticed a large increase in our queue time, and I've tracked it down to the [limit of 180 concurrent jobs](https://docs.github.com/en/actions/reference/usage-limits-billing-and-administration) for an organization. To help be better citizens, I'm proposing changes across a few repositories that will reduce GitHub Actions hours and consumption. I hope these changes are reasonable and I'm happy to talk through them in more detail. - **(you were already doing this, thank you!)** Only run GitHub Actions for pushes and PRs against the main branch of the repository. If your team uses a forking model, this change will not affect you. If your team pushes branches to the repository directly, this changes actions to only run against the primary branches or if you open a Pull Request against a primary branch. - For long-running jobs (especially tests), I added the "Cancel previous" workflow. This is very helpful to prevent a large queue backlog when you are doing rapid development and pushing multiple commits. Without this, GitHub Actions' default behavior is to run all actions on all commits. There are other changes you could make, depending on your project (but I'm not an expert): - If you have tests that should only run when a subset of code changes, consider gating your workflow to particular file paths. For example, we have some jobs that do Terraform linting, but [they only run when Terraform files are changed](https://github.com/google/exposure-notifications-verification-server/blob/c4f59fee71042cf668747e599e7c769fca736554/.github/workflows/terraform.yml#L3-L11). Hopefully these changes are not too controversial and also hopefully you can see how this would reduce actions consumption to be good citizens to fellow Googlers. If you have any questions, feel free to respond here or ping me on chat. Thank you!
sethvargo
requested review from
deepankarb,
jingtang10,
pankajkatoch and
yigit
as code owners
|
Thank you @sethvargo! Appreciate this. We have been struggling with Actions a little bit recently actually. See #73. Have you seen issues like this before and do you have any tips? Thanks! |
|
I believe the Actions runners are restricted to RAM and CPU: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners. |
|
@sethvargo are you saying this is just a hard constraint? is there anything we can do to confirm/debug this? ironically it's blocking this very PR as you can see in the status checks. :( |
|
@deepankarb fyi |
|
@jingtang10 I believe the only other option would be a custom runner (e.g. you configure a VM in GCP as an actions runner). That would give you the ability to specify more RAM/CPU. |
|
Can confirm, free runners are part of a pool that have a very limited ram/cpu budget. These runners are shared amongst all free projects. We're still chatting with GitHub about options. Hosting your own runner would work https://github.blog/2020-08-04-github-actions-self-hosted-runners-on-google-cloud/ |
👋 hello there! I'm a fellow Googler who works on projects that leverage GitHub Actions for CI/CD. Recently I noticed a large increase in our queue time, and I've tracked it down to the limit of 180 concurrent jobs for an organization. To help be better citizens, I'm proposing changes across a few repositories that will reduce GitHub Actions hours and consumption. I hope these changes are reasonable and I'm happy to talk through them in more detail.
(you were already doing this, thank you!) Only run GitHub Actions for pushes and PRs against the main branch of the repository. If your team uses a forking model, this change will not affect you. If your team pushes branches to the repository directly, this changes actions to only run against the primary branches or if you open a Pull Request against a primary branch.
For long-running jobs (especially tests), I added the "Cancel previous" workflow. This is very helpful to prevent a large queue backlog when you are doing rapid development and pushing multiple commits. Without this, GitHub Actions' default behavior is to run all actions on all commits.
There are other changes you could make, depending on your project (but I'm not an expert):
Hopefully these changes are not too controversial and also hopefully you can see how this would reduce actions consumption to be good citizens to fellow Googlers. If you have any questions, feel free to respond here or ping me on chat. Thank you!