Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce GitHub Action VM Size / Resource Consumption (Cost) #2294

Closed
vorburger opened this issue Oct 25, 2023 · 5 comments · Fixed by #2296
Closed

Reduce GitHub Action VM Size / Resource Consumption (Cost) #2294

vorburger opened this issue Oct 25, 2023 · 5 comments · Fixed by #2296
Assignees
Labels
effort:xsmall Extra small effort - 1 day github_actions Pull requests that update GitHub Actions code P1 High priority issue type:build Issues related to code build type:testing Improvements on tests or testing infrastructure

Comments

@vorburger
Copy link
Member

vorburger commented Oct 25, 2023

The GitHub "Runner" which we (me) had set up in #2197 is "large".

The goal of this issue is to explore if it can reduced - ideally without slowing down builds.

@vorburger vorburger added type:testing Improvements on tests or testing infrastructure P1 High priority issue effort:xsmall Extra small effort - 1 day type:build Issues related to code build github_actions Pull requests that update GitHub Actions code labels Oct 25, 2023
@vorburger vorburger self-assigned this Oct 25, 2023
vorburger added a commit to vorburger/android-fhir that referenced this issue Oct 25, 2023
@vorburger
Copy link
Member Author

vorburger commented Oct 25, 2023

My analysis about this:

The https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources which this project originally (before my #2198 PR) used was a VM with 2-cores · 7 GB RAM · 14 GB SSD, and that was definitely too small, and didn't suffice.

CodeQL currently takes ~4-11'.

Build currently takes ~2-12'

#2295 will compare this with a VM with 4-cores · 16 GB RAM · 150 GB SSD...

#2296 will compare this with a VM with 8-cores · 32 GB RAM · 300 GB SSD...

@vorburger
Copy link
Member Author

vorburger commented Oct 25, 2023

#2295 will compare this with a VM with 4-cores · 16 GB RAM · 150 GB SSD...

CodeQL took 9½' with this - about the same as (the high end of) with the huge VM.

Build took only 3' - but that's likely "wrong" (misleading), because all Gradle tasks are UP-TO-DATE; I probably need to make a trivial code change for this to be more realistic... or cherry-pick a random other PR to make a more sensible "real" comparison.

Kokoro failed with an unrelated problem; I've opened new #2297 about that.

#2296 will compare this with a VM with 8-cores · 32 GB RAM · 300 GB SSD...

CodeQL took 6' with this - a bit faster, but I'm not sure how significant this is.

Build again just 2' - same "problem" as above.

@vorburger
Copy link
Member Author

vorburger commented Oct 25, 2023

or cherry-pick a random other PR to make a more sensible "real" comparison.

I'm using #2283 as a semi randomly chosen example (because it seems to have taken a bit longer to build, which is either a pure coincidence, or because the change of that dependency causes some real builds to happen) and have (temporarily) added that to the 2 PRs from above; the results are:

Build of #2283 with the very large VM as-is took ~8½'

CodeQL (ditto) took ~5'

#2295 will compare this with a VM with 4-cores · 16 GB RAM · 150 GB SSD...

Build with #2283 took 20'.

CodeQL (ditto) took 7'.

#2296 will compare this with a VM with 8-cores · 32 GB RAM · 300 GB SSD...

Build with #2283 took 13'.

CodeQL (ditto) took 5'.

@vorburger
Copy link
Member Author

#2299 compare this with a VM with 16-cores · 64 GB RAM · 600 GB SSD:

Build with #2283 took 10'.

CodeQL (ditto) took 5'.

Based on these results, my suggestion would be to just go with the 8-cores one, for now.

I've therefore closed #2299 (16 cores) and #2295 (4 cores) and un-drafted #2296 (8 cores), after rm #2283 from it again.

@vorburger
Copy link
Member Author

BTW The Xmx8g in https://github.com/google/android-fhir/blob/master/gradle.properties is relevant here; significantly more RAM won't be used by Gradle anyway with that.

vorburger added a commit to vorburger/android-fhir that referenced this issue Oct 26, 2023
This is actually MORE (not less) efficient, because
those 2 actions (partially) did the same thing anyway,
so there really is no benefit to separately run them
in parallel, other than increased resource usage, and
more complexity to understand this project's CI.
jingtang10 pushed a commit that referenced this issue Oct 26, 2023
This is actually MORE (not less) efficient, because
those 2 actions (partially) did the same thing anyway,
so there really is no benefit to separately run them
in parallel, other than increased resource usage, and
more complexity to understand this project's CI.
vorburger added a commit that referenced this issue Oct 26, 2023
vorburger added a commit that referenced this issue Oct 26, 2023
vorburger added a commit to vorburger/android-fhir that referenced this issue Oct 26, 2023
vorburger added a commit to vorburger/android-fhir that referenced this issue Oct 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort:xsmall Extra small effort - 1 day github_actions Pull requests that update GitHub Actions code P1 High priority issue type:build Issues related to code build type:testing Improvements on tests or testing infrastructure
Projects
Status: Complete
1 participant