Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce GitHub runner use for CI #5053

Closed
shorowit opened this issue Dec 6, 2023 · 7 comments
Closed

Reduce GitHub runner use for CI #5053

shorowit opened this issue Dec 6, 2023 · 7 comments
Assignees
Labels
Enhancement Request Triage Issue needs to be assessed and labeled, further information on reported might be needed

Comments

@shorowit
Copy link
Contributor

shorowit commented Dec 6, 2023

Enhancement Request

Currently OpenStudio uses 11 runners, totaling about 24 hours worth of runtime, to generate python and C# bindings on the CI. I believe that NREL gets a total of 20 runners for the entire organization. During active OpenStudio development, this has led to severe blockage that causes other repos' actions to queue up for extended periods of time, and can negatively affect other projects getting their work done.

Can we move these over to the Jenkings cluster? Or otherwise reduce/offload them?

@kbenne @wenyikuang

@shorowit shorowit added Enhancement Request Triage Issue needs to be assessed and labeled, further information on reported might be needed labels Dec 6, 2023
@wenyikuang wenyikuang self-assigned this Dec 6, 2023
@kbenne
Copy link
Contributor

kbenne commented Dec 7, 2023

I think this should be a top priority @wenyikuang and @DavidGoldwasser. It is another thing that is good to do now, before we get into another high pressure release cycle. It seems like we should be able to move this over to Jenkins without too much issue, right?

@jmarrec
Copy link
Collaborator

jmarrec commented Dec 11, 2023

We could start by stopping running the workflow for all pushes to develop right now, until we figure out a better plan of action.

And yes, this is movable to Jenkins. With a couple of notes:

  • We should already have the right python version (3.8.10ish), we could do the python bindings for each without feeling the overhead since we already build Python bindings anyways for the CLI now.
  • We probably do not want to do the CSharp bindings on each run
    • this is going to incur a large overhead. We do it for the Windows runner already and it's the worst runtime (but it's the worst independently of the C# bindings already).
    • We only do the windows C# bindings for the x64, not the win32 ones.
    • We probably don't want to bloat the nightly *Nix installers: if we enable BUILD_CSHARP_BINDINGS, they will be packaged.
  • By run in the above, I mean the nightly run, not the incremental runs on PRs.

jmarrec added a commit that referenced this issue Dec 11, 2023
kbenne added a commit that referenced this issue Dec 11, 2023
#5053 - Temporary fix: stop building Python / C# bindings on each push to develop to preserve GHA CI resources
@wenyikuang
Copy link
Collaborator

wenyikuang commented Dec 22, 2023

Hi @jmarrec @kbenne

I believe we could re-used the script we have here with some debugging and configuration update in the Jenkins system to build the CSharp binding release for each time we merge PR to the CSharp branch. After that we may could deprecate the github action.

Or that's not the same case?
I notice it(the os_full_build/csharpbinding) is consuming Mingbo's Openstudio and using the flag of DBUILD_NUGET_PACKAGE instead of DBUILD_CSHARP_BINDINGS.

@shorowit
Copy link
Contributor Author

On a related note, GitHub Linux runners now have 4 cpus instead of 2:
https://github.blog/2024-01-17-github-hosted-runners-double-the-power-for-open-source/

@shorowit
Copy link
Contributor Author

Should we close this issue? Seems like it has been addressed.

@kbenne
Copy link
Contributor

kbenne commented Apr 29, 2024

We do now have openstudio-gems running as an action, which is new, but I don't think it has been a big issue.

The other thing is that I believe the C# bindings/nuget package are setup as an action, but I'm not sure. We'll need @wenyikuang to weight and confirm if we can close this for sure.

@wenyikuang
Copy link
Collaborator

Yeah, C# binding and pypi package are still in github actions.

https://github.com/NREL/OpenStudio/blob/develop/.github/workflows/buildCSharp.yml

I think I could migrant them to come with windows build if there is no hidden issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Request Triage Issue needs to be assessed and labeled, further information on reported might be needed
Projects
None yet
Development

No branches or pull requests

4 participants