Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialize game and test build to ease pressure on RAM, especially in CI #73793

Closed
wants to merge 1 commit into from

Conversation

kevingranade
Copy link
Member

Summary

None

Purpose of change

Some chatter on discord brought some build failures like https://github.com/CleverRaven/Cataclysm-DDA/actions/runs/9078930225/job/24950004752#step:14:746 to my attention, looks like memory exhaustion?

Describe the solution

This adds an "ordering only" dependency on the test executable to new clones of the test and check targets in the makefile, along with an option to enable this behavior.
The effect of this is the test part of the build will not start until the game executable is fully linked, which should ease our memory pressure, ESPECIALLY if the test builds are flying through compilation due to high ccache hit rates and it starts linking the test exe befire the link of the game exe is done.

Describe alternatives you've considered

There are a number of options, potentially splitting the test exe into several pieces could work to limit the peak amount of rAM usage on each link job.
It's possible that building a .so of the game code instead of a .a would reduce the overhead of the test link job?
As for how to structure this feature, it's not TOTALLY necessary to add this to the Makefile, you can invoke it like make -j X bla bla bla cataclysm && make -j X bla bla tests, and you can do that locally or in the tests
I'm not totally against doing it this way, but right now I really want to get some idea of whether this will fix the problem at all.

Testing

The acid test will be if the GCC 9 LTO job passes, but also it might need several runs because I don't know what the pass rate recently is.

Additional context

I've been seeing this a lot on my local system as well, if I end up linking the game and test exe at the same time my computer locks up, this is why I'm working this into the Makefile instead of just editing the gha_compile_only.sh to do the build in stages.

@github-actions github-actions bot added Code: Build Issues regarding different builds and build environments astyled astyled PR, label is assigned by github actions json-styled JSON lint passed, label assigned by github actions BasicBuildPassed This PR builds correctly, label assigned by github actions labels May 15, 2024
@kevingranade
Copy link
Member Author

Well that's definitive, failed the same way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
astyled astyled PR, label is assigned by github actions BasicBuildPassed This PR builds correctly, label assigned by github actions Code: Build Issues regarding different builds and build environments json-styled JSON lint passed, label assigned by github actions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant