lightgbm: restore the workflow and use static OpenMP schedules - #615
Merged
Conversation
#481 merged with a temporary probe job still in place: an omp_probe job that compiles a standalone OpenMP program, and an 'if: github.run_id == "0"' guard that switched the sdist, wheel and publish jobs off. Remove both so the workflow builds again. The probe was investigating the riscv64 segfault in the ranking tests. It found that libgomp's dynamic and guided work-share schedules fault on the runner for a 40-line C program with no LightGBM in it, under both GCC 13 and GCC 14; schedule(static) and 30M concurrent mallocs per thread are clean. That is tracked separately.
luhenry
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#481 merged while a temporary probe was still in the workflow:
build-lightgbm.ymlonmaincarries anomp_probejob and anif: github.run_id == '0'guard that switchespython_sdist,build_wheelsandpublishoff, so it currently builds nothing. This removes both, and adds the patch the probe turned out to be needed.Differs from upstream
dynamicandguidedschedules replaced withstatic— see below.Patches
0002-use-schedule-static-for-the-dynamic-and-guided-OpenMP.patch— Inappropriate [libgomp dynamic and guided OpenMP schedules segfault on the riscv64 runners #617]. Without it the ranking tests segfault insidegomp_iter_guided_next(). riscv64-runner-specific.libgomp's dynamic and guided work-share schedules fault on the runners. A 40-line C program that allocates nothing in the loop body segfaults 3/3 under
schedule(guided)andschedule(dynamic), on the bare runner (GCC 13.3.0) and insidemanylinux_2_39_riscv64(GCC 14.3.1);schedule(static)and 30M concurrentmalloc/freeper thread are clean, and the same binaries pass under QEMU riscv64 and on aarch64 at 64 threads. Details in #617. Only 13 of LightGBM's ~230 parallel regions ask for dynamic or guided; the rest already use static. The cost is load balancing on ranking queries and feature groups. Revert when the runners' toolchain is fixed.Full suite on aarch64, patched and unpatched: 822 passed, 260 skipped, 12 xfailed, 4 xpassed.