CI: add disk usage snapshot before BOLT cmake configuration (testing)#103908
Draft
leshikus wants to merge 1 commit intoClickHouse:masterfrom
Draft
CI: add disk usage snapshot before BOLT cmake configuration (testing)#103908leshikus wants to merge 1 commit intoClickHouse:masterfrom
leshikus wants to merge 1 commit intoClickHouse:masterfrom
Conversation
Add `du / | sort -rn | head -100` and `df -h` right before the BOLT profile collection CMake step to diagnose disk exhaustion. Every cmake compiler-feature test invokes the BOLT-instrumented `clang-21.inst`, which writes a profile file per invocation. With hundreds of tests this may consume several GB before the real build starts and kill the runner silently. The snapshot will confirm whether disk is the culprit. Modifying this file also busts the toolchain cache so the `Build Toolchain (PGO, BOLT)` job runs in the PR workflow.
Contributor
|
Workflow [PR], commit [ab260e7] Summary: ✅ AI ReviewSummaryThis PR adds a pre-BOLT disk snapshot in Missing context
Findings
ClickHouse Rules
Performance & Safety
Final Verdict
|
![clickhouse-gh[bot]](https://avatars.githubusercontent.com/in/928874?s=60&v=4){kind=small}
| # consume several GB before the real build starts. | ||
| if bolt_ok: | ||
| print("=== Disk usage before BOLT cmake configuration ===") | ||
| Shell.check("du / 2>/dev/null | sort -rn | head -100 || true") |
Contributor
There was a problem hiding this comment.
du / 2>/dev/null | sort -rn | head -100 scans the whole root tree and then sorts potentially millions of lines. On a nearly-full runner (the exact case this PR is diagnosing), sort can spill temporary files to disk and make disk pressure worse or fail silently due to the trailing || true.
Please bound this snapshot to avoid self-inflicted pressure, e.g. use du -x --max-depth=<N> / (single filesystem + limited depth) before sorting.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Testing PR to diagnose the BOLT toolchain build failure from #103307.
Every cmake compiler-feature test invokes
clang-21.inst(BOLT-instrumentedclang), which writes a profile file per invocation. With hundreds of tests this
may fill several GB of disk before the real build starts, killing the runner
silently. This PR adds
du / | sort -rn | head -100anddf -hright beforethe BOLT cmake step to confirm whether disk exhaustion is the root cause.
The
build_toolchain.pychange also busts the toolchain cache so theBuild Toolchain (PGO, BOLT) (amd64)job actually runs in the PR workflow.Changelog category (leave one):