fix(dl): pack the modules from build/bin; ccache headroom#43
Conversation
First CI run of build-dl-flavor: the 99-min 11-arch build succeeded (605/605 targets, all 14 CPU variants linked) but the pack step failed instantly — ggml links its modules into build/bin (its own output dir) while create-dl-pack.js scans build/Release beside the addon. The dladdr load model wants every module beside the addon anyway, so scripts/build.js now copies build/bin/*.so* into build/Release in DL mode (fail-loud if none found), and the pack script's error now lists what it DID find plus the seam explanation. Also: ccache max-size 500M → 3G for the DL job — the default was 44% consumed by host objects alone on the cold build. (ccache 4.5.1 installed by the action predates nvcc support, so device compilations aren't cached yet; version pin tracked as a follow-on.)
There was a problem hiding this comment.
Pull request overview
This PR fixes a CI failure in the BACKEND_DL (DL flavor) release pipeline where the pack assembly step could not find CUDA backend modules because ggml outputs shared objects to build/bin while the pack logic scans build/Release.
Changes:
- In DL mode, copy
build/bin/*.so*intobuild/Releaseafter a successful build so pack assembly and the runtime loader can find modules beside the addon. - Improve
create-dl-pack.jsfailure output to list discovered.sofiles and explain thebuild/binvsbuild/Releaseseam. - Increase the DL build job’s ccache size in
release.ymlto provide sufficient headroom for the multi-arch CUDA build.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| scripts/build.js | DL-only post-build copy of shared objects from build/bin into build/Release to satisfy pack/runtime expectations. |
| scripts/create-dl-pack.js | More diagnostic error message when libggml-cuda*.so is missing, including what .so files were found and why. |
| .github/workflows/release.yml | Increase ccache max size for the linux-x64 DL job to reduce cache pressure during the fatbin build. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const binDir = path.join(__dirname, '..', 'build', 'bin'); | ||
| const releaseDir = path.join(__dirname, '..', 'build', 'Release'); | ||
| if (!fs.existsSync(binDir)) { | ||
| console.error(`[lloyal.node] DL: expected module output dir missing: ${binDir}`); | ||
| process.exit(1); | ||
| } |
There was a problem hiding this comment.
Precluded-by-design — leaving as-is. The copy block runs only after execSync(buildCmd) returned success, and cmake-js's success contract IS producing build/Release/lloyal.node (the addon's configured output directory); a failed build exits at the catch before the copy is reached. A missing releaseDir at this point would require something external deleting the directory in the milliseconds between build and copy. The two failure modes that ARE reachable (binDir absent; zero modules matched) already fail loudly with named errors, and even in the hypothetical, copyFileSync's ENOENT carries the exact destination path — comparably diagnosable to an explicit guard.
The action apt-installs 4.5.1 which predates nvcc support — the cold run's stats confirmed 0 device compilations cached. A modern static binary on /usr/local/bin wins PATH resolution for the CMake launchers, so the next cold build populates an nvcc-inclusive cache and re-releases stop repaying the ~90-minute device pass.
Fixes the single failure in dry-run 28769570042's `build-dl-flavor` job.
What happened
The 99-minute 11-arch build succeeded (605/605 targets, all 14 CPU variant modules linked), then `Assemble pack + arch gates` failed instantly: `no libggml-cuda*.so module in build/Release`. Root cause is an output-directory seam — ggml links its shared modules into `build/bin` (its own `RUNTIME_OUTPUT_DIRECTORY`), while the pack script scans `build/Release` beside the addon.
Fix
Unrelated but observed in the same dry-run: the L4 `gpu-tests` failure is the pre-existing lockfile drift from July 1 (version-less optional-dep stubs fail `npm ci`'s sync check; reproduced locally on main, and #42's regenerated lockfile passes cleanly) — #42 fixes it; nothing to do here.
🤖 Generated with Claude Code