ffi: shrink moq-ffi & libmoq staticlibs with LTO (unblocks the moq-go mirror push)#1577
Conversation
…itHub's 100 MB limit The moq-go mirror has been stuck at v0.2.15 since #1549. That PR correctly fixed go/scripts/package.sh to stage moq.h and the Linux staticlibs, but it traded "missing files" for "files too big to push": the unstripped Linux libmoq_ffi.a is ~110 MB, and GitHub's pre-receive hook hard-rejects any file over 100 MB (GH001). Every release push since (v0.2.16, v0.2.17) failed at the publish step, so `go get github.com/moq-dev/moq-go@latest` still resolves to the pre-fix v0.2.15 and consumers' cgo builds fail on the missing header. Enable thin LTO with a single codegen unit for the release artifact in rs/moq-ffi/build.sh (the shared build script for the Go/Swift/Kotlin releases). LTO dead-strips the unused monomorphizations Rust bakes into a staticlib, cutting it ~60% (76 MB -> 28 MB on darwin, ~110 MB -> ~40 MB on Linux) with no source or ABI changes. Verified the full uniffi C export set is preserved and that `go vet`/`build`/`test` link cleanly against the LTO'd lib. Scoped via env vars so a plain `cargo build --release` stays fast. Also add a size guard to go/scripts/package.sh that fails the package step with a file-named error if any staged lib hits 100 MiB, so a future regression fails fast with context instead of dying at `git push` after an hour-long multi-target build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
WalkthroughThis PR coordinates size reduction of moq static libraries across the build and packaging pipeline. The Rust build scripts and Nix overlay now set release profile env vars to thin LTO and a single codegen unit. The Go packaging script measures staged library sizes in MiB, records any that meet or exceed GitHub's 100 MiB per-file limit, and aborts packaging with an error listing oversized libs and guidance to use LTO if any are found. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
libmoq.a carries moq-ffi's whole dependency tree (hang, moq-audio, moq-mux, moq-native, moq-net), so an unstripped build is ~58 MB. It ships as a release tarball and Homebrew bottle rather than a git mirror, so it never hit the 100 MB git push limit that pinned moq-go, but the same thin-LTO treatment roughly halves the artifact for free. Set CARGO_PROFILE_RELEASE_LTO=thin + CODEGEN_UNITS=1 on both build paths: rs/libmoq/build.sh (the Windows cargo path) and the crane derivation in nix/overlay.nix (the Linux/macOS path). Measured: 57.8 MB -> 26.9 MB via cargo, and the nix build drops to 21.3 MB. All 50 hand-written C exports survive LTO (verified identical symbol set), and the header / cmake / pkgconfig artifacts are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Summary
The
moq-dev/moq-gomirror has been stuck at v0.2.15 even though #1549 "fixed" the Go module packaging. The reported root cause (package.shnot stagingmoq.h/ the Linux archives) was already addressed by #1549 - the build and package jobs for v0.2.16 and v0.2.17 succeed and the artifact contains every file. The real blocker is one step later:#1549 traded missing files for files too big to push. The unstripped Linux
libmoq_ffi.ais ~110 MB and GitHub hard-rejects any file over 100 MB (GH001). Both post-fix release runs (moq-ffi-v0.2.16,moq-ffi-v0.2.17) failed at the publish step, sogo get github.com/moq-dev/moq-go@lateststill resolves to the pre-fix v0.2.15 and consumers' cgo builds fail on the missingmoq.h. That's what trips the externalsmoke.sh(go get @latest && CGO_ENABLED=1 go build) and flips the whole run red.Why the obvious alternatives don't work for Go specifically:
So the lib must live in-tree and be under 100 MB. Shrinking the archive is the only lever.
Fix
Enable thin LTO +
codegen-units = 1for the FFI staticlib builds. LTO dead-strips the tens of thousands of unused monomorphizations Rust bakes into a staticlib, with no source or ABI change:moq-ffidarwin arm64.amoq-ffilinux x86_64.alibmoq.a(cargo)libmoq.a(nix build)Applied in two places, scoped via
CARGO_PROFILE_RELEASE_*env vars so a plaincargo build --releasestays fast and a caller can still override:rs/moq-ffi/build.sh- the shared build script for the Go/Swift/Kotlin releases.rs/libmoq/build.sh(Windows cargo path) +nix/overlay.nix(the Linux/macOS crane path) for libmoq.Verification: the full uniffi C export set (263 symbols) and all 50 hand-written libmoq C exports are byte-identical pre/post-LTO, and
go vet/go build/go testlink cleanly under cgo against the LTO'd lib (go/scripts/check.sh). The nix.#libmoqbuild produces a working 21.3 MB.awith header/cmake/pkgconfig intact.Also adds a size guard to
go/scripts/package.sh: if any staged lib hits 100 MiB it fails the package step with a file-named error pointing at the fix, instead of dying atgit pushafter an hour-long multi-target build.Scope
brew install)..sovia Maven) don't hit the git limit, but pick up smaller libs from the sharedmoq-ffi/build.shchange.doc/lib/*need no edits.moq-gst(cdylib plugin) and the binaries (moq-relay/moq-cli) are also LTO candidates but left out of this PR - different distribution, different tradeoffs (compile time vs runtime); happy to do them as a follow-up.Test plan
shellcheck+bash -nclean on all changed scripts;nixfmt --checkclean onoverlay.nixmoq-ffi/build.shstages a 28 MB lib (was 76 MB); all bindings generatego/scripts/check.shwith LTO:go vet/build/testpass against the LTO'd staticlibnix build .#libmoqproduces a working 21.3 MB.a(header/cmake/pkgconfig intact)moq-ffi-v*tag: release-go publish step pushes successfully and the mirror advances past v0.2.15🤖 Generated with Claude Code
(Written by Claude)