Skip to content

ffi: shrink moq-ffi & libmoq staticlibs with LTO (unblocks the moq-go mirror push)#1577

Merged
kixelated merged 2 commits into
mainfrom
claude/competent-mccarthy-1c267a
Jun 1, 2026
Merged

ffi: shrink moq-ffi & libmoq staticlibs with LTO (unblocks the moq-go mirror push)#1577
kixelated merged 2 commits into
mainfrom
claude/competent-mccarthy-1c267a

Conversation

@kixelated
Copy link
Copy Markdown
Collaborator

@kixelated kixelated commented Jun 1, 2026

Summary

The moq-dev/moq-go mirror has been stuck at v0.2.15 even though #1549 "fixed" the Go module packaging. The reported root cause (package.sh not staging moq.h / the Linux archives) was already addressed by #1549 - the build and package jobs for v0.2.16 and v0.2.17 succeed and the artifact contains every file. The real blocker is one step later:

remote: error: File moq/lib/linux_arm64/libmoq_ffi.a is 108.98 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File moq/lib/linux_amd64/libmoq_ffi.a is 109.81 MB; this exceeds GitHub's file size limit of 100.00 MB
 ! [remote rejected] HEAD -> main (pre-receive hook declined)

#1549 traded missing files for files too big to push. The unstripped Linux libmoq_ffi.a is ~110 MB and GitHub hard-rejects any file over 100 MB (GH001). Both post-fix release runs (moq-ffi-v0.2.16, moq-ffi-v0.2.17) failed at the publish step, so go get github.com/moq-dev/moq-go@latest still resolves to the pre-fix v0.2.15 and consumers' cgo builds fail on the missing moq.h. That's what trips the external smoke.sh (go get @latest && CGO_ENABLED=1 go build) and flips the whole run red.

Why the obvious alternatives don't work for Go specifically:

  • Git LFS - the Go module proxy serves the LFS pointer file, not the binary, so cgo would link garbage.
  • Release assets - Go modules can't fetch external assets at build time; cgo needs the lib in the module tree.

So the lib must live in-tree and be under 100 MB. Shrinking the archive is the only lever.

Fix

Enable thin LTO + codegen-units = 1 for the FFI staticlib builds. LTO dead-strips the tens of thousands of unused monomorphizations Rust bakes into a staticlib, with no source or ABI change:

artifact before after (LTO)
moq-ffi darwin arm64 .a 75.8 MB 28.2 MB
moq-ffi linux x86_64 .a ~110 MB ~40 MB (est)
libmoq.a (cargo) 57.8 MB 26.9 MB
libmoq.a (nix build) ~58 MB 21.3 MB

Applied in two places, scoped via CARGO_PROFILE_RELEASE_* env vars so a plain cargo build --release stays fast and a caller can still override:

  • rs/moq-ffi/build.sh - the shared build script for the Go/Swift/Kotlin releases.
  • rs/libmoq/build.sh (Windows cargo path) + nix/overlay.nix (the Linux/macOS crane path) for libmoq.

Verification: the full uniffi C export set (263 symbols) and all 50 hand-written libmoq C exports are byte-identical pre/post-LTO, and go vet/go build/go test link cleanly under cgo against the LTO'd lib (go/scripts/check.sh). The nix .#libmoq build produces a working 21.3 MB .a with header/cmake/pkgconfig intact.

Also adds a size guard to go/scripts/package.sh: if any staged lib hits 100 MiB it fails the package step with a file-named error pointing at the fix, instead of dying at git push after an hour-long multi-target build.

Scope

  • Go is the only wrapper that commits the staticlib into a git mirror, so it's the only one that hit the 100 MB wall - this PR unblocks it.
  • libmoq ships as a release tarball + Homebrew bottle (no git limit), so it was never broken, but gets the same ~2x shrink for free (smaller downloads, faster brew install).
  • Swift (XCFramework as a Release asset, 2 GB limit) and Kotlin (~13 MB .so via Maven) don't hit the git limit, but pick up smaller libs from the shared moq-ffi/build.sh change.
  • Python uses maturin wheels (no git limit), unaffected. No FFI API/ABI change, so the language wrappers and doc/lib/* need no edits.

moq-gst (cdylib plugin) and the binaries (moq-relay/moq-cli) are also LTO candidates but left out of this PR - different distribution, different tradeoffs (compile time vs runtime); happy to do them as a follow-up.

Test plan

  • shellcheck + bash -n clean on all changed scripts; nixfmt --check clean on overlay.nix
  • moq-ffi/build.sh stages a 28 MB lib (was 76 MB); all bindings generate
  • go/scripts/check.sh with LTO: go vet/build/test pass against the LTO'd staticlib
  • 263 uniffi exports + 50 libmoq C exports survive LTO (identical sets)
  • nix build .#libmoq produces a working 21.3 MB .a (header/cmake/pkgconfig intact)
  • Next moq-ffi-v* tag: release-go publish step pushes successfully and the mirror advances past v0.2.15

🤖 Generated with Claude Code

(Written by Claude)

…itHub's 100 MB limit

The moq-go mirror has been stuck at v0.2.15 since #1549. That PR correctly
fixed go/scripts/package.sh to stage moq.h and the Linux staticlibs, but it
traded "missing files" for "files too big to push": the unstripped Linux
libmoq_ffi.a is ~110 MB, and GitHub's pre-receive hook hard-rejects any file
over 100 MB (GH001). Every release push since (v0.2.16, v0.2.17) failed at the
publish step, so `go get github.com/moq-dev/moq-go@latest` still resolves to
the pre-fix v0.2.15 and consumers' cgo builds fail on the missing header.

Enable thin LTO with a single codegen unit for the release artifact in
rs/moq-ffi/build.sh (the shared build script for the Go/Swift/Kotlin
releases). LTO dead-strips the unused monomorphizations Rust bakes into a
staticlib, cutting it ~60% (76 MB -> 28 MB on darwin, ~110 MB -> ~40 MB on
Linux) with no source or ABI changes. Verified the full uniffi C export set is
preserved and that `go vet`/`build`/`test` link cleanly against the LTO'd lib.
Scoped via env vars so a plain `cargo build --release` stays fast.

Also add a size guard to go/scripts/package.sh that fails the package step
with a file-named error if any staged lib hits 100 MiB, so a future regression
fails fast with context instead of dying at `git push` after an hour-long
multi-target build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Jun 1, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9d8a42d0-3031-4a13-8e64-aeaaa40191f5

📥 Commits

Reviewing files that changed from the base of the PR and between 6c23978 and c499809.

📒 Files selected for processing (2)
  • nix/overlay.nix
  • rs/libmoq/build.sh

Walkthrough

This PR coordinates size reduction of moq static libraries across the build and packaging pipeline. The Rust build scripts and Nix overlay now set release profile env vars to thin LTO and a single codegen unit. The Go packaging script measures staged library sizes in MiB, records any that meet or exceed GitHub's 100 MiB per-file limit, and aborts packaging with an error listing oversized libs and guidance to use LTO if any are found.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed Title accurately summarizes the main change: enabling LTO to shrink staticlibs and unblock the Go mirror push under GitHub's 100 MB file limit.
Description check ✅ Passed Description is comprehensive and directly related to the changeset, detailing the problem, solution, scope, and test plan for shrinking staticlibs via LTO.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch claude/competent-mccarthy-1c267a

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

libmoq.a carries moq-ffi's whole dependency tree (hang, moq-audio, moq-mux,
moq-native, moq-net), so an unstripped build is ~58 MB. It ships as a release
tarball and Homebrew bottle rather than a git mirror, so it never hit the
100 MB git push limit that pinned moq-go, but the same thin-LTO treatment
roughly halves the artifact for free.

Set CARGO_PROFILE_RELEASE_LTO=thin + CODEGEN_UNITS=1 on both build paths:
rs/libmoq/build.sh (the Windows cargo path) and the crane derivation in
nix/overlay.nix (the Linux/macOS path). Measured: 57.8 MB -> 26.9 MB via cargo,
and the nix build drops to 21.3 MB. All 50 hand-written C exports survive LTO
(verified identical symbol set), and the header / cmake / pkgconfig artifacts
are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kixelated kixelated changed the title go: shrink moq-ffi staticlib with LTO so the mirror push fits under GitHub's 100 MB limit ffi: shrink moq-ffi & libmoq staticlibs with LTO (unblocks the moq-go mirror push) Jun 1, 2026
@kixelated kixelated merged commit 85cb97d into main Jun 1, 2026
37 of 38 checks passed
@kixelated kixelated deleted the claude/competent-mccarthy-1c267a branch June 1, 2026 18:19
@moq-bot moq-bot Bot mentioned this pull request Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant