Skip to content

wheel: strip non-extern symbols on macOS and Linux#6057

Closed
Grantim wants to merge 3 commits into
masterfrom
wheel/strip-macos
Closed

wheel: strip non-extern symbols on macOS and Linux#6057
Grantim wants to merge 3 commits into
masterfrom
wheel/strip-macos

Conversation

@Grantim
Copy link
Copy Markdown
Contributor

@Grantim Grantim commented May 6, 2026

Summary

The macOS pip wheel ships every Mach-O binary with its __LINKEDIT segment intact. On mrmeshpy.so (124.6 MB) __LINKEDIT is 82.8 MB — 67% of the file; libopenvdb.dylib shows the same: 49 MB of 68.8 MB. The macOS build has no strip step; the Linux build has auditwheel repair but doesn't pass --strip, so the 99 bundled libs in meshlib.libs/ retain ~15 MB of .symtab/.strtab/.debug_* they don't need at runtime.

This PR adds the missing strip step on both platforms:

  • macOSstrip -x between delocate-path and delocate-wheel in scripts/wheel/build_wheel.py. -x removes only local symbols; externs that dyld binds (PyInit_*, library entrypoints) stay intact, so dynamic loading is unaffected. chmod +w first because delocate copies brew dylibs preserving their 0444 mode. delocate-wheel runs after the strip and re-validates / re-signs.
  • Linux--strip flag on auditwheel repair. auditwheel runs strip on each bundled library during the repair step.
  • Windows — no change. PE files don't carry the equivalent symbol payload; PDBs are external and not bundled by delvewheel.

Expected savings from analysis of v3.1.2.192 wheels:

  • macOS .whl (compressed): roughly −30 MB per wheel × 2 architectures
    • mrmeshpy.so 124.6 → ~50 MB uncompressed
    • libopenvdb.dylib 68.8 → ~25 MB uncompressed
  • Linux .whl (compressed): roughly −3-5 MB per wheel × 2 architectures
    • 15 MB of strippable bytes across 99 bundled .so files (mostly .strtab which compresses well, hence the modest net savings)

Test plan

CI under the test-pip-build label produces all 4 wheel artifacts; size delta is visible in the workflow summary.

  • CI builds all 5 pip-build matrices green (macos x86, macos arm64, manylinux x86_64, manylinux aarch64, win amd64)
  • Compare each platform's artifact size to baseline (current master) — confirm ~30 MB drop on macOS, ~3-5 MB on Linux, ~0 on Windows
  • pip install the macOS arm64 wheel in a fresh venv, run import meshlib.mrmeshpy; meshlib.mrmeshpy.makeCube() — should succeed (catches any extern accidentally trimmed)
  • Same for Linux x86_64 wheel
  • Spot-check lief.parse(...).segments on a stripped Mach-O: __LINKEDIT should drop from ~80 MB to ~20-30 MB on mrmeshpy.so
  • Spot-check lief.parse(...).sections on a stripped Linux bundled lib (e.g. libopenvdb.so): .symtab/.strtab should be near zero

🤖 Generated with Claude Code

macOS Mach-O binaries shipped in the meshlib wheel retain all local
symbols by default (`__LINKEDIT` is ~67% of `mrmeshpy.so`). Run
`strip -x` after `delocate-path` and before `delocate-wheel` so the
final wheel ships with externs only — saves roughly 30 MB compressed
per macOS wheel without losing any dyld-visible entrypoint.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Grantim Grantim added the test-pip-build Build Python wheels (and discard them) label May 6, 2026
Grantim and others added 2 commits May 6, 2026 20:07
delocate copies brew dylibs preserving their 0444 mode, so `strip -x`
fails with EACCES. Bump user-write before each strip call.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pass `--strip` to `auditwheel repair` so the bundled deps in
`meshlib.libs/` lose their .symtab/.strtab/.debug_* sections.
~15 MB uncompressed savings across the dep chain (libopenvdb,
libxerces, OCCT TK*, libMR*); compresses to ~3-5 MB on the .whl.

Companion to the macOS strip step. Windows has no analog — PE
files don't carry the equivalent symbol payload (PDBs are external
and not bundled).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Grantim Grantim changed the title wheel: strip non-extern symbols on macOS wheel: strip non-extern symbols on macOS and Linux May 6, 2026
@Grantim
Copy link
Copy Markdown
Contributor Author

Grantim commented May 6, 2026

Abandoning this approach — the wheel-only strip placement was ineffective (modified the staging dir after python -m build had already frozen the wheel) and the lever should live at build/install time so every distribution benefits, not just wheels. Replacing with a two-PR rollout: PR-A adds add_link_options(-Wl,-x) for macOS Release in CompilerOptions.cmake (our libs); PR-B adds vcpkg triplet -Wl,-s (Linux third-party) and CI brew strip (macOS third-party). See investigation in C:/WORK/claude/wheel-size-investigation.md and plan in C:/WORK/claude/strip-plan.md.

@Grantim Grantim closed this May 6, 2026
@Grantim Grantim deleted the wheel/strip-macos branch May 6, 2026 18:57
Grantim added a commit that referenced this pull request May 7, 2026
Companion to PR #6058. PR-A stripped our own libs at link; this strips
the third-party deps that we bundle into wheels/.pkg/NuGet/etc.

- Linux: VCPKG_LINKER_FLAGS_RELEASE -Wl,-s in x64+arm64 triplets so vcpkg
  ports drop .symtab/.strtab/.debug_* during their own release link
  (~15 MB across the bundled chain). Strip-at-link sidesteps the patchelf
  alignment bug that broke #6057's auditwheel --strip attempt.
- macOS: chmod +w + strip -x every brew dylib in install_brew_requirements.sh
  so delocate-wheel/install_name_tool/.pkg copy already-trimmed binaries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Grantim added a commit that referenced this pull request May 7, 2026
Companion to PR #6058. PR-A stripped our own libs at link; this strips
the third-party deps that we bundle into wheels/.pkg/NuGet/etc.

- Linux: VCPKG_LINKER_FLAGS_RELEASE -Wl,-s in x64+arm64 triplets so vcpkg
  ports drop .symtab/.strtab/.debug_* during their own release link
  (~15 MB across the bundled chain). Strip-at-link sidesteps the patchelf
  alignment bug that broke #6057's auditwheel --strip attempt.
- macOS: chmod +w + strip -x every brew dylib in install_brew_requirements.sh
  so delocate-wheel/install_name_tool/.pkg copy already-trimmed binaries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Grantim added a commit that referenced this pull request May 7, 2026
…6063)

* deps: strip third-party libs at install time (vcpkg + brew)

Companion to PR #6058. PR-A stripped our own libs at link; this strips
the third-party deps that we bundle into wheels/.pkg/NuGet/etc.

- Linux: VCPKG_LINKER_FLAGS_RELEASE -Wl,-s in x64+arm64 triplets so vcpkg
  ports drop .symtab/.strtab/.debug_* during their own release link
  (~15 MB across the bundled chain). Strip-at-link sidesteps the patchelf
  alignment bug that broke #6057's auditwheel --strip attempt.
- macOS: chmod +w + strip -x every brew dylib in install_brew_requirements.sh
  so delocate-wheel/install_name_tool/.pkg copy already-trimmed binaries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(brew strip): walk Cellar/ instead of lib/

\$BREW_PREFIX/lib contains only symlinks into Cellar/, so the previous
find -type f matched nothing. Walking Cellar reaches the real .dylib
files. Verified separately: linux vcpkg side stripped libopenvdb.so
.strtab from 2.07 MB to 0; mac libopenvdb.dylib __LINKEDIT was unchanged
because of this bug.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: retrigger CI (prepare-image was cancelled mid-flight)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(brew strip): re-sign dylibs with ad-hoc signature after strip

strip invalidates the existing ad-hoc signature, and dyld then SIGKILLs
any process trying to load the dylib (Apple Silicon enforces this hard).
Symptom on the previous run: python3.10 -m ensurepip got Killed: 9.
Fix: codesign --force --sign - immediately after strip in the same find
chain.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* deps: drop macOS brew strip, scope PR to Linux only

The macOS brew strip ran in CI but produced byte-identical libopenvdb.dylib
in the wheel — strip+codesign on brew bottles appears to be a silent no-op,
likely due to hardened-runtime signature handling. Linux savings (~3 MB
compressed via vcpkg triplet -Wl,-s) are real and worth shipping; macOS
investigation moves to a separate branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants