feat(toolchain): regenerate memcpy.s/memset.s from musl with a generator script

[shuklaayush]

Summary

• Replaces the glibc-derived memcpy.s / memset.s in crates/toolchain/openvm/src/ with musl-libc equivalents (MIT-licensed; matches the v2.0.0 lineage).
• Adds scripts/generate_libc_intrinsics.sh to regenerate both files deterministically from upstream sources.
• Fixes an LLVM jump-table label collision that prevented guest binaries with dense user code from linking under fat LTO.

Why

Two problems with the current state:

1. Licensing: the existing files were compiled from glibc (LGPL-2.1+) but checked in without a license header or attribution. musl-libc (MIT) is permissive and was the original source used in the v2.0.x branch (develop-v2.0.0-rc.1), so switching back is both more permissive and continuity-preserving.

2. Symbol collision under fat LTO: the glibc-derived memcpy.s defined an orphan jump table .LJTI0_0 (without a function prefix). LLVM emits the same name for the first jump table of "function 0" in any translation unit, so user code that happens to lower a match into a jump table — combined with lto = "fat" merging crates into a single LLVM module — causes:

   error: symbol '.LJTI0_0' is already defined
   

   The musl-derived equivalents have no jump table (at -O3 -funroll-loops, musl's memcpy unrolls fully), and the generator script prefixes every per-TU local label (.LBB0_*, .LJTI0_*, .Lfunc_end0) with the function name to make any future drift impossible.

What's in the PR

File	Purpose
scripts/generate_libc_intrinsics.sh	Fetches musl sources at a pinned ref, strips #include lines, compiles with the recipe clang -target riscv64 -march=rv64im -O3 -funroll-loops, applies local-label prefixing, strips .ident, normalizes .file, and embeds musl COPYRIGHT in the file header. Accepts --musl-ref <tag|branch|sha> and --clang <path> with sensible defaults. Idempotent.
crates/toolchain/openvm/src/memcpy.s	Regenerated from musl v1.2.6 (9fa28ece75d8a2191de7c5bb53bed224c5947417).
crates/toolchain/openvm/src/memset.s	Regenerated from musl v1.2.6 (9fa28ece75d8a2191de7c5bb53bed224c5947417).
crates/toolchain/openvm/README.md	Pointer to the generator.

Reproduce

scripts/generate_libc_intrinsics.sh
git diff --exit-code crates/toolchain/openvm/src/memcpy.s crates/toolchain/openvm/src/memset.s

Should produce no diff on a fresh checkout. The .ident directive is stripped during generation, so output bytes are independent of the clang distribution (Homebrew vs Debian vs Apple Clang); the only per-clang-version variation that remains is the .attribute 5 ISA string (e.g. "rv64im" on clang ≤14 vs the canonical "rv64i2p1_m2p0_zmmul1p0" on clang ≥16) — same ISA, different recording.

Test plan

• scripts/generate_libc_intrinsics.sh reruns produce byte-identical output (idempotent).
• Both regenerated files contain no orphan .LJTI0_* / .LBB0_* / .Lfunc_end0 labels — every local label is function-prefixed.
• CI on this PR (build, lint, riscv-test-vector, extension-tests).

🤖 Generated with Claude Code

[github-actions]

Claude finished @shuklaayush's task in 2m 18s —— View job

---

Code Review: PR #2780 — Regenerate memcpy.s/memset.s from musl

• Read changed files and PR diff
• Review scripts/generate_libc_intrinsics.sh
• Review generated memcpy.s and memset.s
• Review README.md changes
• Check for label collision fix correctness
• Post review summary

---

Well-motivated PR. The licensing fix (glibc LGPL → musl MIT) is important, the .LJTI0_0 fat-LTO collision is a real bug, and the generator script is well-engineered. Here's the detailed review:

Looks good

Label collision fix is correct. The old memcpy.s on develop-v2.1.0-rv64 had 3 bare .LJTI0_0 references. The new files have zero unprefixed .LBB0_*, .LJTI0_*, or .Lfunc_end0 labels — confirmed via grep. Every local label is function-prefixed (e.g. .LBBmemcpy0_3, .Lmemsetfunc_end0). The sed patterns at scripts/generate_libc_intrinsics.sh:165-167 correctly handle all three label families.

Annotated tag handling is correct. The ls-remote resolution (lines 72–88) now explicitly looks for peeled ^{} refs first, falling back to the first result only when no peeled ref exists. The earlier automated review's concern about this has been addressed in this version.

Reproducibility is solid. .ident directives are stripped (line 169), .file is normalized to musl_memcpy.c/musl_memset.c (line 168), and the temp dir is cleaned via trap (line 94). set -euo pipefail plus curl -fsSL for HTTP error handling.

License headers are complete. Both generated files embed the full musl MIT license text (COPYRIGHT file), properly attributed. The old files had no license header at all despite being derived from glibc (LGPL).

README is accurate. The --musl-ref and --clang flags match the actual script parameter names.

Nits / observations (non-blocking)

1. BB0_3 in LLVM debug comments (memcpy.s:235):

   # %bb.4:                                #   in Loop: Header=BB0_3 Depth=1
   

   The sed correctly targets only .LBB0_ (with the leading dot), so BB0_3 in #-prefixed comments is left alone. This is harmless — these are comments, not labels — but worth noting for anyone inspecting the output.

2. 32-bit word operations in memcpy. The musl memcpy uses lw/sw (32-bit) on the aligned fast path vs the old glibc version's 64-bit operations. This is inherent to musl's generic-C memcpy.c aligning to sizeof(unsigned int) == 4. For a zkVM where correctness and label safety matter more than raw throughput, this is a fine tradeoff. memset does correctly use sd (64-bit stores) in its bulk fill loop (lines 280–283).

3. .attribute 5 ISA string variation. The PR description correctly notes this is the only remaining per-clang-version variation — "rv64im" on clang ≤14 vs "rv64i2p1_m2p0_zmmul1p0" on clang ≥16. Same ISA, different encoding. Not worth stripping since it doesn't affect linking or correctness.

Summary

The core change is sound and the script is well-crafted. No blocking issues found.

LGTM.

---

[github-actions]

group	app.proof_time_ms	app.cycles	leaf.proof_time_ms
fibonacci	1,876	4,000,051	527
keccak	13,535	14,365,133	2,226
sha2_bench	9,484	11,167,961	1,410
regex	1,614	4,090,656	379
ecrecover	642	112,210	289
pairing	759	592,827	281
kitchen_sink	2,050	1,979,971	433

Note: cells_used metrics omitted because CUDA tracegen does not expose unpadded trace heights.

Commit: 9c8ce6e

Benchmark Workflow