Skip to content

fasl: record shen.*lambdatable* deltas by name instead of serializing live lambdas - #45

Open
pyrex41 wants to merge 1 commit into
mainfrom
fix/fasl-lambdatable
Open

fasl: record shen.*lambdatable* deltas by name instead of serializing live lambdas#45
pyrex41 wants to merge 1 commit into
mainfrom
fix/fasl-lambdatable

Conversation

@pyrex41

@pyrex41 pyrex41 commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Problem

On the S41.2 kernel, every (define ...) in a loaded file runs shen.update-lambdatable, which (set)s shen.*lambdatable* to an assoc list whose entries hold live curried lambda functions. The fasl recorder captured that as an ordinary g (set) record, kdata_ser hit the closure (unserializable KDATA value: function at boot.lua:254), and the whole file became fasl-uncacheable.

In practice that was ~every stdlib and user file (19 of 21 StLib files on a cold boot), so:

  • the ~1s stdlib load re-compiled from Shen sources on every boot, and
  • every user (load) paid full reader+macro+typecheck cost on every run.

This is the dominant share of the ~1.5s cold-start reported for a trivial script (vs ~13ms for shen-cl), and a fixed per-invocation tax on every suite run in batch runners like urdr's Bifrost gate.

Fix

Record a new lt record instead: diff the new table against the current global (the recorder runs before the original set, so the old table is still live) and store only the names whose entry was added/replaced. On replay, rebuild each entry from the live defun the same way shen.update-lambdatable does — shen.lambda-entry (which reads the arity property, already replayed by the preceding p record in stream order) + shen.assoc->. This mirrors the existing lf special case for the older kernel's shen.lambda-form put path.

FASL_FORMAT bumped to SHENFASL5 so stale caches miss cleanly.

Measured (arm64 macOS, LuaJIT 2.1 rolling, S41.2)

before after (warm)
trivial script startup ~1.5s 0.34s
StLib files fasl-cacheable 2/21 21/21

Verification

  • make test: 500 pass / 0 fail across 14 specs
  • run-kernel-tests.lua (canonical certification): 134 passed / 0 failed, ok
  • urdr shen/tests/prng suite: warm/cold stdout identical modulo the by-design run time banner drop; ALL PASS both ways

🤖 Generated with Claude Code

… live lambdas

On the S41.2 kernel every (define ...) in a loaded file runs
shen.update-lambdatable, which (set)s shen.*lambdatable* to an assoc
list whose entries hold live curried lambda functions. The fasl
recorder captured that as an ordinary "g" (set) record, kdata_ser hit
the closure ("unserializable KDATA value: function"), and the whole
file became fasl-uncacheable — in practice ~every stdlib and user file,
so the ~1s stdlib load re-compiled on every boot and every (load) paid
full reader+macro+typecheck cost on every run.

Record a new "lt" record instead: diff the new table against the
current global (the recorder runs before the original set, so the old
table is still live) and store only the NAMES whose entry was
added/replaced. On replay, rebuild each entry from the live defun the
same way shen.update-lambdatable does — shen.lambda-entry (which reads
the arity property, already replayed by the preceding "p" record in
stream order) + shen.assoc->. Mirrors the existing "lf" special case
for the older kernel's shen.lambda-form put path.

Bump FASL_FORMAT to SHENFASL5 so stale caches miss cleanly.

Measured (arm64 macOS, LuaJIT 2.1 rolling): trivial-script startup
1.5s -> 0.34s warm; all 21 StLib files now fasl-hit (previously 19/21
uncacheable). make test 500/0, run-kernel-tests 134/0, warm/cold suite
stdout identical modulo the by-design 'run time' banner drop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant