Conversation
- BUCK files for external-deps (pcre autotools + double-conversion cmake
genrules, parson, esprima/escodegen/estraverse/esutils filegroups),
runtime (libecho, objc sources compiled via -x objective-c since the
system cxx toolchain has no objc support, atoms/webgl genrules, llc'd
invoke-closure-catch trampoline), ejs-llvm, node-compat, node-llvm
(prebuilt addon for now) and lib (babel'd stage0 compiler)
- //:srcdir-tree assembles a --srcdir-shaped tree from the above;
//:ejs.exe.stage{1,2,3} run the bootstrap stages against it
(stage1 = node-hosted stage0 compiler, ejs.exe aliases stage1)
- defs.bzl centralizes triples/platform defines/llvm location
([llvm] section in .buckconfig, defaults to homebrew llvm@16)
- prelude vendored as a submodule pinned to current buck2-prelude
- header include paths in the runtime remapped through exported_headers
so the repo-relative includes (external-deps/pcre/pcre.h etc) resolve
- gen-atoms output compiled as a standalone TU by prepending includes
in the genrule
- nan bumped to 2.28 so node-llvm builds against node 22
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- initTypes was called with triple.pointerSize() (64, truthy) as its is32bit flag, so every 64-bit target used the padded 32-bit EJSObject layout: all module export slot offsets were shifted 8 bytes relative to the C runtime and cross-module binding reads returned garbage - module slot refs now use a non-inbounds gep: imported modules are declared with the generic EJSModule type (exports[1]), so indexing slot > 0 inbounds is poison that llvm 16 optimizers exploit - closure conversion now receives the suffix-stripped module name, so compiling a file with exports as the main input no longer creates bindings keyed 'foo.js' when the module registry has 'foo' Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
api updates in the ejs-llvm and node-llvm bindings: - Intrinsic::getDeclaration -> getOrInsertDeclaration (renamed in 20) - IRBuilder::CreateGlobalStringPtr -> CreateGlobalString (removed in 20) - Type::getPointerTo -> PointerType::getUnqual (removed in 21; everything is an opaque ptr now anyway) - Module::setTargetTriple takes llvm::Triple (changed in 21) - lifetime intrinsics are size-less in 22 - APInt construction passes isSigned/implicitTrunc so negative and oversized js numbers keep their old truncating behavior instead of tripping the (new in 20) assertion .buckconfig llvm prefix now points at homebrew's current llvm keg and the stale -16.0.6 LLVM_SUFFIX default in mk/config.mk is gone (tools come from PATH). verified: node-hosted stage0 compiles the compiler to a working ejs.exe.stage1 via buck2, which compiles and links running programs, all against llvm 22 (llvm-as/opt/llc 22 + libLLVM 22). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- implement MARK_REGISTERS for arm64: spill x19-x28/fp and d8-d15 and scan them as roots. it was previously empty, so any ejsval held only in a callee-saved register during a collection was freed while live. - scan the stack (and generator stacks) for raw, untagged heap pointers too, not just tagged ejsvals: optimized code unboxes closure envs and objects once and keeps/spills the raw pointer. - accept interior pointers in find_page_and_cell (canonicalizing to the cell start before pushing on the worklist): optimized code keeps env slot addresses live with the env base dead. also fixes a page_index off-by-one that could read past page_infos. - zero cells at allocation and skip objects with NULL ops when scanning: any allocation between _ejs_gc_alloc and _ejs_init_object can collect, and recycled cells were full of 0xaf poison. - root _ejs_Iterator_prototype (the add_root call in _ejs_iterator_init_proto rooted _ejs_Generator_prototype instead -- which _ejs_generator_init roots anyway), and register roots for every builtin global ejsval in one place in _ejs_init: the data segment is not scanned, and relying on each *_init to root what it creates is how the iterator prototype ended up freed with everything created later still using it as [[Prototype]]. - skip not-yet-initialized static module objects in mark_from_modules. with these, programs compiled by ejs run to completion with a full collection forced on every allocation (EJS_GC_EVERY_N_ALLOC=1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
without a datalayout, opt folds struct GEPs using llvm's defaults, where i64 is only 4-byte aligned. that computes EJSModule.exports at offset 44 while clang lays it out at 48 for the runtime, so every module slot access was skewed 4 bytes relative to the C side -- the GC scanned the wrong words of each module global and freed objects that were only referenced from module slots. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
parseInt accumulated into a C int, so parseInt("ffffffff", 16) wrapped
to -1. esprima parses hex literals with parseInt, which meant the
self-hosted compiler read 0xffffffff literals in its own source as -1.
Constant.getIntegerValue's 64-bit form then cast that -1.0 to uint32_t,
which saturates to 0 on arm64 (vs wrapping on x86), so stage2 emitted
nanboxing masks of 0x7fff00000000 instead of 0x7fffffffffff and stage3
binaries crashed on their first closure env access. the bindings now
convert with wrapping (ToUint32) semantics as well.
with this the bootstrap reaches a fixed point: stage2 and stage3 are
byte-for-byte identical.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DesugarDestructuring ran before DesugarForOf and rewrote the loop's binding pattern into multiple declarators (of which DesugarForOf only kept the first, with garbage init), so `for (let [k, v] of m)` failed with "undeclared identifier". the destructuring pass now leaves for-of lefts alone, and a second DesugarDestructuring pass runs after DesugarForOf to desugar the `let <pattern> = %iter_next.value` declaration it emits. array, nested, and object patterns in for-of all work now. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
makecontext's variadic args are ints, so the EJSGenerator* was silently truncated to 32 bits and _ejs_generator_start crashed dereferencing it (heap pointers on arm64 macos don't fit). split the pointer across two int args posix-style and reassemble in a trampoline. also grow the generator stack from 64k to 512k; frames from compiled code are large. all 12 generator tests pass now; the full suite is 373 pass / 26 xfail / 0 fail against both stage1 and stage2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
buck2 build //:test-stage3 builds the whole bootstrap chain, assembles a repo-shaped tree (srcdir-tree + test/ + the stage exe + the stage0 compiler), and runs test/tester.js against it. the build fails if any test fails; the output artifact is the full test log. tester.js also learned about ejs.exe.stage3 (-s 3). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Makefile, mk/*, and the per-directory Makefiles (runtime, lib, ejs-llvm, node-compat, node-llvm, external-deps, test, samples, release, packaging) are replaced by the BUCK files: bootstrap stages, external deps, the babel'd stage0 compiler, and the test suite all build with buck2 (see the updated README). the one out-of-band step, building the node-llvm addon with node-gyp, moves from node-llvm/Makefile to node-llvm/build-addon.sh. standalone Makefiles that never used mk/ (test/v8, test/osx-test, test/ios-test-es6, test/mozilla-tests, test/require-test) are left alone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… skeleton
implements the first milestone of EIRProposal.md on the eir branch:
- lib/eir/ops.js: the opcode set with the effect table (the contract
shared by lowering, optimizations, and the abstract interpreter)
- lib/eir/ir.js: Module/Func/Block/Inst; SSA with basic-block arguments
(no phis); explicit per-edge argument lists on terminators
- lib/eir/builder.js: on-the-fly SSA construction (Braun/Buchwald/Hack
CC'13) adapted to block args: write/readVariable, block sealing,
recursive trivial-parameter removal
- lib/eir/printer.js: canonical textual form (deterministic numbering,
golden-testable)
- lib/eir/verifier.js: seal/terminator/edge-arity checks plus
def-dominates-use via Cooper-Harvey-Kennedy dominators
- lib/eir/lower.js: phase-2 skeleton AST->EIR lowering for a whitelisted
subset (literals, locals, globals, operators, member access, calls,
if/while/logical/conditional, return); everything else throws
LowerNotSupported for per-function fallback to the legacy path
- lib/eir/tests.js + //:test-eir: 12 unit tests covering SSA join/loop
parameter insertion and trivial-param removal, lowering shapes, and
verifier rejections
sample: `function g(n) { let i = 0; while (i < n) i = i + 1; return i; }`
lowers to a 4-block CFG whose loop header has exactly one block param
for i -- no allocas, no mem2reg.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
replaces new-cc for the EIR path with a fresh, self-contained scope analysis (lib/eir/scopes.js): every binding gets a unique id (shadowing never aliases SSA variables), references resolve through a side map, and captured bindings get env slots in their declaring function, with parent slots reserved only where a descendant actually reaches past a function. lowering (lib/eir/lower.js) now emits make_env/env_load/env_store/ make_closure directly, using the runtime calling convention (%env, %this, ...params): captured params are stored to the env at entry, function declarations are hoisted as closures, and outer-scope access follows parent slots -- skipping env-less intermediate functions entirely (their incoming env is forwarded to their children's closures). try/catch lowers to invoke-style instructions: inside a protected region any may-throw instruction terminates its block with explicit normal/unwind edges; catch blocks carry the caught exception as a special first parameter that unwind edges don't supply. a nice property falls out of Braun SSA + block args: a variable read in the catch block gets per-throw-site values joined through catch block params, one per unwind edge. also: for/do-while/break/continue/new/this/array/object literals, and eager lowering of child functions so hoisted-but-unreached declarations still get bodies. //:test-eir grows to 25 tests, all green. EIRProposal.md gains a section on block-argument-driven specialization (basic block versioning): analysis at block granularity with types supplied through block arguments, versions cloned per argument-type tuple. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
the emitter (lib/eir/emit.js) maps every EIR value to an LLVM SSA value: block arguments become phis, invoke-style instructions become llvm invokes landing in the catch block's landingpad (begin/end_catch fetch the thrown ejsval), and the only allocas are the outgoing-args scratch area and the &this slot the runtime calling convention wants -- locals never touch memory, so there is nothing for mem2reg to do. it borrows the active LLVMIRVisitor's module/abi/runtime interfaces and atom machinery, so EIR and legacy functions coexist in one compilation unit. --ir (ejs-es6.js flag) turns on candidate collection (eir/integrate.js): v1 candidates are closed top-level function declarations -- every free name must be a real global, since module-scope interop comes later. candidates lower+verify before the legacy passes run; the legacy visitFunction then emits a forwarding thunk to the EIR-emitted function instead of a body. anything unsupported falls back per function via LowerNotSupported. direct recursion is the first language-aware win: a function's self-name binds as a "self" binding and self-calls compile to direct llvm calls (no closure dispatch), forwarding the incoming env. fixes along the way: - node-llvm: CreatePhi now returns the PHINode wrapper (addIncoming); PHINode::Create was a declared-but-undefined shadow of the template's, which -undefined dynamic_lookup turned into a jump to address 0 - scope analysis: var hoists to the function scope (it was block-scoped, breaking v8 function-override semantics); anon function names are uniquified; default/rest params, generators, duplicate and block-level function declarations fall back - the llvm entry block stays terminator-free until the end of emission (legacy cached-literal helpers append initializing stores to it), with a separate prologue block for the guarded argc/args parameter loads - eir sources restyled to avoid a legacy DesugarTemplates miscompile (template-in-arrow-in-template), so stage1 can self-host them validation: //:test-stage0-ir runs the full suite with --ir: 373 pass / 26 xfail / 0 fail. 69/125 top-level functions in the corpus (55%) lower to verified EIR and 62 test files exercise the EIR codegen path; the rest fall back cleanly. //:test-eir, //:test-stage0, //:test-stage1 all green; stage2/stage3 fixed point still holds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
candidates no longer need to be closed:
- named imports from non-native modules lower to module_slot_load
(emitted as the same non-inbounds GEP into the imported module global
the legacy opencoded path uses); const exports fold to their literal,
matching new-cc's constval propagation
- calls to sibling top-level EIR functions compile as direct calls into
the shared per-file EIR module -- no closure dispatch, mutual
recursion included. viability is a fixed point: siblings that fall
back drag their callers with them, value-uses of a sibling or
module-scope reassignment disqualify, and a late lowering failure
abandons the whole file's EIR set before any AST node is tagged
- module-level vars and default/namespace imports still fall back
self-hosted --ir: ejs-llvm gains a PhiNode class (createPhi previously
EJS_NOT_IMPLEMENTED) with addIncoming, mirroring the node-llvm fix, so
the compiled compiler can emit block-arg phis.
two latent legacy-pipeline bugs surfaced by self-hosting the eir
sources, worked around here and left for a proper fix:
- re-exporting an imported binding (export { X } where X is an import)
corrupts the module slot: calls through it die with IsConstructor /
not-a-function asserts. LowerNotSupported is now a plain factory +
marker property (isLowerNotSupported) instead of an Error subclass,
and the re-export in lower.js is gone
- (previously) template-in-arrow-in-template miscompiles
also: the stale make-era ejs-llvm/ejs-llvm-atoms-gen.c shadowed buck's
generated copy via the quoted include (removed; that plus buck2's
content-hash invalidation explained a very confusing afternoon)
validation: //:test-{eir,stage0,stage0-ir,stage1,stage1-ir} all green
(373 pass / 26 xfail each); ~half the corpus's top-level functions and
56 test files exercise EIR codegen, now including sibling direct calls;
stage2/stage3 fixed point holds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- DesugarTemplates never visited a template's substitution expressions
(nor a tagged template's tag), so a template literal nested inside
another template's ${} -- e.g. through an arrow in a .map call --
survived the pass undesugared and crashed codegen
- new-cc registered a module-slot binding for every %moduleSetSlot at
the toplevel, so a re-export (`export { X }` where X is an import)
shadowed the imported binding: every reference to X, including the
setSlot's own right-hand side, read this module's uninitialized slot
instead of the import. the binding is now only registered when the
name isn't already bound to another module's slot
regression tests: template-nested1, reexport1 (with two helper modules
exercising import -> re-export -> use-in-callee).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- self-module exported bindings resolve through module slots on a "%self"
module global: reads via module_slot_load, writes via the new
module_slot_store op (exported consts with literal initializers fold);
- non-exported module-level bindings with primitive literal initializers
that are never reassigned fold to their literal;
- candidates now include exported function declarations and top-level
single-declarator `var f = function () {}` initializers (the var name
doubles as the self name, so anonymous fn-exprs self-recurse directly);
- mod_ctx.imports generalized to mod_ctx.refs ({module, slot, constval?,
writable}); scope analysis tracks free assigned names so writes to
read-only refs fall back at analysis time;
- more early fallback guards (compound assignment, value-position self
references, regex/object literals incl. constval folds) so lowering
can't fail late and abandon a whole file's EIR set.
self-hosted --ir coverage: 50 functions (was 41), 0 late failures.
//:test-{eir,stage0,stage0-ir,stage1,stage1-ir,stage3} green
(375 pass / 26 xfail); stage2/stage3 fixed point holds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…efaults
Syntax expansion aimed at the biggest --ir fallback buckets (which
cascade: every fallen-back module function poisons its callers'
viability):
- update expressions (++/--, pre/post, identifier and member targets;
old value numified via unary_plus);
- compound assignment (+=, -=, ... via a shared operator table; object
and computed key evaluated exactly once);
- untagged template literals (string_concat/ToString chains through the
new call_runtime emit case, matching the legacy inlined default
handler);
- switch statements (document-order strict_eq test chain, fallthrough
bodies, shared case scope, separate break-only target stack so
continue passes through to the enclosing loop);
- for-of (Symbol.iterator/next/done/value protocol, same expansion as
the legacy DesugarForOf pass);
- arrow functions lowered as ordinary closures, with a lexical-`this`
fallback guard; expression bodies supported; `var f = x => ...` is a
module candidate (the var name doubles as the self name);
- default parameters (undefined-check prologue, SSA/env merge);
- fallback guard for closures capturing let/const loop variables, which
would need per-iteration envs EIR doesn't build yet (the legacy
DesugarLetLoopVars handles those correctly).
emit: blocks now emitted in reverse postorder — creation order isn't
dominance-compatible (switch bodies are created before their test
chain), so defs could be emitted after their uses; unreachable blocks
are dropped. cond_br conditions go through to_boolean (it's the only
i1 producer).
Also fixes a latent legacy-pipeline bug found while testing:
DesugarUpdateAssignments kept only the first character of the operator,
so <<=, >>= and >>>= compiled to <, > and > (booleans). shiftassign1
is the regression test; eir-syntax1 covers the new EIR syntax.
self-hosted --ir coverage: 119 functions (was 50), 0 late failures;
self-compile wall time unchanged (68.9s vs 70.6s legacy).
//:test-{eir,stage0,stage0-ir,stage1,stage1-ir,stage3} green
(377 pass / 26 xfail); stage2/stage3 fixed point holds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- for-in via new prop_iter_new/next/current ops wrapping the runtime
property iterator (same expansion as the legacy visitForIn). the
iterator value is opaque (EjsPropIterator, not an ejsval) and stays a
direct instruction reference; prop_iter_next returns the runtime's i8
bool, which the cond_br emit case coerces to i1;
- rest parameters: a trailing RestElement (or legacy fnNode.rest)
declares an ordinary local filled by the new rest_args op — a
branch-free select over argc feeding array_new_copy;
- regex literals lower to make_regexp (fresh RegExp per evaluation via
regexp_new_utf8, matching the legacy visitLiteral);
- exported function/class declarations join mod_ctx.refs, so sibling
functions used as *values* (arr.map(inc), twice(inc, x)) load the
module slot the legacy toplevel stored the closure in — identity-
correct, unlike minting a fresh closure per reference. calls still
prefer the direct-call path.
Also fixes a latent RUNTIME bug this work tripped over: the dense-array
fast path of Array.prototype.slice didn't normalize negative indices
(slice(0, -1) computed a negative count and segfaulted in memmove) and
treated an explicit undefined end as NaN. Fixed per ES6 22.1.3.22;
test/slice-negative1.js is the regression test, eir-syntax2 covers the
new EIR syntax.
self-hosted --ir coverage: 136 functions (was 119), 0 late failures.
//:test-{eir,stage0,stage0-ir,stage1,stage1-ir,stage3} green
(379 pass / 26 xfail); stage2/stage3 byte-identical modulo the
embedded output filename.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rap CI
Namespace imports (`import * as ns`) become refs bound to the module
object:
- ns.member on a JS module resolves to a module_slot_load (const
exports fold) at compile time, mirroring new-cc's visitMemberExpression
rewrite — JS module objects don't answer runtime property lookups for
their exports. ns.member(...) calls pass `this` = undefined, exactly
as the legacy rewrite does;
- native modules ("@llvm") have no link-time global, so their namespace
object is fetched at runtime through module_get (the same fallback the
legacy handleModuleGetExotic uses) and member access stays a runtime
property get;
- default imports resolve through the "default" export slot.
Fixes a serious scope-analysis bug the new coverage exposed: bindings
were declared AFTER their initializer was walked, so a closure created
inside the initializer (`let walk = (n) => { ... walk(el) ... }` —
integrate.js's own collectAssignedNames) resolved its recursive
reference to a GLOBAL, and lowering emitted get_global -> undefined ->
"object not a function" at runtime. Bindings are now declared before
the init walk, and lowering pre-initializes them to undefined so a
direct `let x = x` still reads undefined (matching legacy alloca
semantics). eir-recarrow1 is the regression test.
The bug only bit in the --ir-compiled compiler and sailed through the
whole test matrix: the stage2/stage3 fixed point builds WITHOUT --ir,
so nothing exercised --ir-generated code generating code. New targets
close that hole: //:ejs.exe.stage2-ir (stage1 self-compiles with --ir)
and //:test-bootstrap-ir (that binary must pass the full suite with
--ir). buck-stage.sh takes extra compiler flags to support it.
Also adds --ir-exclude / --ir-exclude-fn debugging flags (file/function
substring filters), which turned the bootstrap miscompile hunt into a
few binary-search self-compiles.
self-hosted --ir coverage: 173 functions (was 136), 0 late failures.
//:test-{eir,stage0,stage0-ir,stage1,stage1-ir,bootstrap-ir,stage3}
green (381 pass / 26 xfail); stage2/stage3 fixed point holds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…wallow
Slot promotion — the biggest --ir fallback bucket was functions
referencing non-exported mutable module state (esprima's lookahead/
state/index, escodegen's json/renumber, ...):
- gather-imports assigns non-exported module-level vars hidden slots on
the same counter as exports, so module-global sizing and GC scanning
need no changes (num_exports is really num_slots). const-literal
declarations stay plain locals (they constant-fold); names with a
nested `var` re-declaration are excluded;
- DesugarImportExport rewrites promoted declarations to %moduleSetSlot,
and new-cc's existing ModuleSlotBinding registration routes every
legacy reference through the slot;
- promoted entries are private: import resolution, re-export, new-cc's
namespace-member rewrite, module-object accessors, and EIR's
ns.member/named-import paths all skip them;
- EIR reads/writes the same slots via module_slot_load/store on "%self"
(const-declared ones are read-only), so mutable module state works
from both pipelines against one storage. non-exported `var f =
function(){}` siblings used as values now also resolve (the slot
holds the one closure, identity-correct);
- EJS_NO_PROMOTE=<substrs> env hook disables promotion per module path
(this bisected every bug below).
Fixes three latent legacy bugs the promotion exposed, one of them big:
1. try/finally SWALLOWED IN-FLIGHT EXCEPTIONS: the finally-only
landingpad branched into the finalizer with a stale cleanup_reason,
so after the finalizer ran, execution fell through to try_merge and
simply continued. Since DesugarLetLoopVars wraps every `for (let
...)` body in try/finally, every for-let loop silently ate
exceptions thrown through it. The landingpad now saves the caught
value and a REASON_EXCEPTION, and the finalizer's dispatch rethrows
via _ejs_throw (a fresh throw: _ejs_rethrow needs an active
exception). test/finallythrow1.js covers throw/nested/break/return
paths.
2. new-cc visitAssignmentExpression double-visited the rhs for module
bindings, double-wrapping closures (%makeClosureNoEnv twice) and
resetting the inner function's scratch_size (crashed on escodegen's
isArray).
3. storeValueInDest didn't handle %moduleGetSlot as a store target
(for-in/for-of loop variables bound to module slots).
EIR additions along the way:
- delete of member expressions (delete_prop -> _ejs_op_delete),
recovering all of esprima;
- hoisted-var semantics: locals are initialized undefined at function
entry, so reads before the declaration statement work;
- reads in unreachable code (after `while (true)`, after a switch whose
cases all return) resolve to undefined instead of throwing.
self-hosted --ir coverage: 286/422 functions (was 173), 0 late
failures; the --ir-built compiler passes the bootstrap smokes.
//:test-{eir,stage0,stage0-ir,stage1,stage1-ir,bootstrap-ir,stage3}
green (383 pass / 26 xfail); stage2/stage3 fixed point holds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Function declarations now promote to module slots exactly like exported
functions (the declaration becomes %moduleSetSlot at its source
position — same hoisting caveat as exports). This breaks the viability
poisoning cascade: a reference to a fallen-back sibling is now just a
slot load of its (thunk) closure, so one unsupported function no longer
drags every transitive caller down with it. `new Node()`-style
constructor uses and functions-as-values resolve the same way.
Self-references in value position or from nested functions are treated
as free module-scope names instead of hard fallbacks — integration
resolves them through the candidate's own slot.
self-hosted --ir coverage: 405/422 functions, 96% (was 286), 0 late
failures. The 17 survivors are genuine feature tails: spread (4),
`arguments` (3), try/finally (2), ObjectPattern decls (2), per-
iteration loop envs (2), and 4 class-declaration references.
//:test-{eir,stage0,stage0-ir,stage1,stage1-ir,bootstrap-ir,stage3}
green (383 pass / 26 xfail); stage2/stage3 fixed point holds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- class declarations promote to module slots like functions (classes
don't hoist, so the %moduleSetSlot rewrite at the source position is
exactly their declaration semantics);
- `arguments` binds to a synthetic per-function local filled by the new
args_obj op (_ejs_arguments_new on the raw argc/args) in the
prologue; arrows resolve to the nearest non-arrow function's binding
and capture it through the env chain like any local;
- shallow object-pattern declarations (`let { a, b: c, d = dflt } = e`)
lower as get_prop_atom chains with SSA default merges. NOTE: EIR
supports pattern defaults but the legacy DesugarDestructuring PANICS
on AssignmentPattern — suite tests stick to the shared subset;
- ast-builder now exports AssignmentPattern (it was never defined, so
`b.AssignmentPattern` comparisons silently never matched).
Also fixes another latent legacy bug: `delete o[k]` (computed key)
read `.property.name` — undefined for anything computed — and deleted
the property literally named "undefined". The key expression is now
evaluated. eir-syntax4 covers arguments/patterns/delete/unreachable-
code shapes against node output.
self-hosted --ir coverage: 413/422 functions (98%), 0 late failures.
The 9 survivors: spread calls (4), per-iteration loop envs (3),
try/finally (2).
//:test-{eir,stage0,stage0-ir,stage1,stage1-ir,bootstrap-ir,stage3}
green (384 pass / 26 xfail); stage2/stage3 fixed point holds
(byte-identical modulo embedded filename).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The keystone gap for retiring the legacy pipeline: try/finally lowers
natively. One finalizer copy on the normal completion path, one in a
synthetic catch block that rethrows, and a copy at every abrupt exit
that crosses the finally boundary (return, break, continue) — tracked
by a finally-context stack recording break/continue/handler depths at
entry. Each exit-site copy runs with the crossed contexts and their
unwind handlers removed, so:
- a return/break inside a finalizer overrides in-flight control
transfer, per spec (the copy just terminates the block first);
- an exception during a finalizer copy propagates without re-running
that finalizer;
- nested finallys run innermost-out.
try now accepts catch and/or finally in any combination (finally-only
included). esprima's parse/tokenize — the last try/finally holdouts —
lower.
self-hosted --ir coverage: 415/422, 0 late failures. Remaining 7:
spread calls (4) and per-iteration loop envs (3).
//:test-{eir,stage0,stage0-ir,stage1,stage1-ir,bootstrap-ir,stage3}
green (385 pass / 26 xfail); stage2/stage3 fixed point holds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The general %-intrinsic mechanism: pre-EIR desugar passes rewrite
constructs lowering has no native form for into %-intrinsic calls, and
EIR lowers those through the table in lib/eir/intrinsics.js (either a
dedicated op or a call_runtime mapping). scopes.js consults the same
table to reject unknown intrinsics early, keeping the late-failure
discipline intact.
DesugarSpread is the first hoisted pass (new preEIRConvert hook, run in
compile() before collectEIRFunctions on both pipelines). It now skips
super(...) calls — it runs before DesugarClasses — and stays in the
legacy list as a post-classes cleanup run for the spreads that survive
super rewriting. %arrayFromSpread lowers to the new array_from_spread
op (scratch-spilled call to _ejs_array_from_iterables).
Two latent legacy bugs fixed on the way:
- node-visitor didn't dispatch ImportDefaultSpecifier /
ImportNamespaceSpecifier, so any generic traversal of a
not-yet-desugared import panicked (nothing traversed one before).
- DesugarSpread's %arrayFromSpread "flattening" discarded its
Array.concat result, silently dropping arguments in calls like
f(...a, [1, ...b]). The special case was wrong anyway (a nested
spread array is a single value argument); it's gone.
Self-hosted --ir: 421/424 lowered, 0 late failures. The 4 spread
fallbacks are gone; the 3 survivors are per-iteration loop envs.
Validated: test-eir (3 new unit tests), test-stage{0,1}{,-ir},
test-bootstrap-ir, test-stage3, test/eir-spread1.js.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…length sinking plans.md P5.3. Three pieces (docs/sinking-plan.md design + results): - lib/eir/sink-flow.ts: flow-sensitive scalar replacement of written object candidates (fold-false guard resolution via the twin-arm argument; Braun-style per-field renaming with minted boxed join params) and partial-escape materialization at a single escape site. All-or-nothing per candidate, plan-before-mutate, FLOW_REGION_CAP bounds the value-lifetime cost. EJS_NO_FLOW_SINK bisects. - arg_len op + _ejs_arg_length runtime helper: rest_args/args_obj used only for .length fold away (arguments is an unmapped argv snapshot; length is synthesized from argc). arg_load recorded-declined: OOB index reads walk the prototype chain, which the accessor epoch does not cover for writable integer data properties. EJS_NO_ARGS_SINK. - optimizer hygiene forced by the self-compile: one scan per round (scanRound gathers the use map + all sink candidates), bisect flags read once per optimizeFunction (process.env is a rebuild-the-environment getter under the self-hosted runtime), and an LOS bounds prefilter in ejs-gc.c — a partial mitigation of the conservative pin-scan cliff root-caused along the way and recorded as gc-P4's first order of business in gc-plan.md. Gates: 205 EIR unit tests; probes types-flowsink1/types-argsink1 node-identical (--types, flag-off, EJS_SHAPES=off, gc-stress, bisect compiles); --types diff lane 474 files 0-divergent (x2); matrix x7 green (stages at 421 pass / 22 xfail); types-bench4 0.04s vs 0.15s A/B (3.75x, node 0.20s); types-bench2 unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…illed The pin-scan cliff (first order of business): all arenas now carve out of ONE PROT_NONE reservation committed 32MB at a time, so the conservative span is fixed and disjoint from the C/LLVM heap forever and arena lookup is a shift into a direct map instead of a per-word bsearch; LOS candidates resolve by binary search over a sorted range array instead of a locked linear walk. A/B on the same tree: the baseline binary sat in the deep slow mode (~10 min per self-compile, ~95% of samples in mark_ejsvals_in_range->find_page_and_cell); the fixed binary does 56-64s across 5 runs — the mmap-layout lottery is gone. Minor pause max 10.9ms; GC ~10% of self-compile wall. Compaction: full-GC conservative hits and registered generators set PINNED; after the sweep, the sparsest pages' unpinned live cells evacuate into denser pages (source set chosen COMPLETELY before any evacuation — one-pass selection let an early destination later become a source via its stale live count), every reference rewrites through the P1 forwarding records (roots incl. shape names, modules, gc-frame chains, remset, all live cells + primstr raw children), and emptied pages return to their arenas. Frag bench: 37.6MB -> 8.3MB (4.5x), idempotent; real self-compile full GCs free ~5k pages each; stress differentials (nursery-off, collect-every-997) byte-identical; EJS_GC_COMPACT=off for A/B. GC.heapSize() added for the gate. Growth target: full_gc_trigger() = EJS_GC_GROWTH% (default 50) of the post-sweep footprint, two-arena floor — replaces the duplicated 60MB constant; with compaction the cadence adapts in both directions. Knob census = 1. Drive-bys: release_to_los leaked the tail page of every freed large object; a young survivor page emptied by a FULL sweep corrupted both page lists (detached from the wrong list head — young_page_freed). Matrix x7 green; docs/gc-p4-results.md has the numbers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ce, slot CSE, devirt The last P5 phase. New passes (lib/eir/cleanup.ts, lib/eir/devirt.ts): - trust-free type lattice over boxed values (const kinds, fixed-result ops, block-param meets to fixpoint) — sound with no oracle, so it fires on flag-off compiles; - cleanup fixpoint (runs after the region passes, foldUnboxOfBox's reason): host-evaluated primitive const folding (string-minting, string relationals, and -0 equality excluded — each argued from a known host/runtime divergence), runtime-mapping typeof folds, typeof x === "T" -> typeof_is (op existed, never minted; emit case added), known-truthiness/never-shaped branch folding, logical_not branch inversion, trivial block-param pruning, and lattice-typed f64 lowering: generic arith over proven numbers computes unboxed with NO guard — a flag-off 50M-iter loop kernel drops 1.3s -> 0.09s (~14x, 2.8x faster than node); - module-slot load CSE (before the region passes): block-local availability + store-to-load forwarding, and a stable-slot tier (single store in the run-once toplevel entry); toplevel receivers stop reloading per access, so shape regions merge at toplevel (the shapes-P3 note); - direct-call devirtualization beyond the self binding: SSA-visible closures and stable %self slots go direct (1,483 sites across the compiler self-compile; esprima alone 878); class-ctor-marked closures and env-using callees decline, fail closed. Pre-existing bugs the new passes flushed out (all fixed): - Map.prototype.delete was an unimplemented 2015 stub (runtime); first compiler-side caller was the CSE kill set — pinned by map6.js; - ejs-llvm had no FP IRBuilder bindings beyond createFAdd (flag-off compiles never emitted f64 before): FSub/FMul/FDiv/FCmpOLT added; - the emitter's double-const cache collided -0 with +0, and the fix's guard itself had to dodge the strict_eq -0===0 tag-compare quirk to survive self-hosting (1/n === -Infinity form); - generator suspension breaks slot stability mid-activation: suspendable functions decline the CSE exemptions. Gates: test-eir (17 new tests) + lowtier green; stages 0-3 + shapes-off green (stage2/3 fixed point holds); --types diff lane 474 files 0-divergent 0-compile-fail; types-bench2 0.21s vs 0.26s off; self-hosted self-compile 57.9s (inside the gc-P4 band). EJS_NO_EIR_CLEANUP / EJS_NO_SLOT_CSE / EJS_NO_DEVIRT bisect. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-O suites + -f/-fno- per-pass flags replace the EJS_* env-var surface (env reverts to debugging-only); pass registry, migration path, and open questions recorded in compiler-plan.md. Planned only — no implementation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, object-owner barrier, per-shape trace masks The Step B core, with the design addendum in gc-plan.md. Shaped slot storage stays a closureenv-shaped region behind the obj->slots ejsval, but born-with-shape allocation now places it INSIDE the object's own cell (32B object + 16B embedded env header + slots, one GC cell) — compiled slot addressing, has_shape guards, and the verifier contract are untouched; embedded-ness is pointer identity, no new header bit. - _ejs_object_new_shaped derives the true shape FIRST (pure, memo'd), then births object + storage as one cell; off-script falls back to the old two-cell / sequential path byte-for-byte. - Constructor results get there via a birth-capacity hint on EJSFunction (one-shot feedback: first construct's field count sizes every later 'this'); works flag-off, no compiler plumbing. - Barrier owner flip: shaped-slot stores remember the wrapper OBJECT (C sites + emitted slot_store); the ordinary Scan walks slot values directly in both modes (embedded storage has no cell of its own), scanning the env edge only when out-of-line. Owners are always cell heads — no interior-pointer remset entries. - Evacuation: the embedded slots ejsval joins minor_fixup_evacuated's self-interior-pointer cases (flat strings, EJSArguments) and is never presented to the precise slot callbacks. - Per-shape trace masks: EJSShape grows f64_mask (built incrementally at intern); the shaped Scan skips f64 slots — precise trace elision. (Emitted f64 slot_store already skipped the barrier; the runtime filter self-elides on numbers.) - Embeds cap at 10 fields (EJS_SHAPE_EMBED_FIELD_MAX) until the 256-byte size class is enabled — 11..14-field cells would round to 256 and take today's LOS routing. Measured: types-bench2 flag-off 2.50->2.37s, object+env cell count halved (8M->4M), requested bytes 305->244MB; --types bench2 is 0.21s on BOTH baseline and this change — the ctor-sinking phases already virtualized the alloc loop, so gc-P5's payoff is flag-off code, self-compile, and footprint. Self-compile (stage2 action) 60.4s vs gc-P4's recorded 62-64s. Gates: matrix x7 green (test-eir, lowtier, stage0-3 incl. byte-identity, shapes-off); embedded-slot stress probe (growth past capacity, ctor hints, dict migration out of embedded, repr flips, old->young slot traffic, enumeration) node-identical under EJS_GC_EVERY_N_ALLOC=7/31/101, PARANOID, NURSERY=off, SHAPES=off; typed-slots/bornshape/poly probes green under the same matrix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ze class; phase closed - lower.ts drops the oracle gate on make_object_shaped literals: key order/count are the site's static truth; without the oracle the static reprs are all-boxed and the runtime's birth derivation supplies true ones. Flag-off literals now allocate single-cell and join the shaped-literal sinking. Semantically identical by construction (make_object was object_create + per-key setprop — exactly new_shaped's screens and fallback). - The 256-byte size class is enabled: ffs(256)=9 > HIGH_LIMIT_BITS had routed 256B cells to the LOS since the beginning; the nursery seam, bump arrays, and emitter idx mapping were already built for 5 classes. HEAP_PAGELISTS_COUNT +1 (indexing documented), three ffs thresholds +1, inline-env emitter cap 128->256. Every cap-14 shape is now single-cell (EJS_SHAPE_EMBED_FIELD_MAX = FIELD_CAP_MAX) and 15..30-slot envs take pages, not the LOS. - gc-p5-results.md records the phase: litbench (escaping-literal loop) 1.56->0.85s (1.84x) flag-off; bench2 flag-off cells 8M->4M, 2.50->2.37s; LOS allocs on a compile workload 85.3k->26.3k (-69%); self-compile parity (60-67s vs 62-64 recorded). The headline correction: shapes-P5's "1.71s bench2 alloc-loop residual" was consumed by the ctor-sinking phases (--types bench2 is 0.21s at phase entry already) — gc-P5's payoff is flag-off code, literal loops, footprint, and LOS pressure. Emitted bump allocation for literals measured at ~6% of an alloc-heavy loop and deferred on that evidence, with the repr-divergence design note recorded. - Gates: matrix x7 green; embedded-slot stress probe + typed-slot/ bornshape/poly probes green under EVERY_N_ALLOC 7/31/101, PARANOID, NURSERY=off, SHAPES=off; --types diff lane 475 files 0-divergent. gc-P5 ticked in gc-plan.md; P6.2 ticked in plans.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Growth past embedded capacity, ctor birth-capacity hints (incl. under-sized), dictionary migration out of embedded storage, repr flips, old->young stores through existing slots, enumeration order. Ran node-identical under EVERY_N_ALLOC 7/31/101, PARANOID, NURSERY=off, SHAPES=off during the phase; lives in the suite as the regression pin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… mark epochs The color macros (SET_*/IS_* on bitmap bytes) and the white/black mask pair become one cell-lifecycle section: every state predicate and transition is an inline function, the encoding private to the block. The mask flip at the end of a full collection is now an explicit mark epoch — white/black are epoch-relative parities and mark_epoch_advance() (single site) ages black into white in O(1). The dead CONCURRENT CAS variants of the color macros go with it. Behavior-preserving: bitmap encodings and transition ordering are unchanged. test-stage0 green; the gc stress lane (EVERY_N_ALLOC 7/31/101 x PARANOID/VERIFY/NURSERY=off/COMPACT=off over the gc tests) matches the baseline failure set exactly (the pre-existing pinned generator-stress bugs, runtime-P1's burn-down list). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…function The root set becomes a real registry: a growable array (O(1) add, swap-with-last remove) with ONE iteration helper that full-GC mark, minor evacuation, compaction fixup, and the debug walks all share — five hand-rolled linked-list walks collapse into root_registry_foreach callbacks. Every collection the runtime initiates for itself now goes through gc_policy(event): the growth trigger on the old-allocation path, the post-minor promotion check, the EVERY_N_ALLOC stress cadences (minor in nursery mode, full in old mode), and the forced allocation-failure collections. Each event preserves its historical baseline/counter resets exactly, so collection schedules are unchanged. test-stage0 green; gc stress lane identical to baseline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s flushed ejs-gc.c (~3.7k lines) becomes six modules plus the internal contract header ejs-gc-internal.h (module map lives there): ejs-gc.c keeps the lifecycle API / allocator entry / cell free path / root registry / collection policy / GC JS object; ejs-gc-heap.c owns arenas, pages, the LOS and find_page_and_cell; ejs-gc-mark.c the worklist and the precise + conservative scanners; ejs-gc-minor.c the nursery; ejs-gc-major.c sweep + compaction + the epoch advance; ejs-gc-debug.c the PROFILE/WATCH/VERIFY/PARANOID machinery. Verified mechanically: every function body extracted from the old file diffs identical (modulo static) against the new tree — the exceptions are _ejs_gc_collect_inner (now calls root_registry_shutdown) and dead page_list_count (dropped). The duplicated tentative definition of heap_size_at_last_gc collapses to one. The TU split shifts codegen, and the stress lanes immediately caught two hazards that ACCIDENTAL conservative pins of stale stack copies had been masking (same lesson as gc-P4's bistable pin scan): - Orphaned old slot storage: growing (or dictionary-migrating) a shaped object's out-of-line slot array disconnects an OLD env cell that still holds its pre-copy slot values; the old-gen walkers (the minor's remset-overflow fallback, EJS_GC_VERIFY, EJS_GC_PARANOID) cannot tell that garbage from live cells and visit the stale slots after the referents move or die. Observed: the promoted env of the still-young rooted Reflect object, orphaned during _ejs_init, whose slot 7 aborted EJS_GC_VERIFY. Fix: shaped_retire_slots queues the retiree for one precise scan at retirement; the next minor rewrites its young refs while still live, leaving the cell inert until swept. - Paranoid checker self-scan: the dying-young-referrer report's raw C-stack sweep read the COLLECTOR's own frames (written after the conservative pin scan) and found the sweep loop's spilled cell cursor. The sweep now floors at the minor's entry frame (paranoid_stack_floor). Gates: test-eir-lowtier + stage0-3 (incl. the stage2/stage3 byte-identity fixed point) + stage1-shapes-off green; the gc stress lane (EVERY_N_ALLOC 7/31/101 x PARANOID/VERIFY/NURSERY=off/ COMPACT=off) matches the phase-entry baseline failure set exactly. test-eir was found red at phase ENTRY (11 pre-existing compiler-side failures from gc-P5's flag-off born-shaped literals; lib/ untouched here) — recorded as compiler-P1.1, not masked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
docs/runtime-p4-results.md records the phase: the cell-lifecycle block + explicit epochs, the root registry + single policy function, the six-file split, the two flushed stack-luck hazards, and the gate results. plans.md marks P6.3 (and with it the P6 milestone) done; runtime-plan.md closes runtime-P4; compiler-plan.md pins the pre-existing test-eir debt found at phase entry as compiler-P1.1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
typeof null is "object" (runtime + fold + typeof_is helpers, which also stop admitting functions as "object"); -0 === 0 (numbers compare before the NaN-box tag in strict/loose eq, SameValue — Object.is(0,0) was false — and SameValueZero); Math.round ties toward +inf; ToNumber's string path is a real StringToNumber (ES whitespace trim, "Infinity" only, 0x/0b/0o); the shift family and ToUint32 lose their UB double->unsigned casts; ToNumber(null)=0 and add tests the ToPrimitive results for stringness; mult/div/mod coerce both sides; uncaught generator-body throws propagate to the caller (invoke_closure_catch at the body boundary, resume sites rethrow on the caller's stack); sparse arrays get real element storage (aligned 512-slot arraylets); getOwnPropertyNames includes non-enumerables, ToObject-coerces, and reports index properties + length. The stress sweep flushed a latent hazard the generator fix made reachable: C-side catchers (_ejs_invoke_closure_catch/_func_catch) left the gc-frame chain head pointing at unwound emitted frames; C wrappers now restore the saved head on catch, closing the same hole for promises/Map/Array.from/iterator helpers. console.log now prints -0 and quotes strings nested in arrays (node inspect parity). Un-pinned: typeof1, math2, sparsearray1, proxy6. Matrix green (424/20/0 per stage lane); stress lanes at the phase-entry baseline; test-eir at exactly the 11 compiler-P1.1 items. docs/runtime-p1-results.md is the record; runtime-plan/plans ticked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Escaping entry points (module exports, closures passed as arguments) now get a typed fast path. The wrapper: a has_tag(number) guard per formal spliced into the generic entry, dispatching via call_typed to a new UNTRUSTED clone flavor (SpecMode.trusted=false) — f64 formals boxed once at entry, ordinary guarded diamonds inside (assume-and- guard gate: only a positive non-number claim declines), boxed result. The entry boxes are structural number proofs, so the formal-rooted diamonds fold trust-free to trusted-clone quality: types-bench5 (exported kernel, every call cross-module) runs 0.34s -> 0.07s user, exact PARITY with types-bench1's closed-world trusted path. The wrapper deliberately does NOT dispatch to the trusted clone (the plan's sketch): maam's value domain is constant propagation, so its claims can be conditioned on analyzed argument CONSTANTS (it prunes y>5 under y=3) — tag guards cannot re-establish that entry state for external callers, numbers included. The same argument exposed a PRE-EXISTING cross-module miscompile: trusted rewrites of call sites HOSTED in escaping functions (external f(7) crossing g's constant-pruned branch -> unguarded unbox of "s"). Fixed by the escape-taint fence: tainted = escaping closures, closed under callee-of-tainted-hosted-site + created-in-tainted-host; no trusted clone for escapees, no trusted rewrite of tainted-hosted sites. Covered code runs only during module init (before any external caller exists), so its trusted machinery keeps its justification; the import-cycle corner is documented residual. No off-switch (it is a soundness fix); EJS_NO_EXPORT_WRAPPER bisects the wrapper alone. types-wrapperfence1 pins the bug (f(7) -> NaN, node-identical). Wrapper compiles take a second optimizeModule pass: the loop-carried number proofs only fit provenNumberAt's depth cap after cleanup's trivial-param pruning. Wrapper-free compiles skip it (byte-pure); flag-off compiles never enter any of this. Gates: eir unit tests 213 pass (11 standing compiler-P1.1 pins only); --types diff lane 476 files / 0 divergent / 1 N/A (tester.js); new probes types-wrapper1 + types-wrapperfence1 + types-bench5 flag-off- identical (specWrapped/specFenced telemetry added to the stats line); matrix: 5 stage lanes 425/20/0, lowtier e2e OK. Details in docs/runtime-p2-results.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Baselines no longer encode any node version's inspect format: a harness-owned serializer (test/harness-console-shim.js) replaces console.log on BOTH sides — node generation runs through harness-run.js, the ejs side compiles a per-test import wrapper — so expected-outs assert on values. node 22.4.0 and 22.23.2 generate byte-identical baselines; CI floats on node 22.x. TZ=UTC pinned by the tester so Date baselines can't go machine-dependent. The years-stale-baseline un-masking flushed real bugs. Fixed: native error prototypes had null [[Prototype]] (instanceof Error false for every subtype; message now non-enumerable per spec), DataView wrongly indexed its buffer (it is not an integer-indexed exotic), typed-array RangeError message aligned with node. Pinned (xfail): Annex B.3.3 block-fundecl hoisting, toLocaleString ICU rounding, Date.prototype-is-ordinary. Un-pinned: number1, date3; esprima1 is generator:none (babel-register never could transpile the external-deps ESM — silently unregenerable before too). Tester repairs: the scheduler skipped the test at index test_threads in BOTH passes (weakmap2 had never actually run); per-test compile TMPDIRs (the shim module made concurrent compiles collide on genFreshFileName temps); baselines regenerate when the harness itself changes. Gates: stage0-3 + shapes-off 424/21/0 each, lowtier OK, test-eir = the standing 11 compiler-P1.1 pins only. docs/runtime-p3-results.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
//lib:generated converts the tsjs ES-module tree to CommonJS with one tsc --allowJs invocation (esModuleInterop matches babel's interop; the @llvm/@node-compat rewrites move to the staging side). test/tester.js becomes strict tester.ts (compiled in the staged tree by buck-test-stage.sh; test/tsconfig.json owns the settings) and the babel-node baseline generator becomes a tsc transpile of each import-syntax test's relative-import closure — directive renamed `generator: esm` across 114 tests, baselines byte-identical (112/112 runnable; the esprima-roundtrip pair is skip-if:true and was unrunnable under babel-node too). runtime/gen-atoms.js becomes gen-atoms.ts via //runtime:gen-atoms-js, output byte-identical on both atoms headers. babel is gone from package.json, the lock, .babelrc, and CI. Gates: tsc -p tsconfig.json + tsc -p test clean; test-eir = the 11 compiler-P1.1 pins only (verified same-by-name); stage0/1/2/3 and shapes-off 424/21/0; lowtier OK. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The optimizer's configuration surface moves from ~20 EJS_* env vars to a gcc/clang-style flag surface backed by one registry table (lib/pass-config.ts): canonical pass name -> config field -> default at each -O level -> help text, with --help and --print-passes generated from it. Passes read the per-run snapshot via passes(), never process.env. Suites: -O0 straight lowering; -O1 the cheap intra-function tier (cleanup, slot CSE, the sinks); -O2 the full pre-P5 default pipeline, byte-for-byte; -O3 = -O2 on the EIR side with LLVM default<O3> (-fllvm-opt=<n> decouples the LLVM level). -f<pass>/-fno-<pass> apply after the suite in command-line order, last-wins. Every EJS_NO_X maps 1:1 to -fno-<kebab(x)>; EJS_EIR_LOWTIER becomes -flowtier; the env reads are deleted (old spellings inert). EJS_FLAGS — tokenized as extra argv, applied last, -O/-f only — is the single env escape for harnesses that don't thread driver flags. Gate before deletion: 35/35 env≡flag A/B pairs byte-identical on one binary (EIR dumps; .ll-level for the emit tier and eir-opt), and default-config dumps byte-identical to a HEAD-built compiler for both flag-off and --types compiles; re-verified after deletion. Consumers: lib/eir/tests.ts uses withPassConfig() instead of env mutation; buck-test-lowtier.sh passes -flowtier. Gates: tsc clean; test-eir 216 pass + the same 11 compiler-P1.1 pins; lowtier e2e green; matrix stage0-3 + stage1-shapes-off all 424/21/0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 11 pinned lib/eir/tests.ts failures were entirely test debt from gc-P5's flag-off born-shaped literals; the recorded "flow-sensitive sinking does not drain make_object_shaped" gap does not exist. Both shaped sinks resolve shapes through the module's shape table, and the tests' lowerAndOptimize helper dropped the module lowerOne returns, so every shaped candidate silently declined in the harness only — the real pipeline always threads the module through optimizeModule (verified end-to-end with --dump-after eir-opt: shaped literals drain). - lowerAndOptimize passes the module to optimizeFunction - op-exact matchers assertContainsOp/assertNotContainsOp (\b regex; underscore is a word char, so \bmake_object\b rejects the shaped op) - expectations moved to the born-shaped contract: literal and class-accessor lowering assert make_object_shaped + shape="…", the flag-off null-oracle test asserts all-boxed shapes, the partial-escape test finds the materialized shaped op, survival assertions op-exact Gates: test-eir all 227 pass; tsc clean; stage0-3 + shapes-off 424/21/0 every lane; lowtier e2e OK. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…olicy //:dist repacks the srcdir-tree + stage2 exe into the installed layout the driver's non---srcdir mode expects (bin/ejs, include/, lib/<triple> archives, node-compat manifest+module); //:test-dist smoke-tests it as a user would (no --srcdir, plus the fail-loudly negative case); CI builds both and uploads per-platform tarballs. The driver now discovers its opt/llc instead of trusting a baked absolute path: LLVM_MAJOR is baked into host-config at build time, and llvm_bindir() resolves env override -> baked bindir -> conventional locations -> PATH, verifying each candidate's `opt --version` major before use — a mismatched opt miscompiles silently (the llvm@16 lesson), so no match means a loud, actionable failure. EJS_LLVM_NO_VERSION_CHECK=1 is the debugging-only escape; the unused llvm-as table entry is gone. docs/release-p1-results.md has the details and follow-ons. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
runtime/ejs-exception.c had `#define spew 1` baked in, so every compiled program logged the full throw/unwind/catch trace (plus the thrown value) to stderr — visible on each import miss while the compiler probes module resolution, i.e. on every @node-compat compile a dist user runs. Off by default now, same compile-time convention as ejs-gc-internal.h. Gates: full matrix (test-eir, stage0-3, shapes-off, lowtier, dist + test-dist) green; the dist smoke log is spew-free. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every package obeys the one structural fact: the driver resolves include/ and lib/ relative to argv[0] and does not chase symlinks, so each puts an absolute-path exec shim on PATH and keeps the layout whole. - dist tarball grows a sh-sourceable dist-info (version/triple/os/llvm major) and ships packaging/install.sh at its root - prefix installer: --prefix copy + bin/ejs shim + --uninstall, with best-effort LLVM-major and linux libuv/libunwind -dev advice (warnings only; the driver stays the fail-loudly authority); smoke-tested as //:test-dist step 4 on all three CI platforms - homebrew: packaging/homebrew/echojs.rb.in + make-formula.sh fills url/sha256/version/llvm-major from a tarball's dist-info; layout lives under libexec with bin.write_exec_script (a link farm is exactly the symlink shape the driver can't follow); depends_on llvm@N rides homebrew-core's versioned alias for the current major - npm wrapper (packaging/npm): postinstall fetches the platform tarball from the v<version> release (EJS_NPM_TARBALL override = the CI/offline/pre-release path), bin/ejs.js spawns dist/bin/ejs - CI: macos smokes the formula via a throwaway tap (install, shim compile, brew test, uninstall) + the npm wrapper; linux smokes npm - removed the 2016 make/llvm-3.4-era debian/, release/trusty64, and npm template bitrot Verified locally (macos arm64): brew tap/install/compile/test/uninstall cycle, npm pack/install/compile, //:dist + //:test-dist green. Hosted asset URLs, the real tap, and npm publish land with release-P3. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The version lives in exactly two files (package.json, the npm wrapper's) plus the git tag; everything else reads it from there. - CHANGELOG.md: Keep-a-Changelog shape, pre-1.0 semver reading documented, seeded with the first-release Unreleased section - packaging/prepare-release.sh: clean-tree check, rolls Unreleased into a dated section (refuses an empty one), stamps both package.jsons via npm version, commits + annotated v-tag; pushing the tag stays a human act - ci.yml's matrix jobs → reusable .github/workflows/bootstrap.yml (workflow_call); CI and Release call the identical workflow, so 'a release is a green matrix' is literal - release.yml on v<semver> tags: version-check (tag == both package.jsons, CHANGELOG section exists) → bootstrap → publish (DRAFT GitHub release with the three tarballs + hosted-URL homebrew formula + npm tgz, changelog section as notes; tap push / npm publish shell-gated on HOMEBREW_TAP_TOKEN / NPM_TOKEN) → clean-machine smokes (bare ubuntu:24.04 container both arches + fresh macos runner: tarball + documented prereqs only, install.sh, compile + run) - npm wrapper renamed @toshok/echojs — the bare name is taken on the registry (unrelated 0.1.4); CI smoke globs follow the scoped pack filename Verified: prepare-release dry run in a scratch clone (stamps, rolls, tags; second cut refuses on empty Unreleased), actionlint clean on all three workflows, make-formula --url mode produces the hosted formula. First pushed tag is the end-to-end proof. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The bare name was taken; @toshok/echojs never shipped. Scoped pack filename (pirouette-echojs-*.tgz) followed through the CI smoke globs and the release publish step. Also: a relative EJS_NPM_TARBALL now resolves against INIT_CWD — postinstall's cwd is the package dir, so relative overrides pointed at nothing. Verified: npm pack + install with a relative EJS_NPM_TARBALL + .bin/ejs compile green; actionlint clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per docs.npmjs.com/trusted-publishers: the publish job gets id-token: write and node 24 + npm@latest (trusted publishing needs npm >= 11.5.1 on node >= 22.14); npm exchanges the GitHub OIDC token for short-lived credentials and generates provenance attestations automatically. Gated on the NPM_TRUSTED_PUBLISHING repo VARIABLE so releases stay green until the publisher is configured on npmjs.com (owner toshok, repo echojs, workflow release.yml — the filename is what the config matches, and the publish step must live in this workflow, not a reusable one, since validation checks the calling workflow). The publish uses the packaging/npm directory rather than the packed tgz so provenance sees the build context; the wrapper's package.json gains repository.directory (provenance matches the repository field exactly) and publishConfig.access=public (scoped first publish). NPM_TOKEN is gone from the workflow and docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The old make-build shipped unprefixed tags up to 0.1.0; the pipeline's v-prefixed tags wouldn't collide, but the version number shouldn't be reused. The tree now carries the to-be-released version between releases, so prepare-release.sh stamps with --allow-same-version (npm version otherwise errors on 'Version not changed' — verified by a scratch-clone 0.2.0 cut with the version pre-stamped). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two independent failures, both now understood end to end: - linux (both arches, every red run since Jul 28): heap_priv.remset_other was declared ejsval** while _ejs_heap.remset is void** (the remset holds object pointers). Apple clang warns on the mismatched assignments; linux clang-22 makes -Wincompatible-pointer-types a default error, so the runtime never compiled. Type corrected (and the mallocs spell sizeof(void*)); no codegen change. Swept every runtime *.c with -Werror=incompatible-pointer-types + -Werror=incompatible-function-pointer-types against buck's argsfile — this was the only instance. - macos (the release-P2 npm smoke): packaging/npm/bin/ejs.js was never committed — the root .gitignore's unanchored 'ejs.js' pattern (guarding the old build's generated driver) swallowed it, so CI's npm pack shipped 3 files and node_modules/.bin/ejs was never created. Pattern anchored to /ejs.js, shim added. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
649c563's message described these changes but staged only the npm shim; this is the remset_other ejsval** -> void** correction (the linux clang-22 default error), the sizeof(void*) malloc spellings, and the /ejs.js gitignore anchor that had swallowed the shim. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ge1 crash _ejs_array_new(n, fill=false) malloc'd the dense element buffer, skipped initializing it, and still published array_length = n. The array scan specop walks [0, length), and callers like splice run GC-capable code (per-element Get/ToString) between the alloc and their stores — so a minor could scan n uninitialized malloc words as ejsvals. On glibc the recycled buffer holds stale pointers into young pages; the mover evacuates the swept, poison-filled cell and later dispatches through the copy's garbage ops. That was the linux bootstrap red: SIGSEGV on x86_64, the page->young assert on arm64, and the flaky per-test stage1 compiler crashes — deterministic in an ubuntu:24.04 container, latent since the gc-P2 mover landed (linux CI's last green predates it; macos survives on allocator-content luck, the window is real on every platform). Fix: hole-fill [0, numElements) unconditionally; the `fill` flag keeps only documentation value. Audited the other element/length publish sites (constructor argc path, push/pop dense, splice, the grow-on-store paths): all hole-fill first, calloc, or have no GC point inside the window. Verified in the container (linux-arm64): the stage2 self-compile that crashed at module 10 passes repeatedly — including under three MALLOC_PERTURB_ patterns — with a poison-evacuation tripwire armed in minor_process_slot; full linux ladder + macos matrix runs in flight. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the hand-rolled curl + zstd download of the facebook/buck2 `latest` release binary in both bootstrap jobs — the action installs the same release and owns the platform selection, so the buck2_triple matrix key and the zstd install-time dependency go away. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New static analysis (courtesy of https://github.com/toshok/echojs-maam), compiler IR, optimization passes, and GC implementation. documentation to come.