JavaScript backend for lengc (Typed-Array linear-memory model)

[savannt]

What

A JavaScript backend for Nimony/lengc, built as a standalone
{.build.}-schedulable plugin over the Leng IR — a third codegen alongside
the existing C and LLVM backends. It follows the Typed-Array / linear-memory
direction @Araq laid out in this thread: JS consumes the same fully-lowered
Leng as the C backend rather than a separate, GC-native semantic model, so
cast, unions, and low-level libraries behave identically across targets and
the two backends can never diverge.

This opened as a pure-compute RFC to settle one question — fully-lowered vs.
GC-target Leng. That's resolved: keep the fully-lowered (C memory model) Leng
and back it with JS Typed Arrays / an ArrayBuffer as byte-addressable
linear memory. The PR has grown into an implementation of that model; the
discussion below traces how it got here.

The model

• One ArrayBuffer is the memory. A pointer is an integer byte offset
  into it. object/array/string/seq/ref are laid out as bytes at
  C-ABI offsets — so cast is transparent and there is only one pointer
  representation.
• A new layout pass (jslayout.nim) computes sizeof/alignment/field
  byte-offsets from the Leng type grammar. The C and LLVM backends both defer
  struct layout to their toolchains, so this is the one genuinely new piece the
  Typed-Array model needs.
• The real Nim allocator, compiled to JS (per @Araq), needing only tiny
  mmap/munmap shims — allocFixed/dealloc/seq growth/ARC atomics all come
  from the pipeline; the runtime provides only the primitives (mmap, stdio,
  the function table).
• JS is emitted as a jsnif tree, not by string concatenation: a JsTag
  vocabulary built over nifcore's TokenBuf, a tiny jsnif → js printer (the
  only place text is produced), and a bottom-up peephole pass that folds the
  x + 0 / x * 1 / constant arithmetic the byte-pointer lowering emits.
• Structure = plugin, not built-in. lengjs <module.c.nif> <out.js> emits
  one artifact per module (the {.build.} contract); jslink <manifest> <out>
  is the {.bundle.} link step — the Ghast-style decoupling.

What runs end-to-end

Real Nim → nimony c → Leng → lengjs → Node, correct results, 0 TODOs on the
user module:

• scalars, arithmetic (incl. checked ops), if/while/case, enums, tuples,
  for-loops, sets, variant objects;
• objects — value, ref/heap (with ARC refcount + destructors), and
  inheritance (base embedded at offset 0, inherited fields, vtable dispatch);
• arrays and buffer indexing, including through a pointer;
• strings — SSO and heap (echo "…" prints, long strings > 14 chars work);
• seqs — @[…], add with reallocating growth, len, indexing;
• closures — captured env + a WASM-style function-dispatch table (_fns[]);
• exceptions — both the ErrorCode ABI (try/except/finally, nested,
  in-loop, except T as e) and the heap ref object of Exception path. Lowered
  via @Araq's jmp/lab → break in labeled blocks — no relooper, since
  Hexer's jmp is forward-only and scoped.

Verification

tests/jsbackend/run.sh golden-diffs hand-authored, self-contained Leng modules
(no system deps) against *.expected.js and runs each under Node — 22 checks.
Broader coverage is validated by compiling real programs through the full
toolchain and bundling with the canonical runtime. The C and LLVM backends are
unaffected (a new plugin + tests only).

Still open (not guessed in code)

• Whole-program {.build.}/{.bundle.} wiring — how every module is routed
  through lengjs and a single jslink scheduled without hand-annotating each
  module.
• The importjs/jsstring interop layer (native Typed-Array data vs. real
  JS values / a handle registry) — on hold until those pragmas exist in sem;
  nothing in-tree references them yet.
• True cycle collection (an upstream ARC concern, not JS-specific).

🤖 Generated with Claude Code

[savannt]

Update — data structures (one step past pure-compute).

Added design-neutral codegen for the lowered data shapes, so the backend now handles object/array values and field/index access, not just scalars:

Leng	JS
(oconstr T (kv f v) …)	{f: v, …}
(aconstr T e0 e1 …)	[e0, e1, …]
(dot obj f)	obj.f
(at arr i) / (pat p i)	arr[i]
(addr x) / (deref x)	x (JS values are references)

These are all 1:1 mappings of the lowered Leng, so they don't pre-empt the M2 design question. Concretely, echo "hello world" now generates zero /*TODO*/s — its string literal lowers to a LongString object constructor and the SSO value {bytes, more: (addr …)}, all of which now emit cleanly. What's left for echo to actually run is purely the runtime: JS shims for the system FFI procs (write/stdout/nimFlushStdStreams) plus an SSO-string decode in the write shim — which is exactly where the consume-lowered-vs-higher-level-Leng question bites.

New self-contained test tests/jsbackend/tdata.c.nif (+ golden + Node functional check: mkpoint(3,4)==7, arrsum()==60), wired into run.sh alongside the existing pure-compute test.

[Araq]

Leng	JS
(addr x) / (deref x)	x (JS values are references)

^ This is not correct at all and it's where most of the complexity comes from in the old JS code generator. Locals which address have been taken must be translated into 1-sized arrays, addr x becomes x then and x becomes x[0].

[savannt]

Reworked addr/deref into the etyBaseIndex fat-pointer model you described ([base, key], deref p → p[0][p[1]], address-taken locals boxed into 1-elem arrays so addr x → [x, 0]). This also handles field/element addresses — addr o.f → [o, "f"], addr a[i] → [a, i] — which dropped the addr-of-location TODOs in the real std/system Leng from 23 to 3 (the 3 left are addr (pat p i) pointer-arithmetic inside cast(ptr uint8, …) in stringimpl's SSO code). Two fat pointers compare component-wise via a small nimPtrEq helper (the =copy self-assignment check); p == nil stays a plain comparison.

On top of that:

• Cross-module importc/exportc resolution via getDeclOrNil, mirroring the C backend's mangleSym: an importc proc/global is referenced by its C name and not emitted, an exportc symbol keeps its C name. So echo's generated code now calls bare fwrite/fprintf/fputc on stdout, and main/cmdCount/cmdLine resolve correctly — the FFI boundary is small and explicit.
• Unsupported nodes now emit a valid placeholder (undefined/*TODO*/) so a never-called function containing one (e.g. the string copyMem path) no longer breaks parsing of the whole bundle.

With those, integer echo runs end-to-end through the actual std/system + std/syncio modules:

$ cat hello.nim                 # echo fib(10); echo 2 + 3
$ nimony c --nimcache:nc hello.nim
$ for f in nc/*/*.c.nif; do lengc js --nimcache:out "$f"; done
$ cat runtime.js out/*.js <(echo 'main(0,[]);') | node
55
5

where runtime.js is ~15 lines (stdout, fprintf, fputc). Module init touches no allocator (it only zero-inits an exception global), and the integer path needs no raw byte addressing, so it just runs.

The one remaining wall is string echo: write lowers to readRawData(s) (a Nim proc returning a pointer into either the inline SSO bytes or the heap buffer) handed to fwrite as a raw byte pointer — byte-addressing into the packed SSO integer. That's exactly the consume-fully-lowered-vs-higher-level-Leng question: a GC-target Leng with strings as JS strings sidesteps it entirely. Does this firm up the case for gating the memory-management passes (destructor/dup/lifter) for a GC backend, the way Nim 2's JS target skips the destructor pass?

The tests/jsbackend/run.sh suite now has 6 self-contained checks (compute / data / addr / addr2 / importc / echo), each a hand-authored Leng module run under Node — needs only bin/lengc + node.

🤖 pushed as commits on js-backend.

[savannt]

Coverage survey — where the backend stands, and how the gaps converge.

I ran ~13 real programs through the full pipeline (nimony c → lengc js → Node) to map what's covered vs. what's blocked. Short version: the design-neutral surface is now broad, and everything still missing bottoms out in the one open question (fully-lowered vs. GC-target Leng).

Runs correctly under Node today (zero user-module /*TODO*/s):

• pure compute, all arithmetic incl. overflow-checked keepovf/ovf
• if / while / case (multi-value branches → switch fall-through)
• enums (→ ints), tuples (→ objects), for loops (→ while + inlined iterator)
• floats, bool short-circuit, arrays + bounds checks (nimIcheckB)
• set membership (c in {'a'..'z'}), object … case variants
• addresses: locals / fields / elements via the fat-pointer model, and pointer equality
• integer echo end-to-end (echo fib(10) → 55) with a ~15-line stdio runtime

Blocked — and each one is the same design decision:

feature	what it lowers to	why it needs the decision
strings	readRawData(s) → fwrite raw byte pointer	byte-addressing into the packed SSO int
seqs / ref / new	allocFixed(sizeof T)	C heap allocation model
closures	env object + allocFixed(sizeof env)	codegen already works — only the env allocation blocks it
exceptions	error-code ABI + jmp/lab (goto) + baseobj	raise sets the exc threadvar and returns Failure; try/except is goto-based, not native try/catch

So the residual TODO tags across the whole stdlib reduce to: sizeof/allocFixed (allocation), addr (pat …) (raw pointer arithmetic), jmp/baseobj (goto-based EH + RTTI). All of them disappear under a GC-target Leng that keeps strings as JS strings, objects GC-managed, and try/except native — which is exactly the "gate hexer's destructor/dup/lifter passes for a GC backend" option, and for exceptions additionally not running the EH→goto lowering.

Concretely: is the plan to branch the JS backend off a higher-level Leng (GC target, native EH), or to reconstruct all of this — allocation, SSO strings, goto — on top of fully-lowered Leng? The former looks dramatically smaller and is where I'd take M2 next; happy to prototype the hexer pass-gating if that's the direction.

🤖 pushed as commits on js-backend (fat pointers, importc/exportc resolution, checked arithmetic, valid-JS placeholders + running integer echo).

[Araq]

We need the low level representation so that all low level libs that happen to use cast can work reliably. This means we base object and array etc on https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Typed_arrays

For JS interop we need all the special pragmas like importjs etc. This means we have the existing "fat pointer" solution for JS objects that happen to use var parameters, we have a jsstring that maps to JS's but usually Nim objects etc are mapped to Typed Arrays.

Note that Nimony now keeps user-level pragmas in its IR verbatim and plugins can access that and plugins can also emit a .bundle instruction so custom backends based on Leng are directly supported. See also Ghast https://github.com/nim-lang/nativenif/pull/73/changes

[savannt]

Got it — that reframes M2 cleanly and resolves the design question I was circling. Let me play it back to make sure I have it right, then sketch the plan.

The model, as I understand it:

• Native data → Typed Arrays. object/array/seq/string are laid out in an ArrayBuffer with typed-array / DataView access at computed byte offsets — a byte-addressable linear memory, the same shape as the C model (and WASM linear memory). That's what makes cast and the low-level libs reliable, and it dissolves the SSO-string wall directly: readRawData(s) becomes an offset into the buffer and fwrite reads those bytes. So we keep the fully-lowered Leng — no hexer pass-gating.
• JS interop → a separate layer. importjs / jsstring etc.: jsstring maps to a native JS string, interop objects stay native JS objects, and the [base, key] fat pointer I already built is the representation for var parameters over those interop objects — not the universal pointer anymore.
• Delivery → a plugin, not an in-tree lengc codegen. A standalone binary that reads the verbatim user pragmas from the IR, parses Leng NIF (paramStr 1) → emits the JS artifact (paramStr 2), scheduled by {.build(…).} as a DAG node and decoupled over the NIF wire — mirroring Ghast / arkham↔nifasm — and able to emit .bundle.

M2 plan on that basis:

1. Restructure the current jscodegen.nim into a standalone Leng→JS plugin along the Ghast layout (parse Leng NIF → translate → render), so it's a real DAG node rather than a lengc subcommand.
2. A struct/array layout pass: field byte-offsets, sizeof / alignment, over ArrayBuffer views — and rework the current {f:v} / [e0,e1] mappings onto that.
3. An allocator over the linear memory — a JS runtime that provides the allocFixed / alloc / dealloc primitives against a growable ArrayBuffer; strings/seqs/refs then fall out of it.
4. Fold the landed pieces back in (fat pointers for interop var params, importc/exportc, arithmetic, control flow) and add importjs / jsstring handling.

Everything already in the branch (fat pointers, importc/exportc resolution via getDeclOrNil, checked arithmetic, control flow, running integer echo) stays valid; it's the object/array-literal mapping that gets reworked onto the buffer.

A few things I'd like to confirm before I start:

1. Allocator — for the linear-memory allocFixed / mimalloc primitives, is the intent to run the existing allocator over a JS ArrayBuffer (WASM-Memory-style, growable), or to provide a minimal JS shim for just the alloc primitives? Anything on the arkham/CPU side I should mirror rather than invent?
2. Exceptions — with fully-lowered Leng the EH is the error-code ABI + jmp/lab goto, so JS would need relooper-style reconstruction (or a per-function state-machine dispatch, since JS has no goto). Is that the expected approach, or does the interop layer give native try/catch a role here?
3. Wiring — should this be scheduled by {.build(…).} exactly like Ghast, and is there a preferred binary name?

Thanks — this is a far more finite target than the fork I was worried about.

[AmjadHD]

• What does cstring translate to? Its C shape or a native JS string?
• What happens if you mix Nim objects and JS objects? Would a register for JS managed memory be needed?
• What happens if you export a Nim object? Does it need a getter/setter interface?

[savannt]

@AmjadHD good questions — my read as the one building this, with the caveat that the final calls on the interop surface are Araq's. All three come down to the same split: Nim-native data lives as bytes in the linear-memory ArrayBuffer (a pointer is an integer offset into it), and anything that must be a real JS value lives outside it.

1. cstring — its C shape: a byte pointer into the buffer (offset to NUL-terminated bytes), not a native JS string. It's a low-level FFI type that code casts and does pointer arithmetic on, so it has to be addressable memory. The native-JS-string case is the separate jsstring type (via importjs), which maps to a real JS string; converting between the two is an explicit copy at the boundary (decode buffer bytes → JS string, or encode back), like the existing cstring/$ conversions.

2. Mixing Nim and JS objects — yes, a register is needed. A Nim object is bytes in the buffer; a JS object is a reference the buffer can't hold. So a field/var of a JS-interop type stores an integer handle, and a JS-side handle table maps handle → live JS value — the classic approach (emscripten's EmVal table; the current Nim JS backend does the equivalent). That table doubles as the GC root set for JS values Nim is holding: an entry is added on the way in from importjs and released when the Nim owner dies. Whether that release is manual, tied to =destroy, or a weak-ref sweep is exactly the memory-management/ownership question I raised above — the handle register is the JS-side half of it.

3. Exporting a Nim object. Since it's bytes in the buffer, a JS caller can't treat it as a native object directly. Two options at the boundary: (a) hand back the offset plus generated accessors — getX(p) / setX(p, v) reading/writing each field at its computed byte offset — or (b) marshal it into a native JS object (a shallow copy) on the way out. (a) keeps it live and mutable and is cheap; (b) is nicer to consume but copies and can desync. Either way it's mechanical, because the field offsets come straight from the layout pass (just landed in 383df28e) — accessor generation is a direct read of objectFields.

Net: two disjoint worlds — the linear-memory buffer and a JS handle table — bridged by importjs/jsstring and explicit marshalling, with the fat-pointer [base, key] form kept for var parameters over JS objects. The one thing still open is the ownership ABI: who frees handles and heap allocations.

[AmjadHD]

• Where should JS (small) primitives be stored? In the linear memory or the registry?
• When exporting a Nim object to JS using generated getters/setters (Option A), how does JS allocate a new instance of that Nim object? Does it call a generated JS factory that invokes the compiled Nim allocator to carve out bytes in the buffer?
• Should we generate ES6 code?

[savannt]

Good follow-ups — the right next-level questions.

Where do JS primitives live — linear memory or the registry? It depends on whether the primitive has a fixed machine representation:

• number / boolean map straight onto Nim scalars (a JS number is an f64; a boolean is a byte), so they live unboxed in linear memory — no registry entry, no indirection; crossing the boundary is just a load/store.
• string / symbol (and arguably bigint) have no fixed in-buffer shape, so they go through the registry as handles — a jsstring field stores an integer handle — unless you explicitly marshal a string to buffer bytes (the cstring direction).

So the registry is only for JS values that can't be a fixed byte pattern; anything with a native scalar shape stays in linear memory. (Same split emscripten draws: numbers pass directly, objects/strings use the EmVal handle table.)

How does JS allocate a Nim object (the Option-A getter/setter case)? Exactly as you sketched: JS can't carve bytes out of the Nim buffer itself — only the compiled allocator knows the heap state — so exporting a constructible object generates an exported factory that calls the Nim allocator for sizeof(T), runs the object's initializer, and returns the offset. The ergonomic wrapper is an ES class whose constructor calls that factory and stashes the offset, with the generated get/set as accessors — so JS writes new Point(1, 2) while underneath it's _nim_Point_new(1, 2) returning an offset. The mirror image — freeing — is a dispose() / FinalizationRegistry that calls the Nim deallocator, which is the ownership-ABI question still open above.

ES6? For this memory model it's less "should we" than "we already must": int64 needs BigInt (ES2020) and the buffer wants a resizable ArrayBuffer (recent), and the backend already emits let/const/"use strict". Given that hard modern floor, ES classes for the export interface (the wrapper above) are the natural fit. The real decision left is the minimum target — Node-only vs browsers, and whether a downlevel/transpile path matters — which I'd defer to you and @Araq; I'd only flag that BigInt + resizable ArrayBuffer already rule out ES5.

I'll follow up shortly with a small runnable spike demonstrating the registry + factory + ES-class export path end-to-end, the same way the string spike showed the linear-memory model — easier to poke at than prose.

[savannt]

Follow-up: the runnable spike is up in 4d6e5eb3 (tests/jsbackend/spike/interop.spike.js), demonstrating the three points end-to-end:

$ node tests/jsbackend/spike/interop.spike.js
interop spike: PASS (registry + allocator factory + ES-class export)

Person { age: int32; name: jsstring } — age is a number stored unboxed in the buffer, name is a string stored as a registry handle; _nim_Person_new is the allocator-backed factory; and class Person is the ES-class export wrapping the offset with get/set accessors + a dispose() that drops the registry root. It runs on the placeholder bump allocator, so the only thing it doesn't pin down is the ownership/free ABI — still the one open question.

[Araq]

As for the allocator question, we compile the native Nim allocator to JS, it needs a tiny emulation for mmap() and munmap() and that's all there is to it. The allocator does not assume a single large block of memory so it could be based on non-growable TypedArrays rather easily too.

[AmjadHD]

Does it generate modules or a single file?

[savannt]

Both, at different stages — the same split as the C backend, plus a link step:

• Per module during codegen. The backend is a {.build.} plugin (lengjs) handed one module's Leng IR (.c.nif), emitting one .js artifact for it — one output per module, exactly like the C backend emits one .c per module. Symbols are already module-suffixed/mangled (e.g. mkpoint_0_tbuffer), so there are no cross-module name clashes.
• Single file after linking. A {.bundle.} linker (jslink) reads the project link manifest and concatenates the runtime + every module artifact + the entry call into one .js. JS function hoisting makes the flat concatenation resolve cross-module references with no import/export wiring, so that combined file is the delivered artifact.

So the final output is a single self-contained file. I went this way because the linear-memory model already makes every module share one ArrayBuffer + runtime, so a single file is the natural unit. The two-tool split keeps it flexible, though — jslink is the only place that decides layout, so a multi-file ES-module mode (one file each, import/export between them) would just be an alternate bundler without touching codegen.

(Verified end-to-end: a real echo fib(10) → four module artifacts → jslink → one file → Node prints 55.)

[savannt]

Progress update: the backend now runs the linear-memory model, not just the pure-compute slice

Since this opened as a pure-compute slice, the interesting work has been moving the shipping backend onto the Typed-Array / linear-memory direction you laid out — so JS is the same lowered Leng as C rather than a separate semantic model. Where it stands now:

One ArrayBuffer is the memory. object / array / string and pointers to them are bytes in one linear buffer (mem), laid out at the C-ABI offsets a small jslayout pass computes. A pointer is a single integer byte offset — addr o.f is p+off, addr a[i] is p+i*stride, deref p is a typed load at p, == is integer ===, nil is offset 0. One cast-transparent representation, so there's no fat-pointer/box form for cast to fail to see through. An address-taken scalar local is spilled to a buffer slot (the C stack model), so it too has a real address.

What runs end-to-end through the full pipeline (nimony c → the .c.nif modules → lengjs per module → jslink bundle → node):

# objects + array + while-loop + bounds check + object-param proc
type Point = object
  x, y: int
proc dist2(p: Point): int = p.x*p.x + p.y*p.y
proc run(): int =
  let p = Point(x: 3, y: 4)
  var a: array[3, int] = [10, 20, 30]
  var s = 0
  var i = 0
  while i < 3:
    s = s + a[i]
    i = i + 1
  result = dist2(p) + s     # 25 + 60
echo run()                  # -> prints 85   (int -> string -> echo, 0 TODOs)

import std/syncio
echo "hi"                   # -> prints hi

Strings echo now. This was the last thing gating a Hello, World. Two things had to fall out of the byte-pointer model rather than be special-cased: (1) the string idiom takes s: ptr string, so a field read lowers to (dot (deref s) bytes) — the dot base is a (deref sym), not a bare symbol — which needed the buffer field-resolution to follow the pointer; and (2) the SSO length is read via deref(cast (ptr u8) (addr …)), so the load width has to come from the (cast (ptr u8) …), not default to a word — otherwise len reads the whole first word and wrongly takes the long-string branch. Both are now resolved through one shared type-resolution path, with a self-contained regression (tstrptr) that pins them.

Your allocator answer folded in cleanly: the runtime provides only mmap/munmap, and the small-string path needs no allocation at all, which is why echo "hi" works before any of the heap-string machinery does.

Still design-gated (all bottom out in the GC/allocator or goto-EH decision, not in the backend surface): seqs, ref/heap objects, closures (the env-passing works; only allocFixed(sizeof env) is gated), exceptions (these lower to an error-code + jmp/lab goto ABI, which needs relooper-style reconstruction for JS), and the long-string (>14 char) heap branch. The design-neutral surface — compute, control flow, enums, tuples, sets, variant objects, arrays, objects, addr/deref, small strings — is essentially complete over the buffer.

Happy to keep this incremental. Full tests/jsbackend suite is green (13 checks: golden + node functional per fixture, plus the bundle link).

[Araq]

produce the JS as NIF-tree with a dedicated JS-whatever enum, the nifcore APIs have been designed with this in mind. Then have it produce a tiny. jsnif -> js emitter. This way the code it not full of stupid string operations we have to keep correct all the time
plus you can easily run a peephole optimizer later on the nifjs etc etc
the runtime costs are tiny if done properly and it's always worth it IME

[savannt]

Done — the backend no longer builds JS by string concatenation. It now emits a jsnif tree and a tiny printer turns that into text (d14144ca).

Two pieces:

• jsnif.nim — a JsTag vocabulary (func/var/if/while/switch/call/bin/member/array/object/…) built over nifcore's TokenBuf (the openTag/closeTag/addStrLit API — I used the ordinal-0 / +1-shim trick your docs describe for the jsonnif/htmlnif adapters, since the pool is private to the backend run). Plus emit, a ~one-line-per-tag printer that is the only place JS text is produced. Parenthesisation and indentation live there, so the translator never thinks about them.

• jscodegen.nim — rewritten to translate Leng → jsnif instead of concatenating. Every "(" & a & " + " & b & ")" is now a jBin node; sub-expressions nest into one shared buffer, so the old captureExpr string-juggling and all the hand-managed parens are gone. The buffer-model type logic (field/pointer resolution, layout) is unchanged — only emission moved onto the tree.

Concretely this kills the bug class it was meant to: the recent defects here were all string-assembly mistakes (an empty mem.() accessor, a stray [base,"field"] fat-pointer string, a wrong load width) — none of those are expressible once the shapes are tree nodes with a typed printer. And, as you said, there's now a tree to run a peephole pass over ((deref (addr x)) → x, redundant temp folding, etc.) whenever that's worth doing.

Verified end-to-end: full tests/jsbackend suite green (13 checks — golden + node-functional per fixture + the bundle link), real echo "hi" prints hi, the real object+array program prints 85 at 0 TODOs, and switch/case (multi-label + default) checks out. Goldens regenerated from the emitter — the only churn is cosmetic (uniform parens).

[savannt]

And the peephole payoff is real already — a first jsnif -> jsnif pass (7988ce77) folds the x + 0 / x * 1 / constant arithmetic the byte-pointer lowering emits (offset-0 fields, byte-stride arrays). It runs bottom-up and shares the input tree's pools, so unchanged subtrees are a memcpy and only folded spots are rebuilt:

mem.u8At((s_0 + 0))              ->  mem.u8At(s_0)
mem.u8At((s_0 + 0) + (1) * 1)    ->  mem.u8At((s_0 + 1))
mem.setU64((hi_0 + 0), 6907906)  ->  mem.setU64(hi_0, 6907906)

Pure tree rewrite, no string handling, trivially extended with more rules. Suite still green, real programs unchanged (hi / 85).

[Araq]

About the exception handling: Here too we make good use of what Hexer already outputs: jmp and lab. These can be directly supported and mapped to break in labeled blocks without a relooper because:

1. jmp is forward-only. Every (jmp L) targets a (lab L) that appears later in the statement stream;
   there are no backward jumps.
2. Scoped — no jump into a construct. A jmp may leave out of enclosing constructs and go forward, but it may never enter a sibling or descend into a nested construct.

These two properties for Leng are currently not strictly enforced but the produced code adheres to them and lengc/arkham will enforce them soon.

[savannt]

Progress update: exceptions (your jmp/lab approach), inheritance, and heap exceptions running end-to-end

Picking up your exception-handling note — I built it exactly the way you described, and both properties held, so no relooper was needed.

jmp/lab → break in labeled blocks (599a9b9e). Ground truth from the pipeline: nimony's ErrorCode model lowers try/except not to try/raise but to exactly the shape you called out — a raising call returns an error tuple, the caller does (if canRaise.err (jmp exlab)), and the handler lives in a dead (if (false) (lab exlab) …) landing pad reachable only through that forward jump. So the lowering is:

• genStmtList linearizes each stmt list, rewriting a dead-if into a flat item stream [else…; jmp SKIP; lab L; handler…; lab SKIP] (a fresh $exs<n> skip label past the handler);
• emitItems wraps that in nested labeled blocks (outermost = last label), so (jmp L) → break L lands right after block L;
• if/while/case stay opaque, so a jmp inside a loop/switch is just a labeled break out of it — which is why forward-only + scoped is all that's required.

Locals live-across a label are hoisted to a function-scoped var (only in procs that contain a jmp, so plain procs are unchanged). Verified e2e under Node: try/except (42 / −1), try/except/finally, sequential and nested try, try-in-a-loop, except T as e binding, and exception subtyping — 0 TODOs.

Object inheritance + RTTI (2ac6bc6d). Needed for the heap-exception (ref object of Exception) path: baseobj conversion is a no-op on the byte offset (base embedded first, like C's .Q); jslayout now walks the base chain so inherited fields keep their front offsets; object-with-trailing-flexarray construction builds vtables in place; vtable dispatch resolves through _fns[mt[i]]. Value- and ref-object inheritance with destructors both run.

Heap exceptions now run end-to-end (65f3d38e). The last blocker was cross-module: the in-flight exc threadvar is address-taken (so boxed to a buffer slot) in system, but a raising module referenced it bare — reading the slot offset instead of the value, because the address-taken pass is per-module. Rather than a whole-program pass, the boxing decision for a module-level global now follows its decl kind (a scalar var/gvar/threadvar has static storage → it lives in a buffer slot in every module), which is visible identically from each module via the lazily-loaded foreign declarations. Consistent by construction, no cross-module scan. With that, raise IOError(...) works:

$ node <bundle for tref_heap_exc.nim>
Caught IOError: disk code=42
done

$ node <bundle for treraise_typed.nim>   # re-raise from a handler, past the enclosing try
outer-caught: inner code=1
top-caught: wrapped: inner code=101

Also landed since the last update (all e2e over the linear-memory buffer, verified through the full toolchain): long strings > 14 chars (0e9547a7), seqs with reallocating growth + the canonical runtime (c24e01cb), ref/heap objects with ARC refcount atomics (dc85a96b), and closures via a WASM-style function-dispatch table (16ee85ac).

Where it stands. Objects (value/ref/inheritance), arrays, strings (SSO + heap), seqs (+ growth), ref/heap objects, closures, and exceptions (both the ErrorCode ABI and the heap ref path) all run end-to-end — "just more Nim over the buffer," with the allocator, atomics, I/O, and the function table as the only runtime shims. Of the original design-gated list, only true cycle collection remains, and that's an upstream ARC concern rather than a JS-backend one. The self-contained tests/jsbackend/run.sh suite is at 22 checks; the C and LLVM backends are untouched.

Still open and not guessed: the whole-program {.build.}/{.bundle.} wiring (how every module is routed through lengjs and the single jslink scheduled automatically), and the importjs/jsstring interop layer — which is on hold until those pragmas exist in sem (nothing references them in-tree yet, so there's no native-vs-interop split to make today).

[savannt]

Addressed the review in 8a63f290 — full tests/jsbackend suite green (13/13).

jsnif tags by construction. Dropped both ids arrays; TagId == JsTag + 1 is now an accessor template (tagId/tagOf), and the pool is built with createTags[JsTag]() (which asserts that alignment). open and the emitter/peephole decode are register moves — no array access.

typenav. jscodegen no longer keeps its own type navigator: exprType delegates to getNominalType, and locals live in the typenav scope stack (openScope/registerParams/closeScope per proc), so the per-proc localTypes leak is gone. typenav itself is untouched.

nifmodules. Removed hasLocalDecl/hasResolvableDecl; the backend now requires a consistent input set and calls getDeclOrNil directly, like jslayout and the C backend.

lengc js subcommand. Removed — lengjs is the plugin. atJS/generateJSBackend are gone.

Native allocator. Done for real: the stdlib builds with -d:nimNativeAlloc, so Nim's own alloc.nim compiles through lengjs (0 TODOs) and runs over mmap/munmap shims carved from the same ArrayBuffer — no more mi_* bump shim, alloc/dealloc/realloc + free-list reuse are real Nim code. Two bugs the allocator was first to hit, both fixed:

• atomic slots are 4-byte on the --bits:32 target (rc: int and pointers) but the shims were i64 — rewrote them at 32-bit and added the missing __atomic_exchange_n/__atomic_compare_exchange_n;
• lengjs was lowering unsigned shr/&/|/^/<</~ to signed JS int32, so a uint32 with the high bit set went negative and broke the TLSF bitmap search (not 0u32 shl k, msbit's x shr a) — now logical >>> + a >>> 0 reinterpret for unsigned types.

munmap is a no-op for now (whole-page reclamation deferred; the allocator still reuses cells within its arenas).

README rewritten for the typed-array / linear-memory model.

[savannt]

Pushed a follow-up commit (f07e7f69) adding the JS value interop foundation (jsffi) — the layer a DOM/host binding sits on. Flagging one design choice before it calcifies.

What it does. A JsValue is an opaque integer handle into a runtime-side value table (the generalisation of the _fns proc-pointer table already in the runtime), so real JS values — strings, objects, functions, later DOM nodes — can cross the linear-memory boundary that native data can't. It rides the existing importc seam: jsffi.nim declares body-less importc procs that lower to calls of runtime bridge functions (_strToJs, _jsGetProp, _jsCall*, …). On top of that: toJs/toStr/toInt/toBool marshalling (strings cross as UTF-8), global/get/set/call, and == as JS ===. A new tffi test exercises it end-to-end (console.log a marshalled string, Math.max round-trip to a Nim int, string JS→Nim, build an object + JSON.stringify); suite 14/14.

The design question for you. The header comment in jscodegen.nim reserves the fat-pointer [base, key] form for "the future importjs/jsstring interop layer." I went with a handle table instead. My reasoning: the fat-pointer form is really about lvalue access into a JS object (obj.prop ≈ [base, "prop"]), whereas a handle is the cleaner primitive for holding an opaque JS value in a Nim variable / int — complementary rather than exclusive (embind / the wasm model use handles). But it is a divergence from what you'd sketched, so if you'd rather the interop go through the fat-pointer path I'll switch it: the user-facing API (toJs/get/call/…) is representation-agnostic, so only the runtime bridge changes, not the surface or the tests.

Known limits (deliberate, documented). Handles are not yet GC-integrated — a kept value must be released (the wrappers release their own transient member-name handles). I intentionally did not add a naive =destroy, since handles are non-owning aliases and auto-release would double-free on copy; doing that correctly (destructor + refcount, or a =copy that dups the JS reference) is the next increment, along with Nim→JS callbacks for events. The module is test-local rather than in lib/ because there's no when defined(js) gating yet and a body-less-importc module in the always-compiled stdlib would fail the C link.

Happy to adjust the direction before building the DOM layer on top.

[Araq]

But it is a divergence from what you'd sketched

That does not matter, your analysis is correct.

[savannt]

Status update — the binding layer on top of jsffi (f07e7f69..af2ee4c2, head af2ee4c2; tests/jsbackend green at 23/23).

Since the jsffi foundation I rounded out the interop core — GC-integrated handles with auto-release, floats, JS arrays, typeof/in/instanceof introspection, variadic apply — and then built the two things it was for: a DOM binding, and a spec-backed generator that emits bindings straight from official WebIDL.

DOM binding + a real test environment. dom.nim is a thin typed facade over jsffi (Document/Element/Node/Event; createElement/appendChild/querySelector/setAttribute/addEventListener/…). For end-to-end tests I added a per-test <test>.env.js mechanism: a preamble (jsdom — the spec-compliant WHATWG DOM) is prepended to the bundle and Node runs with it on NODE_PATH, so a binding is validated against real DOM semantics, not a hand shim. It's gated — a test that needs a DOM env is skipped (not failed) when node_modules is absent, so a bare checkout stays green. jsdom/webidl2/@webref/idl are dev-only; nothing is vendored.

Spec-backed generator (gen/idl2nim.js). DOM/web is too large to hand-write, so dom.nim is really the reference shape for a generator. It reads the curated WebIDL that @webref/idl ships (the official W3C/WHATWG interface data), parses it with the standard webidl2, and emits jsffi bindings in that shape:

• attributes → get/set procs, constructors → newX, operations → call procs, WebIDL types mapped through jsffi's marshalling;
• optional args → one overload per arity; a variadic tail → an openArray spread via apply;
• it flattens the inheritance chain and merges includes mixins, so a generated Element is self-contained — appendChild (Node), addEventListener (EventTarget), querySelector/append/children (the ParentNode mixin), remove (ChildNode) all in one binding;
• given several interfaces it emits one module with interface-typed members, so DOM-tree navigation is typed by name: document.createElement/getElementById/querySelector → Element, ownerDocument → Document, appendChild(node: Node): Node. Since every interface is a JsValue alias, a shared flattened member collapses to one Nim signature, so emission dedups globally.

Each generated binding is exercised end-to-end — URL against native Node, Element/DOMTokenList/the Node/Element/Document module against jsdom.

All of this is application-layer on top of the backend — no codegen changes — so it doubles as a soak test of the Typed-Array/linear-memory model: the interop handle table, strings crossing as UTF-8, Nim procs used as JS callbacks, and the ARC-managed handles all hold up across a realistic binding surface. Nothing needed from you here — flagging for visibility.

[savannt]

Progress update — the standard library now runs end-to-end on the JS backend

Since the last update, the linear-memory model has proven out where it matters most: real std/* modules compile and run unchanged through the pipeline (nimony c --bits:32 → lengjs per module → bundle → node), because they lower to the same object / seq / string / hash shapes the backend already covers — no backend-specific code. New end-to-end tests under tests/jsbackend/:

module	exercised end-to-end
std/tables	initTable, []=, getOrDefault, in, len, pairs
std/sets	HashSet incl/excl, intersection, in-place union / difference
std/strutils	split / strip / replace / repeat / case-mapping / starts-endsWith
std/sequtils	map / filter / all / any / count / deduplicate / foldl / mapIt
std/options	Option[T]: some / none / isSome / get / get(default)
std/math	sqrt / floor / ceil / round / pow / trunc / gcd / …

…plus a word-frequency counter combining strutils + tables + algorithm as a real composed program.

Two fixes fell out of composing those into real programs — the kind of thing isolated fixtures miss:

1. Fixed-width 32-bit wrapping arithmetic. A ≤32-bit +/-/* was emitted as a plain JS a op b, which keeps Number's full (~2⁶⁴) result instead of wrapping to the type width. That silently corrupted string-keyed Table/HashSet: a hash h*prime+c that exceeded 2³² in a JS register got truncated when stored as the table slot's 32-bit hcode, so the stored-vs-recomputed hcode fast-reject mismatched and lookups created duplicate buckets (an 8-distinct-word text produced 15 buckets). Now ≤32-bit +/-/* coerce to the exact type width, with * using Math.imul — the correct low-32-bit product; a plain a*b loses bits above 2⁵³.

2. Runtime C-intrinsic coverage. system/stringimpl.nim's word-at-a-time string comparison importcs __builtin_ctzll/__builtin_clzll/__builtin_bswap64, and std/math importcs the libm functions — all called by C name but never defined in the runtime, so string ordering (e.g. a sort comparator) and any math call threw ReferenceError. Added them to runtime.js: the SWAR bit intrinsics as BigInt ops, and libm mapped onto JS Math with the libm-specific cases spelled out (round = half-away-from-zero, fmod = %, copysign/signbit honour -0).

The JS backend suite is at 31/31 green (each test is a real program compiled and run under node, stdout diffed against a golden). Commits dbcd9d4f, a3dd3225, 7dca4d51.

So the linear-memory thesis now covers objects, arrays, strings (SSO + heap), seqs, refs, closures, exceptions and a real slice of the standard library on top — all "just more Nim over the one ArrayBuffer," with the runtime supplying only the handful of C primitives (allocator mmap/munmap, atomics, stdio, libm, these bit intrinsics).

[savannt]

Relocated to nim-lang/jantak — closing.

Per Araq's decision to house the web-targeting backends in their own repo rather than in-tree, the JavaScript backend now lives in nim-lang/jantak, imported in nim-lang/jantak#2 (together with the WebAssembly backend from #2118). Same code, rebased onto jantak and made to consume nimony as a sibling checkout via --path — the same arrangement nativenif (arkham/nifasm/ghast) uses.

It uses only stock nimony NIF APIs; I verified all three plugins (lengjs/lengwasm/jslink) build against a clean nimony master, so nothing from this PR needs to stay here. The in-tree hastur build-wiring in this PR is obsolete (a future nimony PR could instead teach hastur build to compile them from ../jantak, mirroring buildArkham/../nativenif — optional, best after jantak lands).

Thanks for the reviews — carrying the discussion over to jantak.

[savannt]

I've closed this and withdrawn the branch. This code is not being contributed here — it lives in my own repository under my own license. Please do not reopen or publish it. Thanks.