From fb8addd6833a854a721817f234bfe6b725d05c71 Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Sat, 25 Jul 2026 13:08:48 +0200 Subject: [PATCH 01/31] add one-shot memory spec fix workflow --- .github/workflows/fix-memory-overwrite.yml | 42 ++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 .github/workflows/fix-memory-overwrite.yml diff --git a/.github/workflows/fix-memory-overwrite.yml b/.github/workflows/fix-memory-overwrite.yml new file mode 100644 index 0000000..624839e --- /dev/null +++ b/.github/workflows/fix-memory-overwrite.yml @@ -0,0 +1,42 @@ +name: Apply memory spec correction + +on: + push: + branches: + - agent/fix-host-overwrite-reuse + +permissions: + contents: write + +jobs: + apply-fix: + if: github.actor != 'github-actions[bot]' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + ref: agent/fix-host-overwrite-reuse + + - name: Correct host overwrite semantics + shell: python + run: | + from pathlib import Path + + path = Path("spec/memory.md") + text = path.read_text() + old = "Within a scope's arena, allocation is a single frontier bump — no size classes, no free list, no coalescing. Hosts are not reclaimed one at a time: a host that dies or is overwritten mid-scope (§2.2) becomes dead space in the arena until the scope drains. Reclamation is bulk. When the scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps the scope's chunks and every byte the scope held is released at once, with no per-object teardown pass threaded through the exit. Logical destruction timing is unchanged — a host dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); it is the *memory* that is reclaimed together at drain." + new = "Within a scope's arena, allocation is a single frontier bump — no size classes, no free list, no coalescing. A host is a fixed-size storage slot: overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7). Overwriting therefore consumes no additional arena space.\n\nReclamation is bulk. When the scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps the scope's chunks and every byte the scope held is released at once, with no reachability scan or per-object memory-reclamation pass threaded through the exit. Anything that escaped the scope was already promoted into its destination scope's arena (§3.5), so no surviving object remains in the drained arena. Logical destruction timing is unchanged — a host dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); it is the *memory* that is released together at drain." + if text.count(old) != 1: + raise SystemExit("expected §3.2 text not found exactly once") + path.write_text(text.replace(old, new)) + + - name: Remove one-shot workflow + run: rm .github/workflows/fix-memory-overwrite.yml + + - name: Commit correction + run: | + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add spec/memory.md .github/workflows/fix-memory-overwrite.yml + git commit -m "clarify host overwrite slot reuse" + git push From 78a700db62613f3d6fe85286520cc032281f77a5 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Sat, 25 Jul 2026 11:08:57 +0000 Subject: [PATCH 02/31] clarify host overwrite slot reuse --- .github/workflows/fix-memory-overwrite.yml | 42 ---------------------- spec/memory.md | 4 ++- 2 files changed, 3 insertions(+), 43 deletions(-) delete mode 100644 .github/workflows/fix-memory-overwrite.yml diff --git a/.github/workflows/fix-memory-overwrite.yml b/.github/workflows/fix-memory-overwrite.yml deleted file mode 100644 index 624839e..0000000 --- a/.github/workflows/fix-memory-overwrite.yml +++ /dev/null @@ -1,42 +0,0 @@ -name: Apply memory spec correction - -on: - push: - branches: - - agent/fix-host-overwrite-reuse - -permissions: - contents: write - -jobs: - apply-fix: - if: github.actor != 'github-actions[bot]' - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - with: - ref: agent/fix-host-overwrite-reuse - - - name: Correct host overwrite semantics - shell: python - run: | - from pathlib import Path - - path = Path("spec/memory.md") - text = path.read_text() - old = "Within a scope's arena, allocation is a single frontier bump — no size classes, no free list, no coalescing. Hosts are not reclaimed one at a time: a host that dies or is overwritten mid-scope (§2.2) becomes dead space in the arena until the scope drains. Reclamation is bulk. When the scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps the scope's chunks and every byte the scope held is released at once, with no per-object teardown pass threaded through the exit. Logical destruction timing is unchanged — a host dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); it is the *memory* that is reclaimed together at drain." - new = "Within a scope's arena, allocation is a single frontier bump — no size classes, no free list, no coalescing. A host is a fixed-size storage slot: overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7). Overwriting therefore consumes no additional arena space.\n\nReclamation is bulk. When the scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps the scope's chunks and every byte the scope held is released at once, with no reachability scan or per-object memory-reclamation pass threaded through the exit. Anything that escaped the scope was already promoted into its destination scope's arena (§3.5), so no surviving object remains in the drained arena. Logical destruction timing is unchanged — a host dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); it is the *memory* that is released together at drain." - if text.count(old) != 1: - raise SystemExit("expected §3.2 text not found exactly once") - path.write_text(text.replace(old, new)) - - - name: Remove one-shot workflow - run: rm .github/workflows/fix-memory-overwrite.yml - - - name: Commit correction - run: | - git config user.name "github-actions[bot]" - git config user.email "41898282+github-actions[bot]@users.noreply.github.com" - git add spec/memory.md .github/workflows/fix-memory-overwrite.yml - git commit -m "clarify host overwrite slot reuse" - git push diff --git a/spec/memory.md b/spec/memory.md index f471d75..e59ead8 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -272,7 +272,9 @@ Allocations are 8-byte aligned, so the low bits count 8-byte words: a 1 MiB chun > **Story:** [`stories/memory.md`](../stories/memory.md#the-last-table-problem-and-the-segmented-offset) — "The last table problem, and the segmented offset". ### 3.2 Allocation is a bump; teardown is an unmap -Within a scope's arena, allocation is a single frontier bump — no size classes, no free list, no coalescing. Hosts are not reclaimed one at a time: a host that dies or is overwritten mid-scope (§2.2) becomes dead space in the arena until the scope drains. Reclamation is bulk. When the scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps the scope's chunks and every byte the scope held is released at once, with no per-object teardown pass threaded through the exit. Logical destruction timing is unchanged — a host dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); it is the *memory* that is reclaimed together at drain. +Within a scope's arena, allocation is a single frontier bump — no size classes, no free list, no coalescing. A host is a fixed-size storage slot: overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7). Overwriting therefore consumes no additional arena space. + +Reclamation is bulk. When the scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps the scope's chunks and every byte the scope held is released at once, with no reachability scan or per-object memory-reclamation pass threaded through the exit. Anything that escaped the scope was already promoted into its destination scope's arena (§3.5), so no surviving object remains in the drained arena. Logical destruction timing is unchanged — a host dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); it is the *memory* that is released together at drain. > **Story:** [`stories/memory.md`](../stories/memory.md#when-the-free-stacks-fragment-and-the-arena-takes-the-scope) — "When the free stacks fragment, and the arena takes the scope". From 4da40a7023fbb174b6c20a9e96d23272dcfd23da Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Sat, 25 Jul 2026 14:50:30 +0200 Subject: [PATCH 03/31] Add one-shot memory layout updater --- .../workflows/apply-memory-region-update.yml | 249 ++++++++++++++++++ 1 file changed, 249 insertions(+) create mode 100644 .github/workflows/apply-memory-region-update.yml diff --git a/.github/workflows/apply-memory-region-update.yml b/.github/workflows/apply-memory-region-update.yml new file mode 100644 index 0000000..8aeb6b5 --- /dev/null +++ b/.github/workflows/apply-memory-region-update.yml @@ -0,0 +1,249 @@ +name: Apply memory region update + +on: + push: + branches: + - agent/fix-host-overwrite-reuse + +permissions: + contents: write + +jobs: + update: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + ref: agent/fix-host-overwrite-reuse + fetch-depth: 0 + + - name: Update memory specification and story + shell: bash + run: | + python3 <<'PY' + from pathlib import Path + import re + + def replace_once(text: str, pattern: str, replacement: str, label: str) -> str: + updated, count = re.subn(pattern, replacement, text, count=1, flags=re.S) + if count != 1: + raise RuntimeError(f"expected one replacement for {label}, got {count}") + return updated + + spec_path = Path("spec/memory.md") + spec = spec_path.read_text() + + spec = spec.replace( + "- **`Arena placement`.** A reference-type instance is bump-allocated in the arena of the scope that creates it, and is copied into a parent arena only if it escapes that scope (see §3.5).", + "- **`Regioned arena placement`.** Every scope owns separate fixed-size, dynamic-backing-store, and anchor-cell regions. Statically sized storage is placed inline in the fixed-size region; resizable data uses the dynamic region (see §3).", + ) + + arena_sections = r'''### 3.1 Scope arenas and segmented offsets + The runtime does not reserve one flat region. Each lexical scope owns an **arena** made from three independent allocation regions: + + - The **fixed-size region** stores materialized value-type slots, statically sized reference-type hosts, and the fixed-size handles of dynamic core types. + - The **dynamic region** stores the resizable backing stores behind handles such as `List` and `String`. + - The **anchor-cell region** stores the scope-local anchor cells created for tethered hosts (§4.1). + + Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots, dynamic backing stores, and anchor cells never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. Growing one region never copies or relocates allocations in another. + + ```text + one scope arena + + fixed-size region dynamic region anchor-cell region + ───────────────── ────────────── ────────────────── + [fixed chunk] [dynamic chunk] [anchor chunk] + [fixed chunk] → ... [dynamic chunk] → ... [anchor chunk] → ... + ``` + + The chains are lazy and independent. A scope that uses no dynamic backing store maps no dynamic chunk; a scope that never creates a guest maps no anchor-cell chunk. When the scope drains, every chunk belonging to all three regions is unmapped together. The compiler may optimize away or coalesce physically unobservable storage, but it **MUST** preserve region exclusivity, lifetime, and drain behavior. + + All three regions draw chunk ids from the same chunk directory, so every in-arena location uses the same **`u32` segmented offset**. The `u32` splits into two fields: + + ``` + u32 segmented offset + ┌───────────────┬─────────────────────────┐ + │ chunk id │ in-chunk word offset │ + │ (high bits) │ (low bits) │ + └───────────────┴─────────────────────────┘ + ``` + + Allocations are at least 8-byte aligned, so the low bits count 8-byte words: a 1 MiB chunk holds 2¹⁷ words, so **17 low bits** address any slot in a chunk and the remaining **15 high bits** select one of up to 32768 live chunks — a reach of 32 GiB. The chunk directory maps a chunk id to the chunk's native base address, so an address is materialized only at use as `directory[chunk id] + word offset × 8`. + + Tethers (§4.2), per-host backpointers (§4.2), anchor cells (§4.1), dynamic handles, and size-stack entries (§3.2) use segmented offsets. The value `0` — chunk `0`, word `0` — is the *untethered* sentinel. It costs no reserved memory because anchor cells are allocated only in the anchor-cell region, which never contains that location. Fixed-size payloads may occupy offset `0`. + + > **Story:** [`stories/memory.md`](../stories/memory.md#the-last-table-problem-and-the-segmented-offset) — "The last table problem, and the segmented offset". + + ### 3.2 Allocation is a bump; teardown is an unmap + The fixed-size and anchor-cell regions are pure bump allocators: allocation advances the region's frontier, with no size classes, free lists, or coalescing. A host is a fixed-size storage slot, so overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7); it consumes no new arena space. + + The dynamic region adds exact-size reuse on top of its bump frontier. Dynamic blocks use power-of-two byte sizes beginning at **128 bytes**. Each scope maintains one LIFO **size stack** for every block size that has become reusable. To allocate a dynamic block of size `S`, the runtime first pops `size_stack[S]`; only when that stack is empty does it bump the dynamic frontier, mapping more dynamic chunks as needed. It never satisfies a request from a different size stack and never coalesces neighbouring free blocks. + + Returning a dynamic block pushes its segmented offset onto the stack for that exact byte size. These stacks are shared by all dynamic types in the scope: a 128-byte block previously used by a `List` may later hold string bytes or another list's elements. The stacks affect allocation within the scope only; they require no per-object reclamation when the scope drains. + + Reclamation remains bulk. When the scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps every fixed-size, dynamic, and anchor-cell chunk owned by the scope, with no reachability scan or per-object memory-reclamation pass. Anything that escaped was already placed in storage whose lifetime covers its destination host (§3.5). Logical destruction timing is unchanged — a host dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); the underlying region memory is released together at drain. + + > **Story:** [`stories/memory.md`](../stories/memory.md#when-the-free-stacks-fragment-and-the-arena-takes-the-scope) — "When the free stacks fragment, and the arena takes the scope". + + ''' + spec = replace_once( + spec, + r"### 3\.1 Scope arenas and segmented offsets\n.*?(?=### 3\.3 Value and reference layout follow declaration order)", + arena_sections, + "spec sections 3.1-3.2", + ) + + layout_section = r'''### 3.3 Value and reference layout follow declaration order + Fields are laid out in declaration order. Value types are stored inline. A statically sized reference-type instance is also stored inline in a fixed-size host slot, so value-type slots and reference-type host slots may sit directly beside each other in the fixed-size region. Reference types differ by identity and hosting semantics, not by requiring a separate indirect allocation. + + A reference-type instance carries one `u32` backpointer field of anchor metadata (a segmented offset, §4.2) that remains `0` until the instance is first tethered. A dynamic core type such as `List` occupies a fixed-size handle inline in the same region; only the backing store named by that handle occupies the dynamic region (§3.6). + + ''' + spec = replace_once( + spec, + r"### 3\.3 Value and reference layout follow declaration order\n.*?(?=### 3\.4 Booleans may be packed)", + layout_section, + "spec section 3.3", + ) + + placement_and_dynamic = r'''### 3.5 Statically sized storage uses the fixed-size region + Placement is an implementation decision, not a language-visible property. The arena model places every materialized, statically sized scope slot — value-type storage, a reference-type host, or a dynamic type's fixed-size handle — inline in that scope's fixed-size region. The compiler may keep an unobservable value in registers or otherwise optimize its physical placement, but reference types do not require a separate heap allocation merely because they carry identity. + + When a reference-type instance is rehosted into a longer-lived destination, its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). A dynamic backing store is semantically owned by the current host. The compiler **MUST** place that store in a dynamic region whose lifetime covers every destination into which the handle can be rehosted, so rehosting transfers ownership of the same backing store without copying it. Only growth of the dynamic value may relocate the backing store (§3.6). + + Placement never changes observable semantics: destruction stays deterministic (see [`lifetimes.md`](lifetimes.md) §2), and tethers resolve identically regardless of physical placement (§4), because a tether follows the host's anchor rather than a fixed address. + + > **Story:** [`stories/memory.md`](../stories/memory.md#the-value-world-stays-closed-and-placement-stays-the-compilers) — "The value world stays closed, and placement stays the compiler's". + + ### 3.6 Handle-typed core reference types have fixed footprint + The core dynamically-sized reference types — `List`, `String`, and similar types — are represented as fixed-size **handles**. A handle records the backing store's segmented offset and the metadata needed by the type, such as length and size class. The handle occupies a statically known footprint inline in the fixed-size region; its resizable backing store is a separate allocation in the dynamic region. + + A type that contains a handle-typed field therefore stays statically sized: + + ```zane + type Inventory = #struct { + items List; // fixed-size handle inline; elements in the dynamic region + count Int; + } + ``` + + Dynamic block sizes are byte-based rather than element-type-based. A new list starts with a **128-byte block** — equivalent to sixteen 64-bit words — regardless of `T`. Its element capacity is `floor(block_bytes / stride(T))`. If one element does not fit in 128 bytes, the initial block is the smallest power-of-two block that can hold one element. Keeping the byte classes common allows blocks to be reused across lists with different element types and across other dynamic core types. + + A list grows according to the following rules: + + 1. When its capacity is exhausted, the requested block size is exactly twice its current block size. + 2. The allocator first checks the size stack for that doubled size. If a block is available, it is popped and the live elements are relocated into it. + 3. If that stack is empty and the current backing store is the dynamic frontier allocation with enough contiguous room to double, the frontier is bumped by the additional bytes and the store grows in place. + 4. Otherwise, the doubled block is allocated by bumping the dynamic frontier, mapping more dynamic chunks as needed, and the live elements are relocated into it. + 5. After relocation, the handle's backing-store offset and size class are updated and the old block's offset is pushed onto the size stack for its old byte size. + + Relocation moves or copies elements according to their type's ordinary move rules; the old block becomes reusable only after its previous occupants are no longer live. Guests to the list remain valid because they reach the list's host, whose fixed-size handle now names the current backing store. + + Dynamic chunks and all power-of-two blocks begin at cache-line-aligned addresses. Because the minimum block is 128 bytes and every larger block doubles, both frontier allocations and reused blocks preserve cache-line alignment without mixing backing stores into fixed-size chunks. + + > **Story:** [`stories/memory.md`](../stories/memory.md#the-sentinel-that-costs-nothing-and-the-buffer-that-wanted-a-line) — "The sentinel that costs nothing, and the buffer that wanted a line". + + ''' + spec = replace_once( + spec, + r"### 3\.5 Reference-type instances are placed in their scope's arena\n.*?(?=### 3\.7 Moving a value reuses the destination slot)", + placement_and_dynamic, + "spec sections 3.5-3.6", + ) + + spec = spec.replace( + "Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. A move into a higher scope copies the inline bytes into the destination scope's arena — a promotion (§3.5). Because handle-typed fields (§3.6) keep the moved footprint small, a move relocates only the inline bytes — a handle's backing store never moves. If the moved value is tethered, the move also updates its one anchor cell (§4.5), never the tethers themselves.", + "Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. A move into a higher scope copies the inline bytes into the destination scope's fixed-size region — a promotion (§3.5). Because handle-typed fields (§3.6) keep the moved footprint small, rehosting copies only the handle and transfers ownership of the same backing store; rehosting itself never relocates that store. A dynamic store changes address only through the growth procedure in §3.6. If the moved value is tethered, the move also updates its one anchor cell (§4.5), never the tethers themselves.", + ) + + anchor_section = r'''### 4.1 The anchor cell + Tethers are tracked through per-host **anchor cells** rather than one shared table. An anchor cell is a single **`u32`** holding the current segmented offset (§3.1) of one hosted object; it stores nothing else. + + Anchor storage is **scope-local**, never global. Each scope owns a dedicated anchor-cell region — a separate lazy chunk chain from both its fixed-size and dynamic regions. A scope that never creates a guest allocates no anchor chunk. The first tether to a host bump-allocates its cell in that scope's anchor region (§4.3); minting another cell is one bump and never resizes a monolithic table. + + Keeping cells out of the other two streams preserves dense fixed-size layout and prevents dynamic-buffer history from affecting anchor placement. The region remains compact and heavily reused while live, and all of its chunks disappear with the scope in the same bulk unmap as the other regions. + + > **Story:** [`stories/memory.md`](../stories/memory.md#where-the-cells-live-and-the-scan-that-pays-for-them) — "Where the cells live, and the scan that pays for them". + + ''' + spec = replace_once( + spec, + r"### 4\.1 The anchor cell\n.*?(?=### 4\.2 Tethers are segmented offsets, not pointers)", + anchor_section, + "spec section 4.1", + ) + + spec_path.write_text(spec) + + story_path = Path("stories/memory.md") + story = story_path.read_text() + + placement_story = r'''The opened door is placement. Identity does not require a second heap object: a materialized value slot and a statically sized reference-type host can sit directly beside each other in the scope's fixed-size region. The reference type differs because it carries hosting identity and a backpointer, not because its bytes must be indirect. Dynamic size is kept from leaking upward in the same way: `List`, `String`, and similar types are fixed-size handles inline with the ordinary slots, while only their backing stores occupy the scope's separate dynamic region ([§3.3](../spec/memory.md#33-value-and-reference-layout-follow-declaration-order), [§3.6](../spec/memory.md#36-handle-typed-core-reference-types-have-fixed-footprint)). That separation is what lets a type containing a list remain statically sized and what prevents a growing buffer from shifting any neighbouring value or host. + + Placement remains unobservable. The compiler may keep a value in a register or choose the lifetime region that already covers a later rehosting, provided destruction and guest resolution are unchanged. In particular, a dynamic backing store belongs semantically to its current host but is placed in a dynamic region that outlives every host it can be moved into. Rehosting therefore copies only the fixed-size handle and transfers ownership of the same store; only the container's own growth operation relocates its elements. The cost is the same boundary the earlier design already accepted: raw addresses and layout introspection cannot be language-visible facts if the compiler is free to make these choices. + + ''' + story = replace_once( + story, + r"The opened door is placement\..*?(?=## The kinds collapse into one axis, and `this` becomes a borrow)", + placement_story, + "story placement chapter ending", + ) + + allocator_story = r'''## When the free stacks fragment, and the arena takes the scope + + The allocator first used size-indexed free stacks for everything: round a request, pop the matching stack, and bump a global frontier only when that stack was empty. The operation was O(1), but the global policy let each class hoard memory from every other class. A program could run out of 32-byte slots while 16-byte and 64-byte stacks held abundant space, and borrowing a larger slot merely exchanged external fragmentation for internal waste. + + Scope arenas removed that failure from ordinary storage. Fixed-size values and hosts have stable slots whose lifetimes already match a lexical scope, so their region needs only a frontier; anchor cells have the same append-and-drain shape in their own region. Neither benefits from individual reuse. Overwriting a host reuses its existing slot, and draining the scope unmaps both regions whole. + + Resizable backing stores are the exception the pure bump story had hidden. A list that doubles from 128 to 256 bytes may abandon the old 128-byte block while the scope continues for a long time. Leaving every old buffer stranded until drain makes repeated growth consume memory monotonically, while mixing those buffers among fixed-size slots makes the dense layout depend on container history. The answer was not to restore one global allocator, but to give each scope a third, dynamic region and confine exact-size reuse to it ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets)). Fixed-size, dynamic, and anchor allocations use distinct lazy chunk chains; no chunk contains more than one region, and a scope that never creates dynamic data pays for no dynamic chunk. + + The dynamic region keeps a LIFO stack for each power-of-two byte size, beginning at 128 bytes. Every request checks its exact size stack first and bumps the dynamic frontier only when that stack is empty. There is still no coalescing and no borrowing from a neighbouring class. The important difference from the rejected allocator is scope: these stacks recycle only the backing stores whose churn occurs inside one lifetime, and every stack and chunk disappears together when that scope drains. + + Byte size, rather than element count, defines the classes. A new list asks for 128 bytes — sixteen 64-bit words — whether it stores bytes, integers, or larger records; its capacity is whatever number of `T` elements fit. If one element is larger, the first block is simply the smallest power of two that can contain it. The common byte classes are what make reuse broad: a block returned by one `List` can serve another element type, a string, or any other compatible dynamic core value. + + Growth doubles the block size. The allocator first looks for a reusable block of the doubled size. If none exists and the current buffer is the frontier allocation with room behind it, the frontier advances and the list grows in place. Otherwise a doubled block is bump-allocated at the frontier, the elements move, the handle changes its offset, and the old block is pushed onto its old-size stack. The free stack is therefore the first choice and the frontier the fallback — including the in-place case, which is just a frontier bump that happens not to change the address. + + This reintroduces size-class fragmentation, but only in the one region whose objects genuinely change size, with powers of two that match the doubling policy and a fixed 128-byte floor that maximizes cross-type reuse. The fixed-size and anchor regions retain the arena's strongest property: no free lists, no holes from reassignment, and one bulk unmap at scope drain. The arena did not abolish free stacks everywhere; it confined them to the place where their reuse is worth their fragmentation. + + ''' + story = replace_once( + story, + r"## When the free stacks fragment, and the arena takes the scope\n.*?(?=## The last table problem, and the segmented offset)", + allocator_story, + "story allocator chapter", + ) + + story = story.replace( + "So the fix was not to leave the arena but to stop mixing two things inside it. Cells get their own **region** — a separate chunk chain in the same scope arena, addressed by the same segmented offsets, bump-allocated and bulk-unmapped exactly like the payload region, just not braided through it ([`memory.md` §4.1](https://github.com/zane-lang/spec/blob/2caab48a9f4bafda32d67c3bb902908aea1813ab/spec/memory.md#41-the-anchor-cell)). Payload scans go back to full density because the payload stream holds only payloads, and a payload's position no longer depends on what was tethered before it, so the alignment the compiler wants is the compiler's again. The cells, gathered into their own region, are still compact and still hot — a tether resolution reads a cell from a dense, cache-resident run, just not from the same line as the payload it will then visit. It is arenas the whole way down: two streams, one drain.", + "So the fix was not to leave the arena but to separate the things with different allocation behaviour. The scope now has three regions: fixed-size slots, dynamic backing stores, and anchor cells, each with its own lazy chunk chain and the same segmented-offset directory ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets), [§4.1](../spec/memory.md#41-the-anchor-cell)). Fixed-size scans stay dense because neither cells nor resizable buffers are braided through them; dynamic growth can recycle abandoned blocks without punching holes among hosts; and cells remain compact and hot in their own run. It is arenas the whole way down: three streams, one drain.", + ) + + sentinel_story = r'''## The sentinel that costs nothing, and the buffer that wanted a line + + Separating anchor cells made the zero sentinel free. A tether or backpointer value of `0` means “untethered,” but those values name only anchor cells, and anchor cells live in their own region where chunk `0`, word `0` is never a cell. The fixed-size region may therefore begin at offset zero with no reserved gap ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets)). The sentinel is protected by region identity, not by wasting the first slot. + + Dynamic buffers then asked for a stronger alignment guarantee. They are streamed, copied during growth, and commonly contain elements whose access pattern spans many cache lines. Rather than align an arbitrary buffer after an arbitrary fixed-size payload, the dedicated dynamic region starts from its own chunk boundary and allocates only power-of-two blocks beginning at 128 bytes. Every frontier block is therefore cache-line aligned, and every block later popped from a size stack preserves that alignment ([`memory.md` §3.6](../spec/memory.md#36-handle-typed-core-reference-types-have-fixed-footprint)). + + This is another benefit of refusing to mix regions on one chunk. Small inline values keep ordinary 8-byte alignment without paying cache-line padding, while dynamic backing stores receive cache-line alignment structurally. A scope with no dynamic values maps no dynamic chunk, so the stronger alignment carries no idle page cost. Zero remains a usable fixed-size address, and buffers get the geometry they need without making neighbouring object placement depend on allocation history. + + ''' + story = replace_once( + story, + r"## The sentinel that costs nothing, and the buffer that wanted a line\n.*?(?=## Two vocabularies: host and guest above anchor and tether)", + sentinel_story, + "story sentinel chapter", + ) + + story_path.write_text(story) + PY + + - name: Commit generated update + shell: bash + run: | + rm .github/workflows/apply-memory-region-update.yml + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add spec/memory.md stories/memory.md .github/workflows/apply-memory-region-update.yml + git commit -m "Specify arena regions and dynamic backing-store reuse" + git push From 905cc5bb9bcadd31ccd8641ea7ddf4de8e1b9b21 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Sat, 25 Jul 2026 12:50:38 +0000 Subject: [PATCH 04/31] Specify arena regions and dynamic backing-store reuse --- .../workflows/apply-memory-region-update.yml | 249 ------------------ spec/memory.md | 89 ++++--- stories/memory.md | 34 +-- 3 files changed, 65 insertions(+), 307 deletions(-) delete mode 100644 .github/workflows/apply-memory-region-update.yml diff --git a/.github/workflows/apply-memory-region-update.yml b/.github/workflows/apply-memory-region-update.yml deleted file mode 100644 index 8aeb6b5..0000000 --- a/.github/workflows/apply-memory-region-update.yml +++ /dev/null @@ -1,249 +0,0 @@ -name: Apply memory region update - -on: - push: - branches: - - agent/fix-host-overwrite-reuse - -permissions: - contents: write - -jobs: - update: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - with: - ref: agent/fix-host-overwrite-reuse - fetch-depth: 0 - - - name: Update memory specification and story - shell: bash - run: | - python3 <<'PY' - from pathlib import Path - import re - - def replace_once(text: str, pattern: str, replacement: str, label: str) -> str: - updated, count = re.subn(pattern, replacement, text, count=1, flags=re.S) - if count != 1: - raise RuntimeError(f"expected one replacement for {label}, got {count}") - return updated - - spec_path = Path("spec/memory.md") - spec = spec_path.read_text() - - spec = spec.replace( - "- **`Arena placement`.** A reference-type instance is bump-allocated in the arena of the scope that creates it, and is copied into a parent arena only if it escapes that scope (see §3.5).", - "- **`Regioned arena placement`.** Every scope owns separate fixed-size, dynamic-backing-store, and anchor-cell regions. Statically sized storage is placed inline in the fixed-size region; resizable data uses the dynamic region (see §3).", - ) - - arena_sections = r'''### 3.1 Scope arenas and segmented offsets - The runtime does not reserve one flat region. Each lexical scope owns an **arena** made from three independent allocation regions: - - - The **fixed-size region** stores materialized value-type slots, statically sized reference-type hosts, and the fixed-size handles of dynamic core types. - - The **dynamic region** stores the resizable backing stores behind handles such as `List` and `String`. - - The **anchor-cell region** stores the scope-local anchor cells created for tethered hosts (§4.1). - - Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots, dynamic backing stores, and anchor cells never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. Growing one region never copies or relocates allocations in another. - - ```text - one scope arena - - fixed-size region dynamic region anchor-cell region - ───────────────── ────────────── ────────────────── - [fixed chunk] [dynamic chunk] [anchor chunk] - [fixed chunk] → ... [dynamic chunk] → ... [anchor chunk] → ... - ``` - - The chains are lazy and independent. A scope that uses no dynamic backing store maps no dynamic chunk; a scope that never creates a guest maps no anchor-cell chunk. When the scope drains, every chunk belonging to all three regions is unmapped together. The compiler may optimize away or coalesce physically unobservable storage, but it **MUST** preserve region exclusivity, lifetime, and drain behavior. - - All three regions draw chunk ids from the same chunk directory, so every in-arena location uses the same **`u32` segmented offset**. The `u32` splits into two fields: - - ``` - u32 segmented offset - ┌───────────────┬─────────────────────────┐ - │ chunk id │ in-chunk word offset │ - │ (high bits) │ (low bits) │ - └───────────────┴─────────────────────────┘ - ``` - - Allocations are at least 8-byte aligned, so the low bits count 8-byte words: a 1 MiB chunk holds 2¹⁷ words, so **17 low bits** address any slot in a chunk and the remaining **15 high bits** select one of up to 32768 live chunks — a reach of 32 GiB. The chunk directory maps a chunk id to the chunk's native base address, so an address is materialized only at use as `directory[chunk id] + word offset × 8`. - - Tethers (§4.2), per-host backpointers (§4.2), anchor cells (§4.1), dynamic handles, and size-stack entries (§3.2) use segmented offsets. The value `0` — chunk `0`, word `0` — is the *untethered* sentinel. It costs no reserved memory because anchor cells are allocated only in the anchor-cell region, which never contains that location. Fixed-size payloads may occupy offset `0`. - - > **Story:** [`stories/memory.md`](../stories/memory.md#the-last-table-problem-and-the-segmented-offset) — "The last table problem, and the segmented offset". - - ### 3.2 Allocation is a bump; teardown is an unmap - The fixed-size and anchor-cell regions are pure bump allocators: allocation advances the region's frontier, with no size classes, free lists, or coalescing. A host is a fixed-size storage slot, so overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7); it consumes no new arena space. - - The dynamic region adds exact-size reuse on top of its bump frontier. Dynamic blocks use power-of-two byte sizes beginning at **128 bytes**. Each scope maintains one LIFO **size stack** for every block size that has become reusable. To allocate a dynamic block of size `S`, the runtime first pops `size_stack[S]`; only when that stack is empty does it bump the dynamic frontier, mapping more dynamic chunks as needed. It never satisfies a request from a different size stack and never coalesces neighbouring free blocks. - - Returning a dynamic block pushes its segmented offset onto the stack for that exact byte size. These stacks are shared by all dynamic types in the scope: a 128-byte block previously used by a `List` may later hold string bytes or another list's elements. The stacks affect allocation within the scope only; they require no per-object reclamation when the scope drains. - - Reclamation remains bulk. When the scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps every fixed-size, dynamic, and anchor-cell chunk owned by the scope, with no reachability scan or per-object memory-reclamation pass. Anything that escaped was already placed in storage whose lifetime covers its destination host (§3.5). Logical destruction timing is unchanged — a host dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); the underlying region memory is released together at drain. - - > **Story:** [`stories/memory.md`](../stories/memory.md#when-the-free-stacks-fragment-and-the-arena-takes-the-scope) — "When the free stacks fragment, and the arena takes the scope". - - ''' - spec = replace_once( - spec, - r"### 3\.1 Scope arenas and segmented offsets\n.*?(?=### 3\.3 Value and reference layout follow declaration order)", - arena_sections, - "spec sections 3.1-3.2", - ) - - layout_section = r'''### 3.3 Value and reference layout follow declaration order - Fields are laid out in declaration order. Value types are stored inline. A statically sized reference-type instance is also stored inline in a fixed-size host slot, so value-type slots and reference-type host slots may sit directly beside each other in the fixed-size region. Reference types differ by identity and hosting semantics, not by requiring a separate indirect allocation. - - A reference-type instance carries one `u32` backpointer field of anchor metadata (a segmented offset, §4.2) that remains `0` until the instance is first tethered. A dynamic core type such as `List` occupies a fixed-size handle inline in the same region; only the backing store named by that handle occupies the dynamic region (§3.6). - - ''' - spec = replace_once( - spec, - r"### 3\.3 Value and reference layout follow declaration order\n.*?(?=### 3\.4 Booleans may be packed)", - layout_section, - "spec section 3.3", - ) - - placement_and_dynamic = r'''### 3.5 Statically sized storage uses the fixed-size region - Placement is an implementation decision, not a language-visible property. The arena model places every materialized, statically sized scope slot — value-type storage, a reference-type host, or a dynamic type's fixed-size handle — inline in that scope's fixed-size region. The compiler may keep an unobservable value in registers or otherwise optimize its physical placement, but reference types do not require a separate heap allocation merely because they carry identity. - - When a reference-type instance is rehosted into a longer-lived destination, its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). A dynamic backing store is semantically owned by the current host. The compiler **MUST** place that store in a dynamic region whose lifetime covers every destination into which the handle can be rehosted, so rehosting transfers ownership of the same backing store without copying it. Only growth of the dynamic value may relocate the backing store (§3.6). - - Placement never changes observable semantics: destruction stays deterministic (see [`lifetimes.md`](lifetimes.md) §2), and tethers resolve identically regardless of physical placement (§4), because a tether follows the host's anchor rather than a fixed address. - - > **Story:** [`stories/memory.md`](../stories/memory.md#the-value-world-stays-closed-and-placement-stays-the-compilers) — "The value world stays closed, and placement stays the compiler's". - - ### 3.6 Handle-typed core reference types have fixed footprint - The core dynamically-sized reference types — `List`, `String`, and similar types — are represented as fixed-size **handles**. A handle records the backing store's segmented offset and the metadata needed by the type, such as length and size class. The handle occupies a statically known footprint inline in the fixed-size region; its resizable backing store is a separate allocation in the dynamic region. - - A type that contains a handle-typed field therefore stays statically sized: - - ```zane - type Inventory = #struct { - items List; // fixed-size handle inline; elements in the dynamic region - count Int; - } - ``` - - Dynamic block sizes are byte-based rather than element-type-based. A new list starts with a **128-byte block** — equivalent to sixteen 64-bit words — regardless of `T`. Its element capacity is `floor(block_bytes / stride(T))`. If one element does not fit in 128 bytes, the initial block is the smallest power-of-two block that can hold one element. Keeping the byte classes common allows blocks to be reused across lists with different element types and across other dynamic core types. - - A list grows according to the following rules: - - 1. When its capacity is exhausted, the requested block size is exactly twice its current block size. - 2. The allocator first checks the size stack for that doubled size. If a block is available, it is popped and the live elements are relocated into it. - 3. If that stack is empty and the current backing store is the dynamic frontier allocation with enough contiguous room to double, the frontier is bumped by the additional bytes and the store grows in place. - 4. Otherwise, the doubled block is allocated by bumping the dynamic frontier, mapping more dynamic chunks as needed, and the live elements are relocated into it. - 5. After relocation, the handle's backing-store offset and size class are updated and the old block's offset is pushed onto the size stack for its old byte size. - - Relocation moves or copies elements according to their type's ordinary move rules; the old block becomes reusable only after its previous occupants are no longer live. Guests to the list remain valid because they reach the list's host, whose fixed-size handle now names the current backing store. - - Dynamic chunks and all power-of-two blocks begin at cache-line-aligned addresses. Because the minimum block is 128 bytes and every larger block doubles, both frontier allocations and reused blocks preserve cache-line alignment without mixing backing stores into fixed-size chunks. - - > **Story:** [`stories/memory.md`](../stories/memory.md#the-sentinel-that-costs-nothing-and-the-buffer-that-wanted-a-line) — "The sentinel that costs nothing, and the buffer that wanted a line". - - ''' - spec = replace_once( - spec, - r"### 3\.5 Reference-type instances are placed in their scope's arena\n.*?(?=### 3\.7 Moving a value reuses the destination slot)", - placement_and_dynamic, - "spec sections 3.5-3.6", - ) - - spec = spec.replace( - "Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. A move into a higher scope copies the inline bytes into the destination scope's arena — a promotion (§3.5). Because handle-typed fields (§3.6) keep the moved footprint small, a move relocates only the inline bytes — a handle's backing store never moves. If the moved value is tethered, the move also updates its one anchor cell (§4.5), never the tethers themselves.", - "Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. A move into a higher scope copies the inline bytes into the destination scope's fixed-size region — a promotion (§3.5). Because handle-typed fields (§3.6) keep the moved footprint small, rehosting copies only the handle and transfers ownership of the same backing store; rehosting itself never relocates that store. A dynamic store changes address only through the growth procedure in §3.6. If the moved value is tethered, the move also updates its one anchor cell (§4.5), never the tethers themselves.", - ) - - anchor_section = r'''### 4.1 The anchor cell - Tethers are tracked through per-host **anchor cells** rather than one shared table. An anchor cell is a single **`u32`** holding the current segmented offset (§3.1) of one hosted object; it stores nothing else. - - Anchor storage is **scope-local**, never global. Each scope owns a dedicated anchor-cell region — a separate lazy chunk chain from both its fixed-size and dynamic regions. A scope that never creates a guest allocates no anchor chunk. The first tether to a host bump-allocates its cell in that scope's anchor region (§4.3); minting another cell is one bump and never resizes a monolithic table. - - Keeping cells out of the other two streams preserves dense fixed-size layout and prevents dynamic-buffer history from affecting anchor placement. The region remains compact and heavily reused while live, and all of its chunks disappear with the scope in the same bulk unmap as the other regions. - - > **Story:** [`stories/memory.md`](../stories/memory.md#where-the-cells-live-and-the-scan-that-pays-for-them) — "Where the cells live, and the scan that pays for them". - - ''' - spec = replace_once( - spec, - r"### 4\.1 The anchor cell\n.*?(?=### 4\.2 Tethers are segmented offsets, not pointers)", - anchor_section, - "spec section 4.1", - ) - - spec_path.write_text(spec) - - story_path = Path("stories/memory.md") - story = story_path.read_text() - - placement_story = r'''The opened door is placement. Identity does not require a second heap object: a materialized value slot and a statically sized reference-type host can sit directly beside each other in the scope's fixed-size region. The reference type differs because it carries hosting identity and a backpointer, not because its bytes must be indirect. Dynamic size is kept from leaking upward in the same way: `List`, `String`, and similar types are fixed-size handles inline with the ordinary slots, while only their backing stores occupy the scope's separate dynamic region ([§3.3](../spec/memory.md#33-value-and-reference-layout-follow-declaration-order), [§3.6](../spec/memory.md#36-handle-typed-core-reference-types-have-fixed-footprint)). That separation is what lets a type containing a list remain statically sized and what prevents a growing buffer from shifting any neighbouring value or host. - - Placement remains unobservable. The compiler may keep a value in a register or choose the lifetime region that already covers a later rehosting, provided destruction and guest resolution are unchanged. In particular, a dynamic backing store belongs semantically to its current host but is placed in a dynamic region that outlives every host it can be moved into. Rehosting therefore copies only the fixed-size handle and transfers ownership of the same store; only the container's own growth operation relocates its elements. The cost is the same boundary the earlier design already accepted: raw addresses and layout introspection cannot be language-visible facts if the compiler is free to make these choices. - - ''' - story = replace_once( - story, - r"The opened door is placement\..*?(?=## The kinds collapse into one axis, and `this` becomes a borrow)", - placement_story, - "story placement chapter ending", - ) - - allocator_story = r'''## When the free stacks fragment, and the arena takes the scope - - The allocator first used size-indexed free stacks for everything: round a request, pop the matching stack, and bump a global frontier only when that stack was empty. The operation was O(1), but the global policy let each class hoard memory from every other class. A program could run out of 32-byte slots while 16-byte and 64-byte stacks held abundant space, and borrowing a larger slot merely exchanged external fragmentation for internal waste. - - Scope arenas removed that failure from ordinary storage. Fixed-size values and hosts have stable slots whose lifetimes already match a lexical scope, so their region needs only a frontier; anchor cells have the same append-and-drain shape in their own region. Neither benefits from individual reuse. Overwriting a host reuses its existing slot, and draining the scope unmaps both regions whole. - - Resizable backing stores are the exception the pure bump story had hidden. A list that doubles from 128 to 256 bytes may abandon the old 128-byte block while the scope continues for a long time. Leaving every old buffer stranded until drain makes repeated growth consume memory monotonically, while mixing those buffers among fixed-size slots makes the dense layout depend on container history. The answer was not to restore one global allocator, but to give each scope a third, dynamic region and confine exact-size reuse to it ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets)). Fixed-size, dynamic, and anchor allocations use distinct lazy chunk chains; no chunk contains more than one region, and a scope that never creates dynamic data pays for no dynamic chunk. - - The dynamic region keeps a LIFO stack for each power-of-two byte size, beginning at 128 bytes. Every request checks its exact size stack first and bumps the dynamic frontier only when that stack is empty. There is still no coalescing and no borrowing from a neighbouring class. The important difference from the rejected allocator is scope: these stacks recycle only the backing stores whose churn occurs inside one lifetime, and every stack and chunk disappears together when that scope drains. - - Byte size, rather than element count, defines the classes. A new list asks for 128 bytes — sixteen 64-bit words — whether it stores bytes, integers, or larger records; its capacity is whatever number of `T` elements fit. If one element is larger, the first block is simply the smallest power of two that can contain it. The common byte classes are what make reuse broad: a block returned by one `List` can serve another element type, a string, or any other compatible dynamic core value. - - Growth doubles the block size. The allocator first looks for a reusable block of the doubled size. If none exists and the current buffer is the frontier allocation with room behind it, the frontier advances and the list grows in place. Otherwise a doubled block is bump-allocated at the frontier, the elements move, the handle changes its offset, and the old block is pushed onto its old-size stack. The free stack is therefore the first choice and the frontier the fallback — including the in-place case, which is just a frontier bump that happens not to change the address. - - This reintroduces size-class fragmentation, but only in the one region whose objects genuinely change size, with powers of two that match the doubling policy and a fixed 128-byte floor that maximizes cross-type reuse. The fixed-size and anchor regions retain the arena's strongest property: no free lists, no holes from reassignment, and one bulk unmap at scope drain. The arena did not abolish free stacks everywhere; it confined them to the place where their reuse is worth their fragmentation. - - ''' - story = replace_once( - story, - r"## When the free stacks fragment, and the arena takes the scope\n.*?(?=## The last table problem, and the segmented offset)", - allocator_story, - "story allocator chapter", - ) - - story = story.replace( - "So the fix was not to leave the arena but to stop mixing two things inside it. Cells get their own **region** — a separate chunk chain in the same scope arena, addressed by the same segmented offsets, bump-allocated and bulk-unmapped exactly like the payload region, just not braided through it ([`memory.md` §4.1](https://github.com/zane-lang/spec/blob/2caab48a9f4bafda32d67c3bb902908aea1813ab/spec/memory.md#41-the-anchor-cell)). Payload scans go back to full density because the payload stream holds only payloads, and a payload's position no longer depends on what was tethered before it, so the alignment the compiler wants is the compiler's again. The cells, gathered into their own region, are still compact and still hot — a tether resolution reads a cell from a dense, cache-resident run, just not from the same line as the payload it will then visit. It is arenas the whole way down: two streams, one drain.", - "So the fix was not to leave the arena but to separate the things with different allocation behaviour. The scope now has three regions: fixed-size slots, dynamic backing stores, and anchor cells, each with its own lazy chunk chain and the same segmented-offset directory ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets), [§4.1](../spec/memory.md#41-the-anchor-cell)). Fixed-size scans stay dense because neither cells nor resizable buffers are braided through them; dynamic growth can recycle abandoned blocks without punching holes among hosts; and cells remain compact and hot in their own run. It is arenas the whole way down: three streams, one drain.", - ) - - sentinel_story = r'''## The sentinel that costs nothing, and the buffer that wanted a line - - Separating anchor cells made the zero sentinel free. A tether or backpointer value of `0` means “untethered,” but those values name only anchor cells, and anchor cells live in their own region where chunk `0`, word `0` is never a cell. The fixed-size region may therefore begin at offset zero with no reserved gap ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets)). The sentinel is protected by region identity, not by wasting the first slot. - - Dynamic buffers then asked for a stronger alignment guarantee. They are streamed, copied during growth, and commonly contain elements whose access pattern spans many cache lines. Rather than align an arbitrary buffer after an arbitrary fixed-size payload, the dedicated dynamic region starts from its own chunk boundary and allocates only power-of-two blocks beginning at 128 bytes. Every frontier block is therefore cache-line aligned, and every block later popped from a size stack preserves that alignment ([`memory.md` §3.6](../spec/memory.md#36-handle-typed-core-reference-types-have-fixed-footprint)). - - This is another benefit of refusing to mix regions on one chunk. Small inline values keep ordinary 8-byte alignment without paying cache-line padding, while dynamic backing stores receive cache-line alignment structurally. A scope with no dynamic values maps no dynamic chunk, so the stronger alignment carries no idle page cost. Zero remains a usable fixed-size address, and buffers get the geometry they need without making neighbouring object placement depend on allocation history. - - ''' - story = replace_once( - story, - r"## The sentinel that costs nothing, and the buffer that wanted a line\n.*?(?=## Two vocabularies: host and guest above anchor and tether)", - sentinel_story, - "story sentinel chapter", - ) - - story_path.write_text(story) - PY - - - name: Commit generated update - shell: bash - run: | - rm .github/workflows/apply-memory-region-update.yml - git config user.name "github-actions[bot]" - git config user.email "41898282+github-actions[bot]@users.noreply.github.com" - git add spec/memory.md stories/memory.md .github/workflows/apply-memory-region-update.yml - git commit -m "Specify arena regions and dynamic backing-store reuse" - git push diff --git a/spec/memory.md b/spec/memory.md index e59ead8..30ee5b8 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -15,7 +15,7 @@ Zane eliminates dangling guests by combining single hosting, lexical lifetime ru - **`Repointable guests`.** A guest is non-hosting storage that can point at different hosts over time. - **`Lexical lifetime enforcement`.** Guest assignment and rehosting are checked using declaration scope alone (see [`lifetimes.md`](lifetimes.md) §1). - **`Deterministic destruction`.** Objects are destroyed when their hosting scope drains; there is no tracing garbage collector (see [`lifetimes.md`](lifetimes.md) §2). -- **`Arena placement`.** A reference-type instance is bump-allocated in the arena of the scope that creates it, and is copied into a parent arena only if it escapes that scope (see §3.5). +- **`Regioned arena placement`.** Every scope owns separate fixed-size, dynamic-backing-store, and anchor-cell regions. Statically sized storage is placed inline in the fixed-size region; resizable data uses the dynamic region (see §3). - **`Segmented-offset tethers`.** Internally, each guest is represented by a `u32` tether — a chunk id plus an in-chunk offset — that points at the host's anchor cell, not a raw pointer (see §4.2). The source language and runtime use separate terms: an object lives in a **host**, and a **guest** (`&T`) may access it without storing it or controlling its lifetime. Internally, each guest is represented by a **tether** that resolves through an **anchor**. Moving the object updates the anchor, so existing tethers — and therefore guests — continue to reach it. @@ -226,38 +226,26 @@ if runtimeBool() { ## 3. Memory Layout ### 3.1 Scope arenas and segmented offsets -The runtime does not reserve one flat region. Each lexical scope owns a **bump arena**: a chain of fixed-size **1 MiB chunks** mapped from the OS on demand. Allocation advances one frontier pointer inside the current chunk; when a chunk fills, the runtime maps a fresh 1 MiB chunk, assigns it the next **chunk id**, and makes it current. Growing an arena never copies or relocates live data. +The runtime does not reserve one flat region. Each lexical scope owns an **arena** made from three independent allocation regions: -Scopes nest last-in-first-out, and their arenas nest with them: a scope's chunks are unmapped in full the moment the scope drains (§3.2, [`lifetimes.md`](lifetimes.md) §2.1). Arena granularity is an implementation choice, like boolean packing (§3.4) and placement (§3.5) — the compiler may fold several lexical scopes into one arena. What the language fixes is the observable behavior: memory a scope allocates outlives every guest that can reach it and is released together when the scope drains. +- The **fixed-size region** stores materialized value-type slots, statically sized reference-type hosts, and the fixed-size handles of dynamic core types. +- The **dynamic region** stores the resizable backing stores behind handles such as `List` and `String`. +- The **anchor-cell region** stores the scope-local anchor cells created for tethered hosts (§4.1). -Within an arena, payloads and anchor cells (§4.1) occupy **separate regions** — distinct chunk chains — so a scan over payloads never strides across interleaved cell metadata. Both chains draw chunk ids from the same directory, so a segmented offset addresses either identically. +Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots, dynamic backing stores, and anchor cells never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. Growing one region never copies or relocates allocations in another. ```text one scope arena -payload region anchor-cell region -────────────────── ────────────────── - -payload chunk anchor-cell chunk -+--------------------+ +--------------------+ -| Weapon payload | | Weapon's cell | -| Player payload | | Player's cell | -| Enemy payload | | Enemy's cell | -| ... | | ... | -+--------------------+ +--------------------+ - -payload chunk anchor-cell chunk -+--------------------+ +--------------------+ -| more payloads | | more cells | -+--------------------+ +--------------------+ +fixed-size region dynamic region anchor-cell region +───────────────── ────────────── ────────────────── +[fixed chunk] [dynamic chunk] [anchor chunk] +[fixed chunk] → ... [dynamic chunk] → ... [anchor chunk] → ... ``` -The two regions are separate allocation streams: a scan of payloads does -not step across anchor-cell metadata. Their chunks need not be adjacent in -native memory. Both kinds of chunk have ordinary chunk ids and are resolved -through the same chunk directory. +The chains are lazy and independent. A scope that uses no dynamic backing store maps no dynamic chunk; a scope that never creates a guest maps no anchor-cell chunk. When the scope drains, every chunk belonging to all three regions is unmapped together. The compiler may optimize away or coalesce physically unobservable storage, but it **MUST** preserve region exclusivity, lifetime, and drain behavior. -Every in-arena location is a **`u32` segmented offset**, never a native pointer. The `u32` splits into two fields: +All three regions draw chunk ids from the same chunk directory, so every in-arena location uses the same **`u32` segmented offset**. The `u32` splits into two fields: ``` u32 segmented offset @@ -267,48 +255,65 @@ Every in-arena location is a **`u32` segmented offset**, never a native pointer. └───────────────┴─────────────────────────┘ ``` -Allocations are 8-byte aligned, so the low bits count 8-byte words: a 1 MiB chunk holds 2¹⁷ words, so **17 low bits** address any slot in a chunk and the remaining **15 high bits** select one of up to 32768 live chunks — a reach of 32 GiB. A small **chunk directory** maps a chunk id to that chunk's native base address, so an address is materialized only at use, as `directory[chunk id] + word offset × 8`: splitting the `u32` is a shift and a mask, and the directory lookup is one load. Tethers (§4.2), the per-host backpointer (§4.2), and the anchor cells (§4.1) are all `u32` segmented offsets. The value `0` — chunk `0`, word `0` — is the *untethered* sentinel. It costs no reserved memory: because anchor cells are allocated only in the anchor-cell region (§4.1), which never includes that slot, no cell is ever at `0`, so a `0` backpointer or tether can never name a real cell. Host payloads carry no such restriction and may occupy offset `0` — so a scope's first payload sits at a chunk base, which is why the frontier needs no reserved gap. +Allocations are at least 8-byte aligned, so the low bits count 8-byte words: a 1 MiB chunk holds 2¹⁷ words, so **17 low bits** address any slot in a chunk and the remaining **15 high bits** select one of up to 32768 live chunks — a reach of 32 GiB. The chunk directory maps a chunk id to the chunk's native base address, so an address is materialized only at use as `directory[chunk id] + word offset × 8`. + +Tethers (§4.2), per-host backpointers (§4.2), anchor cells (§4.1), dynamic handles, and size-stack entries (§3.2) use segmented offsets. The value `0` — chunk `0`, word `0` — is the *untethered* sentinel. It costs no reserved memory because anchor cells are allocated only in the anchor-cell region, which never contains that location. Fixed-size payloads may occupy offset `0`. > **Story:** [`stories/memory.md`](../stories/memory.md#the-last-table-problem-and-the-segmented-offset) — "The last table problem, and the segmented offset". ### 3.2 Allocation is a bump; teardown is an unmap -Within a scope's arena, allocation is a single frontier bump — no size classes, no free list, no coalescing. A host is a fixed-size storage slot: overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7). Overwriting therefore consumes no additional arena space. +The fixed-size and anchor-cell regions are pure bump allocators: allocation advances the region's frontier, with no size classes, free lists, or coalescing. A host is a fixed-size storage slot, so overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7); it consumes no new arena space. + +The dynamic region adds exact-size reuse on top of its bump frontier. Dynamic blocks use power-of-two byte sizes beginning at **128 bytes**. Each scope maintains one LIFO **size stack** for every block size that has become reusable. To allocate a dynamic block of size `S`, the runtime first pops `size_stack[S]`; only when that stack is empty does it bump the dynamic frontier, mapping more dynamic chunks as needed. It never satisfies a request from a different size stack and never coalesces neighbouring free blocks. -Reclamation is bulk. When the scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps the scope's chunks and every byte the scope held is released at once, with no reachability scan or per-object memory-reclamation pass threaded through the exit. Anything that escaped the scope was already promoted into its destination scope's arena (§3.5), so no surviving object remains in the drained arena. Logical destruction timing is unchanged — a host dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); it is the *memory* that is released together at drain. +Returning a dynamic block pushes its segmented offset onto the stack for that exact byte size. These stacks are shared by all dynamic types in the scope: a 128-byte block previously used by a `List` may later hold string bytes or another list's elements. The stacks affect allocation within the scope only; they require no per-object reclamation when the scope drains. + +Reclamation remains bulk. When the scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps every fixed-size, dynamic, and anchor-cell chunk owned by the scope, with no reachability scan or per-object memory-reclamation pass. Anything that escaped was already placed in storage whose lifetime covers its destination host (§3.5). Logical destruction timing is unchanged — a host dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); the underlying region memory is released together at drain. > **Story:** [`stories/memory.md`](../stories/memory.md#when-the-free-stacks-fragment-and-the-arena-takes-the-scope) — "When the free stacks fragment, and the arena takes the scope". ### 3.3 Value and reference layout follow declaration order -Fields are laid out in declaration order. Value types are stored inline. A reference-type instance has stable identity and carries one `u32` backpointer field of anchor metadata (a segmented offset, §4.2) that stays `0` until the instance is first tethered. Arena placement of a reference-type instance is covered in §3.5. +Fields are laid out in declaration order. Value types are stored inline. A statically sized reference-type instance is also stored inline in a fixed-size host slot, so value-type slots and reference-type host slots may sit directly beside each other in the fixed-size region. Reference types differ by identity and hosting semantics, not by requiring a separate indirect allocation. + +A reference-type instance carries one `u32` backpointer field of anchor metadata (a segmented offset, §4.2) that remains `0` until the instance is first tethered. A dynamic core type such as `List` occupies a fixed-size handle inline in the same region; only the backing store named by that handle occupies the dynamic region (§3.6). ### 3.4 Booleans may be packed The compiler may pack booleans in structs and arena frames when doing so does not change language semantics. -### 3.5 Reference-type instances are placed in their scope's arena -Placement is an implementation decision, not a language-visible property. A reference-type instance is bump-allocated in the arena of the scope that creates it when both hold: +### 3.5 Statically sized storage uses the fixed-size region +Placement is an implementation decision, not a language-visible property. The arena model places every materialized, statically sized scope slot — value-type storage, a reference-type host, or a dynamic type's fixed-size handle — inline in that scope's fixed-size region. The compiler may keep an unobservable value in registers or otherwise optimize its physical placement, but reference types do not require a separate heap allocation merely because they carry identity. -- its size is statically known, and -- it does not escape that scope in a way a move cannot satisfy. +When a reference-type instance is rehosted into a longer-lived destination, its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). A dynamic backing store is semantically owned by the current host. The compiler **MUST** place that store in a dynamic region whose lifetime covers every destination into which the handle can be rehosted, so rehosting transfers ownership of the same backing store without copying it. Only growth of the dynamic value may relocate the backing store (§3.6). -When an instance escapes — it is moved into a longer-lived host in a parent scope — it is **promoted**: its payload is copied into the destination scope's arena (§3.7). A dynamically-sized instance forces its backing store into the arena the same way (§3.6). Placement never changes observable semantics: destruction stays deterministic (see [`lifetimes.md`](lifetimes.md) §2), and tethers resolve identically regardless of which arena the instance lives in (§4), because a tether resolves through the host's anchor cell rather than a fixed address. This freedom mirrors the boolean-packing rule (§3.4): the compiler may choose the cheaper arena whenever doing so cannot change program meaning. +Placement never changes observable semantics: destruction stays deterministic (see [`lifetimes.md`](lifetimes.md) §2), and tethers resolve identically regardless of physical placement (§4), because a tether follows the host's anchor rather than a fixed address. > **Story:** [`stories/memory.md`](../stories/memory.md#the-value-world-stays-closed-and-placement-stays-the-compilers) — "The value world stays closed, and placement stays the compiler's". ### 3.6 Handle-typed core reference types have fixed footprint -The core dynamically-sized reference types — `List`, `String`, and similar types — are represented as a fixed-size **handle**: a small header (or single segmented offset) whose dynamic backing store lives in the arena. The handle occupies a statically known footprint wherever it is stored. +The core dynamically-sized reference types — `List`, `String`, and similar types — are represented as fixed-size **handles**. A handle records the backing store's segmented offset and the metadata needed by the type, such as length and size class. The handle occupies a statically known footprint inline in the fixed-size region; its resizable backing store is a separate allocation in the dynamic region. -A type that contains a handle-typed field therefore stays statically sized. A type holding a `List` field does not become dynamically sized; it stores the fixed handle inline, and only the backing store behind the handle is a separate arena allocation. +A type that contains a handle-typed field therefore stays statically sized: ```zane type Inventory = #struct { - items List; // fixed-size handle inline; backing store in the arena + items List; // fixed-size handle inline; elements in the dynamic region count Int; } ``` -This is what keeps arena placement (§3.5) broadly applicable: almost every value is statically sized at its own level, so dynamic size appears only inside the backing stores of handle types. +Dynamic block sizes are byte-based rather than element-type-based. A new list starts with a **128-byte block** — equivalent to sixteen 64-bit words — regardless of `T`. Its element capacity is `floor(block_bytes / stride(T))`. If one element does not fit in 128 bytes, the initial block is the smallest power-of-two block that can hold one element. Keeping the byte classes common allows blocks to be reused across lists with different element types and across other dynamic core types. -A backing store is allocated **cache-line-aligned**: before it is placed the arena frontier is advanced to the next cache-line boundary. A backing store is streamed and grown in bulk, and an unaligned base would let its elements straddle cache lines, so sequential access would touch a line more than it needs. Aligning the base packs whole elements within lines. Small inline allocations keep the ordinary 8-byte alignment (§3.1) — cache-line-aligning every small object would waste most of a line per object for no locality gain, since the cost only arises when streaming across many elements. The padding to reach the boundary is at most one line, negligible against a backing store's size. +A list grows according to the following rules: + +1. When its capacity is exhausted, the requested block size is exactly twice its current block size. +2. The allocator first checks the size stack for that doubled size. If a block is available, it is popped and the live elements are relocated into it. +3. If that stack is empty and the current backing store is the dynamic frontier allocation with enough contiguous room to double, the frontier is bumped by the additional bytes and the store grows in place. +4. Otherwise, the doubled block is allocated by bumping the dynamic frontier, mapping more dynamic chunks as needed, and the live elements are relocated into it. +5. After relocation, the handle's backing-store offset and size class are updated and the old block's offset is pushed onto the size stack for its old byte size. + +Relocation moves or copies elements according to their type's ordinary move rules; the old block becomes reusable only after its previous occupants are no longer live. Guests to the list remain valid because they reach the list's host, whose fixed-size handle now names the current backing store. + +Dynamic chunks and all power-of-two blocks begin at cache-line-aligned addresses. Because the minimum block is 128 bytes and every larger block doubles, both frontier allocations and reused blocks preserve cache-line alignment without mixing backing stores into fixed-size chunks. > **Story:** [`stories/memory.md`](../stories/memory.md#the-sentinel-that-costs-nothing-and-the-buffer-that-wanted-a-line) — "The sentinel that costs nothing, and the buffer that wanted a line". @@ -318,16 +323,18 @@ A move transfers hosting into a destination host of the **same type** (see [`lif - Moving into a fresh declaration or a return slot is in-place initialization. - Moving into an already-initialized host first destroys the current occupant, then overwrites the same-size slot. -Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. A move into a higher scope copies the inline bytes into the destination scope's arena — a promotion (§3.5). Because handle-typed fields (§3.6) keep the moved footprint small, a move relocates only the inline bytes — a handle's backing store never moves. If the moved value is tethered, the move also updates its one anchor cell (§4.5), never the tethers themselves. +Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. A move into a higher scope copies the inline bytes into the destination scope's fixed-size region — a promotion (§3.5). Because handle-typed fields (§3.6) keep the moved footprint small, rehosting copies only the handle and transfers ownership of the same backing store; rehosting itself never relocates that store. A dynamic store changes address only through the growth procedure in §3.6. If the moved value is tethered, the move also updates its one anchor cell (§4.5), never the tethers themselves. --- ## 4. Anchors and Tethers ### 4.1 The anchor cell -Tethers are tracked through per-host **anchor cells** rather than one shared table. An anchor cell is a single **`u32`** holding the current segmented offset (§3.1) of one hosted object; it stores nothing else. A cell is an ordinary arena allocation — bump-allocated on the host's first tether (§4.3) — so there is no monolithic table to relocate as anchors accumulate: minting an anchor is one bump, never a resize. +Tethers are tracked through per-host **anchor cells** rather than one shared table. An anchor cell is a single **`u32`** holding the current segmented offset (§3.1) of one hosted object; it stores nothing else. + +Anchor storage is **scope-local**, never global. Each scope owns a dedicated anchor-cell region — a separate lazy chunk chain from both its fixed-size and dynamic regions. A scope that never creates a guest allocates no anchor chunk. The first tether to a host bump-allocates its cell in that scope's anchor region (§4.3); minting another cell is one bump and never resizes a monolithic table. -Cells are bump-allocated in a **dedicated anchor-cell region** of the scope's arena, a separate chunk chain from the one holding payloads (§3.1). Keeping cells out of the payload stream means a scan over payloads never strides across interleaved cell metadata, so iteration stays dense and a payload's placement never depends on how many of its neighbours were tethered first. The cell region is itself compact and heavily reused, so resolving through a cell (§4.4) is a load into hot, cache-resident memory. +Keeping cells out of the other two streams preserves dense fixed-size layout and prevents dynamic-buffer history from affecting anchor placement. The region remains compact and heavily reused while live, and all of its chunks disappear with the scope in the same bulk unmap as the other regions. > **Story:** [`stories/memory.md`](../stories/memory.md#where-the-cells-live-and-the-scan-that-pays-for-them) — "Where the cells live, and the scan that pays for them". diff --git a/stories/memory.md b/stories/memory.md index 6a8afec..e12d0ff 100644 --- a/stories/memory.md +++ b/stories/memory.md @@ -40,7 +40,9 @@ The same rooted-source idea runs through parameters, but with a twist that prote Two last pressures pull in opposite directions — one locks a door, the other opens one — and they are worth telling together because both are about how far the value layer can be trusted to behave. The locked door is the struct. Structs are plain inline values: copied by overwriting bytes, with no anchor and no destruction tracking. That is only sound if a struct can never smuggle in something that *needs* tracking — so a struct field may hold primitives and other structs and nothing else, checked transitively through the whole nested graph ([§2.10](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#210-struct-downstream-enforcement-transitive-struct-field-restrictions)). Let a struct contain a class and a byte-copy would silently duplicate ownership; let it contain an `&` and a byte-copy would silently duplicate ref-tracking state without ever going through the anchor system that makes that state correct. Both break the one invariant that lets struct copies be mechanical, so the closed value world is enforced rather than hoped for — the strictness-buys-speed bargain of the [foundations story](foundations.md#strictness-is-the-performance-model) in miniature. -The opened door is placement. Because the anchor model makes a ref resolve identically no matter *where* its owner physically lives — the ref walks to a cell, and the cell can hold a stack address as easily as a heap one — the compiler is free to put a class instance wherever is cheapest, stack or heap, with no language-visible consequence ([§3.5](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#35-class-instances-may-be-placed-on-the-stack)). It uses the stack whenever the instance is statically sized and does not escape in a way a move cannot satisfy, and is forced to the heap only by genuine dynamic size or escape. The thing that makes this freedom *broad* rather than rare is that dynamic size is kept from leaking upward: the core dynamically-sized types — `List`, `String` — are represented as fixed-size handles whose backing store lives on the heap, so a type containing one stays statically sized and stack-eligible, with only the backing store forced onto the heap ([§3.6](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#36-handle-typed-core-classes-have-fixed-footprint)). Placement, like the boolean-packing latitude beside it, is something the language hands to the compiler precisely because it has been arranged to be unobservable — and it is unobservable only because the anchor indirection, the thing this whole story is built around, already decoupled a ref from any fixed address. The cost is the one the chapter cannot remove: this only holds for as long as the model keeps placement semantically invisible, and every feature that might let a program *observe* where a value physically sits — raw addresses, layout introspection — is a feature this freedom quietly forbids. +The opened door is placement. Identity does not require a second heap object: a materialized value slot and a statically sized reference-type host can sit directly beside each other in the scope's fixed-size region. The reference type differs because it carries hosting identity and a backpointer, not because its bytes must be indirect. Dynamic size is kept from leaking upward in the same way: `List`, `String`, and similar types are fixed-size handles inline with the ordinary slots, while only their backing stores occupy the scope's separate dynamic region ([§3.3](../spec/memory.md#33-value-and-reference-layout-follow-declaration-order), [§3.6](../spec/memory.md#36-handle-typed-core-reference-types-have-fixed-footprint)). That separation is what lets a type containing a list remain statically sized and what prevents a growing buffer from shifting any neighbouring value or host. + +Placement remains unobservable. The compiler may keep a value in a register or choose the lifetime region that already covers a later rehosting, provided destruction and guest resolution are unchanged. In particular, a dynamic backing store belongs semantically to its current host but is placed in a dynamic region that outlives every host it can be moved into. Rehosting therefore copies only the fixed-size handle and transfers ownership of the same store; only the container's own growth operation relocates its elements. The cost is the same boundary the earlier design already accepted: raw addresses and layout introspection cannot be language-visible facts if the compiler is free to make these choices. ## The kinds collapse into one axis, and `this` becomes a borrow @@ -67,13 +69,19 @@ The cost is the ordinary cost of any coined term: a reader meets "tether" and mu ## When the free stacks fragment, and the arena takes the scope -By this point the model's *names* had settled, but the machinery underneath them had not. Allocation, all along, had been served by size-indexed free stacks: round every request up to an 8-byte boundary, give each rounded size its own stack of freed slots, and satisfy a request by popping the matching stack or bumping a frontier when it was empty. It was O(1) and it avoided coalescing, and for a long time that was enough. What it could not avoid was the failure mode built into its own shape. Each size class hoards its freed memory and lends it to no other, so exhausting the 32-byte stack is out-of-memory *for 32-byte objects* — even with the 16-byte and 64-byte stacks sitting on abundant free space they will never surrender. A size-classed allocator fragments along its own class boundaries, and under a churny workload that fragmentation is not a tail risk; it is the steady state. +The allocator first used size-indexed free stacks for everything: round a request, pop the matching stack, and bump a global frontier only when that stack was empty. The operation was O(1), but the global policy let each class hoard memory from every other class. A program could run out of 32-byte slots while 16-byte and 64-byte stacks held abundant space, and borrowing a larger slot merely exchanged external fragmentation for internal waste. + +Scope arenas removed that failure from ordinary storage. Fixed-size values and hosts have stable slots whose lifetimes already match a lexical scope, so their region needs only a frontier; anchor cells have the same append-and-drain shape in their own region. Neither benefits from individual reuse. Overwriting a host reuses its existing slot, and draining the scope unmaps both regions whole. + +Resizable backing stores are the exception the pure bump story had hidden. A list that doubles from 128 to 256 bytes may abandon the old 128-byte block while the scope continues for a long time. Leaving every old buffer stranded until drain makes repeated growth consume memory monotonically, while mixing those buffers among fixed-size slots makes the dense layout depend on container history. The answer was not to restore one global allocator, but to give each scope a third, dynamic region and confine exact-size reuse to it ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets)). Fixed-size, dynamic, and anchor allocations use distinct lazy chunk chains; no chunk contains more than one region, and a scope that never creates dynamic data pays for no dynamic chunk. + +The dynamic region keeps a LIFO stack for each power-of-two byte size, beginning at 128 bytes. Every request checks its exact size stack first and bumps the dynamic frontier only when that stack is empty. There is still no coalescing and no borrowing from a neighbouring class. The important difference from the rejected allocator is scope: these stacks recycle only the backing stores whose churn occurs inside one lifetime, and every stack and chunk disappears together when that scope drains. -The first fix we tried was the cheap one, and we rejected it for making the cure worse than the disease. If the 32-byte class is empty, why not serve the request from the 64-byte class — hand out a larger block and waste the difference? It removes the spurious out-of-memory, but it does so by pouring internal fragmentation into every oversized allocation, and internal fragmentation is precisely what size classes existed to prevent: the whole reason to bucket by size is cache density, and a 64-byte slot holding a 32-byte object is a cache line half full of nothing. We would have traded a fragmentation we could see for one smeared invisibly across the whole heap. +Byte size, rather than element count, defines the classes. A new list asks for 128 bytes — sixteen 64-bit words — whether it stores bytes, integers, or larger records; its capacity is whatever number of `T` elements fit. If one element is larger, the first block is simply the smallest power of two that can contain it. The common byte classes are what make reuse broad: a block returned by one `List` can serve another element type, a string, or any other compatible dynamic core value. -The move that actually dissolved the problem was to stop treating allocation as a global pool at all. Zane already has a strong notion of *when* memory should die: the water-tower scope ([`concurrency.md` §4.1](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/concurrency.md#41-water-tower-lifetime-extension)), the lexical region whose owned objects live exactly until the scope drains. Scopes nest last-in-first-out, and that is exactly the discipline a bump allocator wants: if every scope owns its own **arena** — a region it bump-allocates into and releases whole — then allocation is a single pointer advance and deallocation is a single pointer rewind, because nothing inside a scope outlives the scope. There are no size classes to fragment along, because there are no size classes; a bump arena hands out the next *N* bytes regardless of *N*. The fragmentation problem is not so much solved as made inexpressible ([`memory.md` §3.2](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#32-allocation-is-a-bump-teardown-is-an-unmap)). +Growth doubles the block size. The allocator first looks for a reusable block of the doubled size. If none exists and the current buffer is the frontier allocation with room behind it, the frontier advances and the list grows in place. Otherwise a doubled block is bump-allocated at the frontier, the elements move, the handle changes its offset, and the old block is pushed onto its old-size stack. The free stack is therefore the first choice and the frontier the fallback — including the in-place case, which is just a frontier bump that happens not to change the address. -The honest cost is the one every arena pays, and we took it with eyes open. A bump arena does not reclaim individual objects: an owner overwritten or logically destroyed partway through a scope leaves its bytes stranded as dead space until the whole arena is released. A workload that churns heavily *within* one long-lived scope holds more peak memory than the free stacks would have — handing a reclaimed slot straight back was the free stacks' one real virtue. We judged that a good trade, because the pattern arenas punish (unbounded intra-scope churn) is rarer than the pattern they reward (a scope that allocates, works, and drains), and because bulk release buys something the free stacks never could: teardown with no per-object work at all. When a scope drains its whole arena goes back to the OS in one unmap — no walk over the objects, no drop-glue threaded through the exit, the memory simply gone. +This reintroduces size-class fragmentation, but only in the one region whose objects genuinely change size, with powers of two that match the doubling policy and a fixed 128-byte floor that maximizes cross-type reuse. The fixed-size and anchor regions retain the arena's strongest property: no free lists, no holes from reassignment, and one bulk unmap at scope drain. The arena did not abolish free stacks everywhere; it confined them to the place where their reuse is worth their fragmentation. ## The last table problem, and the segmented offset @@ -95,7 +103,7 @@ The [previous chapter](#the-last-table-problem-and-the-segmented-offset) left a The tempting repair was to go backwards: put the anchors back in one compact heap table, the way the pre-arena model had. That table was never slow to read — it stayed small and hot, and it kept payloads perfectly contiguous because nothing lived between them. But the reason it read so well is exactly the reason it could not stay. A single global table has no scope to be unmapped with; every cell in it has to be handed back one at a time as its owner dies, or the table leaks and grows without bound. That is the per-object teardown the arenas had just abolished — the "free is a no-op" and one-unmap-per-scope wins are wins *because* nothing walks the objects to reclaim them. A global table would have bought back the payload density by selling the teardown, and the teardown was the larger prize. The compactness that made the table attractive was the very thing that forced the per-object free; the two could not be separated. -So the fix was not to leave the arena but to stop mixing two things inside it. Cells get their own **region** — a separate chunk chain in the same scope arena, addressed by the same segmented offsets, bump-allocated and bulk-unmapped exactly like the payload region, just not braided through it ([`memory.md` §4.1](https://github.com/zane-lang/spec/blob/2caab48a9f4bafda32d67c3bb902908aea1813ab/spec/memory.md#41-the-anchor-cell)). Payload scans go back to full density because the payload stream holds only payloads, and a payload's position no longer depends on what was tethered before it, so the alignment the compiler wants is the compiler's again. The cells, gathered into their own region, are still compact and still hot — a tether resolution reads a cell from a dense, cache-resident run, just not from the same line as the payload it will then visit. It is arenas the whole way down: two streams, one drain. +So the fix was not to leave the arena but to separate the things with different allocation behaviour. The scope now has three regions: fixed-size slots, dynamic backing stores, and anchor cells, each with its own lazy chunk chain and the same segmented-offset directory ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets), [§4.1](../spec/memory.md#41-the-anchor-cell)). Fixed-size scans stay dense because neither cells nor resizable buffers are braided through them; dynamic growth can recycle abandoned blocks without punching holes among hosts; and cells remain compact and hot in their own run. It is arenas the whole way down: three streams, one drain. The cost is real and we name it plainly. We gave up the shared-cache-line bonus on the deref itself — the cell and its payload no longer ride into cache together, so a single tether resolution can pay two loads where the interleaved layout often paid one. We judged that the right trade because the payload sweep is the hotter path in the workloads we cared about: densifying the scan that runs over every object beats shaving a load off the deref that runs only when a tether is actually followed. It is the mirror image of the choice the last chapter made, now that we have measured which side of the coin comes up more often. @@ -103,19 +111,11 @@ Separating the region also forced us to finish a sentence the last chapter had l ## The sentinel that costs nothing, and the buffer that wanted a line -The [separate cell region](#where-the-cells-live-and-the-scan-that-pays-for-them) closed the placement question, but it quietly loosened something we had been treating as fixed, and it took a benchmark regression to notice. Growing an owned buffer — appending a hundred thousand small records to a list — ran markedly slower under the arena than under the free-stack allocator it replaced. Nothing about a pure bump-and-fill should have been slower; the frontier just advances. The cause turned out to be one byte of history. - -A CPU moves memory in fixed sixty-four-byte cache lines: a value wholly inside one line costs one line to touch, a value spanning a boundary costs two. Our growth buffer was based at arena offset 8 — not on a line boundary — and a run of 32-byte records laid down from offset 8 puts every other record across a boundary. The write-heavy fill was paying for nearly twice the lines it should. The old free-stack build had, by luck, handed back a line-aligned block; the arena had not. - -The eight-byte skew was not arbitrary, and finding out why turned a one-line fix into a small piece of design. The arena reserves the value zero as the *untethered* sentinel: a backpointer or tether of `0` means "no anchor." In the earliest arena layout cells were bump-allocated from the same frontier as payloads, so the very first cell could land at offset 0 and collide with the sentinel — we prevented that by starting the frontier at 8, one slot in. But the previous chapter had since moved cells into their own region. Once no cell is ever drawn from the payload frontier, no cell can sit at offset 0 — and the only things that carry the sentinel value are backpointers and tethers, which name *cells*. The reservation was guarding a collision that could no longer happen. It was vestigial. - -That reframed the sentinel entirely. Zero is safe not because we hold the slot empty, but because cells live somewhere zero never is. So the payload frontier can start at offset 0 — a chunk base, which is line-aligned — and the sentinel costs no reserved memory at all: a real payload sits at offset 0, still unambiguous, because 0 is only ever *read* as null through a backpointer or tether, and those name cells ([`memory.md` §3.1](https://github.com/zane-lang/spec/blob/848b6ebc51f03e5826c584f026223f0aad2023f7/spec/memory.md#31-scope-arenas-and-segmented-offsets)). The buffer that started all this, being the first allocation in its scope, now lands line-aligned for free. - -Starting at zero only aligns the *first* allocation, though; a buffer created after other objects still lands wherever the frontier happens to be. So the durable rule aligns the thing that actually cares: a dynamically-sized backing store is allocated cache-line-aligned, the frontier bumped to the next line before it is placed ([`memory.md` §3.6](https://github.com/zane-lang/spec/blob/848b6ebc51f03e5826c584f026223f0aad2023f7/spec/memory.md#36-handle-typed-core-reference-types-have-fixed-footprint)). Only backing stores earn it — they are what gets streamed and grown; a small inline object stays 8-byte aligned, since padding every little allocation up to a line would waste most of a line each for locality it will never use. Payloads from zero and backing stores aligned, together, put the fill back on the old model's number. +Separating anchor cells made the zero sentinel free. A tether or backpointer value of `0` means “untethered,” but those values name only anchor cells, and anchor cells live in their own region where chunk `0`, word `0` is never a cell. The fixed-size region may therefore begin at offset zero with no reserved gap ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets)). The sentinel is protected by region identity, not by wasting the first slot. -There was a larger temptation in the same corner, and we turned it down. If a tether and a backpointer only ever name cells, why address them with a full segmented offset — why not a dense ordinal index into the cells, a smaller field? The appeal is real but the arithmetic isn't: the width of a reference is the log of how many things it must name, index or offset alike, so an ordinal cell index is only *smaller* than the `u32` we already use if we also cap the number of simultaneously-tethered objects — a `u16` buys two bytes and a 65,536-cell ceiling. And a dense global index needs one packed anchor array to index into, which is exactly what per-scope bulk teardown refuses to provide: draining a scope frees a block wholesale and promotion inserts a cell into a parent region, so the cells never form one contiguous array without a free-list — the per-object teardown we spent the arena to escape. A per-scope index that keeps the bulk free has to carry its scope's chunk id, at which point it *is* the segmented offset again. The real bit-efficiency — half a native pointer, and eight-byte words rather than bytes — is already banked in the `u32` segmented offset; the further notch pays out only in a bounded profile that promises the cap, so we left it there. +Dynamic buffers then asked for a stronger alignment guarantee. They are streamed, copied during growth, and commonly contain elements whose access pattern spans many cache lines. Rather than align an arbitrary buffer after an arbitrary fixed-size payload, the dedicated dynamic region starts from its own chunk boundary and allocates only power-of-two blocks beginning at 128 bytes. Every frontier block is therefore cache-line aligned, and every block later popped from a size stack preserves that alignment ([`memory.md` §3.6](../spec/memory.md#36-handle-typed-core-reference-types-have-fixed-footprint)). -The cost of what we kept is a rounding: up to a cache line of padding before each backing store, unmeasurable against the store it precedes, and the standing discipline that the payload frontier and the cell region stay separate so that zero keeps its meaning for free. Cheap insurance, for a sentinel that now costs nothing and a fill that no longer straddles. +This is another benefit of refusing to mix regions on one chunk. Small inline values keep ordinary 8-byte alignment without paying cache-line padding, while dynamic backing stores receive cache-line alignment structurally. A scope with no dynamic values maps no dynamic chunk, so the stronger alignment carries no idle page cost. Zero remains a usable fixed-size address, and buffers get the geometry they need without making neighbouring object placement depend on allocation history. ## Two vocabularies: host and guest above anchor and tether From aac419e4cfae5062cd469613b41f675340559a4f Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Sat, 25 Jul 2026 15:18:01 +0200 Subject: [PATCH 05/31] Add one-shot review fix updater --- .github/workflows/apply-review-fixes.yml | 214 +++++++++++++++++++++++ 1 file changed, 214 insertions(+) create mode 100644 .github/workflows/apply-review-fixes.yml diff --git a/.github/workflows/apply-review-fixes.yml b/.github/workflows/apply-review-fixes.yml new file mode 100644 index 0000000..52e0554 --- /dev/null +++ b/.github/workflows/apply-review-fixes.yml @@ -0,0 +1,214 @@ +name: Apply memory review fixes + +on: + push: + branches: + - agent/fix-host-overwrite-reuse + +permissions: + contents: write + +jobs: + update: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + ref: agent/fix-host-overwrite-reuse + fetch-depth: 0 + + - name: Address allocator and promotion review + shell: bash + run: | + python3 <<'PY' + from pathlib import Path + import re + + def replace_once(text: str, old: str, new: str, label: str) -> str: + count = text.count(old) + if count != 1: + raise RuntimeError(f"expected one literal for {label}, got {count}") + return text.replace(old, new, 1) + + def replace_regex(text: str, pattern: str, new: str, label: str) -> str: + updated, count = re.subn(pattern, new, text, count=1, flags=re.S) + if count != 1: + raise RuntimeError(f"expected one regex replacement for {label}, got {count}") + return updated + + spec_path = Path("spec/memory.md") + spec = spec_path.read_text() + + spec = replace_once( + spec, + "Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots, dynamic backing stores, and anchor cells never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. Growing one region never copies or relocates allocations in another.", + "Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots, dynamic backing stores, and anchor cells never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. Growing one region never copies or relocates allocations in another.\n\nAn ordinary allocation never straddles a chunk boundary. A dynamic block of at most 1 MiB is wholly contained in one dynamic chunk; if the remaining bytes in the current chunk cannot hold it, allocation continues in a fresh dynamic chunk. A dynamic block larger than 1 MiB is an **oversized span**: a dedicated contiguous OS mapping containing `block_size / 1 MiB` chunks, all belonging exclusively to that one block and assigned consecutive chunk ids. Its handle stores the segmented offset of the span's first byte and its size class; after resolving that base, element addressing uses an ordinary byte offset across the contiguous mapping. Every constituent chunk also has its own directory entry. Returning an oversized span pushes only its base offset onto the exact-size stack, and the whole span remains mapped for reuse until the scope drains.", + "oversized dynamic span representation", + ) + + old_growth = """A list grows according to the following rules: + +1. When its capacity is exhausted, the requested block size is exactly twice its current block size. +2. The allocator first checks the size stack for that doubled size. If a block is available, it is popped and the live elements are relocated into it. +3. If that stack is empty and the current backing store is the dynamic frontier allocation with enough contiguous room to double, the frontier is bumped by the additional bytes and the store grows in place. +4. Otherwise, the doubled block is allocated by bumping the dynamic frontier, mapping more dynamic chunks as needed, and the live elements are relocated into it. +5. After relocation, the handle's backing-store offset and size class are updated and the old block's offset is pushed onto the size stack for its old byte size. + +Relocation moves or copies elements according to their type's ordinary move rules; the old block becomes reusable only after its previous occupants are no longer live. Guests to the list remain valid because they reach the list's host, whose fixed-size handle now names the current backing store. + +Dynamic chunks and all power-of-two blocks begin at cache-line-aligned addresses. Because the minimum block is 128 bytes and every larger block doubles, both frontier allocations and reused blocks preserve cache-line alignment without mixing backing stores into fixed-size chunks.""" + new_growth = """A list grows according to the following rules: + +1. When its capacity is exhausted, the requested block size is exactly twice its current block size. +2. The allocator first checks the size stack for that doubled size. If a block or oversized span is available, it is popped and the live elements are relocated into it. +3. If that stack is empty, the current backing store is the dynamic frontier allocation, the doubled size is at most 1 MiB, and the additional bytes fit before the current chunk boundary, the frontier is bumped and the store grows in place. +4. Otherwise, a doubled block of at most 1 MiB is bump-allocated wholly within one dynamic chunk. A doubled block larger than 1 MiB is allocated as a fresh dedicated oversized span (§3.1). The live elements are then relocated into the new block or span. +5. After relocation, the handle's backing-store offset and size class are updated and the old block's base offset is pushed onto the stack for its exact old byte size. + +A block never grows in place across a chunk boundary, and an oversized span is never extended in place: further growth relocates into a doubled oversized span after checking that exact-size stack first. Relocation moves or copies elements according to their type's ordinary move rules; the old block becomes reusable only after its previous occupants are no longer live. Guests to the list remain valid because they reach the list's host, whose fixed-size handle now names the current backing store. + +Dynamic chunks, ordinary power-of-two blocks, and oversized spans begin at cache-line-aligned addresses. Because the minimum block is 128 bytes and every larger block doubles, frontier allocations, reused blocks, and dedicated spans preserve cache-line alignment without mixing backing stores into fixed-size chunks.""" + spec = replace_once(spec, old_growth, new_growth, "list growth and oversized spans") + + new_anchor = """### 4.1 The anchor cell +Tethers are tracked through per-host **anchor cells** rather than one shared table. An anchor cell is a single **`u32`**. A canonical cell contains the current segmented offset (§3.1) of its hosted object in the fixed-size region. A source-scope cell left behind by promotion may instead contain the segmented offset of a newer anchor cell and acts as a **forwarding cell** (§4.5). The chunk directory records each chunk's region kind, so resolution distinguishes a payload target from a forwarding target without adding a tag to the cell. + +Anchor storage is **scope-local**, never global. Each scope owns a dedicated anchor-cell region — a separate lazy chunk chain from both its fixed-size and dynamic regions. A scope that never creates a guest allocates no anchor chunk. The first tether to a host bump-allocates its canonical cell in that scope's anchor region (§4.3); promotion may allocate a replacement canonical cell in a destination scope and turn the previous cell into a forwarder (§4.5). No monolithic table is ever resized. + +Keeping cells out of the other two streams preserves dense fixed-size layout and prevents dynamic-buffer history from affecting anchor placement. The region remains compact while live, and all of its chunks disappear with the scope in the same bulk unmap as the other regions. + +> **Story:** [`stories/memory.md`](../stories/memory.md#where-the-cells-live-and-the-scan-that-pays-for-them) — "Where the cells live, and the scan that pays for them". + +""" + spec = replace_regex( + spec, + r"### 4\.1 The anchor cell\n.*?(?=### 4\.2 Tethers are segmented offsets, not pointers)", + new_anchor, + "anchor cell representation", + ) + + spec = replace_once( + spec, + "Every reference-type instance reserves a **`u32` backpointer** field, initialized to `0`; the first tether records the segmented offset of the instance's anchor cell there. The cell is allocated lazily (§4.3), whereas the backpointer field is always present in the layout, so object size is fixed and array layout stays uniform. The backpointer lets a host mint new tethers from the object — `&x` copies the offset — and lets a move locate and update the object's cell (§4.5). It is a single offset, not a list of tethers: the runtime never enumerates the tethers that point at the object, which is what keeps moves O(1) (§4.5).", + "Every reference-type instance reserves a **`u32` backpointer** field, initialized to `0`; the first tether records the segmented offset of the instance's canonical anchor cell there. The cell is allocated lazily (§4.3), whereas the backpointer field is always present in the layout, so object size is fixed and array layout stays uniform. The backpointer always names the newest canonical cell, never a forwarding predecessor. It lets a host mint new tethers by copying that canonical offset and lets a move locate and update the one authoritative cell (§4.5). The runtime never enumerates the tethers or forwarding predecessors that ultimately reach it.", + "canonical backpointer semantics", + ) + spec = replace_once( + spec, + "A tethered reference-type instance therefore costs **12 bytes** across the whole chain: the 4-byte tether (wherever it is stored), the 4-byte anchor cell, and the 4-byte backpointer in the payload.", + "The minimum machinery for one tethered reference-type instance is **12 bytes**: one 4-byte tether, one 4-byte canonical anchor cell, and the 4-byte backpointer in the payload. Each additional tether costs another 4 bytes. Each still-live source scope crossed by promotion may temporarily retain one additional 4-byte forwarding cell until that scope drains (§4.5).", + "anchor cost accounting", + ) + spec = replace_once( + spec, + "A hosted object that never gains a tether consumes no cell: its backpointer field stays `0` and no cell is allocated (it still carries the 4-byte field, §4.2). The first `&` taken on its host bump-allocates a cell in the arena, writes the hosted object's current segmented offset into it, and records the cell's own segmented offset in the object's backpointer. Every subsequent `&` from that host copies the backpointer.", + "A hosted object that never gains a tether consumes no cell: its backpointer field stays `0` and no cell is allocated (it still carries the 4-byte field, §4.2). The first `&` taken on its host bump-allocates a canonical cell in the scope's anchor region, writes the hosted object's current segmented offset into it, and records the cell's own segmented offset in the object's backpointer. Every subsequent `&` from that host copies the canonical backpointer.", + "lazy canonical anchor creation", + ) + spec = replace_once( + spec, + "Resolving a tether reads the anchor cell it points at, reads the hosted object's segmented offset from that cell, materializes the object's address through the chunk directory (§3.1), then accesses the field. Because a cell is never at `0`, a resolution of an untethered `0` never reads a live cell.", + "Resolving a tether reads the anchor cell it points at and inspects the region kind of the cell's target through the chunk directory (§3.1). A target in the fixed-size region is the hosted object's current location. A target in an anchor-cell region is a forwarding cell, so resolution repeats until it reaches the canonical cell whose target is the payload. It then materializes the object's address and accesses the field. Because a cell is never at `0`, a resolution of an untethered `0` never reads a live cell.", + "forwarding-aware tether resolution", + ) + spec = replace_once( + spec, + "The first segmented offset locates the anchor cell. The cell contains the\nsecond segmented offset, which locates the hosted object's current payload.\nBoth use the same chunk-directory resolution rule.", + "The first segmented offset locates an anchor cell. In the common case that cell contains the second segmented offset, which locates the hosted object's current payload. After promotion it may instead locate another anchor cell; resolution follows such forwarding cells until a cell targets the fixed-size region. Every hop uses the same chunk-directory rule, whose region metadata distinguishes forwarding from payload targets.", + "resolution walkthrough forwarding", + ) + spec = replace_once( + spec, + "If the `Weapon` moves or its host is overwritten, the runtime updates only\nthe hosted object's offset stored in the anchor cell. `mainWeapon` continues\nto point at the same cell, and its next access reaches the payload's new location.", + "If the `Weapon` moves or its host is overwritten, the runtime updates only the payload offset in its canonical anchor cell. `mainWeapon` may point directly at that cell or at a source-scope forwarding predecessor; either path reaches the canonical cell and therefore the payload's new location.", + "canonical move resolution", + ) + + promotion_section = """### 4.5 Moves and overwrites update the canonical cell; promotion forwards the old cell +A tether follows an anchor path rather than pointing at a fixed object address. When a host is overwritten in place (§2.2) or its object moves within the same anchor scope, the runtime writes the payload's new segmented offset into the canonical anchor cell named by the payload's backpointer (§4.2). The cell itself does not move, so every tether that reaches it observes the hosted object's current location without per-tether fixup. + +**Promotion** on escape (§3.5) cannot simply reset the backpointer and later mint an unrelated destination cell: source-scope tethers would keep using the old cell while later destination moves updated only the new one. Instead, promotion preserves one forwarding path: + +1. If the payload's backpointer is `0`, promotion copies the payload with a `0` backpointer and allocates no cell. +2. Otherwise, let `old` be the canonical cell named by the source payload's backpointer. Promotion allocates `new` immediately in the destination scope's anchor-cell region. +3. The runtime writes the promoted payload's new fixed-size-region offset into `new`, writes `new`'s segmented offset into `old` (turning `old` into a forwarding cell), and writes `new`'s offset into the promoted payload's backpointer. +4. Existing source-scope tethers continue to point at `old` and resolve through it to `new`; destination-scope tethers minted after promotion point directly at `new`. Every later move or overwrite updates only `new`, the canonical cell, so both paths remain coherent. + +Repeated promotions may form a short chain of one forwarding cell per still-live source scope. Each predecessor is owned by the scope whose tethers can point directly at it and is unmapped only when those tethers expire with that scope. Implementations may path-compress live forwarding cells to the newest canonical cell, but correctness does not depend on compression. Promotion is O(1) in the number of tethers and existing forwarding cells; tether resolution costs one additional dependent load for each uncompressed promotion boundary it crosses. + +This is also how a moved-from symbol stays readable: after a move the symbol downgrades to an `&` — a segmented offset to its scope's cell — and reads follow any forwarding path to the value's current canonical cell (see [`lifetimes.md`](lifetimes.md) §1.6). + +> **Story:** [`stories/memory.md`](../stories/memory.md#the-move-problem-and-the-anchor-that-never-moves) — "The move problem, and the anchor that never moves". [`stories/memory.md`](../stories/memory.md#where-the-cells-live-and-the-scan-that-pays-for-them) — "Where the cells live, and the scan that pays for them". + +""" + spec = replace_regex( + spec, + r"### 4\.5 Moves, overwrites, and promotion update one cell, not all tethers\n.*?(?=### 4\.6 Teardown releases cells in bulk)", + promotion_section, + "promotion forwarding algorithm", + ) + spec = replace_once( + spec, + "Anchor cells are arena allocations, so they are never individually freed. When a scope drains, its chunks — payload and anchor-cell regions alike — are unmapped (§3.2) and its cells vanish together with the hosts and payloads they served.", + "Anchor cells are arena allocations, so they are never individually freed. When a scope drains, its fixed-size, dynamic, and anchor-cell chunks are unmapped (§3.2). Canonical cells, forwarding cells, hosts, and backing stores owned by that scope vanish together.", + "three-region teardown wording", + ) + spec = replace_once( + spec, + "The genuine cost of any anchor scheme is **one extra dependent load per tether resolution** — the cell read — versus an idealized raw pointer that cannot survive moves. Because cells are packed together in the arena's compact anchor-cell region (§4.1), that load usually lands in hot, cache-resident memory, a few cycles at most. It is paid only when resolving a tether; direct access through a host never consults a cell. Across a run of accesses through the same tether with no intervening move, overwrite, or promotion, the compiler resolves the host address once and reuses it, so hot loops do not re-pay the load.", + "The genuine base cost of the anchor scheme is **one extra dependent load per tether resolution** — the canonical cell read — versus an idealized raw pointer that cannot survive moves. An uncompressed forwarding cell left by promotion adds one dependent load for that promotion boundary. Cells are packed in compact scope-local anchor regions (§4.1), and implementations may path-compress forwarding cells, so these loads normally land in hot memory. They are paid only when resolving a tether; direct host access never consults a cell. Across repeated accesses with no intervening move, overwrite, or promotion, the compiler may resolve the host address once and reuse it.", + "forwarding resolution cost", + ) + spec_path.write_text(spec) + + story_path = Path("stories/memory.md") + story = story_path.read_text() + + story = replace_once( + story, + "Growth doubles the block size. The allocator first looks for a reusable block of the doubled size. If none exists and the current buffer is the frontier allocation with room behind it, the frontier advances and the list grows in place. Otherwise a doubled block is bump-allocated at the frontier, the elements move, the handle changes its offset, and the old block is pushed onto its old-size stack. The free stack is therefore the first choice and the frontier the fallback — including the in-place case, which is just a frontier bump that happens not to change the address.", + "Growth doubles the block size. The allocator first looks for a reusable block of the doubled size. If none exists and an ordinary buffer of at most 1 MiB is the frontier allocation with room before its chunk boundary, the frontier advances and the list grows in place. Otherwise the elements move into a new doubled block and the old block is pushed onto its old-size stack. The free stack is therefore the first choice and the frontier the fallback — including the in-place case, which is just a frontier bump that happens not to change the address.", + "story ordinary growth", + ) + story = replace_once( + story, + "This reintroduces size-class fragmentation, but only in the one region whose objects genuinely change size, with powers of two that match the doubling policy and a fixed 128-byte floor that maximizes cross-type reuse. The fixed-size and anchor regions retain the arena's strongest property: no free lists, no holes from reassignment, and one bulk unmap at scope drain. The arena did not abolish free stacks everywhere; it confined them to the place where their reuse is worth their fragmentation.", + "An ordinary block never crosses a 1 MiB chunk boundary. Once doubling asks for more than one chunk, the allocator uses a dedicated contiguous oversized span whose constituent chunks belong only to that block. The handle still names one base segmented offset; the size class tells the runtime how many consecutive chunks the span contains, and indexing proceeds from the resolved contiguous base. Oversized spans are also reused from their exact-size stack first, but they are never extended in place — growth relocates into a doubled span.\n\nThis reintroduces size-class fragmentation, but only in the one region whose objects genuinely change size, with powers of two that match the doubling policy and a fixed 128-byte floor that maximizes cross-type reuse. The fixed-size and anchor regions retain the arena's strongest property: no free lists, no holes from reassignment, and one bulk unmap at scope drain. The arena did not abolish free stacks everywhere; it confined them to the place where their reuse is worth their fragmentation.", + "story oversized spans", + ) + story = replace_once( + story, + "What falls out is a memory model that is uniformly 32-bit and, per tethered object, exactly twelve bytes of machinery: the four-byte tether wherever it is stored, the four-byte anchor cell, and the four-byte backpointer the payload carries home to that cell. The double indirection a tether walks — tether to cell, cell to payload — looks like it should cost two cache misses, and the arena is what makes it cost closer to zero: the cell read is a load into arena memory that is almost always already warm. The first thing we tried for that warmth was to drop each cell right beside the payload that mints it, so the two share a cache line and the second hop is paid for by the first. It worked for the deref, and it opened a loose end the next chapter has to pull on — a cell sitting in the payload stream is a cell a payload-only scan has to step over, and a payload's position starts to depend on how many of its neighbours were tethered before it.", + "What falls out is a memory model whose links are uniformly 32-bit. The minimum tethered case is twelve bytes of machinery — one four-byte tether, one four-byte canonical cell, and the four-byte backpointer the payload carries home to that cell — with four bytes for each additional tether. Promotion may temporarily add one four-byte forwarding cell in each still-live source scope. The usual double indirection — tether to canonical cell, cell to payload — stays compact; a forwarding boundary adds another hot cell read. The first thing we tried for that warmth was to drop each cell right beside the payload that mints it, so the two share a cache line and the second hop is paid for by the first. It worked for the deref, and it opened a loose end the next chapter has to pull on — a cell sitting in the payload stream is a cell a payload-only scan has to step over, and a payload's position starts to depend on how many of its neighbours were tethered before it.", + "story anchor cost", + ) + story = replace_once( + story, + "The one motion this has to survive is escape. A value that outlives its scope is moved into a parent, and under arenas that means its payload is *copied* into the parent's arena — the child arena is about to be unmapped and cannot keep it. The backpointer is what makes that copy invisible: the runtime follows it to the payload's one anchor cell and rewrites the cell with the payload's new location. Every tether still points at that same cell and never learns the payload moved ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)) — the same one-cell-update that made an in-place move O(1) makes a cross-arena promotion O(1) too. When the scope finally drains, its chunks are unmapped and its cells vanish with the objects they served, and the scope rules that governed tethers all along guarantee nothing still pointing could point into them.", + "The one motion this has to survive is escape. A value that outlives its scope is copied into a parent arena, but its source-scope tethers may remain live until the child scope drains. Promotion therefore allocates a new canonical cell in the destination scope, points it at the promoted payload, turns the old canonical cell into a forwarder to the new one, and records the new cell in the payload's backpointer ([`memory.md` §4.5](../spec/memory.md#45-moves-and-overwrites-update-the-canonical-cell-promotion-forwards-the-old-cell)). Old tethers follow the forwarding cell; new tethers point directly at the destination cell; later moves update only that canonical cell and both paths stay current. Promotion remains O(1) in the number of tethers, while each live promotion boundary can add one dependent cell load until its source scope drains.", + "story promotion overview", + ) + story = replace_once( + story, + "The cost is real and we name it plainly. We gave up the shared-cache-line bonus on the deref itself — the cell and its payload no longer ride into cache together, so a single tether resolution can pay two loads where the interleaved layout often paid one. We judged that the right trade because the payload sweep is the hotter path in the workloads we cared about: densifying the scan that runs over every object beats shaving a load off the deref that runs only when a tether is actually followed. It is the mirror image of the choice the last chapter made, now that we have measured which side of the coin comes up more often.", + "The cost is real and we name it plainly. We gave up the shared-cache-line bonus on the deref itself — the cell and its payload no longer ride into cache together, so the common tether resolution pays the canonical cell load, and an uncompressed promotion boundary adds another. We judged that the right trade because the payload sweep is the hotter path in the workloads we cared about, and forwarding appears only when a tethered value crosses scopes. Dense scans beat permanently interleaving metadata, while implementations remain free to path-compress the uncommon forwarding chain.", + "story forwarding cost", + ) + story = replace_once( + story, + "Separating the region also forced us to finish a sentence the last chapter had left dangling. It had described promotion as \"rewrite the one cell and every tether follows,\" which quietly skipped the question of *which arena that cell is in*. Now the answer is unambiguous: the cell lives in the region of the scope that minted it, and on escape that scope is precisely the one about to drain. Every tether already pointing at the cell was taken in that scope or deeper, so none of them outlives it — the lifetime rule that has fenced tethers all along ([`lifetimes.md` §1.1](https://github.com/zane-lang/spec/blob/93bd2f0036b011c9fc876e785bff0d6a4d09465a/spec/lifetimes.md#11--assignment-uses-owner-scope)) does the fencing here too. So promotion updates the old cell to the payload's new home, keeping those doomed-but-still-live tethers reading the promoted copy until their scope ends, and then **resets the payload's backpointer to zero** so the value re-anchors from scratch in its new scope: the next tether taken on it there mints a fresh cell in the destination region, one that finally lives as long as the value does ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/2caab48a9f4bafda32d67c3bb902908aea1813ab/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)). The old cell and its tethers expire together when the source scope drains, and nothing ever reads a cell that has been unmapped. That reset is the honest tax of keeping cells scope-local rather than in one immortal table — a second, cheaper re-anchoring the immortal table would not have needed, paid so that anchors can vanish in the same unmap as everything else.", + "Separating the region also forced us to finish a sentence the last chapter had left dangling. The source cell cannot simply be repointed at the payload and forgotten while the destination later mints an unrelated cell: with only one payload backpointer, a subsequent move would update the destination cell and leave the source path stale. Promotion therefore creates the destination cell immediately. The new cell becomes canonical and points at the promoted payload; the old source cell is rewritten to point at the new cell; and the payload's backpointer is changed to the new cell ([`memory.md` §4.5](../spec/memory.md#45-moves-and-overwrites-update-the-canonical-cell-promotion-forwards-the-old-cell)). Region metadata in the chunk directory tells resolution whether a cell points at a payload or forwards to another cell.\n\nThat gives each scope exactly the path it needs. Existing source tethers keep their old four-byte identity and follow it across the promotion boundary, while destination tethers are minted from the new canonical backpointer. Later moves and overwrites touch only the canonical cell, and every predecessor still leads there. Repeated promotions can form a short chain, one cell per still-live source scope; those cells and the tethers that can name them expire together as their scopes drain, and an implementation may path-compress the chain. The price of scope-local anchors is therefore an occasional extra dependent load, not a stale path or a list of tethers to rewrite.", + "story normative promotion rationale", + ) + story_path.write_text(story) + PY + + - name: Commit review fixes + shell: bash + run: | + rm .github/workflows/apply-review-fixes.yml + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add spec/memory.md stories/memory.md .github/workflows/apply-review-fixes.yml + git commit -m "Address allocator and promotion review" + git push From ef78cde8c8c65b6d60e7746edde68285fae31cb6 Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Sat, 25 Jul 2026 15:20:50 +0200 Subject: [PATCH 06/31] Remove inactive review updater --- .github/workflows/apply-review-fixes.yml | 214 ----------------------- 1 file changed, 214 deletions(-) delete mode 100644 .github/workflows/apply-review-fixes.yml diff --git a/.github/workflows/apply-review-fixes.yml b/.github/workflows/apply-review-fixes.yml deleted file mode 100644 index 52e0554..0000000 --- a/.github/workflows/apply-review-fixes.yml +++ /dev/null @@ -1,214 +0,0 @@ -name: Apply memory review fixes - -on: - push: - branches: - - agent/fix-host-overwrite-reuse - -permissions: - contents: write - -jobs: - update: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - with: - ref: agent/fix-host-overwrite-reuse - fetch-depth: 0 - - - name: Address allocator and promotion review - shell: bash - run: | - python3 <<'PY' - from pathlib import Path - import re - - def replace_once(text: str, old: str, new: str, label: str) -> str: - count = text.count(old) - if count != 1: - raise RuntimeError(f"expected one literal for {label}, got {count}") - return text.replace(old, new, 1) - - def replace_regex(text: str, pattern: str, new: str, label: str) -> str: - updated, count = re.subn(pattern, new, text, count=1, flags=re.S) - if count != 1: - raise RuntimeError(f"expected one regex replacement for {label}, got {count}") - return updated - - spec_path = Path("spec/memory.md") - spec = spec_path.read_text() - - spec = replace_once( - spec, - "Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots, dynamic backing stores, and anchor cells never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. Growing one region never copies or relocates allocations in another.", - "Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots, dynamic backing stores, and anchor cells never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. Growing one region never copies or relocates allocations in another.\n\nAn ordinary allocation never straddles a chunk boundary. A dynamic block of at most 1 MiB is wholly contained in one dynamic chunk; if the remaining bytes in the current chunk cannot hold it, allocation continues in a fresh dynamic chunk. A dynamic block larger than 1 MiB is an **oversized span**: a dedicated contiguous OS mapping containing `block_size / 1 MiB` chunks, all belonging exclusively to that one block and assigned consecutive chunk ids. Its handle stores the segmented offset of the span's first byte and its size class; after resolving that base, element addressing uses an ordinary byte offset across the contiguous mapping. Every constituent chunk also has its own directory entry. Returning an oversized span pushes only its base offset onto the exact-size stack, and the whole span remains mapped for reuse until the scope drains.", - "oversized dynamic span representation", - ) - - old_growth = """A list grows according to the following rules: - -1. When its capacity is exhausted, the requested block size is exactly twice its current block size. -2. The allocator first checks the size stack for that doubled size. If a block is available, it is popped and the live elements are relocated into it. -3. If that stack is empty and the current backing store is the dynamic frontier allocation with enough contiguous room to double, the frontier is bumped by the additional bytes and the store grows in place. -4. Otherwise, the doubled block is allocated by bumping the dynamic frontier, mapping more dynamic chunks as needed, and the live elements are relocated into it. -5. After relocation, the handle's backing-store offset and size class are updated and the old block's offset is pushed onto the size stack for its old byte size. - -Relocation moves or copies elements according to their type's ordinary move rules; the old block becomes reusable only after its previous occupants are no longer live. Guests to the list remain valid because they reach the list's host, whose fixed-size handle now names the current backing store. - -Dynamic chunks and all power-of-two blocks begin at cache-line-aligned addresses. Because the minimum block is 128 bytes and every larger block doubles, both frontier allocations and reused blocks preserve cache-line alignment without mixing backing stores into fixed-size chunks.""" - new_growth = """A list grows according to the following rules: - -1. When its capacity is exhausted, the requested block size is exactly twice its current block size. -2. The allocator first checks the size stack for that doubled size. If a block or oversized span is available, it is popped and the live elements are relocated into it. -3. If that stack is empty, the current backing store is the dynamic frontier allocation, the doubled size is at most 1 MiB, and the additional bytes fit before the current chunk boundary, the frontier is bumped and the store grows in place. -4. Otherwise, a doubled block of at most 1 MiB is bump-allocated wholly within one dynamic chunk. A doubled block larger than 1 MiB is allocated as a fresh dedicated oversized span (§3.1). The live elements are then relocated into the new block or span. -5. After relocation, the handle's backing-store offset and size class are updated and the old block's base offset is pushed onto the stack for its exact old byte size. - -A block never grows in place across a chunk boundary, and an oversized span is never extended in place: further growth relocates into a doubled oversized span after checking that exact-size stack first. Relocation moves or copies elements according to their type's ordinary move rules; the old block becomes reusable only after its previous occupants are no longer live. Guests to the list remain valid because they reach the list's host, whose fixed-size handle now names the current backing store. - -Dynamic chunks, ordinary power-of-two blocks, and oversized spans begin at cache-line-aligned addresses. Because the minimum block is 128 bytes and every larger block doubles, frontier allocations, reused blocks, and dedicated spans preserve cache-line alignment without mixing backing stores into fixed-size chunks.""" - spec = replace_once(spec, old_growth, new_growth, "list growth and oversized spans") - - new_anchor = """### 4.1 The anchor cell -Tethers are tracked through per-host **anchor cells** rather than one shared table. An anchor cell is a single **`u32`**. A canonical cell contains the current segmented offset (§3.1) of its hosted object in the fixed-size region. A source-scope cell left behind by promotion may instead contain the segmented offset of a newer anchor cell and acts as a **forwarding cell** (§4.5). The chunk directory records each chunk's region kind, so resolution distinguishes a payload target from a forwarding target without adding a tag to the cell. - -Anchor storage is **scope-local**, never global. Each scope owns a dedicated anchor-cell region — a separate lazy chunk chain from both its fixed-size and dynamic regions. A scope that never creates a guest allocates no anchor chunk. The first tether to a host bump-allocates its canonical cell in that scope's anchor region (§4.3); promotion may allocate a replacement canonical cell in a destination scope and turn the previous cell into a forwarder (§4.5). No monolithic table is ever resized. - -Keeping cells out of the other two streams preserves dense fixed-size layout and prevents dynamic-buffer history from affecting anchor placement. The region remains compact while live, and all of its chunks disappear with the scope in the same bulk unmap as the other regions. - -> **Story:** [`stories/memory.md`](../stories/memory.md#where-the-cells-live-and-the-scan-that-pays-for-them) — "Where the cells live, and the scan that pays for them". - -""" - spec = replace_regex( - spec, - r"### 4\.1 The anchor cell\n.*?(?=### 4\.2 Tethers are segmented offsets, not pointers)", - new_anchor, - "anchor cell representation", - ) - - spec = replace_once( - spec, - "Every reference-type instance reserves a **`u32` backpointer** field, initialized to `0`; the first tether records the segmented offset of the instance's anchor cell there. The cell is allocated lazily (§4.3), whereas the backpointer field is always present in the layout, so object size is fixed and array layout stays uniform. The backpointer lets a host mint new tethers from the object — `&x` copies the offset — and lets a move locate and update the object's cell (§4.5). It is a single offset, not a list of tethers: the runtime never enumerates the tethers that point at the object, which is what keeps moves O(1) (§4.5).", - "Every reference-type instance reserves a **`u32` backpointer** field, initialized to `0`; the first tether records the segmented offset of the instance's canonical anchor cell there. The cell is allocated lazily (§4.3), whereas the backpointer field is always present in the layout, so object size is fixed and array layout stays uniform. The backpointer always names the newest canonical cell, never a forwarding predecessor. It lets a host mint new tethers by copying that canonical offset and lets a move locate and update the one authoritative cell (§4.5). The runtime never enumerates the tethers or forwarding predecessors that ultimately reach it.", - "canonical backpointer semantics", - ) - spec = replace_once( - spec, - "A tethered reference-type instance therefore costs **12 bytes** across the whole chain: the 4-byte tether (wherever it is stored), the 4-byte anchor cell, and the 4-byte backpointer in the payload.", - "The minimum machinery for one tethered reference-type instance is **12 bytes**: one 4-byte tether, one 4-byte canonical anchor cell, and the 4-byte backpointer in the payload. Each additional tether costs another 4 bytes. Each still-live source scope crossed by promotion may temporarily retain one additional 4-byte forwarding cell until that scope drains (§4.5).", - "anchor cost accounting", - ) - spec = replace_once( - spec, - "A hosted object that never gains a tether consumes no cell: its backpointer field stays `0` and no cell is allocated (it still carries the 4-byte field, §4.2). The first `&` taken on its host bump-allocates a cell in the arena, writes the hosted object's current segmented offset into it, and records the cell's own segmented offset in the object's backpointer. Every subsequent `&` from that host copies the backpointer.", - "A hosted object that never gains a tether consumes no cell: its backpointer field stays `0` and no cell is allocated (it still carries the 4-byte field, §4.2). The first `&` taken on its host bump-allocates a canonical cell in the scope's anchor region, writes the hosted object's current segmented offset into it, and records the cell's own segmented offset in the object's backpointer. Every subsequent `&` from that host copies the canonical backpointer.", - "lazy canonical anchor creation", - ) - spec = replace_once( - spec, - "Resolving a tether reads the anchor cell it points at, reads the hosted object's segmented offset from that cell, materializes the object's address through the chunk directory (§3.1), then accesses the field. Because a cell is never at `0`, a resolution of an untethered `0` never reads a live cell.", - "Resolving a tether reads the anchor cell it points at and inspects the region kind of the cell's target through the chunk directory (§3.1). A target in the fixed-size region is the hosted object's current location. A target in an anchor-cell region is a forwarding cell, so resolution repeats until it reaches the canonical cell whose target is the payload. It then materializes the object's address and accesses the field. Because a cell is never at `0`, a resolution of an untethered `0` never reads a live cell.", - "forwarding-aware tether resolution", - ) - spec = replace_once( - spec, - "The first segmented offset locates the anchor cell. The cell contains the\nsecond segmented offset, which locates the hosted object's current payload.\nBoth use the same chunk-directory resolution rule.", - "The first segmented offset locates an anchor cell. In the common case that cell contains the second segmented offset, which locates the hosted object's current payload. After promotion it may instead locate another anchor cell; resolution follows such forwarding cells until a cell targets the fixed-size region. Every hop uses the same chunk-directory rule, whose region metadata distinguishes forwarding from payload targets.", - "resolution walkthrough forwarding", - ) - spec = replace_once( - spec, - "If the `Weapon` moves or its host is overwritten, the runtime updates only\nthe hosted object's offset stored in the anchor cell. `mainWeapon` continues\nto point at the same cell, and its next access reaches the payload's new location.", - "If the `Weapon` moves or its host is overwritten, the runtime updates only the payload offset in its canonical anchor cell. `mainWeapon` may point directly at that cell or at a source-scope forwarding predecessor; either path reaches the canonical cell and therefore the payload's new location.", - "canonical move resolution", - ) - - promotion_section = """### 4.5 Moves and overwrites update the canonical cell; promotion forwards the old cell -A tether follows an anchor path rather than pointing at a fixed object address. When a host is overwritten in place (§2.2) or its object moves within the same anchor scope, the runtime writes the payload's new segmented offset into the canonical anchor cell named by the payload's backpointer (§4.2). The cell itself does not move, so every tether that reaches it observes the hosted object's current location without per-tether fixup. - -**Promotion** on escape (§3.5) cannot simply reset the backpointer and later mint an unrelated destination cell: source-scope tethers would keep using the old cell while later destination moves updated only the new one. Instead, promotion preserves one forwarding path: - -1. If the payload's backpointer is `0`, promotion copies the payload with a `0` backpointer and allocates no cell. -2. Otherwise, let `old` be the canonical cell named by the source payload's backpointer. Promotion allocates `new` immediately in the destination scope's anchor-cell region. -3. The runtime writes the promoted payload's new fixed-size-region offset into `new`, writes `new`'s segmented offset into `old` (turning `old` into a forwarding cell), and writes `new`'s offset into the promoted payload's backpointer. -4. Existing source-scope tethers continue to point at `old` and resolve through it to `new`; destination-scope tethers minted after promotion point directly at `new`. Every later move or overwrite updates only `new`, the canonical cell, so both paths remain coherent. - -Repeated promotions may form a short chain of one forwarding cell per still-live source scope. Each predecessor is owned by the scope whose tethers can point directly at it and is unmapped only when those tethers expire with that scope. Implementations may path-compress live forwarding cells to the newest canonical cell, but correctness does not depend on compression. Promotion is O(1) in the number of tethers and existing forwarding cells; tether resolution costs one additional dependent load for each uncompressed promotion boundary it crosses. - -This is also how a moved-from symbol stays readable: after a move the symbol downgrades to an `&` — a segmented offset to its scope's cell — and reads follow any forwarding path to the value's current canonical cell (see [`lifetimes.md`](lifetimes.md) §1.6). - -> **Story:** [`stories/memory.md`](../stories/memory.md#the-move-problem-and-the-anchor-that-never-moves) — "The move problem, and the anchor that never moves". [`stories/memory.md`](../stories/memory.md#where-the-cells-live-and-the-scan-that-pays-for-them) — "Where the cells live, and the scan that pays for them". - -""" - spec = replace_regex( - spec, - r"### 4\.5 Moves, overwrites, and promotion update one cell, not all tethers\n.*?(?=### 4\.6 Teardown releases cells in bulk)", - promotion_section, - "promotion forwarding algorithm", - ) - spec = replace_once( - spec, - "Anchor cells are arena allocations, so they are never individually freed. When a scope drains, its chunks — payload and anchor-cell regions alike — are unmapped (§3.2) and its cells vanish together with the hosts and payloads they served.", - "Anchor cells are arena allocations, so they are never individually freed. When a scope drains, its fixed-size, dynamic, and anchor-cell chunks are unmapped (§3.2). Canonical cells, forwarding cells, hosts, and backing stores owned by that scope vanish together.", - "three-region teardown wording", - ) - spec = replace_once( - spec, - "The genuine cost of any anchor scheme is **one extra dependent load per tether resolution** — the cell read — versus an idealized raw pointer that cannot survive moves. Because cells are packed together in the arena's compact anchor-cell region (§4.1), that load usually lands in hot, cache-resident memory, a few cycles at most. It is paid only when resolving a tether; direct access through a host never consults a cell. Across a run of accesses through the same tether with no intervening move, overwrite, or promotion, the compiler resolves the host address once and reuses it, so hot loops do not re-pay the load.", - "The genuine base cost of the anchor scheme is **one extra dependent load per tether resolution** — the canonical cell read — versus an idealized raw pointer that cannot survive moves. An uncompressed forwarding cell left by promotion adds one dependent load for that promotion boundary. Cells are packed in compact scope-local anchor regions (§4.1), and implementations may path-compress forwarding cells, so these loads normally land in hot memory. They are paid only when resolving a tether; direct host access never consults a cell. Across repeated accesses with no intervening move, overwrite, or promotion, the compiler may resolve the host address once and reuse it.", - "forwarding resolution cost", - ) - spec_path.write_text(spec) - - story_path = Path("stories/memory.md") - story = story_path.read_text() - - story = replace_once( - story, - "Growth doubles the block size. The allocator first looks for a reusable block of the doubled size. If none exists and the current buffer is the frontier allocation with room behind it, the frontier advances and the list grows in place. Otherwise a doubled block is bump-allocated at the frontier, the elements move, the handle changes its offset, and the old block is pushed onto its old-size stack. The free stack is therefore the first choice and the frontier the fallback — including the in-place case, which is just a frontier bump that happens not to change the address.", - "Growth doubles the block size. The allocator first looks for a reusable block of the doubled size. If none exists and an ordinary buffer of at most 1 MiB is the frontier allocation with room before its chunk boundary, the frontier advances and the list grows in place. Otherwise the elements move into a new doubled block and the old block is pushed onto its old-size stack. The free stack is therefore the first choice and the frontier the fallback — including the in-place case, which is just a frontier bump that happens not to change the address.", - "story ordinary growth", - ) - story = replace_once( - story, - "This reintroduces size-class fragmentation, but only in the one region whose objects genuinely change size, with powers of two that match the doubling policy and a fixed 128-byte floor that maximizes cross-type reuse. The fixed-size and anchor regions retain the arena's strongest property: no free lists, no holes from reassignment, and one bulk unmap at scope drain. The arena did not abolish free stacks everywhere; it confined them to the place where their reuse is worth their fragmentation.", - "An ordinary block never crosses a 1 MiB chunk boundary. Once doubling asks for more than one chunk, the allocator uses a dedicated contiguous oversized span whose constituent chunks belong only to that block. The handle still names one base segmented offset; the size class tells the runtime how many consecutive chunks the span contains, and indexing proceeds from the resolved contiguous base. Oversized spans are also reused from their exact-size stack first, but they are never extended in place — growth relocates into a doubled span.\n\nThis reintroduces size-class fragmentation, but only in the one region whose objects genuinely change size, with powers of two that match the doubling policy and a fixed 128-byte floor that maximizes cross-type reuse. The fixed-size and anchor regions retain the arena's strongest property: no free lists, no holes from reassignment, and one bulk unmap at scope drain. The arena did not abolish free stacks everywhere; it confined them to the place where their reuse is worth their fragmentation.", - "story oversized spans", - ) - story = replace_once( - story, - "What falls out is a memory model that is uniformly 32-bit and, per tethered object, exactly twelve bytes of machinery: the four-byte tether wherever it is stored, the four-byte anchor cell, and the four-byte backpointer the payload carries home to that cell. The double indirection a tether walks — tether to cell, cell to payload — looks like it should cost two cache misses, and the arena is what makes it cost closer to zero: the cell read is a load into arena memory that is almost always already warm. The first thing we tried for that warmth was to drop each cell right beside the payload that mints it, so the two share a cache line and the second hop is paid for by the first. It worked for the deref, and it opened a loose end the next chapter has to pull on — a cell sitting in the payload stream is a cell a payload-only scan has to step over, and a payload's position starts to depend on how many of its neighbours were tethered before it.", - "What falls out is a memory model whose links are uniformly 32-bit. The minimum tethered case is twelve bytes of machinery — one four-byte tether, one four-byte canonical cell, and the four-byte backpointer the payload carries home to that cell — with four bytes for each additional tether. Promotion may temporarily add one four-byte forwarding cell in each still-live source scope. The usual double indirection — tether to canonical cell, cell to payload — stays compact; a forwarding boundary adds another hot cell read. The first thing we tried for that warmth was to drop each cell right beside the payload that mints it, so the two share a cache line and the second hop is paid for by the first. It worked for the deref, and it opened a loose end the next chapter has to pull on — a cell sitting in the payload stream is a cell a payload-only scan has to step over, and a payload's position starts to depend on how many of its neighbours were tethered before it.", - "story anchor cost", - ) - story = replace_once( - story, - "The one motion this has to survive is escape. A value that outlives its scope is moved into a parent, and under arenas that means its payload is *copied* into the parent's arena — the child arena is about to be unmapped and cannot keep it. The backpointer is what makes that copy invisible: the runtime follows it to the payload's one anchor cell and rewrites the cell with the payload's new location. Every tether still points at that same cell and never learns the payload moved ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)) — the same one-cell-update that made an in-place move O(1) makes a cross-arena promotion O(1) too. When the scope finally drains, its chunks are unmapped and its cells vanish with the objects they served, and the scope rules that governed tethers all along guarantee nothing still pointing could point into them.", - "The one motion this has to survive is escape. A value that outlives its scope is copied into a parent arena, but its source-scope tethers may remain live until the child scope drains. Promotion therefore allocates a new canonical cell in the destination scope, points it at the promoted payload, turns the old canonical cell into a forwarder to the new one, and records the new cell in the payload's backpointer ([`memory.md` §4.5](../spec/memory.md#45-moves-and-overwrites-update-the-canonical-cell-promotion-forwards-the-old-cell)). Old tethers follow the forwarding cell; new tethers point directly at the destination cell; later moves update only that canonical cell and both paths stay current. Promotion remains O(1) in the number of tethers, while each live promotion boundary can add one dependent cell load until its source scope drains.", - "story promotion overview", - ) - story = replace_once( - story, - "The cost is real and we name it plainly. We gave up the shared-cache-line bonus on the deref itself — the cell and its payload no longer ride into cache together, so a single tether resolution can pay two loads where the interleaved layout often paid one. We judged that the right trade because the payload sweep is the hotter path in the workloads we cared about: densifying the scan that runs over every object beats shaving a load off the deref that runs only when a tether is actually followed. It is the mirror image of the choice the last chapter made, now that we have measured which side of the coin comes up more often.", - "The cost is real and we name it plainly. We gave up the shared-cache-line bonus on the deref itself — the cell and its payload no longer ride into cache together, so the common tether resolution pays the canonical cell load, and an uncompressed promotion boundary adds another. We judged that the right trade because the payload sweep is the hotter path in the workloads we cared about, and forwarding appears only when a tethered value crosses scopes. Dense scans beat permanently interleaving metadata, while implementations remain free to path-compress the uncommon forwarding chain.", - "story forwarding cost", - ) - story = replace_once( - story, - "Separating the region also forced us to finish a sentence the last chapter had left dangling. It had described promotion as \"rewrite the one cell and every tether follows,\" which quietly skipped the question of *which arena that cell is in*. Now the answer is unambiguous: the cell lives in the region of the scope that minted it, and on escape that scope is precisely the one about to drain. Every tether already pointing at the cell was taken in that scope or deeper, so none of them outlives it — the lifetime rule that has fenced tethers all along ([`lifetimes.md` §1.1](https://github.com/zane-lang/spec/blob/93bd2f0036b011c9fc876e785bff0d6a4d09465a/spec/lifetimes.md#11--assignment-uses-owner-scope)) does the fencing here too. So promotion updates the old cell to the payload's new home, keeping those doomed-but-still-live tethers reading the promoted copy until their scope ends, and then **resets the payload's backpointer to zero** so the value re-anchors from scratch in its new scope: the next tether taken on it there mints a fresh cell in the destination region, one that finally lives as long as the value does ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/2caab48a9f4bafda32d67c3bb902908aea1813ab/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)). The old cell and its tethers expire together when the source scope drains, and nothing ever reads a cell that has been unmapped. That reset is the honest tax of keeping cells scope-local rather than in one immortal table — a second, cheaper re-anchoring the immortal table would not have needed, paid so that anchors can vanish in the same unmap as everything else.", - "Separating the region also forced us to finish a sentence the last chapter had left dangling. The source cell cannot simply be repointed at the payload and forgotten while the destination later mints an unrelated cell: with only one payload backpointer, a subsequent move would update the destination cell and leave the source path stale. Promotion therefore creates the destination cell immediately. The new cell becomes canonical and points at the promoted payload; the old source cell is rewritten to point at the new cell; and the payload's backpointer is changed to the new cell ([`memory.md` §4.5](../spec/memory.md#45-moves-and-overwrites-update-the-canonical-cell-promotion-forwards-the-old-cell)). Region metadata in the chunk directory tells resolution whether a cell points at a payload or forwards to another cell.\n\nThat gives each scope exactly the path it needs. Existing source tethers keep their old four-byte identity and follow it across the promotion boundary, while destination tethers are minted from the new canonical backpointer. Later moves and overwrites touch only the canonical cell, and every predecessor still leads there. Repeated promotions can form a short chain, one cell per still-live source scope; those cells and the tethers that can name them expire together as their scopes drain, and an implementation may path-compress the chain. The price of scope-local anchors is therefore an occasional extra dependent load, not a stale path or a list of tethers to rewrite.", - "story normative promotion rationale", - ) - story_path.write_text(story) - PY - - - name: Commit review fixes - shell: bash - run: | - rm .github/workflows/apply-review-fixes.yml - git config user.name "github-actions[bot]" - git config user.email "41898282+github-actions[bot]@users.noreply.github.com" - git add spec/memory.md stories/memory.md .github/workflows/apply-review-fixes.yml - git commit -m "Address allocator and promotion review" - git push From 85f29313947e27a5c2619e85291b7626042a67ff Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Sat, 25 Jul 2026 22:37:01 +0200 Subject: [PATCH 07/31] Use global recyclable anchor pool --- spec/memory.md | 163 ++++++++++++++++++++++++------------------------- 1 file changed, 80 insertions(+), 83 deletions(-) diff --git a/spec/memory.md b/spec/memory.md index 30ee5b8..44c0741 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -15,7 +15,7 @@ Zane eliminates dangling guests by combining single hosting, lexical lifetime ru - **`Repointable guests`.** A guest is non-hosting storage that can point at different hosts over time. - **`Lexical lifetime enforcement`.** Guest assignment and rehosting are checked using declaration scope alone (see [`lifetimes.md`](lifetimes.md) §1). - **`Deterministic destruction`.** Objects are destroyed when their hosting scope drains; there is no tracing garbage collector (see [`lifetimes.md`](lifetimes.md) §2). -- **`Regioned arena placement`.** Every scope owns separate fixed-size, dynamic-backing-store, and anchor-cell regions. Statically sized storage is placed inline in the fixed-size region; resizable data uses the dynamic region (see §3). +- **`Regioned arena placement`.** Every scope owns separate fixed-size and dynamic-backing-store regions. Statically sized storage is placed inline in the fixed-size region; resizable data uses the dynamic region. Anchors live outside scope arenas in one runtime-global fixed-slot pool (see §3 and §4). - **`Segmented-offset tethers`.** Internally, each guest is represented by a `u32` tether — a chunk id plus an in-chunk offset — that points at the host's anchor cell, not a raw pointer (see §4.2). The source language and runtime use separate terms: an object lives in a **host**, and a **guest** (`&T`) may access it without storing it or controlling its lifetime. Internally, each guest is represented by a **tether** that resolves through an **anchor**. Moving the object updates the anchor, so existing tethers — and therefore guests — continue to reach it. @@ -61,6 +61,8 @@ pos = Vec2(3, 4) // whole-slot overwrite ### 2.4 `&` is a guest: non-hosting storage `&` creates a **guest**: non-hosting storage that points at a **reference type** only. An `&T` requires `T` to be a reference type — a declared `#struct`/`#variant`/`#enum` — because only a reference type carries the identity (the anchor, §4) that a stable, move-surviving guest needs. A value type is shared by copying it or by a scoped borrow (see [`functions.md`](functions.md) §2.4), never by a stored guest. Writing `&Node` names a guest to a reference type; a bare `&Int` over a value type is ill-formed. +An explicitly declared `&T` slot is **guest-only**: it stores only a tether and can never directly host a `T`. A slot declared as `T` is **host-capable**. After its value is rehosted, that same full-size slot may remain readable in guest state, but it retains the storage needed to host another `T` later. Guest-only and host-capable guest states use the same access semantics, but only the latter can become a host again. + A guest may be declared as: - a local symbol @@ -225,27 +227,28 @@ if runtimeBool() { ## 3. Memory Layout -### 3.1 Scope arenas and segmented offsets -The runtime does not reserve one flat region. Each lexical scope owns an **arena** made from three independent allocation regions: +### 3.1 Scope arenas, the global anchor pool, and segmented offsets +Each lexical scope owns an **arena** made from two independent allocation regions: - The **fixed-size region** stores materialized value-type slots, statically sized reference-type hosts, and the fixed-size handles of dynamic core types. - The **dynamic region** stores the resizable backing stores behind handles such as `List` and `String`. -- The **anchor-cell region** stores the scope-local anchor cells created for tethered hosts (§4.1). -Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots, dynamic backing stores, and anchor cells never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. Growing one region never copies or relocates allocations in another. +Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots and dynamic backing stores never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. -```text -one scope arena +Anchors do not belong to any scope arena. The runtime owns one **global anchor pool**, implemented as a lazy chain of anchor-only 1 MiB pages. Every anchor slot has the same fixed width. The pool maps its first page only when the program creates its first guest and adds another page only when its current frontier and free-address stack cannot satisfy an allocation. -fixed-size region dynamic region anchor-cell region -───────────────── ────────────── ────────────────── -[fixed chunk] [dynamic chunk] [anchor chunk] -[fixed chunk] → ... [dynamic chunk] → ... [anchor chunk] → ... +```text +one scope arena runtime-global anchor pool +────────────────────────────── ────────────────────────── +fixed-size region dynamic region [anchor page] → [anchor page] → ... +[F1] → [F2] [D1] → [D2] ``` -The chains are lazy and independent. A scope that uses no dynamic backing store maps no dynamic chunk; a scope that never creates a guest maps no anchor-cell chunk. When the scope drains, every chunk belonging to all three regions is unmapped together. The compiler may optimize away or coalesce physically unobservable storage, but it **MUST** preserve region exclusivity, lifetime, and drain behavior. +An ordinary dynamic allocation never straddles a chunk boundary. A dynamic block of at most 1 MiB is wholly contained in one dynamic chunk; if the remaining bytes in the current chunk cannot hold it, allocation continues in a fresh dynamic chunk. + +A dynamic block larger than 1 MiB is an **oversized span**: a dedicated contiguous OS mapping made from `block_size / 1 MiB` consecutive dynamic chunks, all belonging exclusively to that block and assigned consecutive chunk ids. Its handle stores the segmented offset of the span's first byte and its size class. After resolving that base, element addressing uses an ordinary byte offset across the contiguous mapping. Every constituent chunk also has a directory entry. Returning an oversized span pushes only its base offset onto the exact-size stack; the complete span remains mapped for reuse until the scope drains. -All three regions draw chunk ids from the same chunk directory, so every in-arena location uses the same **`u32` segmented offset**. The `u32` splits into two fields: +Scope chunks and global anchor pages draw ids from the same chunk directory, so payload locations, dynamic handles, tethers, backpointers, anchor cells, and size-stack entries all use one **`u32` segmented offset**: ``` u32 segmented offset @@ -255,20 +258,22 @@ All three regions draw chunk ids from the same chunk directory, so every in-aren └───────────────┴─────────────────────────┘ ``` -Allocations are at least 8-byte aligned, so the low bits count 8-byte words: a 1 MiB chunk holds 2¹⁷ words, so **17 low bits** address any slot in a chunk and the remaining **15 high bits** select one of up to 32768 live chunks — a reach of 32 GiB. The chunk directory maps a chunk id to the chunk's native base address, so an address is materialized only at use as `directory[chunk id] + word offset × 8`. +Allocations are at least 8-byte aligned, so the low bits count 8-byte words: a 1 MiB chunk holds 2¹⁷ words, so **17 low bits** address any slot in a chunk and the remaining **15 high bits** select one of up to 32768 live chunks — a reach of 32 GiB. The chunk directory maps a chunk id to the chunk's native base address. -Tethers (§4.2), per-host backpointers (§4.2), anchor cells (§4.1), dynamic handles, and size-stack entries (§3.2) use segmented offsets. The value `0` — chunk `0`, word `0` — is the *untethered* sentinel. It costs no reserved memory because anchor cells are allocated only in the anchor-cell region, which never contains that location. Fixed-size payloads may occupy offset `0`. +Tethers (§4.2), per-host backpointers (§4.2), anchor cells (§4.1), dynamic handles, and size-stack entries (§3.2) use segmented offsets. The value `0` is the *untethered* sentinel wherever an anchor identity is expected. The global anchor pool never assigns segmented offset `0`; fixed-size payloads may still occupy it. > **Story:** [`stories/memory.md`](../stories/memory.md#the-last-table-problem-and-the-segmented-offset) — "The last table problem, and the segmented offset". -### 3.2 Allocation is a bump; teardown is an unmap -The fixed-size and anchor-cell regions are pure bump allocators: allocation advances the region's frontier, with no size classes, free lists, or coalescing. A host is a fixed-size storage slot, so overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7); it consumes no new arena space. +### 3.2 Allocation, reuse, and teardown +The fixed-size region is a pure bump allocator. A host is a fixed-size storage slot, so overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7); it consumes no new arena space. -The dynamic region adds exact-size reuse on top of its bump frontier. Dynamic blocks use power-of-two byte sizes beginning at **128 bytes**. Each scope maintains one LIFO **size stack** for every block size that has become reusable. To allocate a dynamic block of size `S`, the runtime first pops `size_stack[S]`; only when that stack is empty does it bump the dynamic frontier, mapping more dynamic chunks as needed. It never satisfies a request from a different size stack and never coalesces neighbouring free blocks. +The dynamic region adds exact-size reuse on top of its bump frontier. Dynamic blocks use power-of-two byte sizes beginning at **128 bytes**. Each scope maintains one LIFO **size stack** for every block size that has become reusable. To allocate a dynamic block of size `S`, the runtime first pops `size_stack[S]`; only when that stack is empty does it bump the dynamic frontier. It never satisfies a request from another size stack and never coalesces neighbouring blocks. -Returning a dynamic block pushes its segmented offset onto the stack for that exact byte size. These stacks are shared by all dynamic types in the scope: a 128-byte block previously used by a `List` may later hold string bytes or another list's elements. The stacks affect allocation within the scope only; they require no per-object reclamation when the scope drains. +Returning a dynamic block pushes its base segmented offset onto the stack for that exact byte size. The stacks are shared by all dynamic types in the scope: a 128-byte block previously used by a `List` may later hold string bytes or another list's elements. An oversized span participates in the same exact-size policy. -Reclamation remains bulk. When the scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps every fixed-size, dynamic, and anchor-cell chunk owned by the scope, with no reachability scan or per-object memory-reclamation pass. Anything that escaped was already placed in storage whose lifetime covers its destination host (§3.5). Logical destruction timing is unchanged — a host dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); the underlying region memory is released together at drain. +The global anchor pool has one LIFO **free-address stack**, because every anchor slot has the same size. Creating an anchor pops that stack first; only when it is empty does allocation bump the global anchor frontier, mapping another anchor page as needed. Returning an anchor pushes its segmented offset onto the same stack. + +When a scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps its fixed-size and dynamic chunks in bulk. Global anchor pages are not tied to scope teardown: individual slots are returned when their hosting lineages end (§4.6). A runtime may unmap a wholly free anchor page, but slot reuse does not depend on page reclamation. > **Story:** [`stories/memory.md`](../stories/memory.md#when-the-free-stacks-fragment-and-the-arena-takes-the-scope) — "When the free stacks fragment, and the arena takes the scope". @@ -306,16 +311,16 @@ Dynamic block sizes are byte-based rather than element-type-based. A new list st A list grows according to the following rules: 1. When its capacity is exhausted, the requested block size is exactly twice its current block size. -2. The allocator first checks the size stack for that doubled size. If a block is available, it is popped and the live elements are relocated into it. -3. If that stack is empty and the current backing store is the dynamic frontier allocation with enough contiguous room to double, the frontier is bumped by the additional bytes and the store grows in place. -4. Otherwise, the doubled block is allocated by bumping the dynamic frontier, mapping more dynamic chunks as needed, and the live elements are relocated into it. -5. After relocation, the handle's backing-store offset and size class are updated and the old block's offset is pushed onto the size stack for its old byte size. +2. The allocator first checks the size stack for that doubled size. If a block or oversized span is available, it is popped and the live elements are relocated into it. +3. If that stack is empty, the current backing store is the dynamic frontier allocation, the doubled size is at most 1 MiB, and the additional bytes fit before the current chunk boundary, the frontier is bumped by the additional bytes and the store grows in place. +4. Otherwise, a doubled block of at most 1 MiB is bump-allocated wholly inside one dynamic chunk. A doubled block larger than 1 MiB is allocated as a fresh dedicated oversized span (§3.1). The live elements are relocated into the new block or span. +5. After relocation, the handle's backing-store offset and size class are updated and the old block's base offset is pushed onto the stack for its exact old byte size. -Relocation moves or copies elements according to their type's ordinary move rules; the old block becomes reusable only after its previous occupants are no longer live. Guests to the list remain valid because they reach the list's host, whose fixed-size handle now names the current backing store. +A block never grows in place across a chunk boundary, and an oversized span is never extended in place: further growth relocates into a doubled oversized span after checking that exact-size stack first. Relocation moves or copies elements according to their type's ordinary move rules; the old block becomes reusable only after its previous occupants are no longer live. Guests to the list remain valid because they reach the list's host, whose fixed-size handle now names the current backing store. -Dynamic chunks and all power-of-two blocks begin at cache-line-aligned addresses. Because the minimum block is 128 bytes and every larger block doubles, both frontier allocations and reused blocks preserve cache-line alignment without mixing backing stores into fixed-size chunks. +Dynamic chunks, ordinary power-of-two blocks, and oversized spans begin at cache-line-aligned addresses. Because the minimum block is 128 bytes and every larger block doubles, frontier allocations, reused blocks, and dedicated spans preserve cache-line alignment without mixing backing stores into fixed-size chunks. -> **Story:** [`stories/memory.md`](../stories/memory.md#the-sentinel-that-costs-nothing-and-the-buffer-that-wanted-a-line) — "The sentinel that costs nothing, and the buffer that wanted a line". +> **Story:** [`stories/memory.md`](../stories/memory.md#the-sentinel-that-costs-one-reserved-identity-and-the-buffer-that-wanted-a-line) — "The sentinel that costs one reserved identity, and the buffer that wanted a line". ### 3.7 Moving a value reuses the destination slot A move transfers hosting into a destination host of the **same type** (see [`lifetimes.md`](lifetimes.md) §1). Because both sides have identical, statically known size, a move is a fixed-size overwrite of the destination slot: @@ -323,39 +328,37 @@ A move transfers hosting into a destination host of the **same type** (see [`lif - Moving into a fresh declaration or a return slot is in-place initialization. - Moving into an already-initialized host first destroys the current occupant, then overwrites the same-size slot. -Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. A move into a higher scope copies the inline bytes into the destination scope's fixed-size region — a promotion (§3.5). Because handle-typed fields (§3.6) keep the moved footprint small, rehosting copies only the handle and transfers ownership of the same backing store; rehosting itself never relocates that store. A dynamic store changes address only through the growth procedure in §3.6. If the moved value is tethered, the move also updates its one anchor cell (§4.5), never the tethers themselves. +Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. A move into a higher scope copies the inline bytes into the destination scope's fixed-size region — a promotion (§3.5). Because handle-typed fields (§3.6) keep the moved footprint small, rehosting copies only the handle and transfers ownership of the same backing store; rehosting itself never relocates that store. A dynamic store changes address only through the growth procedure in §3.6. If the moved value is tethered, the destination inherits the same global anchor identity and updates its one cell (§4.5), never the tethers themselves. --- ## 4. Anchors and Tethers -### 4.1 The anchor cell -Tethers are tracked through per-host **anchor cells** rather than one shared table. An anchor cell is a single **`u32`** holding the current segmented offset (§3.1) of one hosted object; it stores nothing else. - -Anchor storage is **scope-local**, never global. Each scope owns a dedicated anchor-cell region — a separate lazy chunk chain from both its fixed-size and dynamic regions. A scope that never creates a guest allocates no anchor chunk. The first tether to a host bump-allocates its cell in that scope's anchor region (§4.3); minting another cell is one bump and never resizes a monolithic table. +### 4.1 The global anchor pool +Tethers are tracked through **anchor cells** in one runtime-global pool rather than through scope-local anchor regions. An anchor cell is one `u32` holding the current segmented offset (§3.1) of a hosted reference-type value. The cell's own segmented offset is the stable anchor identity for the complete hosting lineage, even when the value is rehosted across scopes. -Keeping cells out of the other two streams preserves dense fixed-size layout and prevents dynamic-buffer history from affecting anchor placement. The region remains compact and heavily reused while live, and all of its chunks disappear with the scope in the same bulk unmap as the other regions. +Anchor pages contain only equal-sized cells. The pool therefore needs one free-address stack and one bump frontier rather than size classes. Pages are allocated lazily and never move while any of their cells are live. > **Story:** [`stories/memory.md`](../stories/memory.md#where-the-cells-live-and-the-scan-that-pays-for-them) — "Where the cells live, and the scan that pays for them". ### 4.2 Tethers are segmented offsets, not pointers -A tether is a **`u32` segmented offset** (§3.1) pointing at the host's anchor cell — not a raw pointer and not a table index. At half the width of a 64-bit pointer, twice as many tethers fit in a cache line, and the 32-bit encoding keeps resolution on cheap 32-bit CPU math. A cell is allocated only on the first tether of a host (§4.3), so cells stay a small fraction of live memory, and the `u32`'s 32 GiB reach (§3.1) sits far beyond any realistic working set. +A tether is a **`u32` segmented offset** (§3.1) naming one global anchor cell — not a raw pointer and not a table index. At half the width of a 64-bit pointer, twice as many tethers fit in a cache line, and the 32-bit encoding keeps resolution on cheap 32-bit CPU math. -The value `0` (chunk `0`, word `0`, §3.1) means *untethered*. A cell is never placed at `0` (§4.1, §3.1), so `0` is never a real cell, and a stray resolution of an untethered `0` traps rather than reading live memory. +Every reference-type payload reserves a `u32` backpointer field initialized to `0`. Once an anchor exists, that field stores the same stable anchor identity that its guests store. Guests and backpointers never store the payload address directly. -Every reference-type instance reserves a **`u32` backpointer** field, initialized to `0`; the first tether records the segmented offset of the instance's anchor cell there. The cell is allocated lazily (§4.3), whereas the backpointer field is always present in the layout, so object size is fixed and array layout stays uniform. The backpointer lets a host mint new tethers from the object — `&x` copies the offset — and lets a move locate and update the object's cell (§4.5). It is a single offset, not a list of tethers: the runtime never enumerates the tethers that point at the object, which is what keeps moves O(1) (§4.5). +An explicitly declared `&T` slot contains only this tether. A host-capable `T` slot that has been rehosted may use the same tether representation while it is in guest state, but retains enough storage to host another `T` later (§2.4). -A tethered reference-type instance therefore costs **12 bytes** across the whole chain: the 4-byte tether (wherever it is stored), the 4-byte anchor cell, and the 4-byte backpointer in the payload. +The minimum machinery for one tethered hosting lineage is **12 bytes of logical data**: one 4-byte tether, one 4-byte anchor cell, and the 4-byte payload backpointer. Each additional guest adds another 4-byte tether. Rehosting adds no cell and no forwarding metadata. > **Story:** [`stories/memory.md`](../stories/memory.md#the-last-table-problem-and-the-segmented-offset) — "The last table problem, and the segmented offset". ### 4.3 Anchors are created lazily -A hosted object that never gains a tether consumes no cell: its backpointer field stays `0` and no cell is allocated (it still carries the 4-byte field, §4.2). The first `&` taken on its host bump-allocates a cell in the arena, writes the hosted object's current segmented offset into it, and records the cell's own segmented offset in the object's backpointer. Every subsequent `&` from that host copies the backpointer. +A hosting lineage that never gains a guest consumes no cell: its payload backpointer remains `0`. The first `&` taken on its host pops the global free-address stack if possible; otherwise it bump-allocates a cell at the global anchor frontier. The runtime writes the payload's current segmented offset into the cell and the cell's identity into the payload backpointer. Every later `&` from that host copies the backpointer. > **Story:** [`stories/memory.md`](../stories/memory.md#finding-the-anchor-and-not-paying-when-there-are-no-refs) — "Finding the anchor, and not paying when there are no refs". ### 4.4 Resolving a tether -Resolving a tether reads the anchor cell it points at, reads the hosted object's segmented offset from that cell, materializes the object's address through the chunk directory (§3.1), then accesses the field. Because a cell is never at `0`, a resolution of an untethered `0` never reads a live cell. +Resolving a tether uses the chunk directory to locate the global anchor cell, reads the hosted payload's current segmented offset from that cell, resolves that offset through the same directory, then accesses the field. Because the pool never allocates cell identity `0`, resolving an untethered `0` traps rather than reading a live cell. Consider reading a field through a tether, where `mainWeapon` is an `&Weapon`: @@ -363,74 +366,65 @@ Consider reading a field through a tether, where `mainWeapon` is an `&Weapon`: dps Float = mainWeapon.dps ``` -`mainWeapon` holds a segmented offset to an anchor cell, not the Weapon's address. Field access uses `.`: it resolves the cell, reads the hosted object's current offset from it, resolves that offset to the object's address, then adds the field offset. The walk is tether → cell → payload offset → payload address → field: - -#### Illustrative resolution walkthrough - -A tether contains a segmented offset to an anchor cell, not directly to the -hosted object. Reading `mainWeapon.dps` therefore resolves two segmented offsets. +The walk is always tether → global anchor cell → payload offset → payload address → field: ```text mainWeapon: &Weapon │ -│ tether segmented offset -│ [ anchor-cell chunk id | anchor-cell word offset ] +│ anchor identity ▼ chunk directory │ ▼ -anchor-cell chunk +global anchor page │ ▼ anchor cell │ -│ hosted object segmented offset -│ [ payload chunk id | payload word offset ] +│ current payload segmented offset ▼ chunk directory │ ▼ -payload chunk +fixed-size chunk │ ▼ Weapon payload │ -│ ordinary field offset within Weapon +│ ordinary field offset ▼ Weapon.dps ``` -The first segmented offset locates the anchor cell. The cell contains the -second segmented offset, which locates the hosted object's current payload. -Both use the same chunk-directory resolution rule. +Moves, overwrites, and promotions update only the current payload offset in that same cell. Guests created before and after a promotion therefore follow an identical path, with no forwarding cells and no promotion-dependent extra hop. -If the `Weapon` moves or its host is overwritten, the runtime updates only -the hosted object's offset stored in the anchor cell. `mainWeapon` continues -to point at the same cell, and its next access reaches the payload's new location. +The added cost over direct host access is one dependent anchor-cell load. Across repeated accesses through the same guest with no intervening move or overwrite, the compiler may resolve the host address once and reuse it. -The `.` reads the field; it never reassigns the Weapon. Rebinding `mainWeapon` itself would only repoint the tether at a different host's cell (subject to the scope rule in [`lifetimes.md`](lifetimes.md) §1.1) — it would not overwrite any field. Splitting each segmented offset is a shift and a mask that fold into machine addressing once the chunk base is in hand, so the encoding costs no arithmetic over a raw-pointer dereference. The added cost is one dependent load: the cell read between the tether and the field, and because cells live packed together in the arena's compact anchor-cell region (§4.1) that load normally lands in hot, cache-resident memory. See §4.8. +### 4.5 Moves, overwrites, and rehosting keep one anchor +A host overwrite, an in-scope move, and a cross-scope rehosting all preserve one anchor identity. -### 4.5 Moves, overwrites, and promotion update one cell, not all tethers -A tether follows the host/anchor path rather than pointing at a fixed object address. When a host is overwritten in place (§2.2) or its object is moved within the scope, the runtime writes the payload's new segmented offset into the object's one anchor cell, located through the backpointer (§4.2). The cell itself does not move, so every existing tether — which points at the cell, not the payload — observes the hosted object's current location on its next resolution with no per-tether fixup. +- **Overwrite:** if the hosting slot already has an anchor, the replacement payload inherits that backpointer and the cell is updated to the replacement's location. Existing guests therefore observe the host's new occupant. Destroying the old occupant does not return the cell, because the hosting lineage continues. +- **Move or rehosting:** after destroying any previous destination occupant and ending its separate hosting lineage, the destination payload inherits the source payload's backpointer. The shared global cell is updated to the destination location, and the destination host assumes responsibility for eventual anchor teardown. The source host-capable slot becomes a guest to the same cell. No anchor is copied, moved, reset, or recreated. +- **Untethered values:** a payload whose backpointer is `0` moves with `0` and still allocates no anchor. -**Promotion** on escape (§3.5) carries one extra step, because the hosted object's anchor cell lives in the anchor-cell region of the scope that minted it (§4.1) — a scope that is about to drain. Every tether that already points at that cell was taken in that scope or deeper ([`lifetimes.md`](lifetimes.md) §1.1), so none of them outlives the cell. On promotion the runtime therefore does two things: it updates the old cell to the payload's new location, so those existing tethers keep resolving to the live promoted copy for the remainder of the source scope, and it **resets the payload's backpointer to `0`**. The reset re-arms lazy allocation (§4.3): the next tether taken in the destination scope mints a fresh cell in the destination arena's cell region — one that lives exactly as long as the promoted value. The old cell and the tethers reading it then expire together when the source scope drains. +Every operation is O(1) in the number of guests. Because the same anchor survives every promotion, source-scope and destination-scope guests remain coherent after all later moves without repointing or forwarding. -This is why relocation, overwrite, and promotion are all **O(1) with respect to the number of tethers**. It is also how a moved-from symbol stays readable: after a move the symbol downgrades to an `&` — a segmented offset to the cell — and reads resolve through the cell to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). +This is also how a moved-from symbol stays readable: after a move the host-capable symbol enters guest state and stores the same tether, so reads resolve through the anchor to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). -> **Story:** [`stories/memory.md`](../stories/memory.md#the-move-problem-and-the-anchor-that-never-moves) — "The move problem, and the anchor that never moves". [`stories/memory.md`](../stories/memory.md#where-the-cells-live-and-the-scan-that-pays-for-them) — "Where the cells live, and the scan that pays for them". +> **Story:** [`stories/memory.md`](../stories/memory.md#the-move-problem-and-the-anchor-that-never-moves) — "The move problem, and the anchor that never moves". -### 4.6 Teardown releases cells in bulk -Anchor cells are arena allocations, so they are never individually freed. When a scope drains, its chunks — payload and anchor-cell regions alike — are unmapped (§3.2) and its cells vanish together with the hosts and payloads they served. +### 4.6 Hosting-lifetime end returns the anchor +An anchor is returned to the global free-address stack when its **hosting lineage** ends. Overwriting only the current occupant does not end that lineage, because the host remains and existing guests follow the replacement. Rehosting transfers teardown responsibility to the destination host; the source slot is now a guest rather than a second host. -Because scope rules keep every tether inside its host's lifetime ([`lifetimes.md`](lifetimes.md) §1.1, §1.4), no live tether can point at a cell that has been unmapped. Destruction therefore creates no dangling-tether state. +At the actual end of the hosting lineage, lexical scope rules guarantee that every guest capable of naming the anchor has already ceased to exist ([`lifetimes.md`](lifetimes.md) §1, [`concurrency.md`](concurrency.md) §4). The runtime may therefore recycle the slot immediately. No generation counter, delayed reuse, or ABA protection is required: a stale guest is not a representable program state. -### 4.7 Why tethers never dangle -A dangling tether would require one of three failures: a host overwrite breaking existing tethers, a tether outliving the host's scope, or an object move leaving tethers pointed at a dead address. The model eliminates each. Host/anchor indirection makes overwrite and move follow the current cell value instead of a stale address (§4.5). The same-or-higher-scope rule keeps every tether inside the host's lifetime envelope ([`lifetimes.md`](lifetimes.md) §1.1). The model is enforced by storage shape and lexical scope, not by runtime borrow tracking. +### 4.7 Why tethers never dangle or misdirect +A dangling or misdirected tether would require a guest to outlive its host, an anchor cell to move, or an anchor slot to be reused while an old guest remains. The model forbids all three. Scope checking proves the first impossible; the global pool gives each live hosting lineage one stable cell identity; and the same scope rule makes immediate slot reuse safe after teardown. -### 4.8 Resolution cost -The segmented encoding adds no arithmetic cost: the shift and mask that split a `u32` into a chunk id and a word offset fold into machine addressing once the chunk base is loaded. The chunk directory is the hottest table in the program — tiny, and normally resident in registers or L1 — so materializing an address from a segmented offset is effectively a single indexed load and add. +### 4.8 Resolution and allocation cost +The segmented encoding adds no meaningful arithmetic cost: the shift and mask that split a `u32` fold into machine addressing once the chunk base is loaded. Tether resolution pays one dependent anchor-cell load beyond direct host access. Rehosting adds no forwarding hop. -The genuine cost of any anchor scheme is **one extra dependent load per tether resolution** — the cell read — versus an idealized raw pointer that cannot survive moves. Because cells are packed together in the arena's compact anchor-cell region (§4.1), that load usually lands in hot, cache-resident memory, a few cycles at most. It is paid only when resolving a tether; direct access through a host never consults a cell. Across a run of accesses through the same tether with no intervening move, overwrite, or promotion, the compiler resolves the host address once and reuses it, so hot loops do not re-pay the load. +A single global free stack and frontier require synchronization under concurrent allocation and teardown. Implementations may use thread-local anchor caches backed by the same global pool without changing anchor identity, reuse order semantics, or lifetime guarantees. --- @@ -451,7 +445,7 @@ The genuine cost of any anchor scheme is **one extra dependent load per tether r | Property | Zane | GC languages | Rust | C/C++ | |---|---|---|---|---| -| Allocation strategy | per-scope bump arenas, bulk teardown | runtime-managed | allocator-dependent | allocator-dependent | +| Allocation strategy | per-scope fixed/dynamic arenas plus a global recyclable anchor pool | runtime-managed | allocator-dependent | allocator-dependent | > **See also:** [`lifetimes.md`](lifetimes.md) §3 for the lifetime and destruction behavior comparison. @@ -463,7 +457,8 @@ The genuine cost of any anchor scheme is **one extra dependent load per tether r |---|---| | Hosting storage | Reference-typed symbols, fields, and container elements are directly initialized and may later be overwritten | | Value type | Mutable in place through a borrowed `mut` receiver; storage may also be overwritten freely | -| `&` (guest) | Non-hosting storage; may be repointed, copied by value, and returned from functions | +| `&` (guest) | Guest-only non-hosting storage; stores one tether, may be repointed, copied by value, and returned, but can never directly host a `T` | +| Host-capable guest state | A slot declared as `T` may become a guest after rehosting while retaining enough storage to host another `T` later | | Place expression | Existing stable storage: a named symbol, a field access of a place, a place-projection subscript of a place, or an `&` parameter | | New `&` value | May be initialized only from a named symbol, a field access of a place, or an `&` parameter; temporaries and `[]` expressions are rejected | | `&` parameter | Declares that the caller must supply an `&`-creating source; the parameter is place-like inside the callee | @@ -474,14 +469,16 @@ The genuine cost of any anchor scheme is **one extra dependent load per tether r | Value-downstream enforcement | Value types may contain only primitives and other value types, transitively — never a reference (`#`) or `&` field | | `&` targets reference types | An `&T` requires `T` to be a reference type; a value is shared by copy or scoped borrow, never by a stored `&` | | Symbol declaration | Must be directly initialized | -| Reference-type placement | Bump-allocated in the creating scope's arena; promoted to a parent arena only on escape — an unobservable choice | -| `&` representation | A guest is represented internally by a `u32` tether: a segmented offset (chunk id + in-chunk offset) to the host's anchor cell; `0` means no tether | -| Addressing | Every location is a `u32` segmented offset resolved through the chunk directory; 8-byte-aligned offsets reach 32 GiB across up to 32768 1 MiB chunks | -| Untethered sentinel | `0` (chunk `0`, word `0`); costs no reserved memory because cells never occupy it — payloads may sit at offset `0` | -| Backing-store alignment | Dynamically-sized backing stores (§3.6) are cache-line-aligned so sequential element access does not straddle lines; small inline allocations stay 8-byte aligned | -| Anchor cell | One `u32` per hosted object that has at least one tether, holding the object's current segmented offset; bump-allocated in the scope's dedicated anchor-cell region, kept out of the payload stream so payload iteration stays dense | -| Backpointer | Each hosted object stores the `u32` segmented offset of its anchor cell for move updates and tether minting; `0` means no cell has been allocated | -| Anchor lifecycle | Lazily allocated on first guest; on promotion the payload re-anchors in the destination arena; released in bulk when the host's scope drains | -| Tethered-instance cost | 12 bytes total: the 4-byte tether, the 4-byte anchor cell, and the 4-byte backpointer | +| Reference-type placement | Bump-allocated in the creating scope's fixed-size region; promoted to a parent region only on escape — an unobservable choice | +| `&` representation | A guest is represented internally by a `u32` tether: a segmented offset (chunk id + in-chunk offset) to the host's global anchor cell; `0` means no tether | +| Addressing | Scope chunks and global anchor pages share one `u32` segmented-offset directory; 8-byte-aligned offsets reach 32 GiB across up to 32768 1 MiB chunks | +| Untethered sentinel | `0`; the global anchor pool reserves this identity, while payloads may still occupy segmented offset `0` | +| Dynamic allocation | Power-of-two byte classes beginning at 128 bytes; exact-size stack first, frontier second; blocks above 1 MiB use dedicated contiguous oversized spans | +| Backing-store alignment | Dynamically-sized backing stores (§3.6) are cache-line-aligned; small inline allocations stay 8-byte aligned | +| Anchor cell | One global-pool `u32` per tethered hosting lineage, holding the current payload segmented offset | +| Backpointer | Each hosted payload stores the stable `u32` identity of its anchor cell for move updates and tether minting; `0` means no cell has been allocated | +| Anchor lifecycle | Lazily allocated on first guest; preserved across overwrite and rehosting; returned to the global free-address stack when the hosting lineage ends | +| Anchor reuse safety | Immediate reuse is safe because lexical scope rules make a live stale guest unrepresentable | +| Tethered-instance cost | Minimum 12 bytes of logical data: one 4-byte tether, one 4-byte anchor cell, and one 4-byte backpointer | > **See also:** [`lifetimes.md`](lifetimes.md) §4 for the summary of scope, move, and destruction rules. From 4eeedb48abc7d77e48f782162bd16f9e55a46a1f Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Sat, 25 Jul 2026 22:38:25 +0200 Subject: [PATCH 08/31] Align memory story with global anchors --- stories/memory.md | 113 +++++++++++++++++----------------------------- 1 file changed, 41 insertions(+), 72 deletions(-) diff --git a/stories/memory.md b/stories/memory.md index e12d0ff..6b0c6b2 100644 --- a/stories/memory.md +++ b/stories/memory.md @@ -4,125 +4,94 @@ ## Safety without a collector and without lifetimes -The starting commitment was a subtraction, not an addition: no tracing garbage collector, and no lifetime annotations. Those are the two roads the rest of the industry took to memory safety, and Zane wanted neither. A collector buys safety by handing the mechanism to a runtime — the managed-altitude cost the [foundations story](foundations.md#the-bet-on-captured-intent) argues against — and pays for it with pauses, headroom, and a non-deterministic moment of death that makes destruction unpredictable. Lifetime annotations, the Rust road, keep determinism but make the programmer carry the proof: a borrow checker that forbids whole patterns and a `'a` vocabulary the source has to speak fluently to compile. We turned both down at once, which sounds like wanting the impossible — safety with neither a runtime watching nor a proof obligation on the author. +The starting commitment was a subtraction: no tracing garbage collector and no lifetime annotations. Zane keeps single hosting and deterministic, scope-driven destruction, but makes guest safety follow from the shape of storage and lexical scope rather than from runtime tracing or source-level lifetime parameters. -What makes that coherent is keeping the half of Rust's model that costs nothing in annotations — single ownership and deterministic, scope-driven destruction — and refusing only the half that does. Every class instance has one owner ([§2.1](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#21-every-class-instance-has-exactly-one-owner)); when its scope drains it dies, with no collector consulted. That gets determinism for free. The hard part is the *other* kind of storage — the non-owning reference, `&` — because that is exactly where Rust spends its annotation budget and its borrow checker. The bet was that a reference's safety could be made to fall out of the *shape* of storage and ordinary lexical scope, rather than out of a separate analysis the author feeds. "Safety from shape" was never the goal in itself; it is what is left over once you refuse both a collector and a lifetime vocabulary. Something still has to make refs safe, and if it is not a runtime and not an annotation, it has to be the structure of the storage forms themselves. The cost of that stance is a real one, and the rest of this story is mostly the working-out of it: refusing the borrow checker means refusing the thing that, in Rust, makes the move problem go away by fiat — and so the move problem becomes ours to solve. +A host is the one storage location that controls a reference-type value's lifetime. A guest is non-hosting and can never extend that lifetime. The compiler checks that every guest remains inside the lifetime envelope of the host it follows. That rule is what makes deterministic teardown and immediate anchor reuse possible later in the design. ## The move problem, and the anchor that never moves -That problem surfaces the instant the two halves meet. Single ownership wants to *move* values — transfer ownership into a return slot, an outer scope, a container — and wants owners to be overwritable in place ([§2.2](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#22-class-owners-are-overwritable-after-initialization)). A non-owning ref, naively, is the value's address. Put those together and you have the classic dangling reference: the ref records an address, the value moves, the address is now stale. This is precisely the situation a borrow checker exists to outlaw — it forbids the move while a borrow is live — and forbidding the move is the ergonomic price we had just refused to pay. So the requirement stated itself plainly: a ref has to keep pointing at the same value *even when that value moves.* +Single hosting wants values to move: into return slots, outer scopes, fields, and containers. A raw address cannot survive that. The solution is one fixed cell containing the value's current location. A guest stores the identity of that cell rather than the value's address. Moving or overwriting the value updates one cell, and every guest follows on its next access. -The move forces the question, but the move also hands over the answer, because the compiler is the one doing the moving — at the moment a value relocates, the compiler knows both where it was and where it now is. The trick is to interpose one level of indirection that itself never moves: a small fixed cell, reachable with confidence at all times, that holds the value's current location. A ref points at *that* cell; the cell points at the value. Now a move is a one-line update — the compiler writes the value's new location into the cell it was always going to touch anyway — and every ref, reading through the cell on its next use, sees the new home with no fixup of its own. The cell earned its name: it is the fixed thing a drifting value is tethered to, the **anchor**. The same single mechanism dissolves three superficially different hazards at once — a move, an in-place owner overwrite, and the readability of a moved-from symbol (which simply downgrades to a ref through the same cell) — because all three are just "the value is somewhere else now," and the anchor is the one place that fact has to be recorded. +Updating guests directly was rejected because it would require enumerating them and make every move O(number of guests). The anchor reverses the cost: moves are O(1), while guest access pays one dependent cell load. -The road not taken here is the obvious one: instead of indirecting through a cell, fix up the refs directly — on a move, walk every ref that points at the value and rewrite it. That keeps the deref one hop shorter, but it forces the owner to *enumerate its refs*, which means carrying a list of them and paying O(number of refs) on every move and every overwrite. The anchor inverts that cost: the owner records *one* link (to its anchor), the move touches *one* cell, and the count of refs never enters the arithmetic — moves and overwrites are O(1) in the number of refs by construction ([§4.5](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#45-moves-and-overwrites-update-one-cell-not-all-refs)). The honest cost lands on the read side, and it is permanent: every dereference through a ref pays one extra dependent load — the cell read sitting between the ref and the value — that a raw pointer would not. We judged a fixed one-load tax on reads a better trade than an unbounded fixup cost on moves, especially since the common case is a value read through a ref far more often than it is moved. That single dependent load, multiplied across the layered dereferencing the design invites, is the pressure that later forces every link to be small ([the indexed table](#from-a-reserved-pool-to-an-indexed-heap-table)). +The cell must remain the same cell for the complete hosting lineage. If promotion moved the cell, every guest would need repointing. If promotion created another independent cell, later moves could update one path and stale the other. A stable global cell avoids both problems: source-scope and destination-scope guests retain one identity throughout all rehosting. -## Finding the anchor, and not paying when there are no refs +## Finding the anchor, and not paying when there are no guests -The anchor solves the move, but it raises two of its own. When a value moves, the compiler has to *find* that value's anchor in order to update it — so the value needs a way back to its cell. And if every value carried an anchor whether or not anything ever referenced it, we would be charging the whole program for a feature most values never use. Both are answered by giving each value a **backpointer**: a single small field that records where its anchor lives ([§4.2](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#42-refs-are-slot-indices-not-pointers)). A move reads the backpointer, finds the cell, updates it — done. And because the backpointer can hold a sentinel meaning "no anchor exists yet," the anchor need not exist until the first ref is actually taken: an unreferenced value keeps its backpointer at the sentinel and consumes no cell at all, while the first `&` lazily mints the anchor and records it ([§4.3](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#43-anchors-are-created-lazily)). The vast majority of values, which are never referenced, pay only for the one backpointer field they carry and nothing more. +Every reference-type payload carries one small backpointer. `0` means no anchor exists. The first guest lazily allocates a cell, writes the payload's current segmented offset into it, and stores the cell's identity in the payload. Later guests copy that identity. -It matters that the backpointer is a *single* link and not a list. The temptation, once a value points back at its anchor, is to let it point back at all its refs — but that is the O(n)-move design from the previous chapter wearing a different hat. One backpointer is exactly enough: it lets a move locate the cell, and it lets the owner mint fresh refs by copying its anchor's identity, without the owner ever once enumerating who points at it. The asymmetry is deliberate — an owner knows about its *anchor*, never about its *refs* — and it is the whole reason the move stays O(1). +One backpointer is sufficient. The payload knows its anchor but never knows or enumerates its guests. Rehosting transfers the same backpointer to the destination and updates the one cell. -## From a reserved pool to an indexed heap table +## Guest-only and host-capable guest storage -Anchors have to live somewhere, and the first home was the lazy one: a fixed region of memory set aside at startup just for anchor cells. It worked, and it bothered us for two reasons that turned out to be the same reason. A fixed region imposes a fixed *cap* — why should a program be allowed only so many simultaneously-referenced values? — and in the common case where few values are referenced, that reserved space sits mostly empty, paid for and unused. Both are the symptom of pinning anchor storage to a size guessed ahead of time. The fix was to stop treating anchors as special and store them the way any dynamically-sized thing is stored: in a growable structure on the heap. That collapses the cap (it grows on demand) and the waste (it is only as big as the live anchors need) together. What remains fixed shrinks to a single **master anchor** — one word at a known location that records where the anchor table currently lives ([§3.1](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#31-single-reserved-memory-region)) — so that when the table relocates to grow, exactly one word is rewritten and no ref or backpointer is disturbed. +Not every guest can become a host. An explicitly declared `&T` contains only a tether and has no room for a `T`; it is permanently guest-only. -Putting the table on the heap forces the last move, and it is the one that shapes the whole layout: once the table can relocate, a backpointer or a ref can no longer be a raw address into it — it has to be an **index**, a position in the table that survives the table moving underneath it. So refs and backpointers stop being pointers and become indices ([§4.2](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#42-refs-are-slot-indices-not-pointers)). That sounds like a concession, but it is where the design pays itself back, because an index can be *small*. The reason to care is cache locality, and cache locality is unusually load-bearing here: the anchor scheme already committed us to layered dereferencing — ref to cell, cell to owner, owner to field, often several such hops per logical access — and the cost of all that indirection is dominated by whether the links it walks are sitting in cache. So the design generalizes the index into a single discipline: the runtime owns one contiguous region, and every location inside it is a `u32` offset from one register-held base rather than a native pointer. A ref, a backpointer, an anchor cell, and the master anchor are all `u32` — half the width of a 64-bit pointer — so twice as many of them fit in a cache line, and the hot anchor table stays resident. The extra dependent load the [anchor scheme](#the-move-problem-and-the-anchor-that-never-moves) imposed is made cheap precisely by making the thing it loads small. The cost is a ceiling: a `u32` caps the program at roughly four billion live anchors and the region at a few gigabytes (stretched by alignment-scaled offsets). We took the ceiling without much hesitation — it sits far past any realistic working set — in exchange for halving the size of the most-walked links in the language. +A slot declared as `T` is different. When its value is rehosted, the slot may remain readable as a guest to the new host, but the physical slot still has the full size and alignment of `T`. It is therefore a **host-capable guest**: it can later be overwritten with another `T` and become a host again. -## Where a new ref may come from +This distinction does not change access syntax, but it matters to layout and assignment. Guest behavior describes how a slot reaches a value; host capability describes how much storage the slot owns. -A ref that always resolves through an anchor is only safe if it never gets *created* pointing at something that has no stable anchor to begin with. So the other half of the model is a restriction on where a new `&` may be born: only from storage that denotes a real, stable, owner-rooted place — a named symbol, a field of a place, or an `&` parameter ([§2.8](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#28-place-expressions-and-new--values)). A temporary — the result of `Engine()` or `makeEngine()` — is rejected as a source, because it has no home for an anchor to track; it exists only for the instant of the expression, and a ref to it would be a ref to nothing the moment the statement ends. The owner form materializes such a temporary into stable storage; the ref form cannot, because there is nothing to make stable. +## Where a new guest may come from -The subtlest case, and the one that looks arbitrary until you see what forces it, is that a subscript `[]` may never *create* a new `&`, even though it can read one that is already stored. The reason is dynamic size. Take a ref to the last element of a list and then pop that element: the element is gone, but the ref would remain, now pointing at a slot the container has reclaimed — a dangling ref minted entirely within the safe language. The container's own operations move and drop elements as it grows and shrinks, and an `&` carved out of an owned element would have no anchor relationship strong enough to survive that. So the language forbids minting an `&` *from* an owned element in the first place; what `weapons[1]` may yield is an `&` only when an `&` was *already* stored there, a value the container holds rather than an interior pointer the container hands out. That keeps element references stable by never letting them come into existence on unstable ground, rather than by trying to track and invalidate them after the fact — the tracking road is exactly the per-element bookkeeping the whole model is built to avoid. +A new `&` may be created only from stable, host-rooted storage: a named symbol, a field of a place, or an existing `&` parameter. Temporaries are rejected because they have no lasting host. A subscript may read a guest already stored in a container, but it cannot mint a guest from a hosting element whose slot might later be removed or reused. -The same rooted-source idea runs through parameters, but with a twist that protects the call site. A parameter declared `&T` demands that the caller hand over a real place; a plain `T` parameter does not, and therefore must not be re-exported as a new `&` from inside the callee ([§2.9](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#29--function-parameters)). The point of splitting the two is to keep the *call* uniform: `consume(e)` and `inspect(e)` read identically, and it is the signature, not the syntax at the call, that decides whether an `&`-rootable source is required. We considered letting the caller's punctuation carry that distinction and rejected it for the same reason the rest of the language pushes such facts into declarations — the obligation belongs to the function's contract, not to a decoration the caller has to remember. A related small rule rounds this out: every symbol must be initialized at its declaration ([§2.11](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#211-symbols-require-direct-initialization)). Once owners are freely overwritable later, a bare uninitialized declaration would reintroduce the maybe-uninitialized storage path the model otherwise never has, so it is simply disallowed — you may reassign all you like, but you may not start from nothing. The cost of this whole cluster is borne by the library author: a function that needs to retain or return a ref must say so with `&` in its signature, and a caller who holds only a temporary or a plain value cannot conjure a ref from it. Expressiveness the C programmer takes for granted — pointer to anything, anywhere — is deliberately not on offer; what you get back is that a ref, once it exists, is known to rest on stable ground. +The restriction is intentionally structural. Instead of dynamically tracking arbitrary interior references, the language prevents unstable guest identities from being created. ## The value world stays closed, and placement stays the compiler's -Two last pressures pull in opposite directions — one locks a door, the other opens one — and they are worth telling together because both are about how far the value layer can be trusted to behave. The locked door is the struct. Structs are plain inline values: copied by overwriting bytes, with no anchor and no destruction tracking. That is only sound if a struct can never smuggle in something that *needs* tracking — so a struct field may hold primitives and other structs and nothing else, checked transitively through the whole nested graph ([§2.10](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#210-struct-downstream-enforcement-transitive-struct-field-restrictions)). Let a struct contain a class and a byte-copy would silently duplicate ownership; let it contain an `&` and a byte-copy would silently duplicate ref-tracking state without ever going through the anchor system that makes that state correct. Both break the one invariant that lets struct copies be mechanical, so the closed value world is enforced rather than hoped for — the strictness-buys-speed bargain of the [foundations story](foundations.md#strictness-is-the-performance-model) in miniature. - -The opened door is placement. Identity does not require a second heap object: a materialized value slot and a statically sized reference-type host can sit directly beside each other in the scope's fixed-size region. The reference type differs because it carries hosting identity and a backpointer, not because its bytes must be indirect. Dynamic size is kept from leaking upward in the same way: `List`, `String`, and similar types are fixed-size handles inline with the ordinary slots, while only their backing stores occupy the scope's separate dynamic region ([§3.3](../spec/memory.md#33-value-and-reference-layout-follow-declaration-order), [§3.6](../spec/memory.md#36-handle-typed-core-reference-types-have-fixed-footprint)). That separation is what lets a type containing a list remain statically sized and what prevents a growing buffer from shifting any neighbouring value or host. - -Placement remains unobservable. The compiler may keep a value in a register or choose the lifetime region that already covers a later rehosting, provided destruction and guest resolution are unchanged. In particular, a dynamic backing store belongs semantically to its current host but is placed in a dynamic region that outlives every host it can be moved into. Rehosting therefore copies only the fixed-size handle and transfers ownership of the same store; only the container's own growth operation relocates its elements. The cost is the same boundary the earlier design already accepted: raw addresses and layout introspection cannot be language-visible facts if the compiler is free to make these choices. - -## The kinds collapse into one axis, and `this` becomes a borrow - -The [previous chapter](#the-value-world-stays-closed-and-placement-stays-the-compilers) drew its line between `struct` and `class`, but that pair was later recognized as one axis wearing two names — the [foundations story](foundations.md#identity-is-opt-in-one-axis-for-value-and-reference) tells why the split collapsed into a single `#` modifier. Seen from the memory model, the collapse barely moves anything, which is the tell that the old two-kind framing was carrying more than its weight: the machinery this document is built on — the backpointer, the anchor cell, lazy allocation, move updates — was never about "class" as a keyword. It was about *identity*. So the rule became "a reference type — a `#`-marked type — is the thing that carries an anchor," and everything the old text said about a "class instance" is now said about a reference-type instance, unchanged in substance. The struct-downstream restriction generalized the same way: it was always enforcing "no identity-bearing or ref-bearing field inside a copied value," so it is now **value-downstream** enforcement ([§2.10](https://github.com/zane-lang/spec/blob/9ab62085ed13f22b0a4897425852327264bd022d/spec/memory.md#210-value-downstream-enforcement-transitive-value-only-field-restriction)) — a value type may contain no reference or `&` field, transitively — which reads as a slightly broader rule but forbids exactly what it always did. - -The one genuinely new thing the axis exposed is that `&` and `#` are the same fact from two sides. A stored, move-surviving `&` needs an anchor to point at, and only a reference type has one, so an `&` references a reference type and nothing else ([§2.4](https://github.com/zane-lang/spec/blob/9ab62085ed13f22b0a4897425852327264bd022d/spec/memory.md#24--is-non-owning-storage)): a persistent handle over a value is written `&#T`, and a bare `&Int` over a value is simply ill-formed. This was implicit in the old model — you could only `&` a class — but naming the value/reference axis made it explicit and, better, *composable*: `#Int` is a reference cell and `&#Int` a handle to it, which is how a value that a program wants several observers to share becomes shareable at all. - -That leaves the question the old model answered by forbidding it: if a value has no anchor, how is one mutated in place? The answer, for a value receiver, is that `this` is a **borrow** of the caller's storage, not a copy — the method is handed the caller's slot for the duration of the call. This is what let value types become mutable at all: the previous design made a struct method return a replacement value precisely because a by-value receiver was a copy whose writes the caller would never see, and a scoped borrow of the caller's slot removes that reason. A *reference* receiver needs no such device — it is an implicit `&`, since a reference type already carries an anchor to point at ([`functions.md` §2.4](https://github.com/zane-lang/spec/blob/f8dc73f7553c016b7e4a5ea85dca68a41b5a4f3e/spec/functions.md#24-mutating-methods-use-mut)); the borrow is strictly the value world's device. The borrow is deliberately *non-escaping* — it cannot be stored as an `&` or returned as one, because a value is not `&`-rootable — so a value stays alias-free even while it is being mutated through, which is the property the concurrency safety rule ([`concurrency.md` §4.2](https://github.com/zane-lang/spec/blob/9ab62085ed13f22b0a4897425852327264bd022d/spec/concurrency.md#42-concurrent-mutation-requires-a-value-typed-receiver)) then leans its whole weight on. The cost is a subtlety a reader must hold: a borrow is a reference in the machine, but it is one the type system refuses to let outlive the call, and that refusal — not the absence of a pointer — is what keeps the value world closed. - -## Naming the tether - -Every chapter before this one has called the non-owning `&` by the plainest word to hand — a "ref", short for reference — and for a long time nothing depended on the word. What made it stop being fine was the type system catching up. Once the value/reference *type* axis settled into its own vocabulary — a value type, a reference type, and the `#` that separates them (the [foundations story](foundations.md#identity-is-opt-in-one-axis-for-value-and-reference) tells that collapse) — the language had two different things both leaning on the root "reference": the `#` *kind* of a type, and the `&` *handle* to one. They are orthogonal — the kind is a property of the owner, the handle is a separate non-owning cell that points at it — and yet "reference type" and "ref" are the same word twice, so a bare "reference" in a sentence could mean either. "ref" *felt* like it separated them, but it never did: it is only "reference" with the end bitten off, carrying the exact collision it appeared to resolve. - -So we went looking for a word that belonged to the value side alone and shared no root with the type axis. Keeping "ref" was the first option and the first rejected, for the reason just given — it does not actually pull the two apart. The next was **`link`**: plain, short, and honest about the semantics, since a link is a non-owning connection with no suggestion of ownership or lifetime. It was the safe choice, and its flaw was that it was *only* safe — "link" says nothing about the shape of this particular relationship, and it walks straight into "linked list" and "linker" the moment a data-structures chapter needs those words. - -The better lead came from a word already in the model. An `&` resolves through an **anchor** — a name chosen chapters ago — and an anchor is a nautical image, so the question asked itself: what, on the value side, is the thing you make fast to an anchor? The precise answers are all real and all unusable. A boat is joined to its anchor by the **rode**, which is exactly the relationship — held fast but not owned, and slippable — but "rode" reads as *road* on the page and looks like a typo. A **mooring pennant** and a **warp** are the same idea under other names, and they lose to "flag" and to "warp speed". The one member of that family that survives ordinary prose is **`tether`**. - -We nearly rejected it too. A tether, we worried, sounds like it *keeps the far thing attached* — as if a live `&` might hold its owner alive, which is the opposite of the rule that a tether can never extend a lifetime. But that reading is backwards, and seeing why is what settled the name. What a thing is tethered *to* is the fixed point; the tether binds and *bounds* the thing on its end, not the anchor. A boat on a tether is held within reach of the anchor; the anchor is not held by the boat. Map that onto the model and it is not a hazard, it is the scope rule stated as an image: an owner is the fixed point, and every `&` taken on it is bound to stay within the owner's reach — a tether may not outlive what it is tied to ([`lifetimes.md` §1.1](https://github.com/zane-lang/spec/blob/93bd2f0036b011c9fc876e785bff0d6a4d09465a/spec/lifetimes.md#11--assignment-uses-owner-scope)). The word we thought fought the semantics turned out to encode them. - -So the value-side handle is a **tether** ([`memory.md` §2.4](https://github.com/zane-lang/spec/blob/93bd2f0036b011c9fc876e785bff0d6a4d09465a/spec/memory.md#24--is-a-tether-non-owning-storage)), and the type axis keeps "reference type" to itself. The name pays a small resonance back on a second look, the kind [the naming guide](../contributing/naming-terms.md) hopes for: a tether is *slack* — it does not pull the owner anywhere, it keeps station beside it, which is exactly how a non-owning handle sits next to the value it watches. And it joins a set already speaking the same dialect — a `verb` acts, a `mould` shapes, a value is `borrow`ed and given back, an `anchor` holds fast — so now a `tether` ties to the anchor without owning it. - -The cost is the ordinary cost of any coined term: a reader meets "tether" and must be told once what it is, where "ref" would have passed without comment — at the price of the collision that started this. And the rename reaches sideways into ordinary words: an owner with a tether on it is now "tethered", one with none "untethered", which quietly retires "referenced" from the value side to keep the split clean. The earlier chapters of this very story still say "ref", because they were written when that was the word, and the history is left standing rather than back-dated; this chapter is where the name changed, not a pretence that it was always so. +Value types contain only value types and primitives, transitively. They carry no anchors and can be copied mechanically. Reference-type hosts, by contrast, carry identity and one backpointer, but their statically sized bytes may still sit inline beside values in the scope's fixed-size region. +Dynamic core types such as `List` and `String` keep fixed-size handles inline. Their variable-sized backing stores live separately, so growing a buffer cannot shift neighbouring hosts or values. Placement is unobservable: the compiler may optimize physical storage as long as destruction, hosting, and guest resolution remain unchanged. ## When the free stacks fragment, and the arena takes the scope -The allocator first used size-indexed free stacks for everything: round a request, pop the matching stack, and bump a global frontier only when that stack was empty. The operation was O(1), but the global policy let each class hoard memory from every other class. A program could run out of 32-byte slots while 16-byte and 64-byte stacks held abundant space, and borrowing a larger slot merely exchanged external fragmentation for internal waste. +A single size-class allocator for every object was rejected because classes hoard memory from each other. Ordinary fixed-size scope storage does not need individual reuse: it can use a bump frontier and disappear in bulk when the scope drains. -Scope arenas removed that failure from ordinary storage. Fixed-size values and hosts have stable slots whose lifetimes already match a lexical scope, so their region needs only a frontier; anchor cells have the same append-and-drain shape in their own region. Neither benefits from individual reuse. Overwriting a host reuses its existing slot, and draining the scope unmaps both regions whole. +Resizable backing stores are different. A list that grows abandons old buffers while the scope may remain active. Each scope therefore has a separate dynamic region with exact-size LIFO stacks. Allocation checks the corresponding size stack first and bumps the dynamic frontier only when that stack is empty. There is no borrowing from neighbouring classes and no coalescing. -Resizable backing stores are the exception the pure bump story had hidden. A list that doubles from 128 to 256 bytes may abandon the old 128-byte block while the scope continues for a long time. Leaving every old buffer stranded until drain makes repeated growth consume memory monotonically, while mixing those buffers among fixed-size slots makes the dense layout depend on container history. The answer was not to restore one global allocator, but to give each scope a third, dynamic region and confine exact-size reuse to it ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets)). Fixed-size, dynamic, and anchor allocations use distinct lazy chunk chains; no chunk contains more than one region, and a scope that never creates dynamic data pays for no dynamic chunk. +Byte size, not element type, defines the classes. A new list starts with 128 bytes regardless of `T`; capacity is derived from `stride(T)`. This makes a returned block reusable by lists of other element types, strings, and other dynamic core values. -The dynamic region keeps a LIFO stack for each power-of-two byte size, beginning at 128 bytes. Every request checks its exact size stack first and bumps the dynamic frontier only when that stack is empty. There is still no coalescing and no borrowing from a neighbouring class. The important difference from the rejected allocator is scope: these stacks recycle only the backing stores whose churn occurs inside one lifetime, and every stack and chunk disappears together when that scope drains. +Growth doubles the byte size. The allocator first looks for a reusable doubled block. If none exists and the current ordinary block is the frontier allocation with room before its 1 MiB chunk boundary, it grows in place. Otherwise the elements move into a new doubled block and the old block enters its exact-size stack. -Byte size, rather than element count, defines the classes. A new list asks for 128 bytes — sixteen 64-bit words — whether it stores bytes, integers, or larger records; its capacity is whatever number of `T` elements fit. If one element is larger, the first block is simply the smallest power of two that can contain it. The common byte classes are what make reuse broad: a block returned by one `List` can serve another element type, a string, or any other compatible dynamic core value. +## The block larger than a page -Growth doubles the block size. The allocator first looks for a reusable block of the doubled size. If none exists and the current buffer is the frontier allocation with room behind it, the frontier advances and the list grows in place. Otherwise a doubled block is bump-allocated at the frontier, the elements move, the handle changes its offset, and the old block is pushed onto its old-size stack. The free stack is therefore the first choice and the frontier the fallback — including the in-place case, which is just a frontier bump that happens not to change the address. +Ordinary dynamic blocks never cross a 1 MiB chunk boundary. Once a power-of-two block exceeds 1 MiB, it becomes an **oversized span**: a dedicated contiguous OS mapping made from consecutive dynamic chunks. -This reintroduces size-class fragmentation, but only in the one region whose objects genuinely change size, with powers of two that match the doubling policy and a fixed 128-byte floor that maximizes cross-type reuse. The fixed-size and anchor regions retain the arena's strongest property: no free lists, no holes from reassignment, and one bulk unmap at scope drain. The arena did not abolish free stacks everywhere; it confined them to the place where their reuse is worth their fragmentation. +The handle still stores one base segmented offset and one size class. Resolving the base produces a contiguous address range, so element indexing continues normally across the constituent chunks. Every chunk also has a directory entry. Oversized spans participate in the same exact-size reuse policy, but they are never extended in place; later growth relocates into a doubled span. -## The last table problem, and the segmented offset +This preserves the simple segmented-offset handle without imposing a one-page maximum on lists and strings. -Bump arenas paid for themselves everywhere but one place, and it was the place the whole model is built around: the anchor. An anchor has to stay reachable and fixed for as long as any tether points at it, and until now the anchors had lived in one growable [heap-resident table](#from-a-reserved-pool-to-an-indexed-heap-table) that a tether indexed. Put arenas underneath that table and its old flaw turns fatal. A monolithic contiguous table, when it fills, has to `realloc` — allocate a larger block and copy every cell across — and that is an O(N) step that also invalidates any native pointer into the array. We had spent the whole design making moves and overwrites O(1); reintroducing an O(N) resize on the anchor path, the hottest indirection in the language, would have handed all of it back. - -The obvious escape was to scatter the cells: stop keeping anchors in one array, allocate each on its own, and let a tether hold the cell's address directly. That kills the resize — there is no array to grow — but it resurrects the cost the [indexed table](#from-a-reserved-pool-to-an-indexed-heap-table) was invented to kill. A raw address is 64 bits. Inflate every tether and every backpointer back to eight bytes and the cache density we bought by making them `u32`s is gone, and every resolution is back on 64-bit pointer math. We would have solved the resize by un-solving the size. - -The two dead ends pointed at the same missing idea: we needed cells that could be *scattered* — so no table ever resizes — yet *addressed narrowly* — so a reference stays 32 bits. The arenas already supplied the first half: a cell is just another bump allocation, dropped in beside the owner that mints it, with no table in sight. The second half is to keep addressing everything with a `u32` but to read that `u32` as a **segmented offset** rather than a flat index — the high bits name a 1 MiB **chunk** and the low bits a word within it ([`memory.md` §3.1](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#31-scope-arenas-and-segmented-offsets)). An arena is a chain of such chunks; when one fills, the runtime maps another from the OS and gives it the next chunk id, and nothing already placed ever moves. A small chunk directory turns a chunk id into a native base, so a reference resolves in a shift, a mask, and one hot directory load. The cell stays four bytes, the tether stays four bytes, and there is no table left to resize ([`memory.md` §4.1](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#41-the-anchor-cell), [§4.2](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#42-tethers-are-segmented-offsets-not-pointers)). +## The last table problem, and the segmented offset -What falls out is a memory model that is uniformly 32-bit and, per tethered object, exactly twelve bytes of machinery: the four-byte tether wherever it is stored, the four-byte anchor cell, and the four-byte backpointer the payload carries home to that cell. The double indirection a tether walks — tether to cell, cell to payload — looks like it should cost two cache misses, and the arena is what makes it cost closer to zero: the cell read is a load into arena memory that is almost always already warm. The first thing we tried for that warmth was to drop each cell right beside the payload that mints it, so the two share a cache line and the second hop is paid for by the first. It worked for the deref, and it opened a loose end the next chapter has to pull on — a cell sitting in the payload stream is a cell a payload-only scan has to step over, and a payload's position starts to depend on how many of its neighbours were tethered before it. +A monolithic growable anchor array would eventually relocate, while native pointers to individually allocated cells would make every tether and backpointer 64 bits. Segmented `u32` offsets avoid both costs. The high bits select a 1 MiB chunk and the low bits select an aligned word inside it. A small chunk directory maps that identity to a native base. -The one motion this has to survive is escape. A value that outlives its scope is moved into a parent, and under arenas that means its payload is *copied* into the parent's arena — the child arena is about to be unmapped and cannot keep it. The backpointer is what makes that copy invisible: the runtime follows it to the payload's one anchor cell and rewrites the cell with the payload's new location. Every tether still points at that same cell and never learns the payload moved ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)) — the same one-cell-update that made an in-place move O(1) makes a cross-arena promotion O(1) too. When the scope finally drains, its chunks are unmapped and its cells vanish with the objects they served, and the scope rules that governed tethers all along guarantee nothing still pointing could point into them. +Scope chunks and anchor pages use the same directory. Tethers, backpointers, payload locations, dynamic handles, and allocator free-stack entries therefore share one compact representation. -The cost is a ceiling, and a lower one than the flat region carried its own version of. Splitting a `u32` into a chunk id and an offset spends bits on structure that a flat offset spent on range: with 1 MiB chunks and 8-byte-aligned slots the arithmetic lands at 32 GiB of live arena across up to 32768 chunks — roomy, but a hard cap, and a program that genuinely needs more cannot have it without widening the reference and paying back the cache density we just secured. The chunk directory is a hop the flat "region base plus offset" did without, too: one more small, hot table on the resolve path. We were content to trade a fixed horizon and a register-resident directory for an allocator that never fragments, never resizes an anchor table, and vaporizes an entire scope's memory in a single unmap. +The value `0` is reserved as “no anchor” wherever an anchor identity is expected. Payload offset zero remains valid; only the global anchor pool refuses to issue cell identity zero. ## Where the cells live, and the scan that pays for them -The [previous chapter](#the-last-table-problem-and-the-segmented-offset) left a cell sitting beside every payload and called the shared cache line a win. It is a win — for the deref. What we had not yet measured was the other side of the same coin, and when we built the layout and ran it on real hardware the coin landed. Two workloads slid backwards. A sweep that reads only payloads — the common case of walking a collection and touching each object's fields — slowed by around a fifth, because the interleaved cells sit between the payloads and a scan that wants only payloads drags the cells through cache anyway; the same objects packed tight would have touched fewer lines. And a write-heavy growth buffer regressed harder still, because once cells share the payload stream the stream's alignment stops being the compiler's to control — a buffer's base now depends on how many cells were minted ahead of it, and a run of entities that should have sat one-per-line ended up straddling two. The shared-line trick had quietly made the *arena's* geometry a function of tether history, and iteration paid for it. +Interleaving anchors with payloads makes fixed-size scans less dense and lets guest-creation history disturb alignment. Anchors therefore live in anchor-only pages outside every scope arena. -The tempting repair was to go backwards: put the anchors back in one compact heap table, the way the pre-arena model had. That table was never slow to read — it stayed small and hot, and it kept payloads perfectly contiguous because nothing lived between them. But the reason it read so well is exactly the reason it could not stay. A single global table has no scope to be unmapped with; every cell in it has to be handed back one at a time as its owner dies, or the table leaks and grows without bound. That is the per-object teardown the arenas had just abolished — the "free is a no-op" and one-unmap-per-scope wins are wins *because* nothing walks the objects to reclaim them. A global table would have bought back the payload density by selling the teardown, and the teardown was the larger prize. The compactness that made the table attractive was the very thing that forced the per-object free; the two could not be separated. +The pool is runtime-global because the cell follows the complete hosting lineage, not whichever scope currently contains the payload. Promotion updates the cell's payload offset and keeps its identity unchanged. No forwarding cell, re-anchoring, or guest repointing is needed. -So the fix was not to leave the arena but to separate the things with different allocation behaviour. The scope now has three regions: fixed-size slots, dynamic backing stores, and anchor cells, each with its own lazy chunk chain and the same segmented-offset directory ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets), [§4.1](../spec/memory.md#41-the-anchor-cell)). Fixed-size scans stay dense because neither cells nor resizable buffers are braided through them; dynamic growth can recycle abandoned blocks without punching holes among hosts; and cells remain compact and hot in their own run. It is arenas the whole way down: three streams, one drain. +All anchor cells have the same size, so the pool has one free-address stack rather than size classes. Allocation pops that stack first and bumps the global frontier only when it is empty. Pages are mapped lazily. -The cost is real and we name it plainly. We gave up the shared-cache-line bonus on the deref itself — the cell and its payload no longer ride into cache together, so a single tether resolution can pay two loads where the interleaved layout often paid one. We judged that the right trade because the payload sweep is the hotter path in the workloads we cared about: densifying the scan that runs over every object beats shaving a load off the deref that runs only when a tether is actually followed. It is the mirror image of the choice the last chapter made, now that we have measured which side of the coin comes up more often. +The global pool does require individual anchor teardown, but the host already supplies an exact teardown event. Overwriting an occupant keeps the hosting lineage alive and retains the cell. Rehosting transfers teardown responsibility to the destination. Only the end of the final hosting lineage returns the address to the stack. -Separating the region also forced us to finish a sentence the last chapter had left dangling. It had described promotion as "rewrite the one cell and every tether follows," which quietly skipped the question of *which arena that cell is in*. Now the answer is unambiguous: the cell lives in the region of the scope that minted it, and on escape that scope is precisely the one about to drain. Every tether already pointing at the cell was taken in that scope or deeper, so none of them outlives it — the lifetime rule that has fenced tethers all along ([`lifetimes.md` §1.1](https://github.com/zane-lang/spec/blob/93bd2f0036b011c9fc876e785bff0d6a4d09465a/spec/lifetimes.md#11--assignment-uses-owner-scope)) does the fencing here too. So promotion updates the old cell to the payload's new home, keeping those doomed-but-still-live tethers reading the promoted copy until their scope ends, and then **resets the payload's backpointer to zero** so the value re-anchors from scratch in its new scope: the next tether taken on it there mints a fresh cell in the destination region, one that finally lives as long as the value does ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/2caab48a9f4bafda32d67c3bb902908aea1813ab/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)). The old cell and its tethers expire together when the source scope drains, and nothing ever reads a cell that has been unmapped. That reset is the honest tax of keeping cells scope-local rather than in one immortal table — a second, cheaper re-anchoring the immortal table would not have needed, paid so that anchors can vanish in the same unmap as everything else. +Immediate reuse needs no generation counter. Lexical scope rules prove that no guest can outlive the host, including guests held by spawned work covered by the water-tower lifetime rules. When the hosting lineage ends, no live guest can still contain the old anchor identity. The ABA state is therefore unrepresentable rather than dynamically detected. -## The sentinel that costs nothing, and the buffer that wanted a line +A concurrent implementation may use thread-local caches backed by the global pool to reduce contention, without changing the semantic identity or lifetime of any anchor. -Separating anchor cells made the zero sentinel free. A tether or backpointer value of `0` means “untethered,” but those values name only anchor cells, and anchor cells live in their own region where chunk `0`, word `0` is never a cell. The fixed-size region may therefore begin at offset zero with no reserved gap ([`memory.md` §3.1](../spec/memory.md#31-scope-arenas-and-segmented-offsets)). The sentinel is protected by region identity, not by wasting the first slot. +## The sentinel that costs one reserved identity, and the buffer that wanted a line -Dynamic buffers then asked for a stronger alignment guarantee. They are streamed, copied during growth, and commonly contain elements whose access pattern spans many cache lines. Rather than align an arbitrary buffer after an arbitrary fixed-size payload, the dedicated dynamic region starts from its own chunk boundary and allocates only power-of-two blocks beginning at 128 bytes. Every frontier block is therefore cache-line aligned, and every block later popped from a size stack preserves that alignment ([`memory.md` §3.6](../spec/memory.md#36-handle-typed-core-reference-types-have-fixed-footprint)). +The global pool reserves anchor identity zero for the untethered sentinel. That costs one unusable slot identity, not one page. Payloads can still occupy segmented offset zero because an anchor cell may legitimately contain that payload location. -This is another benefit of refusing to mix regions on one chunk. Small inline values keep ordinary 8-byte alignment without paying cache-line padding, while dynamic backing stores receive cache-line alignment structurally. A scope with no dynamic values maps no dynamic chunk, so the stronger alignment carries no idle page cost. Zero remains a usable fixed-size address, and buffers get the geometry they need without making neighbouring object placement depend on allocation history. +Dynamic buffers have a separate alignment concern. Their region starts on chunk boundaries and allocates power-of-two blocks beginning at 128 bytes. Ordinary blocks, reused blocks, and oversized spans therefore remain cache-line aligned without forcing the same padding onto small inline values. ## Two vocabularies: host and guest above anchor and tether -With the arena layout settled, one vocabulary problem remained. Calling `&T` a tether had solved the collision with “reference type,” but it left the source language and the runtime sharing one word. The problem is that those are different layers. Source code needs names for the lifetime relationship a programmer reasons about; the memory model needs names for the indirection that keeps that relationship working when an object moves. Using `tether` for both made an implementation choice sound like the meaning of `&T` itself. - -The source pair is now **host** and **guest**. A host is the symbol, field, or container slot that stores a reference-type object — or its hosting handle — and governs the object's lifetime. A guest is an `&T`: it may access the hosted object, but it neither stores that object nor controls how long it lives. When the object moves, it is rehosted, and its guests continue reaching it. The ordinary relationship does useful work here: a host provides both accommodation and the duration of a guest's stay, while a guest may use what is provided but cannot outlast the host. +The source-facing relationship is **host / guest**. A host stores the value and governs its lifetime. A guest may access the hosted value but neither stores it nor controls how long it lives. -The runtime keeps **anchor** and **tether**. Each guest is represented by a tether that resolves through an anchor; moving or rehosting the object updates the anchor, so existing tethers keep working. That vocabulary remains a natural mechanical picture, but it no longer leaks upward into source semantics. The concise model is: *an object lives in a host; a guest may access it; internally, the guest's tether follows the object through its anchor.* +The runtime-facing mechanism is **anchor / tether**. A guest is represented by a tether that names an anchor; the anchor records the payload's current location. Rehosting updates the anchor, so the guest keeps working. -The alternatives each blurred something we wanted to keep sharp. **Owner/tether** named the two halves accurately in isolation but paired source semantics with implementation. **Owner/guest** worked, though “owner” stressed rights and destruction more than residence. **Owner/view** was technically reasonable without being a convincing lived relationship. Proxy, keyholder, delegate, and licensee were variously technical, overloaded, or indirect; “key” also collided with dictionary keys. CC/email language suggested secondary participation, but a CC recipient receives an independent copy rather than live access to one moving object. And keeping **tether** as the name of `&T` remained expressive, but preserved the very overload this split was meant to remove. +The concise model is: *an object lives in a host; a guest may access it; internally, the guest's tether follows the object through its anchor.* From 7b2de03ed6ff890d81a6f369b7517180b91d03a7 Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Sat, 25 Jul 2026 23:20:19 +0200 Subject: [PATCH 09/31] Clarify anchor layout and rehosting conflicts --- spec/memory.md | 57 +++++++++++++++++++++++++++++++++++++------------- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/spec/memory.md b/spec/memory.md index 44c0741..086c905 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -29,9 +29,11 @@ These rules fit together mechanically. Hosts are the only storage that controls ## 2. Hosting and Storage ### 2.1 Every reference-type instance has exactly one host + Every instance of a reference type (a `#`-marked type, see [`types.md`](types.md) §2.1) is hosted by exactly one symbol, field, or container slot at a time. Hosting is the default storage mode for reference values. ### 2.2 Reference-type hosts are overwritable after initialization + Any hosting storage position for a reference-type instance—a symbol, field, or container slot—**MUST** be directly initialized, and **MAY** later be overwritten. ```zane @@ -50,6 +52,7 @@ hosts Array = [Node(), Node()] Rewriting `hosts[1]` replaces the hosted reference-type instance in that slot. Guests to that slot observe the new value because guests follow the host/anchor path, not the original object. ### 2.3 Value types are mutable in place and freely overwritable + Value types have no anchor and no heap identity. A value is mutated in place through a `mut` method whose receiver is a borrow of the value's storage (see [`effects.md`](effects.md) §2.3, [`functions.md`](functions.md) §2.4), and its storage slot may also be reassigned wholesale. Neither operation goes through the anchor system, because a value has no identity to track. ```zane @@ -59,6 +62,7 @@ pos = Vec2(3, 4) // whole-slot overwrite ``` ### 2.4 `&` is a guest: non-hosting storage + `&` creates a **guest**: non-hosting storage that points at a **reference type** only. An `&T` requires `T` to be a reference type — a declared `#struct`/`#variant`/`#enum` — because only a reference type carries the identity (the anchor, §4) that a stable, move-surviving guest needs. A value type is shared by copying it or by a scoped borrow (see [`functions.md`](functions.md) §2.4), never by a stored guest. Writing `&Node` names a guest to a reference type; a bare `&Int` over a value type is ill-formed. An explicitly declared `&T` slot is **guest-only**: it stores only a tether and can never directly host a `T`. A slot declared as `T` is **host-capable**. After its value is rehosted, that same full-size slot may remain readable in guest state, but it retains the storage needed to host another `T` later. Guest-only and host-capable guest states use the same access semantics, but only the latter can become a host again. @@ -76,15 +80,19 @@ An `&` type is legal in storage sites (local symbols, fields, nested storage typ > **Story:** [`stories/memory.md`](../stories/memory.md#two-vocabularies-host-and-guest-above-anchor-and-tether) — "Two vocabularies: host and guest above anchor and tether". ### 2.5 Guests are repointable + An `&` symbol or `&` field may be assigned a different target later, as long as the scope rule in [`lifetimes.md`](lifetimes.md) §1.1 is satisfied. ### 2.6 Guests are independent + Assigning or passing a guest gives the destination its own guest to the same host. Rebinding one guest's storage site later changes only that storage site; it does not retarget other guests that already point to that host. ### 2.7 Guests and hosts use the same surface operations + At use sites, a guest is used with the same surface syntax as a direct host. Method calls, field access, and `mut` calls use the ordinary syntax. The distinction between host and guest matters only at the storage site: a guest stores a non-hosting link, while a host stores the object itself or its hosting slot. ### 2.8 Place expressions and new `&` values + A **place expression** is an expression that denotes an existing, stable storage location. The following are place expressions: @@ -136,6 +144,7 @@ engine Engine() // legal: plain host binding; Engine() temporary is mate > **Story:** [`stories/memory.md`](../stories/memory.md#where-a-new-ref-may-come-from) — "Where a new ref may come from". ### 2.9 Function parameters: borrows and `&` + A **borrow** is non-hosting, non-escaping access to a caller's storage for the duration of a call. Unlike a guest (§2.4), a borrow has no anchor, cannot be stored in a field, and cannot be returned; it exists only while the call runs. Borrowing is the passing mode for **value types**, which have no `&` of their own. A value-type parameter is a **read-only borrow** of the caller's slot, and a value is **copied** only when it is bound into a fresh slot — an assignment, a new declaration, or a field or return store. The one writable borrow is a value-type `mut` receiver (see [`functions.md`](functions.md) §2.4). A **reference type** is passed through the hosting/`&` system instead, in one of two modes: @@ -181,6 +190,7 @@ Void setEngineWrong(this Car, engine Engine) mut { This rule preserves uniform call syntax. The call site writes `consume(e)` or `inspect(e)` regardless of whether the parameter is `&`. The callee's signature determines whether an `&`-creating source is required from the caller. ### 2.10 Value-downstream enforcement (transitive value-only field restriction) + Value types form a closed world of plain value storage. A value-type field may contain primitives (see [`syntax.md`](syntax.md) §2.1) and other value types, but it **MUST NOT** contain a reference type (a `#`-marked type) or an `&`. This rule applies transitively: a value type containing another value type that eventually contains a reference-type or `&` field is also illegal. The same closure forbids a value type from recursing, since a self-reference would need indirection and indirection is a reference. Here, **downstream** means "through nested value-type fields." The restriction is checked recursively through the full value graph. @@ -210,6 +220,7 @@ type BadRef = struct { > **Story:** [`stories/memory.md`](../stories/memory.md#the-value-world-stays-closed-and-placement-stays-the-compilers) — "The value world stays closed, and placement stays the compiler's". ### 2.11 Symbols require direct initialization + Every symbol declaration **MUST** provide its initial value in the declaration itself. Zane does not permit bare symbol declarations followed by conditional or delayed first assignment. ```zane @@ -228,6 +239,7 @@ if runtimeBool() { ## 3. Memory Layout ### 3.1 Scope arenas, the global anchor pool, and segmented offsets + Each lexical scope owns an **arena** made from two independent allocation regions: - The **fixed-size region** stores materialized value-type slots, statically sized reference-type hosts, and the fixed-size handles of dynamic core types. @@ -235,7 +247,7 @@ Each lexical scope owns an **arena** made from two independent allocation region Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots and dynamic backing stores never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. -Anchors do not belong to any scope arena. The runtime owns one **global anchor pool**, implemented as a lazy chain of anchor-only 1 MiB pages. Every anchor slot has the same fixed width. The pool maps its first page only when the program creates its first guest and adds another page only when its current frontier and free-address stack cannot satisfy an allocation. +Anchors do not belong to any scope arena. The runtime owns one **global anchor pool**, implemented as a lazy chain of anchor-only 1 MiB pages. Every anchor occupies an **8-byte-aligned, 8-byte physical slot**: the first four bytes hold the `u32` payload offset and the remaining four bytes are reserved padding. An anchor page therefore contains 131072 addressable slots. The pool maps its first page only when the program creates its first guest and adds another page only when its current frontier and free-address stack cannot satisfy an allocation. ```text one scope arena runtime-global anchor pool @@ -265,27 +277,31 @@ Tethers (§4.2), per-host backpointers (§4.2), anchor cells (§4.1), dynamic ha > **Story:** [`stories/memory.md`](../stories/memory.md#the-last-table-problem-and-the-segmented-offset) — "The last table problem, and the segmented offset". ### 3.2 Allocation, reuse, and teardown + The fixed-size region is a pure bump allocator. A host is a fixed-size storage slot, so overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7); it consumes no new arena space. The dynamic region adds exact-size reuse on top of its bump frontier. Dynamic blocks use power-of-two byte sizes beginning at **128 bytes**. Each scope maintains one LIFO **size stack** for every block size that has become reusable. To allocate a dynamic block of size `S`, the runtime first pops `size_stack[S]`; only when that stack is empty does it bump the dynamic frontier. It never satisfies a request from another size stack and never coalesces neighbouring blocks. Returning a dynamic block pushes its base segmented offset onto the stack for that exact byte size. The stacks are shared by all dynamic types in the scope: a 128-byte block previously used by a `List` may later hold string bytes or another list's elements. An oversized span participates in the same exact-size policy. -The global anchor pool has one LIFO **free-address stack**, because every anchor slot has the same size. Creating an anchor pops that stack first; only when it is empty does allocation bump the global anchor frontier, mapping another anchor page as needed. Returning an anchor pushes its segmented offset onto the same stack. +The global anchor pool has one LIFO **free-address stack**, because every anchor slot has the same size. Creating an anchor pops that stack first; only when it is empty does allocation bump the global anchor frontier, mapping another anchor page as needed. Returning an anchor pushes its segmented offset onto the same stack. Anchor pages remain mapped and retain their chunk-directory entries until runtime shutdown, including when every slot on a page is free; consequently every offset retained by the stack always resolves to its original anchor slot and anchor chunk ids are never repurposed during the run. -When a scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps its fixed-size and dynamic chunks in bulk. Global anchor pages are not tied to scope teardown: individual slots are returned when their hosting lineages end (§4.6). A runtime may unmap a wholly free anchor page, but slot reuse does not depend on page reclamation. +When a scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps its fixed-size and dynamic chunks in bulk. Global anchor pages are not tied to scope teardown: individual slots are returned when their hosting lineages end (§4.6), while the pages themselves remain mapped until runtime shutdown. > **Story:** [`stories/memory.md`](../stories/memory.md#when-the-free-stacks-fragment-and-the-arena-takes-the-scope) — "When the free stacks fragment, and the arena takes the scope". ### 3.3 Value and reference layout follow declaration order + Fields are laid out in declaration order. Value types are stored inline. A statically sized reference-type instance is also stored inline in a fixed-size host slot, so value-type slots and reference-type host slots may sit directly beside each other in the fixed-size region. Reference types differ by identity and hosting semantics, not by requiring a separate indirect allocation. A reference-type instance carries one `u32` backpointer field of anchor metadata (a segmented offset, §4.2) that remains `0` until the instance is first tethered. A dynamic core type such as `List` occupies a fixed-size handle inline in the same region; only the backing store named by that handle occupies the dynamic region (§3.6). ### 3.4 Booleans may be packed + The compiler may pack booleans in structs and arena frames when doing so does not change language semantics. ### 3.5 Statically sized storage uses the fixed-size region + Placement is an implementation decision, not a language-visible property. The arena model places every materialized, statically sized scope slot — value-type storage, a reference-type host, or a dynamic type's fixed-size handle — inline in that scope's fixed-size region. The compiler may keep an unobservable value in registers or otherwise optimize its physical placement, but reference types do not require a separate heap allocation merely because they carry identity. When a reference-type instance is rehosted into a longer-lived destination, its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). A dynamic backing store is semantically owned by the current host. The compiler **MUST** place that store in a dynamic region whose lifetime covers every destination into which the handle can be rehosted, so rehosting transfers ownership of the same backing store without copying it. Only growth of the dynamic value may relocate the backing store (§3.6). @@ -295,6 +311,7 @@ Placement never changes observable semantics: destruction stays deterministic (s > **Story:** [`stories/memory.md`](../stories/memory.md#the-value-world-stays-closed-and-placement-stays-the-compilers) — "The value world stays closed, and placement stays the compiler's". ### 3.6 Handle-typed core reference types have fixed footprint + The core dynamically-sized reference types — `List`, `String`, and similar types — are represented as fixed-size **handles**. A handle records the backing store's segmented offset and the metadata needed by the type, such as length and size class. The handle occupies a statically known footprint inline in the fixed-size region; its resizable backing store is a separate allocation in the dynamic region. A type that contains a handle-typed field therefore stays statically sized: @@ -323,6 +340,7 @@ Dynamic chunks, ordinary power-of-two blocks, and oversized spans begin at cache > **Story:** [`stories/memory.md`](../stories/memory.md#the-sentinel-that-costs-one-reserved-identity-and-the-buffer-that-wanted-a-line) — "The sentinel that costs one reserved identity, and the buffer that wanted a line". ### 3.7 Moving a value reuses the destination slot + A move transfers hosting into a destination host of the **same type** (see [`lifetimes.md`](lifetimes.md) §1). Because both sides have identical, statically known size, a move is a fixed-size overwrite of the destination slot: - Moving into a fresh declaration or a return slot is in-place initialization. @@ -335,29 +353,33 @@ Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md ## 4. Anchors and Tethers ### 4.1 The global anchor pool -Tethers are tracked through **anchor cells** in one runtime-global pool rather than through scope-local anchor regions. An anchor cell is one `u32` holding the current segmented offset (§3.1) of a hosted reference-type value. The cell's own segmented offset is the stable anchor identity for the complete hosting lineage, even when the value is rehosted across scopes. -Anchor pages contain only equal-sized cells. The pool therefore needs one free-address stack and one bump frontier rather than size classes. Pages are allocated lazily and never move while any of their cells are live. +Tethers are tracked through **anchor cells** in one runtime-global pool rather than through scope-local anchor regions. An anchor cell has a 4-byte logical `u32` payload holding the current segmented offset (§3.1) of a hosted reference-type value, but occupies one 8-byte-aligned physical slot so every cell identity is representable by the shared 8-byte-word offset encoding. The cell's own segmented offset is the stable anchor identity for the complete hosting lineage, even when the value is rehosted across scopes. + +Anchor pages contain only equal-sized 8-byte slots. The pool therefore needs one free-address stack and one bump frontier rather than size classes. Pages are allocated lazily, never move, and remain mapped until runtime shutdown. > **Story:** [`stories/memory.md`](../stories/memory.md#where-the-cells-live-and-the-scan-that-pays-for-them) — "Where the cells live, and the scan that pays for them". ### 4.2 Tethers are segmented offsets, not pointers + A tether is a **`u32` segmented offset** (§3.1) naming one global anchor cell — not a raw pointer and not a table index. At half the width of a 64-bit pointer, twice as many tethers fit in a cache line, and the 32-bit encoding keeps resolution on cheap 32-bit CPU math. Every reference-type payload reserves a `u32` backpointer field initialized to `0`. Once an anchor exists, that field stores the same stable anchor identity that its guests store. Guests and backpointers never store the payload address directly. An explicitly declared `&T` slot contains only this tether. A host-capable `T` slot that has been rehosted may use the same tether representation while it is in guest state, but retains enough storage to host another `T` later (§2.4). -The minimum machinery for one tethered hosting lineage is **12 bytes of logical data**: one 4-byte tether, one 4-byte anchor cell, and the 4-byte payload backpointer. Each additional guest adds another 4-byte tether. Rehosting adds no cell and no forwarding metadata. +The minimum physical footprint attributable to one tethered hosting lineage is **16 bytes**: one 4-byte tether, one 8-byte physical anchor slot (containing a 4-byte cell payload), and one 4-byte payload backpointer. Each additional guest adds another 4-byte tether. Rehosting adds no forwarding metadata and, when only one side has live guests, no additional cell. > **Story:** [`stories/memory.md`](../stories/memory.md#the-last-table-problem-and-the-segmented-offset) — "The last table problem, and the segmented offset". ### 4.3 Anchors are created lazily + A hosting lineage that never gains a guest consumes no cell: its payload backpointer remains `0`. The first `&` taken on its host pops the global free-address stack if possible; otherwise it bump-allocates a cell at the global anchor frontier. The runtime writes the payload's current segmented offset into the cell and the cell's identity into the payload backpointer. Every later `&` from that host copies the backpointer. > **Story:** [`stories/memory.md`](../stories/memory.md#finding-the-anchor-and-not-paying-when-there-are-no-refs) — "Finding the anchor, and not paying when there are no refs". ### 4.4 Resolving a tether + Resolving a tether uses the chunk directory to locate the global anchor cell, reads the hosted payload's current segmented offset from that cell, resolves that offset through the same directory, then accesses the field. Because the pool never allocates cell identity `0`, resolving an untethered `0` traps rather than reading a live cell. Consider reading a field through a tether, where `mainWeapon` is an `&Weapon`: @@ -400,28 +422,33 @@ Moves, overwrites, and promotions update only the current payload offset in that The added cost over direct host access is one dependent anchor-cell load. Across repeated accesses through the same guest with no intervening move or overwrite, the compiler may resolve the host address once and reuse it. -### 4.5 Moves, overwrites, and rehosting keep one anchor -A host overwrite, an in-scope move, and a cross-scope rehosting all preserve one anchor identity. +### 4.5 Moves, overwrites, and rehosting keep one canonical anchor + +An overwrite from a newly materialized value and a move from another host are distinct cases. -- **Overwrite:** if the hosting slot already has an anchor, the replacement payload inherits that backpointer and the cell is updated to the replacement's location. Existing guests therefore observe the host's new occupant. Destroying the old occupant does not return the cell, because the hosting lineage continues. -- **Move or rehosting:** after destroying any previous destination occupant and ending its separate hosting lineage, the destination payload inherits the source payload's backpointer. The shared global cell is updated to the destination location, and the destination host assumes responsibility for eventual anchor teardown. The source host-capable slot becomes a guest to the same cell. No anchor is copied, moved, reset, or recreated. -- **Untethered values:** a payload whose backpointer is `0` moves with `0` and still allocates no anchor. +- **Ordinary overwrite:** if the destination hosting slot already has an anchor, the replacement payload inherits that backpointer and the cell is updated to the replacement's location. Existing destination guests therefore observe the new occupant. Destroying the old occupant does not return the cell, because the destination hosting lineage continues. +- **Move or rehosting with guests on only one side:** the anchor named by the live guest set becomes the destination's canonical anchor. If only the source has live guests, its anchor transfers to the destination. If only the destination has live guests, its anchor is preserved and the source's moved-from slot becomes another guest to it. Any noncanonical anchor left from an earlier, now-ended guest set is returned before the move completes. If neither side had live guests but the source slot remains readable in guest state, an anchor is allocated lazily for that new guest. The canonical cell is updated to the destination payload, the destination assumes teardown responsibility, and the source host-capable slot stores its tether. +- **Move or rehosting with live guests on both sides:** the program is ill-formed. The two guest sets name distinct stable identities, and a one-cell payload backpointer cannot preserve both through later moves without forwarding or guest enumeration. The compiler **MUST** reject the operation rather than recycle either referenced anchor. This restriction is determined from lexical guest liveness, not merely from whether a backpointer is nonzero. +- **Consumed untethered temporaries:** a temporary with no source slot that must remain readable may materialize into an untethered destination with backpointer `0` and allocate no anchor. -Every operation is O(1) in the number of guests. Because the same anchor survives every promotion, source-scope and destination-scope guests remain coherent after all later moves without repointing or forwarding. +Every permitted operation is O(1) in the number of guests. Promotion never creates a second live anchor path: it either preserves the sole live identity or is rejected. Source-scope and destination-scope guests therefore remain coherent after all later moves without repointing or forwarding. -This is also how a moved-from symbol stays readable: after a move the host-capable symbol enters guest state and stores the same tether, so reads resolve through the anchor to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). +This is also how a moved-from symbol stays readable: after a permitted move the host-capable symbol enters guest state and stores the canonical tether, so reads resolve through the anchor to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). > **Story:** [`stories/memory.md`](../stories/memory.md#the-move-problem-and-the-anchor-that-never-moves) — "The move problem, and the anchor that never moves". ### 4.6 Hosting-lifetime end returns the anchor + An anchor is returned to the global free-address stack when its **hosting lineage** ends. Overwriting only the current occupant does not end that lineage, because the host remains and existing guests follow the replacement. Rehosting transfers teardown responsibility to the destination host; the source slot is now a guest rather than a second host. At the actual end of the hosting lineage, lexical scope rules guarantee that every guest capable of naming the anchor has already ceased to exist ([`lifetimes.md`](lifetimes.md) §1, [`concurrency.md`](concurrency.md) §4). The runtime may therefore recycle the slot immediately. No generation counter, delayed reuse, or ABA protection is required: a stale guest is not a representable program state. ### 4.7 Why tethers never dangle or misdirect + A dangling or misdirected tether would require a guest to outlive its host, an anchor cell to move, or an anchor slot to be reused while an old guest remains. The model forbids all three. Scope checking proves the first impossible; the global pool gives each live hosting lineage one stable cell identity; and the same scope rule makes immediate slot reuse safe after teardown. ### 4.8 Resolution and allocation cost + The segmented encoding adds no meaningful arithmetic cost: the shift and mask that split a `u32` fold into machine addressing once the chunk base is loaded. Tether resolution pays one dependent anchor-cell load beyond direct host access. Rehosting adds no forwarding hop. A single global free stack and frontier require synchronization under concurrent allocation and teardown. Implementations may use thread-local anchor caches backed by the same global pool without changing anchor identity, reuse order semantics, or lifetime guarantees. @@ -475,10 +502,10 @@ A single global free stack and frontier require synchronization under concurrent | Untethered sentinel | `0`; the global anchor pool reserves this identity, while payloads may still occupy segmented offset `0` | | Dynamic allocation | Power-of-two byte classes beginning at 128 bytes; exact-size stack first, frontier second; blocks above 1 MiB use dedicated contiguous oversized spans | | Backing-store alignment | Dynamically-sized backing stores (§3.6) are cache-line-aligned; small inline allocations stay 8-byte aligned | -| Anchor cell | One global-pool `u32` per tethered hosting lineage, holding the current payload segmented offset | +| Anchor cell | One global-pool 8-byte physical slot per tethered hosting lineage; its 4-byte `u32` payload holds the current payload segmented offset | | Backpointer | Each hosted payload stores the stable `u32` identity of its anchor cell for move updates and tether minting; `0` means no cell has been allocated | | Anchor lifecycle | Lazily allocated on first guest; preserved across overwrite and rehosting; returned to the global free-address stack when the hosting lineage ends | | Anchor reuse safety | Immediate reuse is safe because lexical scope rules make a live stale guest unrepresentable | -| Tethered-instance cost | Minimum 12 bytes of logical data: one 4-byte tether, one 4-byte anchor cell, and one 4-byte backpointer | +| Tethered-instance cost | Minimum 16-byte physical footprint: one 4-byte tether, one 8-byte anchor slot, and one 4-byte backpointer | > **See also:** [`lifetimes.md`](lifetimes.md) §4 for the summary of scope, move, and destruction rules. From cf1e5332771e1a7e5e26a01219c0e1fc8320d58c Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Sat, 25 Jul 2026 23:20:23 +0200 Subject: [PATCH 10/31] Align memory story with anchor rules --- stories/memory.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/stories/memory.md b/stories/memory.md index 6b0c6b2..c3203f3 100644 --- a/stories/memory.md +++ b/stories/memory.md @@ -14,7 +14,9 @@ Single hosting wants values to move: into return slots, outer scopes, fields, an Updating guests directly was rejected because it would require enumerating them and make every move O(number of guests). The anchor reverses the cost: moves are O(1), while guest access pays one dependent cell load. -The cell must remain the same cell for the complete hosting lineage. If promotion moved the cell, every guest would need repointing. If promotion created another independent cell, later moves could update one path and stale the other. A stable global cell avoids both problems: source-scope and destination-scope guests retain one identity throughout all rehosting. +The cell must remain the same cell for the complete hosting lineage. If promotion moved the cell, every guest would need repointing. If promotion created another independent cell, later moves could update one path and stale the other. A stable global cell avoids both problems when at most one side of a move has live guests. + +A move whose source and destination both already have live guests is rejected. Each guest set names a different stable identity, so preserving both would require forwarding or enumerating guests. When only one side has live guests, that side's cell becomes canonical; when neither does but the source slot becomes a readable moved-from guest, the runtime lazily allocates the one cell it now needs. ## Finding the anchor, and not paying when there are no guests @@ -64,7 +66,7 @@ This preserves the simple segmented-offset handle without imposing a one-page ma A monolithic growable anchor array would eventually relocate, while native pointers to individually allocated cells would make every tether and backpointer 64 bits. Segmented `u32` offsets avoid both costs. The high bits select a 1 MiB chunk and the low bits select an aligned word inside it. A small chunk directory maps that identity to a native base. -Scope chunks and anchor pages use the same directory. Tethers, backpointers, payload locations, dynamic handles, and allocator free-stack entries therefore share one compact representation. +Scope chunks and anchor pages use the same directory. Tethers, backpointers, payload locations, dynamic handles, and allocator free-stack entries therefore share one compact representation. Because the low bits count 8-byte words, each anchor uses an 8-byte-aligned physical slot: four bytes for its `u32` payload and four reserved bytes. A 1 MiB anchor page contains 131072 such slots. The value `0` is reserved as “no anchor” wherever an anchor identity is expected. Payload offset zero remains valid; only the global anchor pool refuses to issue cell identity zero. @@ -74,7 +76,7 @@ Interleaving anchors with payloads makes fixed-size scans less dense and lets gu The pool is runtime-global because the cell follows the complete hosting lineage, not whichever scope currently contains the payload. Promotion updates the cell's payload offset and keeps its identity unchanged. No forwarding cell, re-anchoring, or guest repointing is needed. -All anchor cells have the same size, so the pool has one free-address stack rather than size classes. Allocation pops that stack first and bumps the global frontier only when it is empty. Pages are mapped lazily. +All anchor cells have the same physical size, so the pool has one free-address stack rather than size classes. Allocation pops that stack first and bumps the global frontier only when it is empty. Pages are mapped lazily and remain mapped until runtime shutdown, even when wholly free, so offsets retained by the stack never name unmapped or repurposed pages. The global pool does require individual anchor teardown, but the host already supplies an exact teardown event. Overwriting an occupant keeps the hosting lineage alive and retains the cell. Rehosting transfers teardown responsibility to the destination. Only the end of the final hosting lineage returns the address to the stack. From 2feefae409b73efc961e41b54afb6e4c2527efa9 Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Mon, 27 Jul 2026 15:26:14 +0200 Subject: [PATCH 11/31] Preserve memory story and append allocator revision --- stories/memory.md | 131 +++++++++++++++++++++++++++++++--------------- 1 file changed, 88 insertions(+), 43 deletions(-) diff --git a/stories/memory.md b/stories/memory.md index bfa474a..f6e8b7f 100644 --- a/stories/memory.md +++ b/stories/memory.md @@ -4,96 +4,141 @@ ## Safety without a collector and without lifetimes -The starting commitment was a subtraction: no tracing garbage collector and no lifetime annotations. Zane keeps single hosting and deterministic, scope-driven destruction, but makes guest safety follow from the shape of storage and lexical scope rather than from runtime tracing or source-level lifetime parameters. +The starting commitment was a subtraction, not an addition: no tracing garbage collector, and no lifetime annotations. Those are the two roads the rest of the industry took to memory safety, and Zane wanted neither. A collector buys safety by handing the mechanism to a runtime — the managed-altitude cost the [foundations story](foundations.md#the-bet-on-captured-intent) argues against — and pays for it with pauses, headroom, and a non-deterministic moment of death that makes destruction unpredictable. Lifetime annotations, the Rust road, keep determinism but make the programmer carry the proof: a borrow checker that forbids whole patterns and a `'a` vocabulary the source has to speak fluently to compile. We turned both down at once, which sounds like wanting the impossible — safety with neither a runtime watching nor a proof obligation on the author. -A host is the one storage location that controls a reference-type value's lifetime. A guest is non-hosting and can never extend that lifetime. The compiler checks that every guest remains inside the lifetime envelope of the host it follows. That rule is what makes deterministic teardown and immediate anchor reuse possible later in the design. +What makes that coherent is keeping the half of Rust's model that costs nothing in annotations — single ownership and deterministic, scope-driven destruction — and refusing only the half that does. Every class instance has one owner ([§2.1](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#21-every-class-instance-has-exactly-one-owner)); when its scope drains it dies, with no collector consulted. That gets determinism for free. The hard part is the *other* kind of storage — the non-owning reference, `&` — because that is exactly where Rust spends its annotation budget and its borrow checker. The bet was that a reference's safety could be made to fall out of the *shape* of storage and ordinary lexical scope, rather than out of a separate analysis the author feeds. "Safety from shape" was never the goal in itself; it is what is left over once you refuse both a collector and a lifetime vocabulary. Something still has to make refs safe, and if it is not a runtime and not an annotation, it has to be the structure of the storage forms themselves. The cost of that stance is a real one, and the rest of this story is mostly the working-out of it: refusing the borrow checker means refusing the thing that, in Rust, makes the move problem go away by fiat — and so the move problem becomes ours to solve. ## The move problem, and the anchor that never moves -Single hosting wants values to move: into return slots, outer scopes, fields, and containers. A raw address cannot survive that. The solution is one fixed cell containing the value's current location. A guest stores the identity of that cell rather than the value's address. Moving or overwriting the value updates one cell, and every guest follows on its next access. +That problem surfaces the instant the two halves meet. Single ownership wants to *move* values — transfer ownership into a return slot, an outer scope, a container — and wants owners to be overwritable in place ([§2.2](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#22-class-owners-are-overwritable-after-initialization)). A non-owning ref, naively, is the value's address. Put those together and you have the classic dangling reference: the ref records an address, the value moves, the address is now stale. This is precisely the situation a borrow checker exists to outlaw — it forbids the move while a borrow is live — and forbidding the move is the ergonomic price we had just refused to pay. So the requirement stated itself plainly: a ref has to keep pointing at the same value *even when that value moves.* -Updating guests directly was rejected because it would require enumerating them and make every move O(number of guests). The anchor reverses the cost: moves are O(1), while guest access pays one dependent cell load. +The move forces the question, but the move also hands over the answer, because the compiler is the one doing the moving — at the moment a value relocates, the compiler knows both where it was and where it now is. The trick is to interpose one level of indirection that itself never moves: a small fixed cell, reachable with confidence at all times, that holds the value's current location. A ref points at *that* cell; the cell points at the value. Now a move is a one-line update — the compiler writes the value's new location into the cell it was always going to touch anyway — and every ref, reading through the cell on its next use, sees the new home with no fixup of its own. The cell earned its name: it is the fixed thing a drifting value is tethered to, the **anchor**. The same single mechanism dissolves three superficially different hazards at once — a move, an in-place owner overwrite, and the readability of a moved-from symbol (which simply downgrades to a ref through the same cell) — because all three are just "the value is somewhere else now," and the anchor is the one place that fact has to be recorded. -The cell must remain the same cell for the complete hosting lineage. If promotion moved the cell, every guest would need repointing. If promotion created another independent cell, later moves could update one path and stale the other. A stable global cell avoids both problems when at most one side of a move has live guests. +The road not taken here is the obvious one: instead of indirecting through a cell, fix up the refs directly — on a move, walk every ref that points at the value and rewrite it. That keeps the deref one hop shorter, but it forces the owner to *enumerate its refs*, which means carrying a list of them and paying O(number of refs) on every move and every overwrite. The anchor inverts that cost: the owner records *one* link (to its anchor), the move touches *one* cell, and the count of refs never enters the arithmetic — moves and overwrites are O(1) in the number of refs by construction ([§4.5](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#45-moves-and-overwrites-update-one-cell-not-all-refs)). The honest cost lands on the read side, and it is permanent: every dereference through a ref pays one extra dependent load — the cell read sitting between the ref and the value — that a raw pointer would not. We judged a fixed one-load tax on reads a better trade than an unbounded fixup cost on moves, especially since the common case is a value read through a ref far more often than it is moved. That single dependent load, multiplied across the layered dereferencing the design invites, is the pressure that later forces every link to be small ([the indexed table](#from-a-reserved-pool-to-an-indexed-heap-table)). -A move whose source and destination both already have live guests is rejected. Each guest set names a different stable identity, so preserving both would require forwarding or enumerating guests. When only one side has live guests, that side's cell becomes canonical; when neither does but the source slot becomes a readable moved-from guest, the runtime lazily allocates the one cell it now needs. +## Finding the anchor, and not paying when there are no refs -## Finding the anchor, and not paying when there are no guests +The anchor solves the move, but it raises two of its own. When a value moves, the compiler has to *find* that value's anchor in order to update it — so the value needs a way back to its cell. And if every value carried an anchor whether or not anything ever referenced it, we would be charging the whole program for a feature most values never use. Both are answered by giving each value a **backpointer**: a single small field that records where its anchor lives ([§4.2](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#42-refs-are-slot-indices-not-pointers)). A move reads the backpointer, finds the cell, updates it — done. And because the backpointer can hold a sentinel meaning "no anchor exists yet," the anchor need not exist until the first ref is actually taken: an unreferenced value keeps its backpointer at the sentinel and consumes no cell at all, while the first `&` lazily mints the anchor and records it ([§4.3](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#43-anchors-are-created-lazily)). The vast majority of values, which are never referenced, pay only for the one backpointer field they carry and nothing more. -Every reference-type payload carries one small backpointer. `0` means no anchor exists. The first guest lazily allocates a cell, writes the payload's current segmented offset into it, and stores the cell's identity in the payload. Later guests copy that identity. +It matters that the backpointer is a *single* link and not a list. The temptation, once a value points back at its anchor, is to let it point back at all its refs — but that is the O(n)-move design from the previous chapter wearing a different hat. One backpointer is exactly enough: it lets a move locate the cell, and it lets the owner mint fresh refs by copying its anchor's identity, without the owner ever once enumerating who points at it. The asymmetry is deliberate — an owner knows about its *anchor*, never about its *refs* — and it is the whole reason the move stays O(1). -One backpointer is sufficient. The payload knows its anchor but never knows or enumerates its guests. Rehosting transfers the same backpointer to the destination and updates the one cell. +## From a reserved pool to an indexed heap table -## Guest-only and host-capable guest storage +Anchors have to live somewhere, and the first home was the lazy one: a fixed region of memory set aside at startup just for anchor cells. It worked, and it bothered us for two reasons that turned out to be the same reason. A fixed region imposes a fixed *cap* — why should a program be allowed only so many simultaneously-referenced values? — and in the common case where few values are referenced, that reserved space sits mostly empty, paid for and unused. Both are the symptom of pinning anchor storage to a size guessed ahead of time. The fix was to stop treating anchors as special and store them the way any dynamically-sized thing is stored: in a growable structure on the heap. That collapses the cap (it grows on demand) and the waste (it is only as big as the live anchors need) together. What remains fixed shrinks to a single **master anchor** — one word at a known location that records where the anchor table currently lives ([§3.1](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#31-single-reserved-memory-region)) — so that when the table relocates to grow, exactly one word is rewritten and no ref or backpointer is disturbed. -Not every guest can become a host. An explicitly declared `&T` contains only a tether and has no room for a `T`; it is permanently guest-only. +Putting the table on the heap forces the last move, and it is the one that shapes the whole layout: once the table can relocate, a backpointer or a ref can no longer be a raw address into it — it has to be an **index**, a position in the table that survives the table moving underneath it. So refs and backpointers stop being pointers and become indices ([§4.2](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#42-refs-are-slot-indices-not-pointers)). That sounds like a concession, but it is where the design pays itself back, because an index can be *small*. The reason to care is cache locality, and cache locality is unusually load-bearing here: the anchor scheme already committed us to layered dereferencing — ref to cell, cell to owner, owner to field, often several such hops per logical access — and the cost of all that indirection is dominated by whether the links it walks are sitting in cache. So the design generalizes the index into a single discipline: the runtime owns one contiguous region, and every location inside it is a `u32` offset from one register-held base rather than a native pointer. A ref, a backpointer, an anchor cell, and the master anchor are all `u32` — half the width of a 64-bit pointer — so twice as many of them fit in a cache line, and the hot anchor table stays resident. The extra dependent load the [anchor scheme](#the-move-problem-and-the-anchor-that-never-moves) imposed is made cheap precisely by making the thing it loads small. The cost is a ceiling: a `u32` caps the program at roughly four billion live anchors and the region at a few gigabytes (stretched by alignment-scaled offsets). We took the ceiling without much hesitation — it sits far past any realistic working set — in exchange for halving the size of the most-walked links in the language. -A slot declared as `T` is different. When its value is rehosted, the slot may remain readable as a guest to the new host, but the physical slot still has the full size and alignment of `T`. It is therefore a **host-capable guest**: it can later be overwritten with another `T` and become a host again. +## Where a new ref may come from -This distinction does not change access syntax, but it matters to layout and assignment. Guest behavior describes how a slot reaches a value; host capability describes how much storage the slot owns. +A ref that always resolves through an anchor is only safe if it never gets *created* pointing at something that has no stable anchor to begin with. So the other half of the model is a restriction on where a new `&` may be born: only from storage that denotes a real, stable, owner-rooted place — a named symbol, a field of a place, or an `&` parameter ([§2.8](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#28-place-expressions-and-new--values)). A temporary — the result of `Engine()` or `makeEngine()` — is rejected as a source, because it has no home for an anchor to track; it exists only for the instant of the expression, and a ref to it would be a ref to nothing the moment the statement ends. The owner form materializes such a temporary into stable storage; the ref form cannot, because there is nothing to make stable. -## Where a new guest may come from +The subtlest case, and the one that looks arbitrary until you see what forces it, is that a subscript `[]` may never *create* a new `&`, even though it can read one that is already stored. The reason is dynamic size. Take a ref to the last element of a list and then pop that element: the element is gone, but the ref would remain, now pointing at a slot the container has reclaimed — a dangling ref minted entirely within the safe language. The container's own operations move and drop elements as it grows and shrinks, and an `&` carved out of an owned element would have no anchor relationship strong enough to survive that. So the language forbids minting an `&` *from* an owned element in the first place; what `weapons[1]` may yield is an `&` only when an `&` was *already* stored there, a value the container holds rather than an interior pointer the container hands out. That keeps element references stable by never letting them come into existence on unstable ground, rather than by trying to track and invalidate them after the fact — the tracking road is exactly the per-element bookkeeping the whole model is built to avoid. -A new `&` may be created only from stable, host-rooted storage: a named symbol, a field of a place, or an existing `&` parameter. Temporaries are rejected because they have no lasting host. A subscript may read a guest already stored in a container, but it cannot mint a guest from a hosting element whose slot might later be removed or reused. - -The restriction is intentionally structural. Instead of dynamically tracking arbitrary interior references, the language prevents unstable guest identities from being created. +The same rooted-source idea runs through parameters, but with a twist that protects the call site. A parameter declared `&T` demands that the caller hand over a real place; a plain `T` parameter does not, and therefore must not be re-exported as a new `&` from inside the callee ([§2.9](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#29--function-parameters)). The point of splitting the two is to keep the *call* uniform: `consume(e)` and `inspect(e)` read identically, and it is the signature, not the syntax at the call, that decides whether an `&`-rootable source is required. We considered letting the caller's punctuation carry that distinction and rejected it for the same reason the rest of the language pushes such facts into declarations — the obligation belongs to the function's contract, not to a decoration the caller has to remember. A related small rule rounds this out: every symbol must be initialized at its declaration ([§2.11](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#211-symbols-require-direct-initialization)). Once owners are freely overwritable later, a bare uninitialized declaration would reintroduce the maybe-uninitialized storage path the model otherwise never has, so it is simply disallowed — you may reassign all you like, but you may not start from nothing. The cost of this whole cluster is borne by the library author: a function that needs to retain or return a ref must say so with `&` in its signature, and a caller who holds only a temporary or a plain value cannot conjure a ref from it. Expressiveness the C programmer takes for granted — pointer to anything, anywhere — is deliberately not on offer; what you get back is that a ref, once it exists, is known to rest on stable ground. ## The value world stays closed, and placement stays the compiler's -Value types contain only value types and primitives, transitively. They carry no anchors and can be copied mechanically. Reference-type hosts, by contrast, carry identity and one backpointer, but their statically sized bytes may still sit inline beside values in the scope's fixed-size region. +Two last pressures pull in opposite directions — one locks a door, the other opens one — and they are worth telling together because both are about how far the value layer can be trusted to behave. The locked door is the struct. Structs are plain inline values: copied by overwriting bytes, with no anchor and no destruction tracking. That is only sound if a struct can never smuggle in something that *needs* tracking — so a struct field may hold primitives and other structs and nothing else, checked transitively through the whole nested graph ([§2.10](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#210-struct-downstream-enforcement-transitive-struct-field-restrictions)). Let a struct contain a class and a byte-copy would silently duplicate ownership; let it contain an `&` and a byte-copy would silently duplicate ref-tracking state without ever going through the anchor system that makes that state correct. Both break the one invariant that lets struct copies be mechanical, so the closed value world is enforced rather than hoped for — the strictness-buys-speed bargain of the [foundations story](foundations.md#strictness-is-the-performance-model) in miniature. -Dynamically-sized reference types such as `List` and `String` keep fixed-size handles inline. Their variable-sized backing stores live separately, so growing a buffer cannot shift neighbouring hosts or values. Placement is unobservable: the compiler may optimize physical storage as long as destruction, hosting, and guest resolution remain unchanged. +The opened door is placement. Because the anchor model makes a ref resolve identically no matter *where* its owner physically lives — the ref walks to a cell, and the cell can hold a stack address as easily as a heap one — the compiler is free to put a class instance wherever is cheapest, stack or heap, with no language-visible consequence ([§3.5](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#35-class-instances-may-be-placed-on-the-stack)). It uses the stack whenever the instance is statically sized and does not escape in a way a move cannot satisfy, and is forced to the heap only by genuine dynamic size or escape. The thing that makes this freedom *broad* rather than rare is that dynamic size is kept from leaking upward: dynamically-sized reference types such as `List` and `String` are represented as fixed-size handles whose backing stores live separately, so a type containing one stays statically sized, with only the backing store requiring dynamic storage ([§3.6](../spec/memory.md#36-handle-typed-dynamic-reference-types-have-fixed-footprint)). Placement, like the boolean-packing latitude beside it, is something the language hands to the compiler precisely because it has been arranged to be unobservable — and it is unobservable only because the anchor indirection, the thing this whole story is built around, already decoupled a ref from any fixed address. The cost is the one the chapter cannot remove: this only holds for as long as the model keeps placement semantically invisible, and every feature that might let a program *observe* where a value physically sits — raw addresses, layout introspection — is a feature this freedom quietly forbids. -## When the free stacks fragment, and the arena takes the scope +## The kinds collapse into one axis, and `this` becomes a borrow + +The [previous chapter](#the-value-world-stays-closed-and-placement-stays-the-compilers) drew its line between `struct` and `class`, but that pair was later recognized as one axis wearing two names — the [foundations story](foundations.md#identity-is-opt-in-one-axis-for-value-and-reference) tells why the split collapsed into a single `#` modifier. Seen from the memory model, the collapse barely moves anything, which is the tell that the old two-kind framing was carrying more than its weight: the machinery this document is built on — the backpointer, the anchor cell, lazy allocation, move updates — was never about "class" as a keyword. It was about *identity*. So the rule became "a reference type — a `#`-marked type — is the thing that carries an anchor," and everything the old text said about a "class instance" is now said about a reference-type instance, unchanged in substance. The struct-downstream restriction generalized the same way: it was always enforcing "no identity-bearing or ref-bearing field inside a copied value," so it is now **value-downstream** enforcement ([§2.10](https://github.com/zane-lang/spec/blob/9ab62085ed13f22b0a4897425852327264bd022d/spec/memory.md#210-value-downstream-enforcement-transitive-value-only-field-restriction)) — a value type may contain no reference or `&` field, transitively — which reads as a slightly broader rule but forbids exactly what it always did. + +The one genuinely new thing the axis exposed is that `&` and `#` are the same fact from two sides. A stored, move-surviving `&` needs an anchor to point at, and only a reference type has one, so an `&` references a reference type and nothing else ([§2.4](https://github.com/zane-lang/spec/blob/9ab62085ed13f22b0a4897425852327264bd022d/spec/memory.md#24--is-non-owning-storage)): a persistent handle over a value is written `&#T`, and a bare `&Int` over a value is simply ill-formed. This was implicit in the old model — you could only `&` a class — but naming the value/reference axis made it explicit and, better, *composable*: `#Int` is a reference cell and `&#Int` a handle to it, which is how a value that a program wants several observers to share becomes shareable at all. + +That leaves the question the old model answered by forbidding it: if a value has no anchor, how is one mutated in place? The answer, for a value receiver, is that `this` is a **borrow** of the caller's storage, not a copy — the method is handed the caller's slot for the duration of the call. This is what let value types become mutable at all: the previous design made a struct method return a replacement value precisely because a by-value receiver was a copy whose writes the caller would never see, and a scoped borrow of the caller's slot removes that reason. A *reference* receiver needs no such device — it is an implicit `&`, since a reference type already carries an anchor to point at ([`functions.md` §2.4](https://github.com/zane-lang/spec/blob/f8dc73f7553c016b7e4a5ea85dca68a41b5a4f3e/spec/functions.md#24-mutating-methods-use-mut)); the borrow is strictly the value world's device. The borrow is deliberately *non-escaping* — it cannot be stored as an `&` or returned as one, because a value is not `&`-rootable — so a value stays alias-free even while it is being mutated through, which is the property the concurrency safety rule ([`concurrency.md` §4.2](https://github.com/zane-lang/spec/blob/9ab62085ed13f22b0a4897425852327264bd022d/spec/concurrency.md#42-concurrent-mutation-requires-a-value-typed-receiver)) then leans its whole weight on. The cost is a subtlety a reader must hold: a borrow is a reference in the machine, but it is one the type system refuses to let outlive the call, and that refusal — not the absence of a pointer — is what keeps the value world closed. + +## Naming the tether + +Every chapter before this one has called the non-owning `&` by the plainest word to hand — a "ref", short for reference — and for a long time nothing depended on the word. What made it stop being fine was the type system catching up. Once the value/reference *type* axis settled into its own vocabulary — a value type, a reference type, and the `#` that separates them (the [foundations story](foundations.md#identity-is-opt-in-one-axis-for-value-and-reference) tells that collapse) — the language had two different things both leaning on the root "reference": the `#` *kind* of a type, and the `&` *handle* to one. They are orthogonal — the kind is a property of the owner, the handle is a separate non-owning cell that points at it — and yet "reference type" and "ref" are the same word twice, so a bare "reference" in a sentence could mean either. "ref" *felt* like it separated them, but it never did: it is only "reference" with the end bitten off, carrying the exact collision it appeared to resolve. -A single size-class allocator for every object was rejected because classes hoard memory from each other. Ordinary fixed-size scope storage does not need individual reuse: it can use a bump frontier and disappear in bulk when the scope drains. +So we went looking for a word that belonged to the value side alone and shared no root with the type axis. Keeping "ref" was the first option and the first rejected, for the reason just given — it does not actually pull the two apart. The next was **`link`**: plain, short, and honest about the semantics, since a link is a non-owning connection with no suggestion of ownership or lifetime. It was the safe choice, and its flaw was that it was *only* safe — "link" says nothing about the shape of this particular relationship, and it walks straight into "linked list" and "linker" the moment a data-structures chapter needs those words. -Resizable backing stores are different. A list that grows abandons old buffers while the scope may remain active. Each scope therefore has a separate dynamic region with exact-size LIFO stacks. Allocation checks the corresponding size stack first and bumps the dynamic frontier only when that stack is empty. There is no borrowing from neighbouring classes and no coalescing. +The better lead came from a word already in the model. An `&` resolves through an **anchor** — a name chosen chapters ago — and an anchor is a nautical image, so the question asked itself: what, on the value side, is the thing you make fast to an anchor? The precise answers are all real and all unusable. A boat is joined to its anchor by the **rode**, which is exactly the relationship — held fast but not owned, and slippable — but "rode" reads as *road* on the page and looks like a typo. A **mooring pennant** and a **warp** are the same idea under other names, and they lose to "flag" and to "warp speed". The one member of that family that survives ordinary prose is **`tether`**. -Byte size, not element type, defines the classes. A new list starts with 128 bytes regardless of `T`; capacity is derived from `stride(T)`. This makes a returned block reusable by lists of other element types, strings, and other dynamically-sized reference values. +We nearly rejected it too. A tether, we worried, sounds like it *keeps the far thing attached* — as if a live `&` might hold its owner alive, which is the opposite of the rule that a tether can never extend a lifetime. But that reading is backwards, and seeing why is what settled the name. What a thing is tethered *to* is the fixed point; the tether binds and *bounds* the thing on its end, not the anchor. A boat on a tether is held within reach of the anchor; the anchor is not held by the boat. Map that onto the model and it is not a hazard, it is the scope rule stated as an image: an owner is the fixed point, and every `&` taken on it is bound to stay within the owner's reach — a tether may not outlive what it is tied to ([`lifetimes.md` §1.1](https://github.com/zane-lang/spec/blob/93bd2f0036b011c9fc876e785bff0d6a4d09465a/spec/lifetimes.md#11--assignment-uses-owner-scope)). The word we thought fought the semantics turned out to encode them. -Growth doubles the byte size. The allocator first looks for a reusable doubled block. If none exists and the current ordinary block is the frontier allocation with room before its 1 MiB chunk boundary, it grows in place. Otherwise the elements move into a new doubled block and the old block enters its exact-size stack. +So the value-side handle is a **tether** ([`memory.md` §2.4](https://github.com/zane-lang/spec/blob/93bd2f0036b011c9fc876e785bff0d6a4d09465a/spec/memory.md#24--is-a-tether-non-owning-storage)), and the type axis keeps "reference type" to itself. The name pays a small resonance back on a second look, the kind [the naming guide](../contributing/naming-terms.md) hopes for: a tether is *slack* — it does not pull the owner anywhere, it keeps station beside it, which is exactly how a non-owning handle sits next to the value it watches. And it joins a set already speaking the same dialect — a `verb` acts, a `mould` shapes, a value is `borrow`ed and given back, an `anchor` holds fast — so now a `tether` ties to the anchor without owning it. -## The block larger than a page +The cost is the ordinary cost of any coined term: a reader meets "tether" and must be told once what it is, where "ref" would have passed without comment — at the price of the collision that started this. And the rename reaches sideways into ordinary words: an owner with a tether on it is now "tethered", one with none "untethered", which quietly retires "referenced" from the value side to keep the split clean. The earlier chapters of this very story still say "ref", because they were written when that was the word, and the history is left standing rather than back-dated; this chapter is where the name changed, not a pretence that it was always so. -Ordinary dynamic blocks never cross a 1 MiB chunk boundary. Once a power-of-two block exceeds 1 MiB, it becomes an **oversized span**: a dedicated contiguous OS mapping made from consecutive dynamic chunks. -The handle still stores one base segmented offset and one size class. Resolving the base produces a contiguous address range, so element indexing continues normally across the constituent chunks. Every chunk also has a directory entry. Oversized spans participate in the same exact-size reuse policy, but they are never extended in place; later growth relocates into a doubled span. +## When the free stacks fragment, and the arena takes the scope + +By this point the model's *names* had settled, but the machinery underneath them had not. Allocation, all along, had been served by size-indexed free stacks: round every request up to an 8-byte boundary, give each rounded size its own stack of freed slots, and satisfy a request by popping the matching stack or bumping a frontier when it was empty. It was O(1) and it avoided coalescing, and for a long time that was enough. What it could not avoid was the failure mode built into its own shape. Each size class hoards its freed memory and lends it to no other, so exhausting the 32-byte stack is out-of-memory *for 32-byte objects* — even with the 16-byte and 64-byte stacks sitting on abundant free space they will never surrender. A size-classed allocator fragments along its own class boundaries, and under a churny workload that fragmentation is not a tail risk; it is the steady state. + +The first fix we tried was the cheap one, and we rejected it for making the cure worse than the disease. If the 32-byte class is empty, why not serve the request from the 64-byte class — hand out a larger block and waste the difference? It removes the spurious out-of-memory, but it does so by pouring internal fragmentation into every oversized allocation, and internal fragmentation is precisely what size classes existed to prevent: the whole reason to bucket by size is cache density, and a 64-byte slot holding a 32-byte object is a cache line half full of nothing. We would have traded a fragmentation we could see for one smeared invisibly across the whole heap. + +The move that actually dissolved the problem was to stop treating allocation as a global pool at all. Zane already has a strong notion of *when* memory should die: the water-tower scope ([`concurrency.md` §4.1](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/concurrency.md#41-water-tower-lifetime-extension)), the lexical region whose owned objects live exactly until the scope drains. Scopes nest last-in-first-out, and that is exactly the discipline a bump allocator wants: if every scope owns its own **arena** — a region it bump-allocates into and releases whole — then allocation is a single pointer advance and deallocation is a single pointer rewind, because nothing inside a scope outlives the scope. There are no size classes to fragment along, because there are no size classes; a bump arena hands out the next *N* bytes regardless of *N*. The fragmentation problem is not so much solved as made inexpressible ([`memory.md` §3.2](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#32-allocation-is-a-bump-teardown-is-an-unmap)). -This preserves the simple segmented-offset handle without imposing a one-page maximum on lists and strings. +The honest cost is the one every arena pays, and we took it with eyes open. A bump arena does not reclaim individual objects: an owner overwritten or logically destroyed partway through a scope leaves its bytes stranded as dead space until the whole arena is released. A workload that churns heavily *within* one long-lived scope holds more peak memory than the free stacks would have — handing a reclaimed slot straight back was the free stacks' one real virtue. We judged that a good trade, because the pattern arenas punish (unbounded intra-scope churn) is rarer than the pattern they reward (a scope that allocates, works, and drains), and because bulk release buys something the free stacks never could: teardown with no per-object work at all. When a scope drains its whole arena goes back to the OS in one unmap — no walk over the objects, no drop-glue threaded through the exit, the memory simply gone. ## The last table problem, and the segmented offset -A monolithic growable anchor array would eventually relocate, while native pointers to individually allocated cells would make every tether and backpointer 64 bits. Segmented `u32` offsets avoid both costs. The high bits select a 1 MiB chunk and the low bits select an aligned word inside it. A small chunk directory maps that identity to a native base. +Bump arenas paid for themselves everywhere but one place, and it was the place the whole model is built around: the anchor. An anchor has to stay reachable and fixed for as long as any tether points at it, and until now the anchors had lived in one growable [heap-resident table](#from-a-reserved-pool-to-an-indexed-heap-table) that a tether indexed. Put arenas underneath that table and its old flaw turns fatal. A monolithic contiguous table, when it fills, has to `realloc` — allocate a larger block and copy every cell across — and that is an O(N) step that also invalidates any native pointer into the array. We had spent the whole design making moves and overwrites O(1); reintroducing an O(N) resize on the anchor path, the hottest indirection in the language, would have handed all of it back. + +The obvious escape was to scatter the cells: stop keeping anchors in one array, allocate each on its own, and let a tether hold the cell's address directly. That kills the resize — there is no array to grow — but it resurrects the cost the [indexed table](#from-a-reserved-pool-to-an-indexed-heap-table) was invented to kill. A raw address is 64 bits. Inflate every tether and every backpointer back to eight bytes and the cache density we bought by making them `u32`s is gone, and every resolution is back on 64-bit pointer math. We would have solved the resize by un-solving the size. + +The two dead ends pointed at the same missing idea: we needed cells that could be *scattered* — so no table ever resizes — yet *addressed narrowly* — so a reference stays 32 bits. The arenas already supplied the first half: a cell is just another bump allocation, dropped in beside the owner that mints it, with no table in sight. The second half is to keep addressing everything with a `u32` but to read that `u32` as a **segmented offset** rather than a flat index — the high bits name a 1 MiB **chunk** and the low bits a word within it ([`memory.md` §3.1](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#31-scope-arenas-and-segmented-offsets)). An arena is a chain of such chunks; when one fills, the runtime maps another from the OS and gives it the next chunk id, and nothing already placed ever moves. A small chunk directory turns a chunk id into a native base, so a reference resolves in a shift, a mask, and one hot directory load. The cell stays four bytes, the tether stays four bytes, and there is no table left to resize ([`memory.md` §4.1](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#41-the-anchor-cell), [§4.2](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#42-tethers-are-segmented-offsets-not-pointers)). + +What falls out is a memory model that is uniformly 32-bit and, per tethered object, exactly twelve bytes of machinery: the four-byte tether wherever it is stored, the four-byte anchor cell, and the four-byte backpointer the payload carries home to that cell. The double indirection a tether walks — tether to cell, cell to payload — looks like it should cost two cache misses, and the arena is what makes it cost closer to zero: the cell read is a load into arena memory that is almost always already warm. The first thing we tried for that warmth was to drop each cell right beside the payload that mints it, so the two share a cache line and the second hop is paid for by the first. It worked for the deref, and it opened a loose end the next chapter has to pull on — a cell sitting in the payload stream is a cell a payload-only scan has to step over, and a payload's position starts to depend on how many of its neighbours were tethered before it. -Scope chunks and anchor pages use the same directory. Tethers, backpointers, payload locations, dynamic handles, and allocator free-stack entries therefore share one compact representation. Because the low bits count 8-byte words, each anchor uses an 8-byte-aligned physical slot: four bytes for its `u32` payload and four reserved bytes. A 1 MiB anchor page contains 131072 such slots. +The one motion this has to survive is escape. A value that outlives its scope is moved into a parent, and under arenas that means its payload is *copied* into the parent's arena — the child arena is about to be unmapped and cannot keep it. The backpointer is what makes that copy invisible: the runtime follows it to the payload's one anchor cell and rewrites the cell with the payload's new location. Every tether still points at that same cell and never learns the payload moved ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)) — the same one-cell-update that made an in-place move O(1) makes a cross-arena promotion O(1) too. When the scope finally drains, its chunks are unmapped and its cells vanish with the objects they served, and the scope rules that governed tethers all along guarantee nothing still pointing could point into them. -The value `0` is reserved as “no anchor” wherever an anchor identity is expected. Payload offset zero remains valid; only the global anchor pool refuses to issue cell identity zero. +The cost is a ceiling, and a lower one than the flat region carried its own version of. Splitting a `u32` into a chunk id and an offset spends bits on structure that a flat offset spent on range: with 1 MiB chunks and 8-byte-aligned slots the arithmetic lands at 32 GiB of live arena across up to 32768 chunks — roomy, but a hard cap, and a program that genuinely needs more cannot have it without widening the reference and paying back the cache density we just secured. The chunk directory is a hop the flat "region base plus offset" did without, too: one more small, hot table on the resolve path. We were content to trade a fixed horizon and a register-resident directory for an allocator that never fragments, never resizes an anchor table, and vaporizes an entire scope's memory in a single unmap. ## Where the cells live, and the scan that pays for them -Interleaving anchors with payloads makes fixed-size scans less dense and lets guest-creation history disturb alignment. Anchors therefore live in anchor-only pages outside every scope arena. +The [previous chapter](#the-last-table-problem-and-the-segmented-offset) left a cell sitting beside every payload and called the shared cache line a win. It is a win — for the deref. What we had not yet measured was the other side of the same coin, and when we built the layout and ran it on real hardware the coin landed. Two workloads slid backwards. A sweep that reads only payloads — the common case of walking a collection and touching each object's fields — slowed by around a fifth, because the interleaved cells sit between the payloads and a scan that wants only payloads drags the cells through cache anyway; the same objects packed tight would have touched fewer lines. And a write-heavy growth buffer regressed harder still, because once cells share the payload stream the stream's alignment stops being the compiler's to control — a buffer's base now depends on how many cells were minted ahead of it, and a run of entities that should have sat one-per-line ended up straddling two. The shared-line trick had quietly made the *arena's* geometry a function of tether history, and iteration paid for it. -The pool is runtime-global because the cell follows the complete hosting lineage, not whichever scope currently contains the payload. Promotion updates the cell's payload offset and keeps its identity unchanged. No forwarding cell, re-anchoring, or guest repointing is needed. +The tempting repair was to go backwards: put the anchors back in one compact heap table, the way the pre-arena model had. That table was never slow to read — it stayed small and hot, and it kept payloads perfectly contiguous because nothing lived between them. But the reason it read so well is exactly the reason it could not stay. A single global table has no scope to be unmapped with; every cell in it has to be handed back one at a time as its owner dies, or the table leaks and grows without bound. That is the per-object teardown the arenas had just abolished — the "free is a no-op" and one-unmap-per-scope wins are wins *because* nothing walks the objects to reclaim them. A global table would have bought back the payload density by selling the teardown, and the teardown was the larger prize. The compactness that made the table attractive was the very thing that forced the per-object free; the two could not be separated. -All anchor cells have the same physical size, so the pool has one free-address stack rather than size classes. Allocation pops that stack first and bumps the global frontier only when it is empty. Pages are mapped lazily and remain mapped until runtime shutdown, even when wholly free, so offsets retained by the stack never name unmapped or repurposed pages. +So the fix was not to leave the arena but to stop mixing two things inside it. Cells get their own **region** — a separate chunk chain in the same scope arena, addressed by the same segmented offsets, bump-allocated and bulk-unmapped exactly like the payload region, just not braided through it ([`memory.md` §4.1](https://github.com/zane-lang/spec/blob/2caab48a9f4bafda32d67c3bb902908aea1813ab/spec/memory.md#41-the-anchor-cell)). Payload scans go back to full density because the payload stream holds only payloads, and a payload's position no longer depends on what was tethered before it, so the alignment the compiler wants is the compiler's again. The cells, gathered into their own region, are still compact and still hot — a tether resolution reads a cell from a dense, cache-resident run, just not from the same line as the payload it will then visit. It is arenas the whole way down: two streams, one drain. -The global pool does require individual anchor teardown, but the host already supplies an exact teardown event. Overwriting an occupant keeps the hosting lineage alive and retains the cell. Rehosting transfers teardown responsibility to the destination. Only the end of the final hosting lineage returns the address to the stack. +The cost is real and we name it plainly. We gave up the shared-cache-line bonus on the deref itself — the cell and its payload no longer ride into cache together, so a single tether resolution can pay two loads where the interleaved layout often paid one. We judged that the right trade because the payload sweep is the hotter path in the workloads we cared about: densifying the scan that runs over every object beats shaving a load off the deref that runs only when a tether is actually followed. It is the mirror image of the choice the last chapter made, now that we have measured which side of the coin comes up more often. -Immediate reuse needs no generation counter. Lexical scope rules prove that no guest can outlive the host, including guests held by spawned work covered by the water-tower lifetime rules. When the hosting lineage ends, no live guest can still contain the old anchor identity. The ABA state is therefore unrepresentable rather than dynamically detected. +Separating the region also forced us to finish a sentence the last chapter had left dangling. It had described promotion as "rewrite the one cell and every tether follows," which quietly skipped the question of *which arena that cell is in*. Now the answer is unambiguous: the cell lives in the region of the scope that minted it, and on escape that scope is precisely the one about to drain. Every tether already pointing at the cell was taken in that scope or deeper, so none of them outlives it — the lifetime rule that has fenced tethers all along ([`lifetimes.md` §1.1](https://github.com/zane-lang/spec/blob/93bd2f0036b011c9fc876e785bff0d6a4d09465a/spec/lifetimes.md#11--assignment-uses-owner-scope)) does the fencing here too. So promotion updates the old cell to the payload's new home, keeping those doomed-but-still-live tethers reading the promoted copy until their scope ends, and then **resets the payload's backpointer to zero** so the value re-anchors from scratch in its new scope: the next tether taken on it there mints a fresh cell in the destination region, one that finally lives as long as the value does ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/2caab48a9f4bafda32d67c3bb902908aea1813ab/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)). The old cell and its tethers expire together when the source scope drains, and nothing ever reads a cell that has been unmapped. That reset is the honest tax of keeping cells scope-local rather than in one immortal table — a second, cheaper re-anchoring the immortal table would not have needed, paid so that anchors can vanish in the same unmap as everything else. -A concurrent implementation may use thread-local caches backed by the global pool to reduce contention, without changing the semantic identity or lifetime of any anchor. +## The sentinel that costs nothing, and the buffer that wanted a line -## The sentinel that costs one reserved identity, and the buffer that wanted a line +The [separate cell region](#where-the-cells-live-and-the-scan-that-pays-for-them) closed the placement question, but it quietly loosened something we had been treating as fixed, and it took a benchmark regression to notice. Growing an owned buffer — appending a hundred thousand small records to a list — ran markedly slower under the arena than under the free-stack allocator it replaced. Nothing about a pure bump-and-fill should have been slower; the frontier just advances. The cause turned out to be one byte of history. -The global pool reserves anchor identity zero for the untethered sentinel. That costs one unusable slot identity, not one page. Payloads can still occupy segmented offset zero because an anchor cell may legitimately contain that payload location. +A CPU moves memory in fixed sixty-four-byte cache lines: a value wholly inside one line costs one line to touch, a value spanning a boundary costs two. Our growth buffer was based at arena offset 8 — not on a line boundary — and a run of 32-byte records laid down from offset 8 puts every other record across a boundary. The write-heavy fill was paying for nearly twice the lines it should. The old free-stack build had, by luck, handed back a line-aligned block; the arena had not. -Dynamic buffers have a separate alignment concern. Their region starts on chunk boundaries and allocates power-of-two blocks beginning at 128 bytes. Ordinary blocks, reused blocks, and oversized spans therefore remain cache-line aligned without forcing the same padding onto small inline values. +The eight-byte skew was not arbitrary, and finding out why turned a one-line fix into a small piece of design. The arena reserves the value zero as the *untethered* sentinel: a backpointer or tether of `0` means "no anchor." In the earliest arena layout cells were bump-allocated from the same frontier as payloads, so the very first cell could land at offset 0 and collide with the sentinel — we prevented that by starting the frontier at 8, one slot in. But the previous chapter had since moved cells into their own region. Once no cell is ever drawn from the payload frontier, no cell can sit at offset 0 — and the only things that carry the sentinel value are backpointers and tethers, which name *cells*. The reservation was guarding a collision that could no longer happen. It was vestigial. + +That reframed the sentinel entirely. Zero is safe not because we hold the slot empty, but because cells live somewhere zero never is. So the payload frontier can start at offset 0 — a chunk base, which is line-aligned — and the sentinel costs no reserved memory at all: a real payload sits at offset 0, still unambiguous, because 0 is only ever *read* as null through a backpointer or tether, and those name cells ([`memory.md` §3.1](https://github.com/zane-lang/spec/blob/848b6ebc51f03e5826c584f026223f0aad2023f7/spec/memory.md#31-scope-arenas-and-segmented-offsets)). The buffer that started all this, being the first allocation in its scope, now lands line-aligned for free. + +Starting at zero only aligns the *first* allocation, though; a buffer created after other objects still lands wherever the frontier happens to be. So the durable rule aligns the thing that actually cares: a dynamically-sized backing store is allocated cache-line-aligned, the frontier bumped to the next line before it is placed ([`memory.md` §3.6](https://github.com/zane-lang/spec/blob/848b6ebc51f03e5826c584f026223f0aad2023f7/spec/memory.md#36-handle-typed-core-reference-types-have-fixed-footprint)). Only backing stores earn it — they are what gets streamed and grown; a small inline object stays 8-byte aligned, since padding every little allocation up to a line would waste most of a line each for locality it will never use. Payloads from zero and backing stores aligned, together, put the fill back on the old model's number. + +There was a larger temptation in the same corner, and we turned it down. If a tether and a backpointer only ever name cells, why address them with a full segmented offset — why not a dense ordinal index into the cells, a smaller field? The appeal is real but the arithmetic isn't: the width of a reference is the log of how many things it must name, index or offset alike, so an ordinal cell index is only *smaller* than the `u32` we already use if we also cap the number of simultaneously-tethered objects — a `u16` buys two bytes and a 65,536-cell ceiling. And a dense global index needs one packed anchor array to index into, which is exactly what per-scope bulk teardown refuses to provide: draining a scope frees a block wholesale and promotion inserts a cell into a parent region, so the cells never form one contiguous array without a free-list — the per-object teardown we spent the arena to escape. A per-scope index that keeps the bulk free has to carry its scope's chunk id, at which point it *is* the segmented offset again. The real bit-efficiency — half a native pointer, and eight-byte words rather than bytes — is already banked in the `u32` segmented offset; the further notch pays out only in a bounded profile that promises the cap, so we left it there. + +The cost of what we kept is a rounding: up to a cache line of padding before each backing store, unmeasurable against the store it precedes, and the standing discipline that the payload frontier and the cell region stay separate so that zero keeps its meaning for free. Cheap insurance, for a sentinel that now costs nothing and a fill that no longer straddles. + +## Two payload streams, and the anchor that leaves the scope + +The scope arena survived, but the pure-bump conclusion did not survive unchanged. Fixed-size values, reference-type hosts, and dynamic handles still fit the original rule: append them densely and reclaim their chunks when the scope drains. Resizable backing stores do not. A list can abandon several buffers while its scope remains alive, so treating those buffers like ordinary fixed-size payloads strands exactly the kind of reusable holes that matter. The arena therefore split into two lazy chunk chains per scope: one fixed-size region that remains a pure bump allocator, and one dynamic region that may reuse backing-store blocks. A chunk belongs to exactly one region, and a scope that never allocates a backing store never maps a dynamic chunk. + +The dynamic region brings back free stacks in the one place where their fragmentation is controlled rather than global. Blocks use shared power-of-two byte classes beginning at 128 bytes, independent of element type. Allocation checks the exact-size LIFO stack first and bumps the frontier only when that stack is empty. A full list requests exactly twice its current byte size; it grows in place only when it is the frontier allocation and the added bytes fit before the chunk boundary. Otherwise its elements relocate into a reusable doubled block or a newly bumped one, and the old block enters its exact-size stack. Blocks above 1 MiB become dedicated contiguous oversized spans, addressed by one base segmented offset and reused through the same exact-size rule. The cost is dead space between size classes and until scope teardown, but reuse is confined to the buffers whose repeated growth creates it. + +The scope-local anchor region turned out to have a deeper problem than teardown. Promotion could leave source guests naming the old cell while destination guests named a new one; a later move had only one payload backpointer and could update only one path. Moving the old cell would merely force every existing guest to be repointed. The final design therefore takes anchors out of scopes entirely. One runtime-global pool grows through anchor-only pages, and one anchor identity follows a hosting lineage through overwrites and rehosting. Promotion updates that same cell rather than forwarding, recreating, or moving it. + +Each anchor occupies an 8-byte-aligned physical slot: four bytes hold the payload's segmented offset and four are reserved so every slot is addressable by the shared 8-byte-word encoding. The pool uses one free-address stack because all slots have the same size; allocation pops it before bumping the global frontier. Anchor pages remain mapped until runtime shutdown, even when wholly free, so an offset retained by the stack can never name an unmapped or repurposed page. The minimum physical machinery for one tethered lineage is therefore sixteen bytes: a four-byte tether, an eight-byte anchor slot, and a four-byte payload backpointer. + +Individual teardown is the price of taking anchors out of scope arenas, but the host already provides the exact event needed. Overwriting an occupant preserves the lineage and its anchor; rehosting transfers teardown responsibility; only the end of the lineage returns the slot. Lexical lifetime rules prove that no guest can survive that event, so immediate slot reuse needs neither a generation counter nor delayed reclamation. A concurrent runtime may put thread-local caches in front of the same global pool without changing identity. + +One canonical cell also exposes the only move the model cannot represent. If source and destination both have live guest sets, each set already names a different stable identity; preserving both through later moves would require forwarding or guest enumeration. Such a move is rejected. When only one side has live guests, its anchor becomes canonical. When neither does but the moved-from host-capable slot becomes a guest, the runtime creates the one anchor that new guest needs. This also sharpens the storage distinction: an explicit `&T` is guest-only and holds only a tether, while a slot declared `T` keeps its full storage after rehosting and may later host another `T`. + +The sentinel changes by one small accounting detail. Segmented offset zero remains a valid payload location, but the global pool never issues anchor identity zero. The sentinel therefore costs one unusable anchor-slot identity rather than forcing either payload region away from its naturally aligned chunk base. Dynamic blocks begin at 128 bytes and preserve cache-line alignment through doubling, reuse, and oversized spans. ## Two vocabularies: host and guest above anchor and tether -The source-facing relationship is **host / guest**. A host stores the value and governs its lifetime. A guest may access the hosted value but neither stores it nor controls how long it lives. +With the arena layout settled, one vocabulary problem remained. Calling `&T` a tether had solved the collision with “reference type,” but it left the source language and the runtime sharing one word. The problem is that those are different layers. Source code needs names for the lifetime relationship a programmer reasons about; the memory model needs names for the indirection that keeps that relationship working when an object moves. Using `tether` for both made an implementation choice sound like the meaning of `&T` itself. + +The source pair is now **host** and **guest**. A host is the symbol, field, or container slot that stores a reference-type object — or its hosting handle — and governs the object's lifetime. A guest is an `&T`: it may access the hosted object, but it neither stores that object nor controls how long it lives. When the object moves, it is rehosted, and its guests continue reaching it. The ordinary relationship does useful work here: a host provides both accommodation and the duration of a guest's stay, while a guest may use what is provided but cannot outlast the host. -The runtime-facing mechanism is **anchor / tether**. A guest is represented by a tether that names an anchor; the anchor records the payload's current location. Rehosting updates the anchor, so the guest keeps working. +The runtime keeps **anchor** and **tether**. Each guest is represented by a tether that resolves through an anchor; moving or rehosting the object updates the anchor, so existing tethers keep working. That vocabulary remains a natural mechanical picture, but it no longer leaks upward into source semantics. The concise model is: *an object lives in a host; a guest may access it; internally, the guest's tether follows the object through its anchor.* -The concise model is: *an object lives in a host; a guest may access it; internally, the guest's tether follows the object through its anchor.* +The alternatives each blurred something we wanted to keep sharp. **Owner/tether** named the two halves accurately in isolation but paired source semantics with implementation. **Owner/guest** worked, though “owner” stressed rights and destruction more than residence. **Owner/view** was technically reasonable without being a convincing lived relationship. Proxy, keyholder, delegate, and licensee were variously technical, overloaded, or indirect; “key” also collided with dictionary keys. CC/email language suggested secondary participation, but a CC recipient receives an independent copy rather than live access to one moving object. And keeping **tether** as the name of `&T` remained expressive, but preserved the very overload this split was meant to remove. From b5cab00d0939c09b1d11492743f99ecbb35c3679 Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Tue, 28 Jul 2026 15:52:24 +0200 Subject: [PATCH 12/31] docs: anchor pool and move-guest liveness rules (#149) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * docs: fix review defects in the arena and global-anchor change Repair the integration half of the memory-model revision and the cross-references it left stale. - Restore the story pointer at memory.md §3.6; it named a chapter heading that no longer exists, so the link resolved to nothing. - Point the sections whose rules the revision changed (§3.1, §3.2, §3.6, §4.1, §4.5, §4.6) at the new story chapter; §4.6 had no pointer at all. - Restore the story's placement chapter to its previous wording: the chapter predates this change, and its in-prose spec reference is a commit-pinned permalink rather than a living relative link. - Update the glossary entries for anchor cell and arena placement, which still described a scope-local anchor region. - Record the both-sides-guests move restriction in lifetimes.md, the home of move legality, as §1.10 with a summary row. - Restore the arena-granularity latitude and the destruction-timing distinction dropped from §3.1 and §3.2, and note that the fixed-size region reclaims nothing individually. - State how the anchor pool avoids issuing identity 0, restore the address formula, drop a contrast with storage the language has no form for, and square the segmented-offset diagram. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016YnNCXtguHg4tSbLCtTT2c * docs: sharpen the scope-drain guarantee and the move-liveness summary Address review on the promotion and move-liveness wording. - State the drain guarantee as reachability: a scope's memory is released at drain and no guest resolves into released memory, with an escaping value promoted out first and its guests following the canonical anchor. The previous phrasing claimed source-arena memory outlives every guest that can reach it, which promotion breaks. - Say in the glossary that promotion copies the fixed-size bytes — the inline payload or the handle — while a dynamic backing store transfers without being copied. - Complete the move-liveness summary row with the case where neither side has live guests: a moved-from slot that stays readable anchors lazily. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016YnNCXtguHg4tSbLCtTT2c --------- Co-authored-by: Claude --- README.md | 2 +- spec/glossary.md | 6 +++--- spec/lifetimes.md | 25 +++++++++++++++++++++++++ spec/memory.md | 26 ++++++++++++++++++-------- stories/memory.md | 2 +- 5 files changed, 48 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 4edd3ee..39e82bf 100644 --- a/README.md +++ b/README.md @@ -66,7 +66,7 @@ The spec states *what* the language does; the **why** lives in a parallel set of | [`stories/adt.md`](stories/adt.md) | [`spec/adt.md`](spec/adt.md) — splitting `enum` from `variant` against the hype, the shared struct body, escaping the matcher machine with case overloads and the turn to a central `match` block, matching variants rather than patterns, keeping enum data outside the members, reducing a match group to sugar for one arm per case, and building a variant by naming a case rather than calling a constructor | | [`stories/generics.md`](stories/generics.md) | [`spec/generics.md`](spec/generics.md) — the parameter model, the `<>`/`()` split, size-in-the-type, and the deferred features | | [`stories/dependencies.md`](stories/dependencies.md) | [`spec/dependencies.md`](spec/dependencies.md) — URL identity, the manifest/resolution split, prebuilt distribution, symbol-rewriting, the browsable global cache, the package-graph acyclicity rule, opt-in remapping, and why `core` became a bundled implementation package | -| [`stories/memory.md`](stories/memory.md) | [`spec/memory.md`](spec/memory.md) — the no-GC-no-lifetimes goal, the move problem and the anchor, lazy backpointer creation, the indexed heap table, the rooted-guest rules and the host/guest terminology split, the collapse to one value/reference axis with a borrowed receiver, and the shift to segmented chunked bump arenas | +| [`stories/memory.md`](stories/memory.md) | [`spec/memory.md`](spec/memory.md) — the no-GC-no-lifetimes goal, the move problem and the anchor, lazy backpointer creation, the indexed heap table, the rooted-guest rules and the host/guest terminology split, the collapse to one value/reference axis with a borrowed receiver, the shift to segmented chunked bump arenas, and the split into fixed-size and dynamic regions with anchors moved to a runtime-global recyclable pool | | [`stories/lifetimes.md`](stories/lifetimes.md) | [`spec/lifetimes.md`](spec/lifetimes.md) — lexical scope in place of a borrow checker, what may be moved, the declaration-block rule that kills flow analysis, downgrade instead of use-after-move, parameter-rooted returned guests, and why each strict rule is the minimal guard against one specific memory corruption | | [`stories/effects.md`](stories/effects.md) | [`spec/effects.md`](spec/effects.md) — inferring effects instead of annotating them, receiver-scoped `mut`, capabilities in place of ambient I/O, the four-level ladder and the Total-Pure/Pure split, what deliberately is not an effect, and mutation through a borrowed receiver | | [`stories/concurrency.md`](stories/concurrency.md) | [`spec/concurrency.md`](spec/concurrency.md) — the parallelism/concurrency split and the refusal of `async` coloring, why `spawn` marks only a call, water-tower lifetimes, signature-based safety without locks, and value-typed mutation closing the aliased-write gap | diff --git a/spec/glossary.md b/spec/glossary.md index c879ed1..995f11c 100644 --- a/spec/glossary.md +++ b/spec/glossary.md @@ -175,7 +175,7 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`functions.md`](functions.md) §1 ### 3.23 anchor cell -- **Meaning:** A runtime `u32` cell holding the current segmented offset of one hosted object — the stable indirection point through which tethers resolve. It is bump-allocated when the first guest is created, in a dedicated anchor-cell region of the host's scope arena. +- **Meaning:** A runtime cell whose `u32` payload holds the current segmented offset of one hosted object — the stable indirection point through which tethers resolve. It occupies one 8-byte slot in the runtime-global anchor pool, allocated when the first guest is created and returned to the pool's free-address stack when the hosting lineage ends. Its own segmented offset is the anchor identity that a whole hosting lineage keeps, across overwrite and rehosting. - **Why this name:** The cell is the fixed point that lets a moving object remain reachable: rehosting updates the cell while existing tethers keep pointing to it. - **Canonical home:** [`memory.md`](memory.md) §4.1 @@ -185,8 +185,8 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`memory.md`](memory.md) §4.2 ### 3.25 arena placement -- **Meaning:** A reference-type instance is bump-allocated in the arena of the scope that creates it, and is copied (promoted) into a parent arena only if it escapes that scope. Only dynamic size or escape changes where an instance lives. Placement is an unobservable implementation choice. -- **Why this name:** Placement is a choice among **arenas** — the per-scope bump regions — rather than between a stack and a heap; the creating scope's arena is the default, a parent arena the fallback on escape. +- **Meaning:** A scope's arena has two regions: statically sized storage — value slots, reference-type hosts, and dynamic handles — is bump-allocated inline in the fixed-size region of the scope that creates it, while a resizable backing store goes in that scope's dynamic region. An instance that escapes is **promoted**: its fixed-size bytes — the inline payload, or the handle of a dynamically-sized type — are copied into a parent arena, while a dynamic backing store transfers to the new host without being copied. Placement is an unobservable implementation choice. +- **Why this name:** Placement is a choice among **arenas** — the per-scope regions — rather than between a stack and a heap; the creating scope's arena is the default, a parent arena the fallback on escape. - **Canonical home:** [`memory.md`](memory.md) §3.5 ### 3.26 capability marker diff --git a/spec/lifetimes.md b/spec/lifetimes.md index 4eea393..122a12c 100644 --- a/spec/lifetimes.md +++ b/spec/lifetimes.md @@ -214,6 +214,30 @@ Because a floated result is kept rather than dropped, no guest dangles and no ho > **Story:** [`stories/lifetimes.md`](../stories/lifetimes.md#the-signature-is-the-whole-contract-retiring-inferred-consumption) — "The signature is the whole contract: retiring inferred consumption". +### 1.10 A move needs live guests on at most one side + +A move into an already-initialized host is rejected when **both** the source and the destination have live guests at that point. One hosting lineage keeps one anchor identity ([`memory.md`](memory.md) §4.5), and two live guest sets name two identities that the single canonical cell cannot carry forward. The compiler decides this from lexical guest liveness alone, the same way it decides guest assignment (§1.1). + +```zane +a Node() +b Node() +ra &Node = a +b = a // legal: only the source has a live guest; its anchor becomes canonical +ra:inspect() // ra reaches the value in its new home, b +``` + +```zane +c Node() +d Node() +rc &Node = c +rd &Node = d +d = c // ILLEGAL: both sides have live guests +rc:inspect() +rd:inspect() +``` + +Every permitted move stays O(1) in the number of guests, and the guests on the surviving side keep reaching the value in its new home (§1.6). + --- ## 2. Lifetime and Destruction @@ -254,6 +278,7 @@ Because scope rules (§1.1) prevent guests from outliving their hosts, the runti | Move-source | A direct host symbol (local or parameter) or a hosting verb result; not an `&`, field, container element, or other access path | | Move declaration-block restriction | A direct host symbol may only be moved in the exact lexical block where it was declared; parameters may be moved at the body top level | | Move destination scope | Destination host must be in the same or a higher lexical scope than the source host | +| Move guest liveness | A move into an initialized host is rejected when both the source and the destination have live guests | | Post-move downgrade | After a move, the source symbol downgrades to an `&` and remains readable but is no longer a move-source | | Parameter scope | A reference parameter belongs to the call-site scope, not the body, so a value passed by hosting access outlives the call | | Hosting argument | A verb takes a **guest** (`&T`, caller keeps it), **relays** the host (`T` and returns a hosting handle, caller may bind it to host again), or **consumes** it (`T`, no host returned, caller keeps a guest); passing to a plain `T` downgrades the caller to a guest whatever the body does | diff --git a/spec/memory.md b/spec/memory.md index b2623a5..5991141 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -250,6 +250,8 @@ Each lexical scope owns an **arena** made from two independent allocation region Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots and dynamic backing stores never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. +Scopes nest last-in-first-out, and their arenas nest with them: both regions of a scope are unmapped in full the moment the scope drains (§3.2, [`lifetimes.md`](lifetimes.md) §2.1). Arena granularity is an implementation choice, like boolean packing (§3.4) and placement (§3.5) — the compiler may fold several lexical scopes into one arena. What the language fixes is the observable behavior: a scope's memory is released together when that scope drains, and no guest ever resolves into released memory. A value that escapes is promoted out of the draining scope first (§3.5, §3.7), and its guests reach the promoted value through the canonical anchor (§4.5). + Anchors do not belong to any scope arena. The runtime owns one **global anchor pool**, implemented as a lazy chain of anchor-only 1 MiB pages. Every anchor occupies an **8-byte-aligned, 8-byte physical slot**: the first four bytes hold the `u32` payload offset and the remaining four bytes are reserved padding. An anchor page therefore contains 131072 addressable slots. The pool maps its first page only when the program creates its first guest and adds another page only when its current frontier and free-address stack cannot satisfy an allocation. ```text @@ -267,21 +269,22 @@ Scope chunks and global anchor pages draw ids from the same chunk directory, so ``` u32 segmented offset - ┌───────────────┬─────────────────────────┐ + ┌───────────────┬──────────────────────────┐ │ chunk id │ in-chunk word offset │ │ (high bits) │ (low bits) │ - └───────────────┴─────────────────────────┘ + └───────────────┴──────────────────────────┘ ``` -Allocations are at least 8-byte aligned, so the low bits count 8-byte words: a 1 MiB chunk holds 2¹⁷ words, so **17 low bits** address any slot in a chunk and the remaining **15 high bits** select one of up to 32768 live chunks — a reach of 32 GiB. The chunk directory maps a chunk id to the chunk's native base address. +Allocations are at least 8-byte aligned, so the low bits count 8-byte words: a 1 MiB chunk holds 2¹⁷ words, so **17 low bits** address any slot in a chunk and the remaining **15 high bits** select one of up to 32768 live chunks — a reach of 32 GiB. The chunk directory maps a chunk id to the chunk's native base address, so an address is materialized only at use, as `directory[chunk id] + word offset × 8`: splitting the `u32` is a shift and a mask, and the directory lookup is one load. -Tethers (§4.2), per-host backpointers (§4.2), anchor cells (§4.1), dynamic handles, and size-stack entries (§3.2) use segmented offsets. The value `0` is the *untethered* sentinel wherever an anchor identity is expected. The global anchor pool never assigns segmented offset `0`; fixed-size payloads may still occupy it. +Tethers (§4.2), per-host backpointers (§4.2), anchor cells (§4.1), dynamic handles, and size-stack entries (§3.2) use segmented offsets. The value `0` is the *untethered* sentinel wherever an anchor identity is expected. The global anchor pool never issues `0` as an anchor identity: if its first page is assigned chunk id `0`, that page's first slot is left permanently unused. Payloads carry no such restriction and may occupy segmented offset `0`, so a region's first allocation sits at a chunk base. > **Story:** [`stories/memory.md`](../stories/memory.md#the-last-table-problem-and-the-segmented-offset) — "The last table problem, and the segmented offset". +> **Story:** [`stories/memory.md`](../stories/memory.md#two-payload-streams-and-the-anchor-that-leaves-the-scope) — "Two payload streams, and the anchor that leaves the scope". ### 3.2 Allocation, reuse, and teardown -The fixed-size region is a pure bump allocator. A host is a fixed-size storage slot, so overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7); it consumes no new arena space. +The fixed-size region is a pure bump allocator: no size classes, no free list, no coalescing. A host is a fixed-size storage slot, so overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7); it consumes no new arena space. Nothing in this region is reclaimed individually — a slot whose occupant dies before the scope drains stays dead space until teardown. The dynamic region adds exact-size reuse on top of its bump frontier. Dynamic blocks use power-of-two byte sizes beginning at **128 bytes**. Each scope maintains one LIFO **size stack** for every block size that has become reusable. To allocate a dynamic block of size `S`, the runtime first pops `size_stack[S]`; only when that stack is empty does it bump the dynamic frontier. It never satisfies a request from another size stack and never coalesces neighbouring blocks. @@ -289,9 +292,10 @@ Returning a dynamic block pushes its base segmented offset onto the stack for th The global anchor pool has one LIFO **free-address stack**, because every anchor slot has the same size. Creating an anchor pops that stack first; only when it is empty does allocation bump the global anchor frontier, mapping another anchor page as needed. Returning an anchor pushes its segmented offset onto the same stack. Anchor pages remain mapped and retain their chunk-directory entries until runtime shutdown, including when every slot on a page is free; consequently every offset retained by the stack always resolves to its original anchor slot and anchor chunk ids are never repurposed during the run. -When a scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps its fixed-size and dynamic chunks in bulk. Global anchor pages are not tied to scope teardown: individual slots are returned when their hosting lineages end (§4.6), while the pages themselves remain mapped until runtime shutdown. +When a scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps its fixed-size and dynamic chunks in bulk, with no per-object teardown pass threaded through the exit. Logical destruction timing is independent of this: a value dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); it is the *memory* that is reclaimed together at drain. Global anchor pages are not tied to scope teardown: individual slots are returned when their hosting lineages end (§4.6), while the pages themselves remain mapped until runtime shutdown. > **Story:** [`stories/memory.md`](../stories/memory.md#when-the-free-stacks-fragment-and-the-arena-takes-the-scope) — "When the free stacks fragment, and the arena takes the scope". +> **Story:** [`stories/memory.md`](../stories/memory.md#two-payload-streams-and-the-anchor-that-leaves-the-scope) — "Two payload streams, and the anchor that leaves the scope". ### 3.3 Value and reference layout follow declaration order @@ -340,7 +344,8 @@ A block never grows in place across a chunk boundary, and an oversized span is n Dynamic chunks, ordinary power-of-two blocks, and oversized spans begin at cache-line-aligned addresses. Because the minimum block is 128 bytes and every larger block doubles, frontier allocations, reused blocks, and dedicated spans preserve cache-line alignment without mixing backing stores into fixed-size chunks. -> **Story:** [`stories/memory.md`](../stories/memory.md#the-sentinel-that-costs-one-reserved-identity-and-the-buffer-that-wanted-a-line) — "The sentinel that costs one reserved identity, and the buffer that wanted a line". +> **Story:** [`stories/memory.md`](../stories/memory.md#the-sentinel-that-costs-nothing-and-the-buffer-that-wanted-a-line) — "The sentinel that costs nothing, and the buffer that wanted a line". +> **Story:** [`stories/memory.md`](../stories/memory.md#two-payload-streams-and-the-anchor-that-leaves-the-scope) — "Two payload streams, and the anchor that leaves the scope". ### 3.7 Moving a value reuses the destination slot @@ -357,11 +362,12 @@ Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md ### 4.1 The global anchor pool -Tethers are tracked through **anchor cells** in one runtime-global pool rather than through scope-local anchor regions. An anchor cell has a 4-byte logical `u32` payload holding the current segmented offset (§3.1) of a hosted reference-type value, but occupies one 8-byte-aligned physical slot so every cell identity is representable by the shared 8-byte-word offset encoding. The cell's own segmented offset is the stable anchor identity for the complete hosting lineage, even when the value is rehosted across scopes. +Tethers are tracked through **anchor cells** in one runtime-global pool. An anchor cell has a 4-byte logical `u32` payload holding the current segmented offset (§3.1) of a hosted reference-type value, but occupies one 8-byte-aligned physical slot so every cell identity is representable by the shared 8-byte-word offset encoding. The cell's own segmented offset is the stable anchor identity for the complete hosting lineage, even when the value is rehosted across scopes. Anchor pages contain only equal-sized 8-byte slots. The pool therefore needs one free-address stack and one bump frontier rather than size classes. Pages are allocated lazily, never move, and remain mapped until runtime shutdown. > **Story:** [`stories/memory.md`](../stories/memory.md#where-the-cells-live-and-the-scan-that-pays-for-them) — "Where the cells live, and the scan that pays for them". +> **Story:** [`stories/memory.md`](../stories/memory.md#two-payload-streams-and-the-anchor-that-leaves-the-scope) — "Two payload streams, and the anchor that leaves the scope". ### 4.2 Tethers are segmented offsets, not pointers @@ -439,6 +445,7 @@ Every permitted operation is O(1) in the number of guests. Promotion never creat This is also how a moved-from symbol stays readable: after a permitted move the host-capable symbol enters guest state and stores the canonical tether, so reads resolve through the anchor to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). > **Story:** [`stories/memory.md`](../stories/memory.md#the-move-problem-and-the-anchor-that-never-moves) — "The move problem, and the anchor that never moves". +> **Story:** [`stories/memory.md`](../stories/memory.md#two-payload-streams-and-the-anchor-that-leaves-the-scope) — "Two payload streams, and the anchor that leaves the scope". ### 4.6 Hosting-lifetime end returns the anchor @@ -446,6 +453,8 @@ An anchor is returned to the global free-address stack when its **hosting lineag At the actual end of the hosting lineage, lexical scope rules guarantee that every guest capable of naming the anchor has already ceased to exist ([`lifetimes.md`](lifetimes.md) §1, [`concurrency.md`](concurrency.md) §4). The runtime may therefore recycle the slot immediately. No generation counter, delayed reuse, or ABA protection is required: a stale guest is not a representable program state. +> **Story:** [`stories/memory.md`](../stories/memory.md#two-payload-streams-and-the-anchor-that-leaves-the-scope) — "Two payload streams, and the anchor that leaves the scope". + ### 4.7 Why tethers never dangle or misdirect A dangling or misdirected tether would require a guest to outlive its host, an anchor cell to move, or an anchor slot to be reused while an old guest remains. The model forbids all three. Scope checking proves the first impossible; the global pool gives each live hosting lineage one stable cell identity; and the same scope rule makes immediate slot reuse safe after teardown. @@ -509,6 +518,7 @@ A single global free stack and frontier require synchronization under concurrent | Backpointer | Each hosted payload stores the stable `u32` identity of its anchor cell for move updates and tether minting; `0` means no cell has been allocated | | Anchor lifecycle | Lazily allocated on first guest; preserved across overwrite and rehosting; returned to the global free-address stack when the hosting lineage ends | | Anchor reuse safety | Immediate reuse is safe because lexical scope rules make a live stale guest unrepresentable | +| Move guest liveness | A move is rejected when both sides have live guests; with live guests on exactly one side, that side's anchor becomes the canonical one; with none on either side, a moved-from slot that stays readable anchors lazily (see [`lifetimes.md`](lifetimes.md) §1.10) | | Tethered-instance cost | Minimum 16-byte physical footprint: one 4-byte tether, one 8-byte anchor slot, and one 4-byte backpointer | > **See also:** [`lifetimes.md`](lifetimes.md) §4 for the summary of scope, move, and destruction rules. diff --git a/stories/memory.md b/stories/memory.md index f6e8b7f..0ddc8d7 100644 --- a/stories/memory.md +++ b/stories/memory.md @@ -40,7 +40,7 @@ The same rooted-source idea runs through parameters, but with a twist that prote Two last pressures pull in opposite directions — one locks a door, the other opens one — and they are worth telling together because both are about how far the value layer can be trusted to behave. The locked door is the struct. Structs are plain inline values: copied by overwriting bytes, with no anchor and no destruction tracking. That is only sound if a struct can never smuggle in something that *needs* tracking — so a struct field may hold primitives and other structs and nothing else, checked transitively through the whole nested graph ([§2.10](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#210-struct-downstream-enforcement-transitive-struct-field-restrictions)). Let a struct contain a class and a byte-copy would silently duplicate ownership; let it contain an `&` and a byte-copy would silently duplicate ref-tracking state without ever going through the anchor system that makes that state correct. Both break the one invariant that lets struct copies be mechanical, so the closed value world is enforced rather than hoped for — the strictness-buys-speed bargain of the [foundations story](foundations.md#strictness-is-the-performance-model) in miniature. -The opened door is placement. Because the anchor model makes a ref resolve identically no matter *where* its owner physically lives — the ref walks to a cell, and the cell can hold a stack address as easily as a heap one — the compiler is free to put a class instance wherever is cheapest, stack or heap, with no language-visible consequence ([§3.5](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#35-class-instances-may-be-placed-on-the-stack)). It uses the stack whenever the instance is statically sized and does not escape in a way a move cannot satisfy, and is forced to the heap only by genuine dynamic size or escape. The thing that makes this freedom *broad* rather than rare is that dynamic size is kept from leaking upward: dynamically-sized reference types such as `List` and `String` are represented as fixed-size handles whose backing stores live separately, so a type containing one stays statically sized, with only the backing store requiring dynamic storage ([§3.6](../spec/memory.md#36-handle-typed-dynamic-reference-types-have-fixed-footprint)). Placement, like the boolean-packing latitude beside it, is something the language hands to the compiler precisely because it has been arranged to be unobservable — and it is unobservable only because the anchor indirection, the thing this whole story is built around, already decoupled a ref from any fixed address. The cost is the one the chapter cannot remove: this only holds for as long as the model keeps placement semantically invisible, and every feature that might let a program *observe* where a value physically sits — raw addresses, layout introspection — is a feature this freedom quietly forbids. +The opened door is placement. Because the anchor model makes a ref resolve identically no matter *where* its owner physically lives — the ref walks to a cell, and the cell can hold a stack address as easily as a heap one — the compiler is free to put a class instance wherever is cheapest, stack or heap, with no language-visible consequence ([§3.5](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#35-class-instances-may-be-placed-on-the-stack)). It uses the stack whenever the instance is statically sized and does not escape in a way a move cannot satisfy, and is forced to the heap only by genuine dynamic size or escape. The thing that makes this freedom *broad* rather than rare is that dynamic size is kept from leaking upward: the core dynamically-sized types — `List`, `String` — are represented as fixed-size handles whose backing store lives on the heap, so a type containing one stays statically sized and stack-eligible, with only the backing store forced onto the heap ([§3.6](https://github.com/zane-lang/spec/blob/9abc6748ebcbf3b27a011f7729b78f98f110d9f8/spec/memory.md#36-handle-typed-core-classes-have-fixed-footprint)). Placement, like the boolean-packing latitude beside it, is something the language hands to the compiler precisely because it has been arranged to be unobservable — and it is unobservable only because the anchor indirection, the thing this whole story is built around, already decoupled a ref from any fixed address. The cost is the one the chapter cannot remove: this only holds for as long as the model keeps placement semantically invisible, and every feature that might let a program *observe* where a value physically sits — raw addresses, layout introspection — is a feature this freedom quietly forbids. ## The kinds collapse into one axis, and `this` becomes a borrow From 6140d1aa4fc1bffe35bd1bd52e72c5bffa7614a6 Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Tue, 28 Jul 2026 16:43:49 +0200 Subject: [PATCH 13/31] Clarify rehosting storage relocation Define rehosting as copying the complete hosted representation into destination-owned storage. Inline bytes move into the destination slot, dynamic backing stores relocate into equal-size destination-region allocations, old source storage ceases to be live, and the source host-capable slot becomes a guest through the canonical tether. Keep anchor bookkeeping O(1) in the number of guests while stating the physical relocation cost explicitly. Align the normative memory model, glossary, and design story, and complete the glossary's verb list with operators and lambdas. --- spec/glossary.md | 4 ++-- spec/memory.md | 16 ++++++++-------- stories/memory.md | 6 ++++-- 3 files changed, 14 insertions(+), 12 deletions(-) diff --git a/spec/glossary.md b/spec/glossary.md index 995f11c..91a61fb 100644 --- a/spec/glossary.md +++ b/spec/glossary.md @@ -75,7 +75,7 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`memory.md`](memory.md) §2.10 ### 3.3 unified type parameters -- **Meaning:** A type or number parameter is a *type parameter* (`name Type`, an uppercase name such as `T`, ranging over types) or a *number parameter* (`name Number`, a lowercase name such as `n`, ranging over compile-time numbers and resolving to a number value in body positions). A type definition declares its parameters in a `<>` header (their order is applied positionally at use sites); a verb — function, method, or constructor — has no header and introduces each parameter inline within its value parameters, at the parameter's first marked occurrence. Parameters are referenced by bare name; casing carries the kind. +- **Meaning:** A type or number parameter is a *type parameter* (`name Type`, an uppercase name such as `T`, ranging over types) or a *number parameter* (`name Number`, a lowercase name such as `n`, ranging over compile-time numbers and resolving to a number value in body positions). A type definition declares its parameters in a `<>` header (their order is applied positionally at use sites); a verb — function, method, operator, constructor, or lambda — has no header and introduces each parameter inline within its value parameters, at the parameter's first marked occurrence. Parameters are referenced by bare name; casing carries the kind. - **Why this name:** Type and number parameters share one concept-and-reference system (the `Type`/`Number` concepts, bare references, and the casing rule) across types and verbs; only the introduction site differs — a header for types, which are applied positionally, and inline for verbs, whose parameters are always inferred. - **Canonical home:** [`generics.md`](generics.md) §3 @@ -185,7 +185,7 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`memory.md`](memory.md) §4.2 ### 3.25 arena placement -- **Meaning:** A scope's arena has two regions: statically sized storage — value slots, reference-type hosts, and dynamic handles — is bump-allocated inline in the fixed-size region of the scope that creates it, while a resizable backing store goes in that scope's dynamic region. An instance that escapes is **promoted**: its fixed-size bytes — the inline payload, or the handle of a dynamically-sized type — are copied into a parent arena, while a dynamic backing store transfers to the new host without being copied. Placement is an unobservable implementation choice. +- **Meaning:** A scope's arena has two regions: statically sized storage — value slots, reference-type hosts, and dynamic handles — is bump-allocated inline in the fixed-size region of the scope that creates it, while a resizable backing store goes in that scope's dynamic region. Rehosting copies the complete hosted representation into destination-owned storage: inline bytes move into the destination fixed-size region, each dynamic backing store is relocated into an equal-size destination-region allocation, and the old source storage ceases to be live. The source host-capable slot then stores the canonical tether as a guest. Placement is an unobservable implementation choice. - **Why this name:** Placement is a choice among **arenas** — the per-scope regions — rather than between a stack and a heap; the creating scope's arena is the default, a parent arena the fallback on escape. - **Canonical home:** [`memory.md`](memory.md) §3.5 diff --git a/spec/memory.md b/spec/memory.md index 5991141..a9eafc2 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -284,7 +284,7 @@ Tethers (§4.2), per-host backpointers (§4.2), anchor cells (§4.1), dynamic ha ### 3.2 Allocation, reuse, and teardown -The fixed-size region is a pure bump allocator: no size classes, no free list, no coalescing. A host is a fixed-size storage slot, so overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7); it consumes no new arena space. Nothing in this region is reclaimed individually — a slot whose occupant dies before the scope drains stays dead space until teardown. +The fixed-size region is a pure bump allocator: no size classes, no free list, no coalescing. A host has a fixed-size storage slot, so overwriting it destroys the current occupant and initializes the replacement directly in the same slot (§2.2, §3.7); the overwrite consumes no new space in the fixed-size region. Any dynamic backing stores owned by the destroyed occupant are returned to their exact-size stacks before the replacement becomes live. Nothing in the fixed-size region is reclaimed individually — bytes in a slot that cease to be live before the scope drains remain dead space until teardown. The dynamic region adds exact-size reuse on top of its bump frontier. Dynamic blocks use power-of-two byte sizes beginning at **128 bytes**. Each scope maintains one LIFO **size stack** for every block size that has become reusable. To allocate a dynamic block of size `S`, the runtime first pops `size_stack[S]`; only when that stack is empty does it bump the dynamic frontier. It never satisfies a request from another size stack and never coalesces neighbouring blocks. @@ -311,7 +311,7 @@ The compiler may pack booleans in structs and arena frames when doing so does no Placement is an implementation decision, not a language-visible property. The arena model places every materialized, statically sized scope slot — value-type storage, a reference-type host, or a dynamic type's fixed-size handle — inline in that scope's fixed-size region. The compiler may keep an unobservable value in registers or otherwise optimize its physical placement, but reference types do not require a separate heap allocation merely because they carry identity. -When a reference-type instance is rehosted into a longer-lived destination, its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). A dynamic backing store is semantically owned by the current host. The compiler **MUST** place that store in a dynamic region whose lifetime covers every destination into which the handle can be rehosted, so rehosting transfers ownership of the same backing store without copying it. Only growth of the dynamic value may relocate the backing store (§3.6). +When a reference-type instance is rehosted, all storage owned by that host is relocated into storage owned by the destination. Its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). For every dynamic backing store, the runtime allocates an equal-size block or oversized span in the destination scope's dynamic region, relocates the live contents into it according to their ordinary move rules, updates the copied handle, and then returns the old source block or span to its exact-size stack. Anchored reference-type values hosted inside a relocated backing store update their own canonical anchor cells as their host locations change. A promotion therefore completes before the source scope may drain and leaves no destination handle pointing into source-scope memory. Placement never changes observable semantics: destruction stays deterministic (see [`lifetimes.md`](lifetimes.md) §2), and tethers resolve identically regardless of physical placement (§4), because a tether follows the host's anchor rather than a fixed address. @@ -354,7 +354,7 @@ A move transfers hosting into a destination host of the **same type** (see [`lif - Moving into a fresh declaration or a return slot is in-place initialization. - Moving into an already-initialized host first destroys the current occupant, then overwrites the same-size slot. -Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. A move into a higher scope copies the inline bytes into the destination scope's fixed-size region — a promotion (§3.5). Because handle-typed fields (§3.6) keep the moved footprint small, rehosting copies only the handle and transfers ownership of the same backing store; rehosting itself never relocates that store. A dynamic store changes address only through the growth procedure in §3.6. If the moved value is tethered, the destination inherits the same global anchor identity and updates its one cell (§4.5), never the tethers themselves. +Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. Rehosting copies the complete hosted representation into destination-owned storage. The inline payload or handle is copied into the destination's fixed-size slot. Each dynamic backing store is relocated into an equal-size destination-region block or oversized span as specified in §3.5; after its live contents and any contained anchor locations have been updated, the old store is returned to the source scope's exact-size stack. The source payload bytes then cease to be live. Its host-capable slot is rewritten into guest state and stores the canonical tether; the rest of that full-size slot is dead until the slot is overwritten or its scope drains. If the moved value is tethered, the destination inherits the same global anchor identity and updates its one cell (§4.5), never the tethers themselves. --- @@ -436,11 +436,11 @@ The added cost over direct host access is one dependent anchor-cell load. Across An overwrite from a newly materialized value and a move from another host are distinct cases. - **Ordinary overwrite:** if the destination hosting slot already has an anchor, the replacement payload inherits that backpointer and the cell is updated to the replacement's location. Existing destination guests therefore observe the new occupant. Destroying the old occupant does not return the cell, because the destination hosting lineage continues. -- **Move or rehosting with guests on only one side:** the anchor named by the live guest set becomes the destination's canonical anchor. If only the source has live guests, its anchor transfers to the destination. If only the destination has live guests, its anchor is preserved and the source's moved-from slot becomes another guest to it. Any noncanonical anchor left from an earlier, now-ended guest set is returned before the move completes. If neither side had live guests but the source slot remains readable in guest state, an anchor is allocated lazily for that new guest. The canonical cell is updated to the destination payload, the destination assumes teardown responsibility, and the source host-capable slot stores its tether. +- **Move or rehosting with guests on only one side:** the anchor named by the live guest set becomes the destination's canonical anchor. If only the source has live guests, its anchor transfers to the destination. If only the destination has live guests, its anchor is preserved and the source's moved-from slot becomes another guest to it. Any noncanonical anchor left from an earlier, now-ended guest set is returned before the move completes. If neither side had live guests but the source slot remains readable in guest state, an anchor is allocated lazily for that new guest. The complete hosted representation is first relocated into destination-owned storage (§3.5, §3.7). The old source payload and backing-store bytes then cease to be live, the canonical cell is updated to the destination payload, the destination assumes teardown responsibility, and the source host-capable slot stores only its canonical tether in guest state. - **Move or rehosting with live guests on both sides:** the program is ill-formed. The two guest sets name distinct stable identities, and a one-cell payload backpointer cannot preserve both through later moves without forwarding or guest enumeration. The compiler **MUST** reject the operation rather than recycle either referenced anchor. This restriction is determined from lexical guest liveness, not merely from whether a backpointer is nonzero. - **Consumed untethered temporaries:** a temporary with no source slot that must remain readable may materialize into an untethered destination with backpointer `0` and allocate no anchor. -Every permitted operation is O(1) in the number of guests. Promotion never creates a second live anchor path: it either preserves the sole live identity or is rejected. Source-scope and destination-scope guests therefore remain coherent after all later moves without repointing or forwarding. +Anchor bookkeeping for every permitted operation is O(1) in the number of guests. Physical rehosting is proportional to the bytes or elements relocated and may update the canonical anchors of contained reference-type hosts, but it never enumerates guests. Promotion never creates a second live anchor path: it either preserves the sole live identity or is rejected. Source-scope and destination-scope guests therefore remain coherent after all later moves without repointing or forwarding. This is also how a moved-from symbol stays readable: after a permitted move the host-capable symbol enters guest state and stores the canonical tether, so reads resolve through the anchor to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). @@ -461,7 +461,7 @@ A dangling or misdirected tether would require a guest to outlive its host, an a ### 4.8 Resolution and allocation cost -The segmented encoding adds no meaningful arithmetic cost: the shift and mask that split a `u32` fold into machine addressing once the chunk base is loaded. Tether resolution pays one dependent anchor-cell load beyond direct host access. Rehosting adds no forwarding hop. +The segmented encoding adds no meaningful arithmetic cost: the shift and mask that split a `u32` fold into machine addressing once the chunk base is loaded. Tether resolution pays one dependent anchor-cell load beyond direct host access. Rehosting adds no forwarding hop and no guest-enumeration cost; its physical relocation cost remains proportional to the representation moved. A single global free stack and frontier require synchronization under concurrent allocation and teardown. Implementations may use thread-local anchor caches backed by the same global pool without changing anchor identity, reuse order semantics, or lifetime guarantees. @@ -497,7 +497,7 @@ A single global free stack and frontier require synchronization under concurrent | Hosting storage | Reference-typed symbols, fields, and container elements are directly initialized and may later be overwritten | | Value type | Mutable in place through a borrowed `mut` receiver; storage may also be overwritten freely | | `&` (guest) | Guest-only non-hosting storage; stores one tether, may be repointed, copied by value, and returned, but can never directly host a `T` | -| Host-capable guest state | A slot declared as `T` may become a guest after rehosting while retaining enough storage to host another `T` later | +| Host-capable guest state | After rehosting, the old hosted bytes cease to be live and a slot declared as `T` stores the canonical tether as a guest while retaining enough storage to host another `T` later | | Place expression | Existing stable storage: a named symbol, a field access of a place, a place-projection subscript of a place, or an `&` parameter | | New `&` value | May be initialized only from a named symbol, a field access of a place, or an `&` parameter; temporaries and `[]` expressions are rejected | | `&` parameter | Declares that the caller must supply an `&`-creating source; the parameter is place-like inside the callee | @@ -508,7 +508,7 @@ A single global free stack and frontier require synchronization under concurrent | Value-downstream enforcement | Value types may contain only primitives and other value types, transitively — never a reference (`#`) or `&` field | | `&` targets reference types | An `&T` requires `T` to be a reference type; a value is shared by copy or scoped borrow, never by a stored `&` | | Symbol declaration | Must be directly initialized | -| Reference-type placement | Bump-allocated in the creating scope's fixed-size region; promoted to a parent region only on escape — an unobservable choice | +| Reference-type placement | Inline storage is bump-allocated in the creating scope's fixed-size region; rehosting copies inline bytes and every owned dynamic backing store into destination-owned regions before source storage is retired | | `&` representation | A guest is represented internally by a `u32` tether: a segmented offset (chunk id + in-chunk offset) to the host's global anchor cell; `0` means no tether | | Addressing | Scope chunks and global anchor pages share one `u32` segmented-offset directory; 8-byte-aligned offsets reach 32 GiB across up to 32768 1 MiB chunks | | Untethered sentinel | `0`; the global anchor pool reserves this identity, while payloads may still occupy segmented offset `0` | diff --git a/stories/memory.md b/stories/memory.md index 0ddc8d7..c72ddae 100644 --- a/stories/memory.md +++ b/stories/memory.md @@ -85,7 +85,7 @@ The two dead ends pointed at the same missing idea: we needed cells that could b What falls out is a memory model that is uniformly 32-bit and, per tethered object, exactly twelve bytes of machinery: the four-byte tether wherever it is stored, the four-byte anchor cell, and the four-byte backpointer the payload carries home to that cell. The double indirection a tether walks — tether to cell, cell to payload — looks like it should cost two cache misses, and the arena is what makes it cost closer to zero: the cell read is a load into arena memory that is almost always already warm. The first thing we tried for that warmth was to drop each cell right beside the payload that mints it, so the two share a cache line and the second hop is paid for by the first. It worked for the deref, and it opened a loose end the next chapter has to pull on — a cell sitting in the payload stream is a cell a payload-only scan has to step over, and a payload's position starts to depend on how many of its neighbours were tethered before it. -The one motion this has to survive is escape. A value that outlives its scope is moved into a parent, and under arenas that means its payload is *copied* into the parent's arena — the child arena is about to be unmapped and cannot keep it. The backpointer is what makes that copy invisible: the runtime follows it to the payload's one anchor cell and rewrites the cell with the payload's new location. Every tether still points at that same cell and never learns the payload moved ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)) — the same one-cell-update that made an in-place move O(1) makes a cross-arena promotion O(1) too. When the scope finally drains, its chunks are unmapped and its cells vanish with the objects they served, and the scope rules that governed tethers all along guarantee nothing still pointing could point into them. +The one motion this has to survive is escape. A value that outlives its scope is moved into a parent, and under arenas that means its payload is *copied* into the parent's arena — the child arena is about to be unmapped and cannot keep it. The backpointer is what makes that copy invisible: the runtime follows it to the payload's one anchor cell and rewrites the cell with the payload's new location. Every tether still points at that same cell and never learns the payload moved ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)). The one-cell update keeps anchor bookkeeping O(1) in the number of tethers; physical promotion still costs proportionally to the representation copied. When the scope finally drains, its chunks are unmapped and its cells vanish with the objects they served, and the scope rules that governed tethers all along guarantee nothing still pointing could point into them. The cost is a ceiling, and a lower one than the flat region carried its own version of. Splitting a `u32` into a chunk id and an offset spends bits on structure that a flat offset spent on range: with 1 MiB chunks and 8-byte-aligned slots the arithmetic lands at 32 GiB of live arena across up to 32768 chunks — roomy, but a hard cap, and a program that genuinely needs more cannot have it without widening the reference and paying back the cache density we just secured. The chunk directory is a hop the flat "region base plus offset" did without, too: one more small, hot table on the resolve path. We were content to trade a fixed horizon and a register-resident directory for an allocator that never fragments, never resizes an anchor table, and vaporizes an entire scope's memory in a single unmap. @@ -123,7 +123,9 @@ The scope arena survived, but the pure-bump conclusion did not survive unchanged The dynamic region brings back free stacks in the one place where their fragmentation is controlled rather than global. Blocks use shared power-of-two byte classes beginning at 128 bytes, independent of element type. Allocation checks the exact-size LIFO stack first and bumps the frontier only when that stack is empty. A full list requests exactly twice its current byte size; it grows in place only when it is the frontier allocation and the added bytes fit before the chunk boundary. Otherwise its elements relocate into a reusable doubled block or a newly bumped one, and the old block enters its exact-size stack. Blocks above 1 MiB become dedicated contiguous oversized spans, addressed by one base segmented offset and reused through the same exact-size rule. The cost is dead space between size classes and until scope teardown, but reuse is confined to the buffers whose repeated growth creates it. -The scope-local anchor region turned out to have a deeper problem than teardown. Promotion could leave source guests naming the old cell while destination guests named a new one; a later move had only one payload backpointer and could update only one path. Moving the old cell would merely force every existing guest to be repointed. The final design therefore takes anchors out of scopes entirely. One runtime-global pool grows through anchor-only pages, and one anchor identity follows a hosting lineage through overwrites and rehosting. Promotion updates that same cell rather than forwarding, recreating, or moving it. +The scope-local anchor region turned out to have a deeper problem than teardown. Promotion could leave source guests naming the old cell while destination guests named a new one; a later move had only one payload backpointer and could update only one path. Moving the old cell would merely force every existing guest to be repointed. The final design therefore takes anchors out of scopes entirely. One runtime-global pool grows through anchor-only pages, and one anchor identity follows a hosting lineage through overwrites and rehosting. Promotion first relocates the complete hosted representation into destination-owned storage and then updates that same cell rather than forwarding, recreating, or moving it. + +That distinction matters for dynamic values. Rehosting copies the inline payload or handle into the destination host and relocates every owned backing store into an equal-size block or oversized span in the destination scope's dynamic region. The live contents move under their ordinary rules, including anchor updates for contained reference-type hosts; the old dynamic allocations are returned to the source scope's exact-size stacks. The old source payload bytes cease to be live, and its host-capable slot stores only the canonical tether as a guest. The anchor work remains O(1) in the number of guests, but the physical move is proportional to the bytes or elements relocated. Each anchor occupies an 8-byte-aligned physical slot: four bytes hold the payload's segmented offset and four are reserved so every slot is addressable by the shared 8-byte-word encoding. The pool uses one free-address stack because all slots have the same size; allocation pops it before bumping the global frontier. Anchor pages remain mapped until runtime shutdown, even when wholly free, so an offset retained by the stack can never name an unmapped or repurposed page. The minimum physical machinery for one tethered lineage is therefore sixteen bytes: a four-byte tether, an eight-byte anchor slot, and a four-byte payload backpointer. From a717afcbc8cba03fbd10c74f28afe0351f95f796 Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Wed, 29 Jul 2026 18:30:32 +0200 Subject: [PATCH 14/31] Add temporary forwarding-anchor update workflow --- .../tmp-forwarding-anchor-update.yml | 225 ++++++++++++++++++ 1 file changed, 225 insertions(+) create mode 100644 .github/workflows/tmp-forwarding-anchor-update.yml diff --git a/.github/workflows/tmp-forwarding-anchor-update.yml b/.github/workflows/tmp-forwarding-anchor-update.yml new file mode 100644 index 0000000..bb274c0 --- /dev/null +++ b/.github/workflows/tmp-forwarding-anchor-update.yml @@ -0,0 +1,225 @@ +name: Temporary forwarding-anchor update + +on: + push: + branches: + - agent/fix-host-overwrite-reuse + +permissions: + contents: write + +jobs: + update: + if: github.actor != 'github-actions[bot]' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + ref: agent/fix-host-overwrite-reuse + + - name: Update forwarding-anchor specification + shell: bash + run: | + python - <<'PY' + from pathlib import Path + + def replace_once(path: str, old: str, new: str) -> None: + file = Path(path) + text = file.read_text() + count = text.count(old) + if count != 1: + raise SystemExit(f"{path}: expected one match, found {count}: {old[:80]!r}") + file.write_text(text.replace(old, new)) + + replace_once( + "spec/memory.md", + "Anchors do not belong to any scope arena. The runtime owns one **global anchor pool**, implemented as a lazy chain of anchor-only 1 MiB pages. Every anchor occupies an **8-byte-aligned, 8-byte physical slot**: the first four bytes hold the `u32` payload offset and the remaining four bytes are reserved padding. An anchor page therefore contains 131072 addressable slots. The pool maps its first page only when the program creates its first guest and adds another page only when its current frontier and free-address stack cannot satisfy an allocation.", + "Anchors do not belong to any scope arena. The runtime owns one **global anchor pool**, implemented as a lazy chain of anchor-only 1 MiB pages. Every anchor occupies an **8-byte-aligned, 8-byte physical slot**: the first four bytes hold a `u32` target segmented offset and the remaining four bytes identify whether that target is a hosted payload or another anchor. An anchor page therefore contains 131072 addressable slots. The pool maps its first page only when the program creates its first guest and adds another page only when its current frontier and free-address stack cannot satisfy an allocation." + ) + + replace_once( + "spec/memory.md", + "When a reference-type instance is rehosted, all storage owned by that host is relocated into storage owned by the destination. Its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). For every dynamic backing store, the runtime allocates an equal-size block or oversized span in the destination scope's dynamic region, relocates the live contents into it according to their ordinary move rules, updates the copied handle, and then returns the old source block or span to its exact-size stack. Anchored reference-type values hosted inside a relocated backing store update their own canonical anchor cells as their host locations change. A promotion therefore completes before the source scope may drain and leaves no destination handle pointing into source-scope memory.", + "When a reference-type instance is rehosted, all storage owned by that host is relocated into storage owned by the destination. Its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). For every dynamic backing store, the runtime allocates an equal-size block or oversized span in the destination scope's dynamic region, relocates the live contents into it according to their ordinary move rules, updates the copied handle, and then returns the old source block or span to its exact-size stack. Anchored reference-type values hosted inside a relocated backing store update their terminal payload anchors or install forwarding anchors as their host locations and identities merge. A promotion therefore completes before the source scope may drain and leaves no destination handle pointing into source-scope memory." + ) + + replace_once( + "spec/memory.md", + "Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. Rehosting copies the complete hosted representation into destination-owned storage. The inline payload or handle is copied into the destination's fixed-size slot. Each dynamic backing store is relocated into an equal-size destination-region block or oversized span as specified in §3.5; after its live contents and any contained anchor locations have been updated, the old store is returned to the source scope's exact-size stack. The source payload bytes then cease to be live. Its host-capable slot is rewritten into guest state and stores the canonical tether; the rest of that full-size slot is dead until the slot is overwritten or its scope drains. If the moved value is tethered, the destination inherits the same global anchor identity and updates its one cell (§4.5), never the tethers themselves.", + "Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. Rehosting copies the complete hosted representation into destination-owned storage. The inline payload or handle is copied into the destination's fixed-size slot. Each dynamic backing store is relocated into an equal-size destination-region block or oversized span as specified in §3.5; after its live contents and any contained anchor locations have been updated, the old store is returned to the source scope's exact-size stack. The source payload bytes then cease to be live. Its host-capable slot is rewritten into guest state and stores a tether to the destination's terminal anchor; the rest of that full-size slot is dead until the slot is overwritten or its scope drains. Existing source guests continue through the source anchor, which either becomes the destination's payload anchor or forwards to the destination's existing anchor (§4.5); no guest is enumerated or rewritten." + ) + + replace_once( + "spec/memory.md", + "Tethers are tracked through **anchor cells** in one runtime-global pool. An anchor cell has a 4-byte logical `u32` payload holding the current segmented offset (§3.1) of a hosted reference-type value, but occupies one 8-byte-aligned physical slot so every cell identity is representable by the shared 8-byte-word offset encoding. The cell's own segmented offset is the stable anchor identity for the complete hosting lineage, even when the value is rehosted across scopes.", + "Tethers are tracked through **anchor cells** in one runtime-global pool. An anchor cell occupies one 8-byte-aligned physical slot so every cell identity is representable by the shared 8-byte-word offset encoding. Its first `u32` is a segmented target offset; its second `u32` identifies the target as either a hosted payload or another anchor. A payload anchor is the terminal identity stored in the hosted payload's backpointer. A forwarding anchor preserves an older guest identity after two hosting lineages merge, without changing any existing tether." + ) + + replace_once( + "spec/memory.md", + "Every reference-type payload reserves a `u32` backpointer field initialized to `0`. Once an anchor exists, that field stores the same stable anchor identity that its guests store. Guests and backpointers never store the payload address directly.", + "Every reference-type payload reserves a `u32` backpointer field initialized to `0`. Once an anchor exists, that field stores the terminal payload-anchor identity. Guests may store either that terminal identity or an older identity that forwards to it. Neither guests nor backpointers store the payload address directly." + ) + + replace_once( + "spec/memory.md", + "The minimum physical footprint attributable to one tethered hosting lineage is **16 bytes**: one 4-byte tether, one 8-byte physical anchor slot (containing a 4-byte cell payload), and one 4-byte payload backpointer. Each additional guest adds another 4-byte tether. Rehosting adds no forwarding metadata and, when only one side has live guests, no additional cell.", + "The minimum physical footprint attributable to one tethered hosting lineage is **16 bytes**: one 4-byte tether, one 8-byte physical anchor slot, and one 4-byte payload backpointer. Each additional guest adds another 4-byte tether. Merging two already-anchored hosting lineages reuses both existing cells: the destination cell remains the terminal payload anchor and the source cell becomes a forwarding anchor." + ) + + replace_once( + "spec/memory.md", + "Resolving a tether uses the chunk directory to locate the global anchor cell, reads the hosted payload's current segmented offset from that cell, resolves that offset through the same directory, then accesses the field. Because the pool never allocates cell identity `0`, resolving an untethered `0` traps rather than reading a live cell.", + "Resolving a tether uses the chunk directory to locate its global anchor cell. If the cell forwards, resolution repeats with the target anchor until it reaches a payload anchor; it then resolves that cell's payload offset through the same directory and accesses the field. Forwarding chains cannot cycle because a move only redirects a superseded source identity toward the destination's terminal identity. The runtime may path-compress visited forwarding cells. Because the pool never allocates cell identity `0`, resolving an untethered `0` traps rather than reading a live cell." + ) + + replace_once( + "spec/memory.md", + "Moves, overwrites, and promotions update only the current payload offset in that same cell. Guests created before and after a promotion therefore follow an identical path, with no forwarding cells and no promotion-dependent extra hop.\n\nThe added cost over direct host access is one dependent anchor-cell load. Across repeated accesses through the same guest with no intervening move or overwrite, the compiler may resolve the host address once and reuse it.", + "Ordinary overwrites and moves without an existing destination identity update one payload anchor. When two anchored hosting lineages merge, the destination anchor remains terminal and the source anchor forwards to it. Existing source guests therefore gain a forwarding hop, while destination guests and newly minted guests continue to use the terminal anchor directly. Copying or rebinding a guest through a forwarding tether resolves and stores the terminal identity, so an obsolete anchor identity cannot escape the lexical scope in which it was originally usable.\n\nThe added cost over direct host access is one dependent anchor-cell load for a terminal tether and one load per uncompressed forwarding hop for an older tether. Across repeated accesses through the same guest with no intervening move or overwrite, the compiler may resolve the host address once and reuse it; the runtime may also compress the chain." + ) + + old_45 = """### 4.5 Moves, overwrites, and rehosting keep one canonical anchor + +An overwrite from a newly materialized value and a move from another host are distinct cases. + +- **Ordinary overwrite:** if the destination hosting slot already has an anchor, the replacement payload inherits that backpointer and the cell is updated to the replacement's location. Existing destination guests therefore observe the new occupant. Destroying the old occupant does not return the cell, because the destination hosting lineage continues. +- **Move or rehosting with guests on only one side:** the anchor named by the live guest set becomes the destination's canonical anchor. If only the source has live guests, its anchor transfers to the destination. If only the destination has live guests, its anchor is preserved and the source's moved-from slot becomes another guest to it. Any noncanonical anchor left from an earlier, now-ended guest set is returned before the move completes. If neither side had live guests but the source slot remains readable in guest state, an anchor is allocated lazily for that new guest. The complete hosted representation is first relocated into destination-owned storage (§3.5, §3.7). The old source payload and backing-store bytes then cease to be live, the canonical cell is updated to the destination payload, the destination assumes teardown responsibility, and the source host-capable slot stores only its canonical tether in guest state. +- **Move or rehosting with live guests on both sides:** the program is ill-formed. The two guest sets name distinct stable identities, and a one-cell payload backpointer cannot preserve both through later moves without forwarding or guest enumeration. The compiler **MUST** reject the operation rather than recycle either referenced anchor. This restriction is determined from lexical guest liveness, not merely from whether a backpointer is nonzero. +- **Consumed untethered temporaries:** a temporary with no source slot that must remain readable may materialize into an untethered destination with backpointer `0` and allocate no anchor. + +Anchor bookkeeping for every permitted operation is O(1) in the number of guests. Physical rehosting is proportional to the bytes or elements relocated and may update the canonical anchors of contained reference-type hosts, but it never enumerates guests. Promotion never creates a second live anchor path: it either preserves the sole live identity or is rejected. Source-scope and destination-scope guests therefore remain coherent after all later moves without repointing or forwarding. + +This is also how a moved-from symbol stays readable: after a permitted move the host-capable symbol enters guest state and stores the canonical tether, so reads resolve through the anchor to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). +""" + new_45 = """### 4.5 Moves and overwrites may merge anchor identities + +An overwrite from a newly materialized value and a move from another host are distinct cases. + +- **Ordinary overwrite:** if the destination hosting slot already has a payload anchor, the replacement payload inherits that backpointer and the cell is updated to the replacement's location. Existing destination guests therefore observe the new occupant. Destroying the old occupant does not return the cell, because the destination hosting lineage continues. +- **Move into a fresh or untethered destination:** if the source already has a payload anchor, that cell follows the value into the destination and remains terminal. If no anchor exists but the moved-from source slot must remain readable as a guest, the runtime allocates one for the value after relocation. The source host-capable slot stores a tether to the terminal anchor. +- **Move into an anchored destination:** the destination payload anchor remains terminal, because the destination host identity survives replacement. If the source has a different payload anchor, the runtime changes that source cell into a forwarding anchor targeting the destination cell. Existing source guests continue through the forwarding cell; existing destination guests continue directly through the destination cell. The moved payload stores the destination identity in its backpointer, and the moved-from source slot stores that same terminal tether. If resolving both identities already reaches the same terminal anchor, no new forwarding edge is installed. +- **Consumed untethered temporaries:** a temporary with no source slot that must remain readable may materialize into an untethered destination with backpointer `0` and allocate no anchor. + +The same rules apply recursively to reference-type hosts contained in a relocated representation. Their destination host identities survive replacement, and any distinct source identities forward to them. Anchor bookkeeping is O(1) for each merged host and never enumerates guests; physical rehosting remains proportional to the bytes, elements, and contained hosts relocated. + +This is also how a moved-from symbol stays readable: after a move the host-capable symbol enters guest state and stores the terminal tether, so reads resolve through the anchor path to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). +""" + replace_once("spec/memory.md", old_45, new_45) + + old_46 = """### 4.6 Hosting-lifetime end returns the anchor + +An anchor is returned to the global free-address stack when its **hosting lineage** ends. Overwriting only the current occupant does not end that lineage, because the host remains and existing guests follow the replacement. Rehosting transfers teardown responsibility to the destination host; the source slot is now a guest rather than a second host. + +At the actual end of the hosting lineage, lexical scope rules guarantee that every guest capable of naming the anchor has already ceased to exist ([`lifetimes.md`](lifetimes.md) §1, [`concurrency.md`](concurrency.md) §4). The runtime may therefore recycle the slot immediately. No generation counter, delayed reuse, or ABA protection is required: a stale guest is not a representable program state. +""" + new_46 = """### 4.6 Payload and forwarding anchors retire at different events + +A terminal payload anchor is returned to the global free-address stack when its **hosting lineage** ends. Overwriting only the current occupant does not end that lineage, because the destination host remains and existing destination guests follow the replacement. Rehosting transfers teardown responsibility to the destination host. + +A source anchor converted into a forwarder may still be named by guests created before the move, so it is not returned when the source stops hosting. Instead, the runtime associates it with the lexical scope of that former source host and returns it when that scope drains. Every guest that could already contain that obsolete identity is then dead by the ordinary scope rules. Copying or rebinding such a guest stores the terminal identity (§4.4), so the forwarding identity cannot newly escape its retirement scope. + +These two retirement rules require neither reference counting nor guest enumeration. When either kind of anchor is returned, no live guest can still name it; immediate reuse therefore needs no generation counter, delayed reuse, or ABA protection. +""" + replace_once("spec/memory.md", old_46, new_46) + + replace_once( + "spec/memory.md", + "A dangling or misdirected tether would require a guest to outlive its host, an anchor cell to move, or an anchor slot to be reused while an old guest remains. The model forbids all three. Scope checking proves the first impossible; the global pool gives each live hosting lineage one stable cell identity; and the same scope rule makes immediate slot reuse safe after teardown.", + "A dangling or misdirected tether would require a guest to outlive the hosted value, a forwarding chain to cycle, or an anchor slot to be reused while an old guest remains. The model forbids all three. Scope checking prevents the first; forwarding always points from a superseded source identity toward the destination's terminal identity, preventing cycles; and the separate payload-anchor and forwarding-anchor retirement rules make slot reuse safe." + ) + + replace_once( + "spec/memory.md", + "The segmented encoding adds no meaningful arithmetic cost: the shift and mask that split a `u32` fold into machine addressing once the chunk base is loaded. Tether resolution pays one dependent anchor-cell load beyond direct host access. Rehosting adds no forwarding hop and no guest-enumeration cost; its physical relocation cost remains proportional to the representation moved.", + "The segmented encoding adds no meaningful arithmetic cost: the shift and mask that split a `u32` fold into machine addressing once the chunk base is loaded. A terminal tether pays one dependent anchor-cell load beyond direct host access; an older identity pays one additional load per uncompressed forwarding hop. Rehosting never enumerates guests, and path compression makes repeated traversal of a chain amortized toward the terminal case. Physical relocation cost remains proportional to the representation moved." + ) + + replace_once( + "spec/memory.md", + "| Host-capable guest state | After rehosting, the old hosted bytes cease to be live and a slot declared as `T` stores the canonical tether as a guest while retaining enough storage to host another `T` later |", + "| Host-capable guest state | After rehosting, the old hosted bytes cease to be live and a slot declared as `T` stores the terminal tether as a guest while retaining enough storage to host another `T` later |" + ) + replace_once( + "spec/memory.md", + "| Anchor cell | One global-pool 8-byte physical slot per tethered hosting lineage; its 4-byte `u32` payload holds the current payload segmented offset |\n| Backpointer | Each hosted payload stores the stable `u32` identity of its anchor cell for move updates and tether minting; `0` means no cell has been allocated |\n| Anchor lifecycle | Lazily allocated on first guest; preserved across overwrite and rehosting; returned to the global free-address stack when the hosting lineage ends |\n| Anchor reuse safety | Immediate reuse is safe because lexical scope rules make a live stale guest unrepresentable |", + "| Anchor cell | One global-pool 8-byte physical slot containing a `u32` target and a payload/forwarding kind; a forwarding cell targets another anchor |\n| Backpointer | Each hosted payload stores the terminal payload-anchor identity for move updates and tether minting; `0` means no cell has been allocated |\n| Anchor merging | Moving into an anchored destination preserves the destination anchor and converts a distinct source anchor into a forwarder; no guest is enumerated |\n| Anchor lifecycle | A payload anchor returns when its hosting lineage ends; a forwarding anchor returns when its former source-host scope drains |\n| Anchor reuse safety | Guest canonicalization and lexical scope rules ensure no live tether names a returned slot |" + ) + + old_lifetime = """### 1.10 A move needs live guests on at most one side + +A move into an already-initialized host is rejected when **both** the source and the destination have live guests at that point. One hosting lineage keeps one anchor identity ([`memory.md`](memory.md) §4.5), and two live guest sets name two identities that the single canonical cell cannot carry forward. The compiler decides this from lexical guest liveness alone, the same way it decides guest assignment (§1.1). + +```zane +a Node() +b Node() +ra &Node = a +b = a // legal: only the source has a live guest; its anchor becomes canonical +ra:inspect() // ra reaches the value in its new home, b +``` + +```zane +c Node() +d Node() +rc &Node = c +rd &Node = d +d = c // ILLEGAL: both sides have live guests +rc:inspect() +rd:inspect() +``` + +Every permitted move stays O(1) in the number of guests, and the guests on the surviving side keep reaching the value in its new home (§1.6). + +""" + replace_once("spec/lifetimes.md", old_lifetime, "") + replace_once( + "spec/lifetimes.md", + "| Move guest liveness | A move into an initialized host is rejected when both the source and the destination have live guests |\n", + "" + ) + + replace_once( + "spec/glossary.md", + "- **Meaning:** A runtime cell whose `u32` payload holds the current segmented offset of one hosted object — the stable indirection point through which tethers resolve. It occupies one 8-byte slot in the runtime-global anchor pool, allocated when the first guest is created and returned to the pool's free-address stack when the hosting lineage ends. Its own segmented offset is the anchor identity that a whole hosting lineage keeps, across overwrite and rehosting.\n- **Why this name:** The cell is the fixed point that lets a moving object remain reachable: rehosting updates the cell while existing tethers keep pointing to it.", + "- **Meaning:** An 8-byte runtime cell in the global anchor pool containing a `u32` target and a kind. A payload anchor targets a hosted object's segmented offset; a forwarding anchor targets another anchor after two hosting identities merge. A guest's `u32` tether names an anchor cell and follows forwarding cells until it reaches the terminal payload anchor.\n- **Why this name:** The cell is a stable point through which an older guest identity can remain attached to a moving value, either directly or through another anchor." + ) + replace_once( + "spec/glossary.md", + "- **Meaning:** The internal representation of a guest: a `u32` segmented offset pointing at the host's anchor cell, not a raw pointer. The value `0` means no tether. A tether is a runtime mechanism, distinct from the source-facing `&T` guest (§3.33).", + "- **Meaning:** The internal representation of a guest: a `u32` segmented offset pointing at an anchor cell, not a raw pointer. The cell may directly target the hosted payload or forward to another anchor. The value `0` means no tether. A tether is a runtime mechanism, distinct from the source-facing `&T` guest (§3.33)." + ) + + old_story = """The scope-local anchor region turned out to have a deeper problem than teardown. Promotion could leave source guests naming the old cell while destination guests named a new one; a later move had only one payload backpointer and could update only one path. Moving the old cell would merely force every existing guest to be repointed. The final design therefore takes anchors out of scopes entirely. One runtime-global pool grows through anchor-only pages, and one anchor identity follows a hosting lineage through overwrites and rehosting. Promotion first relocates the complete hosted representation into destination-owned storage and then updates that same cell rather than forwarding, recreating, or moving it. + +That distinction matters for dynamic values. Rehosting copies the inline payload or handle into the destination host and relocates every owned backing store into an equal-size block or oversized span in the destination scope's dynamic region. The live contents move under their ordinary rules, including anchor updates for contained reference-type hosts; the old dynamic allocations are returned to the source scope's exact-size stacks. The old source payload bytes cease to be live, and its host-capable slot stores only the canonical tether as a guest. The anchor work remains O(1) in the number of guests, but the physical move is proportional to the bytes or elements relocated. + +Each anchor occupies an 8-byte-aligned physical slot: four bytes hold the payload's segmented offset and four are reserved so every slot is addressable by the shared 8-byte-word encoding. The pool uses one free-address stack because all slots have the same size; allocation pops it before bumping the global frontier. Anchor pages remain mapped until runtime shutdown, even when wholly free, so an offset retained by the stack can never name an unmapped or repurposed page. The minimum physical machinery for one tethered lineage is therefore sixteen bytes: a four-byte tether, an eight-byte anchor slot, and a four-byte payload backpointer. + +Individual teardown is the price of taking anchors out of scope arenas, but the host already provides the exact event needed. Overwriting an occupant preserves the lineage and its anchor; rehosting transfers teardown responsibility; only the end of the lineage returns the slot. Lexical lifetime rules prove that no guest can survive that event, so immediate slot reuse needs neither a generation counter nor delayed reclamation. A concurrent runtime may put thread-local caches in front of the same global pool without changing identity. + +One canonical cell also exposes the only move the model cannot represent. If source and destination both have live guest sets, each set already names a different stable identity; preserving both through later moves would require forwarding or guest enumeration. Such a move is rejected. When only one side has live guests, its anchor becomes canonical. When neither does but the moved-from host-capable slot becomes a guest, the runtime creates the one anchor that new guest needs. This also sharpens the storage distinction: an explicit `&T` is guest-only and holds only a tether, while a slot declared `T` keeps its full storage after rehosting and may later host another `T`. +""" + new_story = """The scope-local anchor region turned out to have a deeper problem than teardown. Promotion could leave source guests naming the old cell while destination guests named a new one; a later move had only one payload backpointer and could update only one path. Moving either cell would merely force every existing guest on that side to be repointed. The final design therefore takes anchors out of scopes entirely. One runtime-global pool grows through anchor-only pages, and anchor cells may target either a payload or another anchor. + +That extra target kind makes identity merging mechanical. A move into an already-anchored destination destroys its old occupant but preserves the destination host identity, so the destination anchor remains the terminal payload anchor. The source anchor changes into a forwarding cell that targets it. Old source guests walk source anchor to destination anchor to payload; destination guests and newly created guests go directly to the destination anchor. Nothing in the source language exposes the chain, and no guest is enumerated or rewritten. + +The same rule applies inside dynamic values. Rehosting copies the inline payload or handle into the destination host and relocates every owned backing store into an equal-size block or oversized span in the destination scope's dynamic region. Contained reference-type hosts merge their anchor identities in the same way. The old source payload bytes cease to be live, and its host-capable slot stores the terminal tether as a guest. Anchor work remains O(1) per merged host, while the physical move is proportional to the bytes, elements, and contained hosts relocated. + +Each anchor occupies an 8-byte-aligned physical slot: four bytes hold a target segmented offset and four identify whether that target is a payload or another anchor. The pool uses one free-address stack because all slots have the same size; allocation pops it before bumping the global frontier. Anchor pages remain mapped until runtime shutdown, even when wholly free, so an offset retained by the stack can never name an unmapped or repurposed page. The minimum physical machinery for one directly tethered lineage remains sixteen bytes: a four-byte tether, an eight-byte anchor slot, and a four-byte payload backpointer. + +The chain has a natural direction. A superseded source identity always forwards toward the destination identity, and moves only target the same or a higher scope, so a forwarding edge never points toward a shorter-lived host. Resolution may compress the path. Copying or rebinding an old guest stores the terminal identity rather than propagating the obsolete one, which means a forwarding anchor cannot newly escape the lexical scope that originally contained its source host. + +That gives the two cell kinds different retirement events. A terminal payload anchor returns when the final hosting lineage ends. A forwarding anchor remains only until its former source-host scope drains, when every old guest that could still contain that identity is dead. Neither case requires reference counting or guest enumeration, and both permit immediate free-stack reuse. The earlier restriction on moves with guests on both sides disappears entirely: the two identities simply become a chain. An explicit `&T` remains guest-only and holds only a tether, while a slot declared `T` keeps its full storage after rehosting and may later host another `T`. +""" + replace_once("stories/memory.md", old_story, new_story) + PY + + - name: Commit update and remove temporary workflow + shell: bash + run: | + rm .github/workflows/tmp-forwarding-anchor-update.yml + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add spec/memory.md spec/lifetimes.md spec/glossary.md stories/memory.md .github/workflows/tmp-forwarding-anchor-update.yml + git commit -m "Specify forwarding anchors for merged identities" + git push origin HEAD:agent/fix-host-overwrite-reuse From ff46f0fee338e77a83bf16d5428b6e76551ed8df Mon Sep 17 00:00:00 2001 From: Manuel Stieger <149385373+TheLazyCat00@users.noreply.github.com> Date: Wed, 29 Jul 2026 19:29:48 +0200 Subject: [PATCH 15/31] Specify forwarding anchors for merged host identities --- .../tmp-forwarding-anchor-update.yml | 225 ------------------ spec/glossary.md | 8 +- spec/lifetimes.md | 25 -- spec/memory.md | 70 +++--- stories/memory.md | 12 +- 5 files changed, 47 insertions(+), 293 deletions(-) delete mode 100644 .github/workflows/tmp-forwarding-anchor-update.yml diff --git a/.github/workflows/tmp-forwarding-anchor-update.yml b/.github/workflows/tmp-forwarding-anchor-update.yml deleted file mode 100644 index bb274c0..0000000 --- a/.github/workflows/tmp-forwarding-anchor-update.yml +++ /dev/null @@ -1,225 +0,0 @@ -name: Temporary forwarding-anchor update - -on: - push: - branches: - - agent/fix-host-overwrite-reuse - -permissions: - contents: write - -jobs: - update: - if: github.actor != 'github-actions[bot]' - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - with: - ref: agent/fix-host-overwrite-reuse - - - name: Update forwarding-anchor specification - shell: bash - run: | - python - <<'PY' - from pathlib import Path - - def replace_once(path: str, old: str, new: str) -> None: - file = Path(path) - text = file.read_text() - count = text.count(old) - if count != 1: - raise SystemExit(f"{path}: expected one match, found {count}: {old[:80]!r}") - file.write_text(text.replace(old, new)) - - replace_once( - "spec/memory.md", - "Anchors do not belong to any scope arena. The runtime owns one **global anchor pool**, implemented as a lazy chain of anchor-only 1 MiB pages. Every anchor occupies an **8-byte-aligned, 8-byte physical slot**: the first four bytes hold the `u32` payload offset and the remaining four bytes are reserved padding. An anchor page therefore contains 131072 addressable slots. The pool maps its first page only when the program creates its first guest and adds another page only when its current frontier and free-address stack cannot satisfy an allocation.", - "Anchors do not belong to any scope arena. The runtime owns one **global anchor pool**, implemented as a lazy chain of anchor-only 1 MiB pages. Every anchor occupies an **8-byte-aligned, 8-byte physical slot**: the first four bytes hold a `u32` target segmented offset and the remaining four bytes identify whether that target is a hosted payload or another anchor. An anchor page therefore contains 131072 addressable slots. The pool maps its first page only when the program creates its first guest and adds another page only when its current frontier and free-address stack cannot satisfy an allocation." - ) - - replace_once( - "spec/memory.md", - "When a reference-type instance is rehosted, all storage owned by that host is relocated into storage owned by the destination. Its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). For every dynamic backing store, the runtime allocates an equal-size block or oversized span in the destination scope's dynamic region, relocates the live contents into it according to their ordinary move rules, updates the copied handle, and then returns the old source block or span to its exact-size stack. Anchored reference-type values hosted inside a relocated backing store update their own canonical anchor cells as their host locations change. A promotion therefore completes before the source scope may drain and leaves no destination handle pointing into source-scope memory.", - "When a reference-type instance is rehosted, all storage owned by that host is relocated into storage owned by the destination. Its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). For every dynamic backing store, the runtime allocates an equal-size block or oversized span in the destination scope's dynamic region, relocates the live contents into it according to their ordinary move rules, updates the copied handle, and then returns the old source block or span to its exact-size stack. Anchored reference-type values hosted inside a relocated backing store update their terminal payload anchors or install forwarding anchors as their host locations and identities merge. A promotion therefore completes before the source scope may drain and leaves no destination handle pointing into source-scope memory." - ) - - replace_once( - "spec/memory.md", - "Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. Rehosting copies the complete hosted representation into destination-owned storage. The inline payload or handle is copied into the destination's fixed-size slot. Each dynamic backing store is relocated into an equal-size destination-region block or oversized span as specified in §3.5; after its live contents and any contained anchor locations have been updated, the old store is returned to the source scope's exact-size stack. The source payload bytes then cease to be live. Its host-capable slot is rewritten into guest state and stores the canonical tether; the rest of that full-size slot is dead until the slot is overwritten or its scope drains. If the moved value is tethered, the destination inherits the same global anchor identity and updates its one cell (§4.5), never the tethers themselves.", - "Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. Rehosting copies the complete hosted representation into destination-owned storage. The inline payload or handle is copied into the destination's fixed-size slot. Each dynamic backing store is relocated into an equal-size destination-region block or oversized span as specified in §3.5; after its live contents and any contained anchor locations have been updated, the old store is returned to the source scope's exact-size stack. The source payload bytes then cease to be live. Its host-capable slot is rewritten into guest state and stores a tether to the destination's terminal anchor; the rest of that full-size slot is dead until the slot is overwritten or its scope drains. Existing source guests continue through the source anchor, which either becomes the destination's payload anchor or forwards to the destination's existing anchor (§4.5); no guest is enumerated or rewritten." - ) - - replace_once( - "spec/memory.md", - "Tethers are tracked through **anchor cells** in one runtime-global pool. An anchor cell has a 4-byte logical `u32` payload holding the current segmented offset (§3.1) of a hosted reference-type value, but occupies one 8-byte-aligned physical slot so every cell identity is representable by the shared 8-byte-word offset encoding. The cell's own segmented offset is the stable anchor identity for the complete hosting lineage, even when the value is rehosted across scopes.", - "Tethers are tracked through **anchor cells** in one runtime-global pool. An anchor cell occupies one 8-byte-aligned physical slot so every cell identity is representable by the shared 8-byte-word offset encoding. Its first `u32` is a segmented target offset; its second `u32` identifies the target as either a hosted payload or another anchor. A payload anchor is the terminal identity stored in the hosted payload's backpointer. A forwarding anchor preserves an older guest identity after two hosting lineages merge, without changing any existing tether." - ) - - replace_once( - "spec/memory.md", - "Every reference-type payload reserves a `u32` backpointer field initialized to `0`. Once an anchor exists, that field stores the same stable anchor identity that its guests store. Guests and backpointers never store the payload address directly.", - "Every reference-type payload reserves a `u32` backpointer field initialized to `0`. Once an anchor exists, that field stores the terminal payload-anchor identity. Guests may store either that terminal identity or an older identity that forwards to it. Neither guests nor backpointers store the payload address directly." - ) - - replace_once( - "spec/memory.md", - "The minimum physical footprint attributable to one tethered hosting lineage is **16 bytes**: one 4-byte tether, one 8-byte physical anchor slot (containing a 4-byte cell payload), and one 4-byte payload backpointer. Each additional guest adds another 4-byte tether. Rehosting adds no forwarding metadata and, when only one side has live guests, no additional cell.", - "The minimum physical footprint attributable to one tethered hosting lineage is **16 bytes**: one 4-byte tether, one 8-byte physical anchor slot, and one 4-byte payload backpointer. Each additional guest adds another 4-byte tether. Merging two already-anchored hosting lineages reuses both existing cells: the destination cell remains the terminal payload anchor and the source cell becomes a forwarding anchor." - ) - - replace_once( - "spec/memory.md", - "Resolving a tether uses the chunk directory to locate the global anchor cell, reads the hosted payload's current segmented offset from that cell, resolves that offset through the same directory, then accesses the field. Because the pool never allocates cell identity `0`, resolving an untethered `0` traps rather than reading a live cell.", - "Resolving a tether uses the chunk directory to locate its global anchor cell. If the cell forwards, resolution repeats with the target anchor until it reaches a payload anchor; it then resolves that cell's payload offset through the same directory and accesses the field. Forwarding chains cannot cycle because a move only redirects a superseded source identity toward the destination's terminal identity. The runtime may path-compress visited forwarding cells. Because the pool never allocates cell identity `0`, resolving an untethered `0` traps rather than reading a live cell." - ) - - replace_once( - "spec/memory.md", - "Moves, overwrites, and promotions update only the current payload offset in that same cell. Guests created before and after a promotion therefore follow an identical path, with no forwarding cells and no promotion-dependent extra hop.\n\nThe added cost over direct host access is one dependent anchor-cell load. Across repeated accesses through the same guest with no intervening move or overwrite, the compiler may resolve the host address once and reuse it.", - "Ordinary overwrites and moves without an existing destination identity update one payload anchor. When two anchored hosting lineages merge, the destination anchor remains terminal and the source anchor forwards to it. Existing source guests therefore gain a forwarding hop, while destination guests and newly minted guests continue to use the terminal anchor directly. Copying or rebinding a guest through a forwarding tether resolves and stores the terminal identity, so an obsolete anchor identity cannot escape the lexical scope in which it was originally usable.\n\nThe added cost over direct host access is one dependent anchor-cell load for a terminal tether and one load per uncompressed forwarding hop for an older tether. Across repeated accesses through the same guest with no intervening move or overwrite, the compiler may resolve the host address once and reuse it; the runtime may also compress the chain." - ) - - old_45 = """### 4.5 Moves, overwrites, and rehosting keep one canonical anchor - -An overwrite from a newly materialized value and a move from another host are distinct cases. - -- **Ordinary overwrite:** if the destination hosting slot already has an anchor, the replacement payload inherits that backpointer and the cell is updated to the replacement's location. Existing destination guests therefore observe the new occupant. Destroying the old occupant does not return the cell, because the destination hosting lineage continues. -- **Move or rehosting with guests on only one side:** the anchor named by the live guest set becomes the destination's canonical anchor. If only the source has live guests, its anchor transfers to the destination. If only the destination has live guests, its anchor is preserved and the source's moved-from slot becomes another guest to it. Any noncanonical anchor left from an earlier, now-ended guest set is returned before the move completes. If neither side had live guests but the source slot remains readable in guest state, an anchor is allocated lazily for that new guest. The complete hosted representation is first relocated into destination-owned storage (§3.5, §3.7). The old source payload and backing-store bytes then cease to be live, the canonical cell is updated to the destination payload, the destination assumes teardown responsibility, and the source host-capable slot stores only its canonical tether in guest state. -- **Move or rehosting with live guests on both sides:** the program is ill-formed. The two guest sets name distinct stable identities, and a one-cell payload backpointer cannot preserve both through later moves without forwarding or guest enumeration. The compiler **MUST** reject the operation rather than recycle either referenced anchor. This restriction is determined from lexical guest liveness, not merely from whether a backpointer is nonzero. -- **Consumed untethered temporaries:** a temporary with no source slot that must remain readable may materialize into an untethered destination with backpointer `0` and allocate no anchor. - -Anchor bookkeeping for every permitted operation is O(1) in the number of guests. Physical rehosting is proportional to the bytes or elements relocated and may update the canonical anchors of contained reference-type hosts, but it never enumerates guests. Promotion never creates a second live anchor path: it either preserves the sole live identity or is rejected. Source-scope and destination-scope guests therefore remain coherent after all later moves without repointing or forwarding. - -This is also how a moved-from symbol stays readable: after a permitted move the host-capable symbol enters guest state and stores the canonical tether, so reads resolve through the anchor to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). -""" - new_45 = """### 4.5 Moves and overwrites may merge anchor identities - -An overwrite from a newly materialized value and a move from another host are distinct cases. - -- **Ordinary overwrite:** if the destination hosting slot already has a payload anchor, the replacement payload inherits that backpointer and the cell is updated to the replacement's location. Existing destination guests therefore observe the new occupant. Destroying the old occupant does not return the cell, because the destination hosting lineage continues. -- **Move into a fresh or untethered destination:** if the source already has a payload anchor, that cell follows the value into the destination and remains terminal. If no anchor exists but the moved-from source slot must remain readable as a guest, the runtime allocates one for the value after relocation. The source host-capable slot stores a tether to the terminal anchor. -- **Move into an anchored destination:** the destination payload anchor remains terminal, because the destination host identity survives replacement. If the source has a different payload anchor, the runtime changes that source cell into a forwarding anchor targeting the destination cell. Existing source guests continue through the forwarding cell; existing destination guests continue directly through the destination cell. The moved payload stores the destination identity in its backpointer, and the moved-from source slot stores that same terminal tether. If resolving both identities already reaches the same terminal anchor, no new forwarding edge is installed. -- **Consumed untethered temporaries:** a temporary with no source slot that must remain readable may materialize into an untethered destination with backpointer `0` and allocate no anchor. - -The same rules apply recursively to reference-type hosts contained in a relocated representation. Their destination host identities survive replacement, and any distinct source identities forward to them. Anchor bookkeeping is O(1) for each merged host and never enumerates guests; physical rehosting remains proportional to the bytes, elements, and contained hosts relocated. - -This is also how a moved-from symbol stays readable: after a move the host-capable symbol enters guest state and stores the terminal tether, so reads resolve through the anchor path to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). -""" - replace_once("spec/memory.md", old_45, new_45) - - old_46 = """### 4.6 Hosting-lifetime end returns the anchor - -An anchor is returned to the global free-address stack when its **hosting lineage** ends. Overwriting only the current occupant does not end that lineage, because the host remains and existing guests follow the replacement. Rehosting transfers teardown responsibility to the destination host; the source slot is now a guest rather than a second host. - -At the actual end of the hosting lineage, lexical scope rules guarantee that every guest capable of naming the anchor has already ceased to exist ([`lifetimes.md`](lifetimes.md) §1, [`concurrency.md`](concurrency.md) §4). The runtime may therefore recycle the slot immediately. No generation counter, delayed reuse, or ABA protection is required: a stale guest is not a representable program state. -""" - new_46 = """### 4.6 Payload and forwarding anchors retire at different events - -A terminal payload anchor is returned to the global free-address stack when its **hosting lineage** ends. Overwriting only the current occupant does not end that lineage, because the destination host remains and existing destination guests follow the replacement. Rehosting transfers teardown responsibility to the destination host. - -A source anchor converted into a forwarder may still be named by guests created before the move, so it is not returned when the source stops hosting. Instead, the runtime associates it with the lexical scope of that former source host and returns it when that scope drains. Every guest that could already contain that obsolete identity is then dead by the ordinary scope rules. Copying or rebinding such a guest stores the terminal identity (§4.4), so the forwarding identity cannot newly escape its retirement scope. - -These two retirement rules require neither reference counting nor guest enumeration. When either kind of anchor is returned, no live guest can still name it; immediate reuse therefore needs no generation counter, delayed reuse, or ABA protection. -""" - replace_once("spec/memory.md", old_46, new_46) - - replace_once( - "spec/memory.md", - "A dangling or misdirected tether would require a guest to outlive its host, an anchor cell to move, or an anchor slot to be reused while an old guest remains. The model forbids all three. Scope checking proves the first impossible; the global pool gives each live hosting lineage one stable cell identity; and the same scope rule makes immediate slot reuse safe after teardown.", - "A dangling or misdirected tether would require a guest to outlive the hosted value, a forwarding chain to cycle, or an anchor slot to be reused while an old guest remains. The model forbids all three. Scope checking prevents the first; forwarding always points from a superseded source identity toward the destination's terminal identity, preventing cycles; and the separate payload-anchor and forwarding-anchor retirement rules make slot reuse safe." - ) - - replace_once( - "spec/memory.md", - "The segmented encoding adds no meaningful arithmetic cost: the shift and mask that split a `u32` fold into machine addressing once the chunk base is loaded. Tether resolution pays one dependent anchor-cell load beyond direct host access. Rehosting adds no forwarding hop and no guest-enumeration cost; its physical relocation cost remains proportional to the representation moved.", - "The segmented encoding adds no meaningful arithmetic cost: the shift and mask that split a `u32` fold into machine addressing once the chunk base is loaded. A terminal tether pays one dependent anchor-cell load beyond direct host access; an older identity pays one additional load per uncompressed forwarding hop. Rehosting never enumerates guests, and path compression makes repeated traversal of a chain amortized toward the terminal case. Physical relocation cost remains proportional to the representation moved." - ) - - replace_once( - "spec/memory.md", - "| Host-capable guest state | After rehosting, the old hosted bytes cease to be live and a slot declared as `T` stores the canonical tether as a guest while retaining enough storage to host another `T` later |", - "| Host-capable guest state | After rehosting, the old hosted bytes cease to be live and a slot declared as `T` stores the terminal tether as a guest while retaining enough storage to host another `T` later |" - ) - replace_once( - "spec/memory.md", - "| Anchor cell | One global-pool 8-byte physical slot per tethered hosting lineage; its 4-byte `u32` payload holds the current payload segmented offset |\n| Backpointer | Each hosted payload stores the stable `u32` identity of its anchor cell for move updates and tether minting; `0` means no cell has been allocated |\n| Anchor lifecycle | Lazily allocated on first guest; preserved across overwrite and rehosting; returned to the global free-address stack when the hosting lineage ends |\n| Anchor reuse safety | Immediate reuse is safe because lexical scope rules make a live stale guest unrepresentable |", - "| Anchor cell | One global-pool 8-byte physical slot containing a `u32` target and a payload/forwarding kind; a forwarding cell targets another anchor |\n| Backpointer | Each hosted payload stores the terminal payload-anchor identity for move updates and tether minting; `0` means no cell has been allocated |\n| Anchor merging | Moving into an anchored destination preserves the destination anchor and converts a distinct source anchor into a forwarder; no guest is enumerated |\n| Anchor lifecycle | A payload anchor returns when its hosting lineage ends; a forwarding anchor returns when its former source-host scope drains |\n| Anchor reuse safety | Guest canonicalization and lexical scope rules ensure no live tether names a returned slot |" - ) - - old_lifetime = """### 1.10 A move needs live guests on at most one side - -A move into an already-initialized host is rejected when **both** the source and the destination have live guests at that point. One hosting lineage keeps one anchor identity ([`memory.md`](memory.md) §4.5), and two live guest sets name two identities that the single canonical cell cannot carry forward. The compiler decides this from lexical guest liveness alone, the same way it decides guest assignment (§1.1). - -```zane -a Node() -b Node() -ra &Node = a -b = a // legal: only the source has a live guest; its anchor becomes canonical -ra:inspect() // ra reaches the value in its new home, b -``` - -```zane -c Node() -d Node() -rc &Node = c -rd &Node = d -d = c // ILLEGAL: both sides have live guests -rc:inspect() -rd:inspect() -``` - -Every permitted move stays O(1) in the number of guests, and the guests on the surviving side keep reaching the value in its new home (§1.6). - -""" - replace_once("spec/lifetimes.md", old_lifetime, "") - replace_once( - "spec/lifetimes.md", - "| Move guest liveness | A move into an initialized host is rejected when both the source and the destination have live guests |\n", - "" - ) - - replace_once( - "spec/glossary.md", - "- **Meaning:** A runtime cell whose `u32` payload holds the current segmented offset of one hosted object — the stable indirection point through which tethers resolve. It occupies one 8-byte slot in the runtime-global anchor pool, allocated when the first guest is created and returned to the pool's free-address stack when the hosting lineage ends. Its own segmented offset is the anchor identity that a whole hosting lineage keeps, across overwrite and rehosting.\n- **Why this name:** The cell is the fixed point that lets a moving object remain reachable: rehosting updates the cell while existing tethers keep pointing to it.", - "- **Meaning:** An 8-byte runtime cell in the global anchor pool containing a `u32` target and a kind. A payload anchor targets a hosted object's segmented offset; a forwarding anchor targets another anchor after two hosting identities merge. A guest's `u32` tether names an anchor cell and follows forwarding cells until it reaches the terminal payload anchor.\n- **Why this name:** The cell is a stable point through which an older guest identity can remain attached to a moving value, either directly or through another anchor." - ) - replace_once( - "spec/glossary.md", - "- **Meaning:** The internal representation of a guest: a `u32` segmented offset pointing at the host's anchor cell, not a raw pointer. The value `0` means no tether. A tether is a runtime mechanism, distinct from the source-facing `&T` guest (§3.33).", - "- **Meaning:** The internal representation of a guest: a `u32` segmented offset pointing at an anchor cell, not a raw pointer. The cell may directly target the hosted payload or forward to another anchor. The value `0` means no tether. A tether is a runtime mechanism, distinct from the source-facing `&T` guest (§3.33)." - ) - - old_story = """The scope-local anchor region turned out to have a deeper problem than teardown. Promotion could leave source guests naming the old cell while destination guests named a new one; a later move had only one payload backpointer and could update only one path. Moving the old cell would merely force every existing guest to be repointed. The final design therefore takes anchors out of scopes entirely. One runtime-global pool grows through anchor-only pages, and one anchor identity follows a hosting lineage through overwrites and rehosting. Promotion first relocates the complete hosted representation into destination-owned storage and then updates that same cell rather than forwarding, recreating, or moving it. - -That distinction matters for dynamic values. Rehosting copies the inline payload or handle into the destination host and relocates every owned backing store into an equal-size block or oversized span in the destination scope's dynamic region. The live contents move under their ordinary rules, including anchor updates for contained reference-type hosts; the old dynamic allocations are returned to the source scope's exact-size stacks. The old source payload bytes cease to be live, and its host-capable slot stores only the canonical tether as a guest. The anchor work remains O(1) in the number of guests, but the physical move is proportional to the bytes or elements relocated. - -Each anchor occupies an 8-byte-aligned physical slot: four bytes hold the payload's segmented offset and four are reserved so every slot is addressable by the shared 8-byte-word encoding. The pool uses one free-address stack because all slots have the same size; allocation pops it before bumping the global frontier. Anchor pages remain mapped until runtime shutdown, even when wholly free, so an offset retained by the stack can never name an unmapped or repurposed page. The minimum physical machinery for one tethered lineage is therefore sixteen bytes: a four-byte tether, an eight-byte anchor slot, and a four-byte payload backpointer. - -Individual teardown is the price of taking anchors out of scope arenas, but the host already provides the exact event needed. Overwriting an occupant preserves the lineage and its anchor; rehosting transfers teardown responsibility; only the end of the lineage returns the slot. Lexical lifetime rules prove that no guest can survive that event, so immediate slot reuse needs neither a generation counter nor delayed reclamation. A concurrent runtime may put thread-local caches in front of the same global pool without changing identity. - -One canonical cell also exposes the only move the model cannot represent. If source and destination both have live guest sets, each set already names a different stable identity; preserving both through later moves would require forwarding or guest enumeration. Such a move is rejected. When only one side has live guests, its anchor becomes canonical. When neither does but the moved-from host-capable slot becomes a guest, the runtime creates the one anchor that new guest needs. This also sharpens the storage distinction: an explicit `&T` is guest-only and holds only a tether, while a slot declared `T` keeps its full storage after rehosting and may later host another `T`. -""" - new_story = """The scope-local anchor region turned out to have a deeper problem than teardown. Promotion could leave source guests naming the old cell while destination guests named a new one; a later move had only one payload backpointer and could update only one path. Moving either cell would merely force every existing guest on that side to be repointed. The final design therefore takes anchors out of scopes entirely. One runtime-global pool grows through anchor-only pages, and anchor cells may target either a payload or another anchor. - -That extra target kind makes identity merging mechanical. A move into an already-anchored destination destroys its old occupant but preserves the destination host identity, so the destination anchor remains the terminal payload anchor. The source anchor changes into a forwarding cell that targets it. Old source guests walk source anchor to destination anchor to payload; destination guests and newly created guests go directly to the destination anchor. Nothing in the source language exposes the chain, and no guest is enumerated or rewritten. - -The same rule applies inside dynamic values. Rehosting copies the inline payload or handle into the destination host and relocates every owned backing store into an equal-size block or oversized span in the destination scope's dynamic region. Contained reference-type hosts merge their anchor identities in the same way. The old source payload bytes cease to be live, and its host-capable slot stores the terminal tether as a guest. Anchor work remains O(1) per merged host, while the physical move is proportional to the bytes, elements, and contained hosts relocated. - -Each anchor occupies an 8-byte-aligned physical slot: four bytes hold a target segmented offset and four identify whether that target is a payload or another anchor. The pool uses one free-address stack because all slots have the same size; allocation pops it before bumping the global frontier. Anchor pages remain mapped until runtime shutdown, even when wholly free, so an offset retained by the stack can never name an unmapped or repurposed page. The minimum physical machinery for one directly tethered lineage remains sixteen bytes: a four-byte tether, an eight-byte anchor slot, and a four-byte payload backpointer. - -The chain has a natural direction. A superseded source identity always forwards toward the destination identity, and moves only target the same or a higher scope, so a forwarding edge never points toward a shorter-lived host. Resolution may compress the path. Copying or rebinding an old guest stores the terminal identity rather than propagating the obsolete one, which means a forwarding anchor cannot newly escape the lexical scope that originally contained its source host. - -That gives the two cell kinds different retirement events. A terminal payload anchor returns when the final hosting lineage ends. A forwarding anchor remains only until its former source-host scope drains, when every old guest that could still contain that identity is dead. Neither case requires reference counting or guest enumeration, and both permit immediate free-stack reuse. The earlier restriction on moves with guests on both sides disappears entirely: the two identities simply become a chain. An explicit `&T` remains guest-only and holds only a tether, while a slot declared `T` keeps its full storage after rehosting and may later host another `T`. -""" - replace_once("stories/memory.md", old_story, new_story) - PY - - - name: Commit update and remove temporary workflow - shell: bash - run: | - rm .github/workflows/tmp-forwarding-anchor-update.yml - git config user.name "github-actions[bot]" - git config user.email "41898282+github-actions[bot]@users.noreply.github.com" - git add spec/memory.md spec/lifetimes.md spec/glossary.md stories/memory.md .github/workflows/tmp-forwarding-anchor-update.yml - git commit -m "Specify forwarding anchors for merged identities" - git push origin HEAD:agent/fix-host-overwrite-reuse diff --git a/spec/glossary.md b/spec/glossary.md index 91a61fb..28b560a 100644 --- a/spec/glossary.md +++ b/spec/glossary.md @@ -175,17 +175,17 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`functions.md`](functions.md) §1 ### 3.23 anchor cell -- **Meaning:** A runtime cell whose `u32` payload holds the current segmented offset of one hosted object — the stable indirection point through which tethers resolve. It occupies one 8-byte slot in the runtime-global anchor pool, allocated when the first guest is created and returned to the pool's free-address stack when the hosting lineage ends. Its own segmented offset is the anchor identity that a whole hosting lineage keeps, across overwrite and rehosting. -- **Why this name:** The cell is the fixed point that lets a moving object remain reachable: rehosting updates the cell while existing tethers keep pointing to it. +- **Meaning:** An 8-byte runtime cell in the global anchor pool containing a `u32` target and a kind. A payload anchor targets a hosted object's segmented offset; a forwarding anchor targets another anchor after two hosting identities merge. A guest's `u32` tether names an anchor cell and follows forwarding cells until it reaches the terminal payload anchor. +- **Why this name:** The cell is a stable point through which an older guest identity can remain attached to a moving value, either directly or through another anchor. - **Canonical home:** [`memory.md`](memory.md) §4.1 ### 3.24 segmented-offset tether -- **Meaning:** The internal representation of a guest: a `u32` segmented offset pointing at the host's anchor cell, not a raw pointer. The value `0` means no tether. A tether is a runtime mechanism, distinct from the source-facing `&T` guest (§3.33). +- **Meaning:** The internal representation of a guest: a `u32` segmented offset pointing at an anchor cell, not a raw pointer. The cell may directly target the hosted payload or forward to another anchor. The value `0` means no tether. A tether is a runtime mechanism, distinct from the source-facing `&T` guest (§3.33). - **Why this name:** The tether connects a guest's stored representation to the anchor through which it reaches the hosted object. - **Canonical home:** [`memory.md`](memory.md) §4.2 ### 3.25 arena placement -- **Meaning:** A scope's arena has two regions: statically sized storage — value slots, reference-type hosts, and dynamic handles — is bump-allocated inline in the fixed-size region of the scope that creates it, while a resizable backing store goes in that scope's dynamic region. Rehosting copies the complete hosted representation into destination-owned storage: inline bytes move into the destination fixed-size region, each dynamic backing store is relocated into an equal-size destination-region allocation, and the old source storage ceases to be live. The source host-capable slot then stores the canonical tether as a guest. Placement is an unobservable implementation choice. +- **Meaning:** A scope's arena has two regions: statically sized storage — value slots, reference-type hosts, and dynamic handles — is bump-allocated inline in the fixed-size region of the scope that creates it, while a resizable backing store goes in that scope's dynamic region. Rehosting copies the complete hosted representation into destination-owned storage: inline bytes move into the destination fixed-size region, each dynamic backing store is relocated into an equal-size destination-region allocation, and the old source storage ceases to be live. The source host-capable slot then stores the terminal tether as a guest. Placement is an unobservable implementation choice. - **Why this name:** Placement is a choice among **arenas** — the per-scope regions — rather than between a stack and a heap; the creating scope's arena is the default, a parent arena the fallback on escape. - **Canonical home:** [`memory.md`](memory.md) §3.5 diff --git a/spec/lifetimes.md b/spec/lifetimes.md index 122a12c..4eea393 100644 --- a/spec/lifetimes.md +++ b/spec/lifetimes.md @@ -214,30 +214,6 @@ Because a floated result is kept rather than dropped, no guest dangles and no ho > **Story:** [`stories/lifetimes.md`](../stories/lifetimes.md#the-signature-is-the-whole-contract-retiring-inferred-consumption) — "The signature is the whole contract: retiring inferred consumption". -### 1.10 A move needs live guests on at most one side - -A move into an already-initialized host is rejected when **both** the source and the destination have live guests at that point. One hosting lineage keeps one anchor identity ([`memory.md`](memory.md) §4.5), and two live guest sets name two identities that the single canonical cell cannot carry forward. The compiler decides this from lexical guest liveness alone, the same way it decides guest assignment (§1.1). - -```zane -a Node() -b Node() -ra &Node = a -b = a // legal: only the source has a live guest; its anchor becomes canonical -ra:inspect() // ra reaches the value in its new home, b -``` - -```zane -c Node() -d Node() -rc &Node = c -rd &Node = d -d = c // ILLEGAL: both sides have live guests -rc:inspect() -rd:inspect() -``` - -Every permitted move stays O(1) in the number of guests, and the guests on the surviving side keep reaching the value in its new home (§1.6). - --- ## 2. Lifetime and Destruction @@ -278,7 +254,6 @@ Because scope rules (§1.1) prevent guests from outliving their hosts, the runti | Move-source | A direct host symbol (local or parameter) or a hosting verb result; not an `&`, field, container element, or other access path | | Move declaration-block restriction | A direct host symbol may only be moved in the exact lexical block where it was declared; parameters may be moved at the body top level | | Move destination scope | Destination host must be in the same or a higher lexical scope than the source host | -| Move guest liveness | A move into an initialized host is rejected when both the source and the destination have live guests | | Post-move downgrade | After a move, the source symbol downgrades to an `&` and remains readable but is no longer a move-source | | Parameter scope | A reference parameter belongs to the call-site scope, not the body, so a value passed by hosting access outlives the call | | Hosting argument | A verb takes a **guest** (`&T`, caller keeps it), **relays** the host (`T` and returns a hosting handle, caller may bind it to host again), or **consumes** it (`T`, no host returned, caller keeps a guest); passing to a plain `T` downgrades the caller to a guest whatever the body does | diff --git a/spec/memory.md b/spec/memory.md index a9eafc2..b1911da 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -16,9 +16,9 @@ Zane eliminates dangling guests by combining single hosting, lexical lifetime ru - **`Lexical lifetime enforcement`.** Guest assignment and rehosting are checked using declaration scope alone (see [`lifetimes.md`](lifetimes.md) §1). - **`Deterministic destruction`.** Objects are destroyed when their hosting scope drains; there is no tracing garbage collector (see [`lifetimes.md`](lifetimes.md) §2). - **`Regioned arena placement`.** Every scope owns separate fixed-size and dynamic-backing-store regions. Statically sized storage is placed inline in the fixed-size region; resizable data uses the dynamic region. Anchors live outside scope arenas in one runtime-global fixed-slot pool (see §3 and §4). -- **`Segmented-offset tethers`.** Internally, each guest is represented by a `u32` tether — a chunk id plus an in-chunk offset — that points at the host's anchor cell, not a raw pointer (see §4.2). +- **`Segmented-offset tethers`.** Internally, each guest is represented by a `u32` tether — a chunk id plus an in-chunk offset — that points at an anchor cell in the host's identity path, not a raw pointer (see §4.2). -The source language and runtime use separate terms: an object lives in a **host**, and a **guest** (`&T`) may access it without storing it or controlling its lifetime. Internally, each guest is represented by a **tether** that resolves through an **anchor**. Moving the object updates the anchor, so existing tethers — and therefore guests — continue to reach it. +The source language and runtime use separate terms: an object lives in a **host**, and a **guest** (`&T`) may access it without storing it or controlling its lifetime. Internally, each guest is represented by a **tether** that resolves through an **anchor**. Moving the object updates its terminal anchor or links an older anchor to the destination anchor, so existing tethers — and therefore guests — continue to reach it. These rules fit together mechanically. Hosts are the only storage that controls destruction. A guest may point only at an existing place, never a temporary. Lexical scope checks ensure the host outlives every guest derived from it. When an object is rehosted or a host is overwritten, guests stay valid. Internally, their tethers follow the host's anchor rather than a fixed object address. @@ -85,7 +85,7 @@ An `&` symbol or `&` field may be assigned a different target later, as long as ### 2.6 Guests are independent -Assigning or passing a guest gives the destination its own guest to the same host. Rebinding one guest's storage site later changes only that storage site; it does not retarget other guests that already point to that host. +Assigning or passing a guest gives the destination its own guest to the same host. The runtime may resolve a forwarding tether and store the terminal anchor identity in the new guest; this canonicalization is unobservable. Rebinding one guest's storage site later changes only that storage site; it does not retarget other guests that already point to that host. ### 2.7 Guests and hosts use the same surface operations @@ -250,9 +250,9 @@ Each lexical scope owns an **arena** made from two independent allocation region Each region is a separate chain of fixed-size **1 MiB chunks** mapped from the OS on demand. A chunk belongs to exactly one region: fixed-size slots and dynamic backing stores never coexist in the same chunk. A region maps no chunk until its first allocation. When its current chunk cannot satisfy an allocation, the runtime maps another chunk for that region, assigns it the next **chunk id**, and makes it current. -Scopes nest last-in-first-out, and their arenas nest with them: both regions of a scope are unmapped in full the moment the scope drains (§3.2, [`lifetimes.md`](lifetimes.md) §2.1). Arena granularity is an implementation choice, like boolean packing (§3.4) and placement (§3.5) — the compiler may fold several lexical scopes into one arena. What the language fixes is the observable behavior: a scope's memory is released together when that scope drains, and no guest ever resolves into released memory. A value that escapes is promoted out of the draining scope first (§3.5, §3.7), and its guests reach the promoted value through the canonical anchor (§4.5). +Scopes nest last-in-first-out, and their arenas nest with them: both regions of a scope are unmapped in full the moment the scope drains (§3.2, [`lifetimes.md`](lifetimes.md) §2.1). Arena granularity is an implementation choice, like boolean packing (§3.4) and placement (§3.5) — the compiler may fold several lexical scopes into one arena. What the language fixes is the observable behavior: a scope's memory is released together when that scope drains, and no guest ever resolves into released memory. A value that escapes is promoted out of the draining scope first (§3.5, §3.7), and its guests reach the promoted value through the terminal anchor path (§4.5). -Anchors do not belong to any scope arena. The runtime owns one **global anchor pool**, implemented as a lazy chain of anchor-only 1 MiB pages. Every anchor occupies an **8-byte-aligned, 8-byte physical slot**: the first four bytes hold the `u32` payload offset and the remaining four bytes are reserved padding. An anchor page therefore contains 131072 addressable slots. The pool maps its first page only when the program creates its first guest and adds another page only when its current frontier and free-address stack cannot satisfy an allocation. +Anchors do not belong to any scope arena. The runtime owns one **global anchor pool**, implemented as a lazy chain of anchor-only 1 MiB pages. Every anchor occupies an **8-byte-aligned, 8-byte physical slot**: the first four bytes hold a `u32` target segmented offset and the remaining four bytes identify whether that target is a hosted payload or another anchor. An anchor page therefore contains 131072 addressable slots. The pool maps its first page only when the program creates its first guest and adds another page only when its current frontier and free-address stack cannot satisfy an allocation. ```text one scope arena runtime-global anchor pool @@ -292,7 +292,7 @@ Returning a dynamic block pushes its base segmented offset onto the stack for th The global anchor pool has one LIFO **free-address stack**, because every anchor slot has the same size. Creating an anchor pops that stack first; only when it is empty does allocation bump the global anchor frontier, mapping another anchor page as needed. Returning an anchor pushes its segmented offset onto the same stack. Anchor pages remain mapped and retain their chunk-directory entries until runtime shutdown, including when every slot on a page is free; consequently every offset retained by the stack always resolves to its original anchor slot and anchor chunk ids are never repurposed during the run. -When a scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps its fixed-size and dynamic chunks in bulk, with no per-object teardown pass threaded through the exit. Logical destruction timing is independent of this: a value dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); it is the *memory* that is reclaimed together at drain. Global anchor pages are not tied to scope teardown: individual slots are returned when their hosting lineages end (§4.6), while the pages themselves remain mapped until runtime shutdown. +When a scope drains — after all its spawned work completes ([`concurrency.md`](concurrency.md) §4.1) — the runtime unmaps its fixed-size and dynamic chunks in bulk, with no per-object teardown pass threaded through the exit. Logical destruction timing is independent of this: a value dies when its host, container, or scope does ([`lifetimes.md`](lifetimes.md) §2.1); it is the *memory* that is reclaimed together at drain. Global anchor pages are not tied to scope teardown: terminal payload anchors are returned when their hosting lineages end, while forwarding anchors are returned from the former source scope's retirement stack when that scope drains (§4.6). The pages themselves remain mapped until runtime shutdown. > **Story:** [`stories/memory.md`](../stories/memory.md#when-the-free-stacks-fragment-and-the-arena-takes-the-scope) — "When the free stacks fragment, and the arena takes the scope". > **Story:** [`stories/memory.md`](../stories/memory.md#two-payload-streams-and-the-anchor-that-leaves-the-scope) — "Two payload streams, and the anchor that leaves the scope". @@ -311,7 +311,7 @@ The compiler may pack booleans in structs and arena frames when doing so does no Placement is an implementation decision, not a language-visible property. The arena model places every materialized, statically sized scope slot — value-type storage, a reference-type host, or a dynamic type's fixed-size handle — inline in that scope's fixed-size region. The compiler may keep an unobservable value in registers or otherwise optimize its physical placement, but reference types do not require a separate heap allocation merely because they carry identity. -When a reference-type instance is rehosted, all storage owned by that host is relocated into storage owned by the destination. Its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). For every dynamic backing store, the runtime allocates an equal-size block or oversized span in the destination scope's dynamic region, relocates the live contents into it according to their ordinary move rules, updates the copied handle, and then returns the old source block or span to its exact-size stack. Anchored reference-type values hosted inside a relocated backing store update their own canonical anchor cells as their host locations change. A promotion therefore completes before the source scope may drain and leaves no destination handle pointing into source-scope memory. +When a reference-type instance is rehosted, all storage owned by that host is relocated into storage owned by the destination. Its statically sized inline bytes are copied into the destination host's fixed-size slot (§3.7). For every dynamic backing store, the runtime allocates an equal-size block or oversized span in the destination scope's dynamic region, relocates the live contents into it according to their ordinary move rules, updates the copied handle, and then returns the old source block or span to its exact-size stack. Anchored reference-type hosts inside a relocated backing store apply the same identity-merging rule as the outer host: a destination identity remains terminal and a distinct source identity forwards to it (§4.5). A promotion therefore completes before the source scope may drain and leaves no destination handle pointing into source-scope memory. Placement never changes observable semantics: destruction stays deterministic (see [`lifetimes.md`](lifetimes.md) §2), and tethers resolve identically regardless of physical placement (§4), because a tether follows the host's anchor rather than a fixed address. @@ -354,7 +354,7 @@ A move transfers hosting into a destination host of the **same type** (see [`lif - Moving into a fresh declaration or a return slot is in-place initialization. - Moving into an already-initialized host first destroys the current occupant, then overwrites the same-size slot. -Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. Rehosting copies the complete hosted representation into destination-owned storage. The inline payload or handle is copied into the destination's fixed-size slot. Each dynamic backing store is relocated into an equal-size destination-region block or oversized span as specified in §3.5; after its live contents and any contained anchor locations have been updated, the old store is returned to the source scope's exact-size stack. The source payload bytes then cease to be live. Its host-capable slot is rewritten into guest state and stores the canonical tether; the rest of that full-size slot is dead until the slot is overwritten or its scope drains. If the moved value is tethered, the destination inherits the same global anchor identity and updates its one cell (§4.5), never the tethers themselves. +Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md) §1.4), so the destination always outlives the source and its slot already exists. Rehosting copies the complete hosted representation into destination-owned storage. The inline payload or handle is copied into the destination's fixed-size slot. Each dynamic backing store is relocated into an equal-size destination-region block or oversized span as specified in §3.5; after its live contents and any contained host identities have been updated, the old store is returned to the source scope's exact-size stack. The source payload bytes then cease to be live. Its host-capable slot is rewritten into guest state and stores a tether to the terminal anchor; the rest of that full-size slot is dead until the slot is overwritten or its scope drains. If both source and destination already have distinct anchor identities, the destination identity remains terminal and the source identity becomes a forwarding anchor (§4.5). Existing tethers are never enumerated or rewritten. --- @@ -362,7 +362,7 @@ Moves only ever target the same or a higher scope ([`lifetimes.md`](lifetimes.md ### 4.1 The global anchor pool -Tethers are tracked through **anchor cells** in one runtime-global pool. An anchor cell has a 4-byte logical `u32` payload holding the current segmented offset (§3.1) of a hosted reference-type value, but occupies one 8-byte-aligned physical slot so every cell identity is representable by the shared 8-byte-word offset encoding. The cell's own segmented offset is the stable anchor identity for the complete hosting lineage, even when the value is rehosted across scopes. +Tethers are tracked through **anchor cells** in one runtime-global pool. An anchor cell occupies one 8-byte-aligned physical slot so every cell identity is representable by the shared 8-byte-word offset encoding. Its first `u32` is a segmented target offset; its second `u32` identifies the target as either a hosted payload or another anchor. A **payload anchor** terminates at the currently hosted value. A **forwarding anchor** preserves an older guest identity after two hosting identities merge, without changing any existing tether. Anchor pages contain only equal-sized 8-byte slots. The pool therefore needs one free-address stack and one bump frontier rather than size classes. Pages are allocated lazily, never move, and remain mapped until runtime shutdown. @@ -373,11 +373,11 @@ Anchor pages contain only equal-sized 8-byte slots. The pool therefore needs one A tether is a **`u32` segmented offset** (§3.1) naming one global anchor cell — not a raw pointer and not a table index. At half the width of a 64-bit pointer, twice as many tethers fit in a cache line, and the 32-bit encoding keeps resolution on cheap 32-bit CPU math. -Every reference-type payload reserves a `u32` backpointer field initialized to `0`. Once an anchor exists, that field stores the same stable anchor identity that its guests store. Guests and backpointers never store the payload address directly. +Every reference-type payload reserves a `u32` backpointer field initialized to `0`. Once an anchor exists, that field stores the terminal payload-anchor identity. Guests may store either that terminal identity or an older identity that forwards to it. Neither guests nor backpointers store the payload address directly. An explicitly declared `&T` slot contains only this tether. A host-capable `T` slot that has been rehosted may use the same tether representation while it is in guest state, but retains enough storage to host another `T` later (§2.4). -The minimum physical footprint attributable to one tethered hosting lineage is **16 bytes**: one 4-byte tether, one 8-byte physical anchor slot (containing a 4-byte cell payload), and one 4-byte payload backpointer. Each additional guest adds another 4-byte tether. Rehosting adds no forwarding metadata and, when only one side has live guests, no additional cell. +The minimum physical footprint attributable to one directly tethered hosting lineage is **16 bytes**: one 4-byte tether, one 8-byte physical anchor slot, and one 4-byte payload backpointer. Each additional guest adds another 4-byte tether. Merging two already-anchored hosting identities allocates no new cell: the destination cell remains the payload anchor and the existing source cell becomes a forwarder until its former source scope drains. > **Story:** [`stories/memory.md`](../stories/memory.md#the-last-table-problem-and-the-segmented-offset) — "The last table problem, and the segmented offset". @@ -389,7 +389,7 @@ A hosting lineage that never gains a guest consumes no cell: its payload backpoi ### 4.4 Resolving a tether -Resolving a tether uses the chunk directory to locate the global anchor cell, reads the hosted payload's current segmented offset from that cell, resolves that offset through the same directory, then accesses the field. Because the pool never allocates cell identity `0`, resolving an untethered `0` traps rather than reading a live cell. +Resolving a tether uses the chunk directory to locate its global anchor cell. If the cell is a forwarder, resolution repeats with the target anchor until it reaches a payload anchor; it then resolves that cell's payload offset through the same directory and accesses the field. Forwarding chains cannot cycle because a move redirects a superseded source identity toward the same- or longer-lived destination identity. The runtime may path-compress visited forwarding cells. Because the pool never allocates cell identity `0`, resolving an untethered `0` traps rather than reading a live cell. Consider reading a field through a tether, where `mainWeapon` is an `&Weapon`: @@ -397,7 +397,7 @@ Consider reading a field through a tether, where `mainWeapon` is an `&Weapon`: dps Float = mainWeapon.dps ``` -The walk is always tether → global anchor cell → payload offset → payload address → field: +The terminal case is tether → global anchor cell → payload offset → payload address → field. An older tether may first cross one or more forwarding anchor cells: ```text mainWeapon: &Weapon @@ -427,41 +427,43 @@ Weapon payload Weapon.dps ``` -Moves, overwrites, and promotions update only the current payload offset in that same cell. Guests created before and after a promotion therefore follow an identical path, with no forwarding cells and no promotion-dependent extra hop. +Ordinary overwrites and moves into untethered destinations update one payload anchor. When two anchored hosting identities merge, the destination anchor remains terminal and the source anchor forwards to it. Existing source guests therefore gain a forwarding hop, while destination guests and newly minted guests continue to use the terminal anchor directly. Assigning or passing a guest resolves its tether and stores the terminal identity in the new guest, so an obsolete identity cannot newly escape its former source-host scope. -The added cost over direct host access is one dependent anchor-cell load. Across repeated accesses through the same guest with no intervening move or overwrite, the compiler may resolve the host address once and reuse it. +The added cost over direct host access is one dependent anchor-cell load for a terminal tether and one load per uncompressed forwarding hop for an older tether. Across repeated accesses through the same guest with no intervening move or overwrite, the compiler may resolve the host address once and reuse it; the runtime may also compress the anchor path. -### 4.5 Moves, overwrites, and rehosting keep one canonical anchor +### 4.5 Moves and overwrites may merge anchor identities An overwrite from a newly materialized value and a move from another host are distinct cases. -- **Ordinary overwrite:** if the destination hosting slot already has an anchor, the replacement payload inherits that backpointer and the cell is updated to the replacement's location. Existing destination guests therefore observe the new occupant. Destroying the old occupant does not return the cell, because the destination hosting lineage continues. -- **Move or rehosting with guests on only one side:** the anchor named by the live guest set becomes the destination's canonical anchor. If only the source has live guests, its anchor transfers to the destination. If only the destination has live guests, its anchor is preserved and the source's moved-from slot becomes another guest to it. Any noncanonical anchor left from an earlier, now-ended guest set is returned before the move completes. If neither side had live guests but the source slot remains readable in guest state, an anchor is allocated lazily for that new guest. The complete hosted representation is first relocated into destination-owned storage (§3.5, §3.7). The old source payload and backing-store bytes then cease to be live, the canonical cell is updated to the destination payload, the destination assumes teardown responsibility, and the source host-capable slot stores only its canonical tether in guest state. -- **Move or rehosting with live guests on both sides:** the program is ill-formed. The two guest sets name distinct stable identities, and a one-cell payload backpointer cannot preserve both through later moves without forwarding or guest enumeration. The compiler **MUST** reject the operation rather than recycle either referenced anchor. This restriction is determined from lexical guest liveness, not merely from whether a backpointer is nonzero. +- **Ordinary overwrite:** if the destination hosting slot already has a payload anchor, the replacement payload inherits that backpointer and the cell is updated to the replacement's location. Existing destination guests therefore observe the new occupant. Destroying the old occupant does not return the cell, because the destination hosting identity continues. +- **Move into a fresh or untethered destination:** if the source already has a payload anchor, that cell follows the value into the destination and remains terminal. If no anchor exists but the moved-from source slot must remain readable as a guest, the runtime allocates one for the value after relocation. The source host-capable slot stores a tether to the terminal anchor. +- **Move into an anchored destination:** the destination payload anchor remains terminal, because the destination host identity survives replacement. If the source has a different payload anchor, the runtime changes that source cell into a forwarding anchor targeting the destination cell and records the forwarder on the former source scope's retirement stack. Existing source guests continue through the forwarding cell; existing destination guests continue directly through the destination cell. The moved payload stores the destination identity in its backpointer, and the moved-from source slot stores that same terminal tether. If resolving both identities already reaches the same terminal anchor, no new forwarding edge is installed. - **Consumed untethered temporaries:** a temporary with no source slot that must remain readable may materialize into an untethered destination with backpointer `0` and allocate no anchor. -Anchor bookkeeping for every permitted operation is O(1) in the number of guests. Physical rehosting is proportional to the bytes or elements relocated and may update the canonical anchors of contained reference-type hosts, but it never enumerates guests. Promotion never creates a second live anchor path: it either preserves the sole live identity or is rejected. Source-scope and destination-scope guests therefore remain coherent after all later moves without repointing or forwarding. +The same rules apply recursively to reference-type hosts contained in a relocated representation, including hosts inside dynamic backing stores. Their destination host identities survive replacement, and any distinct source identities forward to them. Anchor bookkeeping is O(1) for each merged host and never enumerates guests; physical rehosting remains proportional to the bytes, elements, and contained hosts relocated. -This is also how a moved-from symbol stays readable: after a permitted move the host-capable symbol enters guest state and stores the canonical tether, so reads resolve through the anchor to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). +This is also how a moved-from symbol stays readable: after a move the host-capable symbol enters guest state and stores the terminal tether, so reads resolve through the anchor path to the value's new home (see [`lifetimes.md`](lifetimes.md) §1.6). > **Story:** [`stories/memory.md`](../stories/memory.md#the-move-problem-and-the-anchor-that-never-moves) — "The move problem, and the anchor that never moves". > **Story:** [`stories/memory.md`](../stories/memory.md#two-payload-streams-and-the-anchor-that-leaves-the-scope) — "Two payload streams, and the anchor that leaves the scope". -### 4.6 Hosting-lifetime end returns the anchor +### 4.6 Payload and forwarding anchors retire at different events -An anchor is returned to the global free-address stack when its **hosting lineage** ends. Overwriting only the current occupant does not end that lineage, because the host remains and existing guests follow the replacement. Rehosting transfers teardown responsibility to the destination host; the source slot is now a guest rather than a second host. +A terminal payload anchor is returned to the global free-address stack when its **hosting identity** ends. Overwriting only the current occupant does not end that identity, because the destination host remains and existing destination guests follow the replacement. Rehosting transfers teardown responsibility to the destination host. -At the actual end of the hosting lineage, lexical scope rules guarantee that every guest capable of naming the anchor has already ceased to exist ([`lifetimes.md`](lifetimes.md) §1, [`concurrency.md`](concurrency.md) §4). The runtime may therefore recycle the slot immediately. No generation counter, delayed reuse, or ABA protection is required: a stale guest is not a representable program state. +A source anchor converted into a forwarder may still be named by guests created before the move, so it is not returned when the source stops hosting. Instead, the runtime pushes it onto a retirement stack owned by the lexical scope of that former source host and returns it when that scope drains. Every guest that could already contain that obsolete identity is then dead by the ordinary scope rules. Assigning or passing such a guest stores the terminal identity (§2.6, §4.4), so the forwarding identity cannot newly escape its retirement scope. + +Forwarding edges always point from a former source identity toward a destination identity in the same or a higher lexical scope. A forwarder therefore never depends on an anchor retired before it; at a shared scope drain all identities from that scope may be returned together. These retirement rules require neither reference counting nor guest enumeration. When either kind of anchor is returned, no live guest can still name it, so immediate reuse needs no generation counter, delayed reuse, or ABA protection. > **Story:** [`stories/memory.md`](../stories/memory.md#two-payload-streams-and-the-anchor-that-leaves-the-scope) — "Two payload streams, and the anchor that leaves the scope". ### 4.7 Why tethers never dangle or misdirect -A dangling or misdirected tether would require a guest to outlive its host, an anchor cell to move, or an anchor slot to be reused while an old guest remains. The model forbids all three. Scope checking proves the first impossible; the global pool gives each live hosting lineage one stable cell identity; and the same scope rule makes immediate slot reuse safe after teardown. +A dangling or misdirected tether would require a guest to outlive the hosted value, a forwarding chain to cycle, or an anchor slot to be reused while an old guest remains. The model forbids all three. Scope checking prevents the first; forwarding always points from a superseded source identity toward a same- or longer-lived destination identity, preventing cycles; and the separate payload-anchor and forwarding-anchor retirement rules make slot reuse safe. ### 4.8 Resolution and allocation cost -The segmented encoding adds no meaningful arithmetic cost: the shift and mask that split a `u32` fold into machine addressing once the chunk base is loaded. Tether resolution pays one dependent anchor-cell load beyond direct host access. Rehosting adds no forwarding hop and no guest-enumeration cost; its physical relocation cost remains proportional to the representation moved. +The segmented encoding adds no meaningful arithmetic cost: the shift and mask that split a `u32` fold into machine addressing once the chunk base is loaded. A terminal tether pays one dependent anchor-cell load beyond direct host access; an older identity pays one additional load per uncompressed forwarding hop. Rehosting never enumerates guests, and path compression makes repeated traversal of a chain approach the terminal case. Physical relocation cost remains proportional to the representation moved. A single global free stack and frontier require synchronization under concurrent allocation and teardown. Implementations may use thread-local anchor caches backed by the same global pool without changing anchor identity, reuse order semantics, or lifetime guarantees. @@ -497,7 +499,7 @@ A single global free stack and frontier require synchronization under concurrent | Hosting storage | Reference-typed symbols, fields, and container elements are directly initialized and may later be overwritten | | Value type | Mutable in place through a borrowed `mut` receiver; storage may also be overwritten freely | | `&` (guest) | Guest-only non-hosting storage; stores one tether, may be repointed, copied by value, and returned, but can never directly host a `T` | -| Host-capable guest state | After rehosting, the old hosted bytes cease to be live and a slot declared as `T` stores the canonical tether as a guest while retaining enough storage to host another `T` later | +| Host-capable guest state | After rehosting, the old hosted bytes cease to be live and a slot declared as `T` stores the terminal tether as a guest while retaining enough storage to host another `T` later | | Place expression | Existing stable storage: a named symbol, a field access of a place, a place-projection subscript of a place, or an `&` parameter | | New `&` value | May be initialized only from a named symbol, a field access of a place, or an `&` parameter; temporaries and `[]` expressions are rejected | | `&` parameter | Declares that the caller must supply an `&`-creating source; the parameter is place-like inside the callee | @@ -509,16 +511,16 @@ A single global free stack and frontier require synchronization under concurrent | `&` targets reference types | An `&T` requires `T` to be a reference type; a value is shared by copy or scoped borrow, never by a stored `&` | | Symbol declaration | Must be directly initialized | | Reference-type placement | Inline storage is bump-allocated in the creating scope's fixed-size region; rehosting copies inline bytes and every owned dynamic backing store into destination-owned regions before source storage is retired | -| `&` representation | A guest is represented internally by a `u32` tether: a segmented offset (chunk id + in-chunk offset) to the host's global anchor cell; `0` means no tether | +| `&` representation | A guest is represented internally by a `u32` tether: a segmented offset to a global anchor cell, which may terminate at a payload or forward to another anchor; `0` means no tether | | Addressing | Scope chunks and global anchor pages share one `u32` segmented-offset directory; 8-byte-aligned offsets reach 32 GiB across up to 32768 1 MiB chunks | | Untethered sentinel | `0`; the global anchor pool reserves this identity, while payloads may still occupy segmented offset `0` | | Dynamic allocation | Power-of-two byte classes beginning at 128 bytes; exact-size stack first, frontier second; blocks above 1 MiB use dedicated contiguous oversized spans | | Backing-store alignment | Dynamically-sized backing stores (§3.6) are cache-line-aligned; small inline allocations stay 8-byte aligned | -| Anchor cell | One global-pool 8-byte physical slot per tethered hosting lineage; its 4-byte `u32` payload holds the current payload segmented offset | -| Backpointer | Each hosted payload stores the stable `u32` identity of its anchor cell for move updates and tether minting; `0` means no cell has been allocated | -| Anchor lifecycle | Lazily allocated on first guest; preserved across overwrite and rehosting; returned to the global free-address stack when the hosting lineage ends | -| Anchor reuse safety | Immediate reuse is safe because lexical scope rules make a live stale guest unrepresentable | -| Move guest liveness | A move is rejected when both sides have live guests; with live guests on exactly one side, that side's anchor becomes the canonical one; with none on either side, a moved-from slot that stays readable anchors lazily (see [`lifetimes.md`](lifetimes.md) §1.10) | -| Tethered-instance cost | Minimum 16-byte physical footprint: one 4-byte tether, one 8-byte anchor slot, and one 4-byte backpointer | +| Anchor cell | One global-pool 8-byte physical slot containing a `u32` target and a payload/forwarding kind; a forwarding cell targets another anchor | +| Backpointer | Each hosted payload stores the terminal payload-anchor identity for move updates and tether minting; `0` means no cell has been allocated | +| Anchor merging | Moving into an anchored destination preserves the destination anchor and converts a distinct source anchor into a forwarder; no guest is enumerated | +| Anchor lifecycle | A payload anchor returns when its hosting identity ends; a forwarding anchor returns when its former source-host scope drains | +| Anchor reuse safety | Guest canonicalization and lexical scope rules ensure no live tether names a returned slot | +| Tethered-instance cost | Minimum 16-byte direct footprint: one 4-byte tether, one 8-byte anchor slot, and one 4-byte backpointer; each retained historical identity uses one existing 8-byte forwarding slot until its retirement scope drains | > **See also:** [`lifetimes.md`](lifetimes.md) §4 for the summary of scope, move, and destruction rules. diff --git a/stories/memory.md b/stories/memory.md index c72ddae..7d91da8 100644 --- a/stories/memory.md +++ b/stories/memory.md @@ -123,15 +123,17 @@ The scope arena survived, but the pure-bump conclusion did not survive unchanged The dynamic region brings back free stacks in the one place where their fragmentation is controlled rather than global. Blocks use shared power-of-two byte classes beginning at 128 bytes, independent of element type. Allocation checks the exact-size LIFO stack first and bumps the frontier only when that stack is empty. A full list requests exactly twice its current byte size; it grows in place only when it is the frontier allocation and the added bytes fit before the chunk boundary. Otherwise its elements relocate into a reusable doubled block or a newly bumped one, and the old block enters its exact-size stack. Blocks above 1 MiB become dedicated contiguous oversized spans, addressed by one base segmented offset and reused through the same exact-size rule. The cost is dead space between size classes and until scope teardown, but reuse is confined to the buffers whose repeated growth creates it. -The scope-local anchor region turned out to have a deeper problem than teardown. Promotion could leave source guests naming the old cell while destination guests named a new one; a later move had only one payload backpointer and could update only one path. Moving the old cell would merely force every existing guest to be repointed. The final design therefore takes anchors out of scopes entirely. One runtime-global pool grows through anchor-only pages, and one anchor identity follows a hosting lineage through overwrites and rehosting. Promotion first relocates the complete hosted representation into destination-owned storage and then updates that same cell rather than forwarding, recreating, or moving it. +The scope-local anchor region turned out to have a deeper problem than teardown. Promotion could leave source guests naming the old cell while destination guests named a new one; a later move had only one payload backpointer and could update only one path. Moving either cell would merely force every existing guest on that side to be repointed. The final design therefore takes anchors out of scopes entirely. One runtime-global pool grows through anchor-only pages, and an anchor cell may target either a payload or another anchor. -That distinction matters for dynamic values. Rehosting copies the inline payload or handle into the destination host and relocates every owned backing store into an equal-size block or oversized span in the destination scope's dynamic region. The live contents move under their ordinary rules, including anchor updates for contained reference-type hosts; the old dynamic allocations are returned to the source scope's exact-size stacks. The old source payload bytes cease to be live, and its host-capable slot stores only the canonical tether as a guest. The anchor work remains O(1) in the number of guests, but the physical move is proportional to the bytes or elements relocated. +That second target kind makes identity merging mechanical. A move into an already-anchored destination destroys its old occupant but preserves the destination host identity, so the destination anchor remains the terminal payload anchor. A distinct source anchor changes into a forwarding cell that targets it. Old source guests walk source anchor to destination anchor to payload; destination guests and newly created guests go directly to the destination anchor. Nothing in the source language exposes the chain, and no guest is enumerated or rewritten. -Each anchor occupies an 8-byte-aligned physical slot: four bytes hold the payload's segmented offset and four are reserved so every slot is addressable by the shared 8-byte-word encoding. The pool uses one free-address stack because all slots have the same size; allocation pops it before bumping the global frontier. Anchor pages remain mapped until runtime shutdown, even when wholly free, so an offset retained by the stack can never name an unmapped or repurposed page. The minimum physical machinery for one tethered lineage is therefore sixteen bytes: a four-byte tether, an eight-byte anchor slot, and a four-byte payload backpointer. +The same rule applies inside dynamic values. Rehosting copies the inline payload or handle into the destination host and relocates every owned backing store into an equal-size block or oversized span in the destination scope's dynamic region. Contained reference-type hosts merge their anchor identities in the same way. The old source payload bytes cease to be live, and its host-capable slot stores the terminal tether as a guest. Anchor work remains O(1) per merged host, while the physical move is proportional to the bytes, elements, and contained hosts relocated. -Individual teardown is the price of taking anchors out of scope arenas, but the host already provides the exact event needed. Overwriting an occupant preserves the lineage and its anchor; rehosting transfers teardown responsibility; only the end of the lineage returns the slot. Lexical lifetime rules prove that no guest can survive that event, so immediate slot reuse needs neither a generation counter nor delayed reclamation. A concurrent runtime may put thread-local caches in front of the same global pool without changing identity. +Each anchor occupies an 8-byte-aligned physical slot: four bytes hold a target segmented offset and four identify whether that target is a payload or another anchor. The pool uses one free-address stack because all slots have the same size; allocation pops it before bumping the global frontier. Anchor pages remain mapped until runtime shutdown, even when wholly free, so an offset retained by the stack can never name an unmapped or repurposed page. The minimum physical machinery for one directly tethered identity remains sixteen bytes: a four-byte tether, an eight-byte anchor slot, and a four-byte payload backpointer. -One canonical cell also exposes the only move the model cannot represent. If source and destination both have live guest sets, each set already names a different stable identity; preserving both through later moves would require forwarding or guest enumeration. Such a move is rejected. When only one side has live guests, its anchor becomes canonical. When neither does but the moved-from host-capable slot becomes a guest, the runtime creates the one anchor that new guest needs. This also sharpens the storage distinction: an explicit `&T` is guest-only and holds only a tether, while a slot declared `T` keeps its full storage after rehosting and may later host another `T`. +The chain has a natural direction. A superseded source identity always forwards toward the destination identity, and moves only target the same or a higher scope, so a forwarding edge never points toward a shorter-lived host. Resolution may compress the path. Copying or rebinding an old guest stores the terminal identity rather than propagating the obsolete one, which means a forwarding anchor cannot newly escape the lexical scope that originally contained its source host. + +That gives the two cell kinds different retirement events. A terminal payload anchor returns when its final hosting identity ends. A forwarding anchor is pushed onto its former source scope's retirement stack and remains only until that scope drains, when every old guest that could still contain the identity is dead. Neither case requires reference counting or guest enumeration, and both permit immediate free-stack reuse. The earlier restriction on moves with guests on both sides disappears entirely: the two identities simply become a runtime chain. An explicit `&T` remains guest-only and holds only a tether, while a slot declared `T` keeps its full storage after rehosting and may later host another `T`. The sentinel changes by one small accounting detail. Segmented offset zero remains a valid payload location, but the global pool never issues anchor identity zero. The sentinel therefore costs one unusable anchor-slot identity rather than forcing either payload region away from its naturally aligned chunk base. Dynamic blocks begin at 128 bytes and preserve cache-line alignment through doubling, reuse, and oversized spans. From b10eaedcf4725c21b5ea52347f87364f868ace8f Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 21:14:39 +0000 Subject: [PATCH 16/31] docs: bare symbols are no longer guest sources; add the 'T borrow mode MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A new `&` may now be minted only from a field access or an `&T` parameter. A bare symbol stays a place expression but is no longer a guest source, so nothing can point at a local's own hosting slot and the slot stays free to be overwritten or moved from. This is what makes the aliasing case that started this — a guest to `main`, then `second = main` — a compile error at the line that mints the guest rather than a question about what the guest then denotes. Removing that source leaves a hole where a bare symbol needs to reach a callee, so reference types gain a third passing mode: `'T`, a borrow. The three modes are now `T` (swallow), `&T` (guest), and `'T` (borrow, any place including a bare symbol, read and mut for the call only, never stored, returned, or moved). A reference receiver written bare is an implicit `'T` borrow; `this &T` is the guest receiver a method writes when it stores or returns the receiver. Consequences carried through: a returned `&T` must be rooted in an `&T` parameter; a field reached through a borrow is not a guest source either; overloads may not differ only by passing mode; an `&` field and a recursive `#variant` case are fed from a field or an `&` parameter, so a recursive structure is rooted in a field rather than a bare local. The anchor system is unchanged. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- CLAUDE.md | 24 +++++++- spec/adt.md | 26 ++++++++- spec/concurrency.md | 2 +- spec/effects.md | 2 +- spec/foundations.md | 3 +- spec/functions.md | 66 ++++++++++++++-------- spec/glossary.md | 18 ++++-- spec/lexical.md | 4 +- spec/lifetimes.md | 51 +++++++++++------ spec/memory.md | 135 ++++++++++++++++++++++++++++++-------------- spec/syntax.md | 37 ++++++++++-- spec/types.md | 12 ++-- 12 files changed, 278 insertions(+), 102 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 239091d..80c34b2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -43,13 +43,33 @@ The generics system was unified into a `<>`-header / `()`-call model (canonical home `spec/generics.md`, casing rules `spec/lexical.md`). Several pre-redesign forms are now illegal and must never reappear. Grep for them — none should hit: +```sh +grep -RIn -E "Array\[|\[size\]|Array[0-9]+|Matrix10|\[rows\]|\[cols\]|inferred type generic|type-parameter symbol|root form" spec/ ``` -grep -nE "Array\[|\[size\]|Array[0-9]+|Matrix10|\[rows\]|\[cols\]|'[A-Z]|inferred type generic|type-parameter symbol|root form" spec/*.md -``` + +`'[A-Z]` used to be on that list — it is **not** any more. A leading `'` is now +the **borrow** type marker (`'Node`), canonical home `spec/memory.md` §2.9, +surface form `spec/syntax.md` §2.3. Do not re-add it to the retired-forms grep. The only legitimate stray `<...>` is `Result` in `spec/error-handling.md` — Rust's type named as a comparison, not Zane's. +A second guard covers the memory model. A **bare symbol is not a guest source** +(`spec/memory.md` §2.8.1), so a spec example that mints an `&` from one is a +bug. Eyeball every hit of: + +```sh +grep -RIn -E "&[A-Z][A-Za-z0-9]* *= *[a-z][A-Za-z0-9]*$" spec/ +``` + +Every surviving hit must be a field access (`= car.engine`) or an `&T` +parameter — never a bare local. Run these with `-R` on the directory, not a +`spec/*.md` glob plus a bare directory argument: `grep` prints +`bench/: Is a directory` and silently skips it otherwise. + +Stories are exempt from both greps: `stories/` records the language as it was +at each turn and is never rewritten to match the present spec. + If the grep hits an old form, stop and rewrite it in the unified system. If a cross-reference target moved (renumbered `§`), fix the reference in every doc that uses it, then re-grep for the old numbers. If the change conflicts with diff --git a/spec/adt.md b/spec/adt.md index a38b401..07e0863 100644 --- a/spec/adt.md +++ b/spec/adt.md @@ -115,7 +115,7 @@ Naming a case takes its payload whole; to reach a nested case, write another cas An `enum` member is the payloadless degenerate of the same form: `Colors.red` selects a case that carries no payload, so it is written with no argument list (§2). A payload-carrying case is called; a payloadless one is selected. -A recursive `#variant` case boxes through `&` (§4), and its construction follows the ordinary reference rules: `Expr.flip(r)` takes an `&Expr`, and its argument must be a source that may create a new `&` (see [`memory.md`](memory.md) §2.8), exactly as an `&` field of a `#struct` requires (see [`types.md`](types.md) §3.9). +A recursive `#variant` case boxes through `&` (§4), and its construction follows the ordinary reference rules: `Expr.flip(r)` takes an `&Expr`, and its argument must be a guest source (see [`memory.md`](memory.md) §2.8), exactly as an `&` field of a `#struct` requires (see [`types.md`](types.md) §3.9). A bare symbol is not one, so the child a recursive case points at is reached through a field or through an `&Expr` parameter (§4.1). **Shared surface, different mechanism.** The `Type.member(args)` form — in both its long (`e Expr = Expr.intLit("5")`) and short (`e Expr.intLit("5")`) declaration — is exactly the surface a **named constructor** on a product type uses (see [`types.md`](types.md) §3.4): `v Vector2.diagonal(Float(3))` reads and declares just like `e Expr.intLit("5")`. The resemblance is purely **syntactic**. A named constructor is a declared *verb* that builds through `init{ }`; naming a variant case is built-in syntax with no verb behind it. They share a spelling, not a mechanism. @@ -131,6 +131,30 @@ A directly inline self-reference would have infinite size, which the uniform-str - The `#` modifier is what carries recursion: a plain `variant` is the sum mould's value form, laid out inline, while a `#variant` is its reference form — carrying a tag, boxing its recursive cases through `&`, and placed by the ordinary reference-type rules ([`memory.md`](memory.md) §3.5). A recursive sum such as `Expr` is a `#variant`. - Indirection is always **explicit `&`**. There is no hidden auto-boxing, matching Zane's stance that hosting and guests are explicit. +### 4.1 A recursive structure is rooted in a field + +Because a recursive member is an `&`, filling it needs a **guest source**, and a bare symbol is not one ([`memory.md`](memory.md) §2.8.1). A recursive structure is therefore rooted in a field rather than in a bare local: the node a case points at is hosted by a field, and the guest is minted from that field access. + +```zane +type Tree = #struct { + root Expr; +} + +tree Tree(Expr.intLit("5")) // the first node is hosted by a field +outer Expr.flip(tree.root) // legal: `tree.root` is a field access on a place + +leaf Expr.intLit("5") +bad Expr.flip(leaf) // ILLEGAL: `leaf` is a bare symbol, not a guest source +``` + +A verb that builds recursively takes its child as an `&` parameter, which is itself a guest source, so the chain continues without further ceremony: + +```zane +Expr negate(inner &Expr) => Expr.flip(inner) +``` + +This is the same requirement an `&` field of any `#struct` carries; recursion is not a special case. What it means in practice is that the *root* of a recursive structure lives in a field of the type that owns the structure — which is where a root belongs anyway, since that field is what keeps the whole shape alive. + > **Story:** [`stories/adt.md`](../stories/adt.md#one-body-product-or-sum) — "One body, product or sum". --- diff --git a/spec/concurrency.md b/spec/concurrency.md index dfbca3f..f3b2620 100644 --- a/spec/concurrency.md +++ b/spec/concurrency.md @@ -121,7 +121,7 @@ A spawned call may **mutate** state only through a value-typed receiver. A spawn A direct consequence is that reference types are never mutated by spawned work, so every concurrent **read** of the reference-typed object graph is safe by construction. ### 4.3 Single writer per storage location -For any one storage location, at most one live spawned call may hold a **mutable borrow** — the `!` receiver of a spawned `mut` call. Two spawned calls that mutably borrow the same location are a compile-time error. Because value types carry no `&`, a location's identity is unambiguous — there is no hidden alias to obscure that two receivers denote the same slot — so this disjointness is checked at the spawn site by inspecting the receivers, not by tracing the program. The hosting scope may not access a location while a live spawn holds its mutable borrow; the borrow is released when that spawn completes (§4.1). +For any one storage location, at most one live spawned call may hold a **mutable borrow** — the `!` receiver of a spawned `mut` call. By §4.2 that receiver is always value-typed, so the borrows this rule counts are value borrows; a reference-type `'T` borrow ([`memory.md`](memory.md) §2.9) never reaches a spawned `mut` receiver. Two spawned calls that mutably borrow the same location are a compile-time error. Because value types carry no `&`, a location's identity is unambiguous — there is no hidden alias to obscure that two receivers denote the same slot — so this disjointness is checked at the spawn site by inspecting the receivers, not by tracing the program. The hosting scope may not access a location while a live spawn holds its mutable borrow; the borrow is released when that spawn completes (§4.1). ### 4.4 Reads take a coherent snapshot A spawned call may read a value that another live spawn is mutating; the read observes a **coherent snapshot** of the value rather than blocking. Reading a shared value into a fresh binding — `snap VarType = shared` — is what takes the snapshot, and the copy is tear-free even when the writer is mid-update. This replaces lock-based serialization for in-memory value state, so a real-time reader never waits on a writer. Serialization still applies to external, capability-backed resources (§4.5). diff --git a/spec/effects.md b/spec/effects.md index 1a1bf7b..bc5b4ac 100644 --- a/spec/effects.md +++ b/spec/effects.md @@ -31,7 +31,7 @@ A side effect is any observable interaction beyond returning a value, including: A capability is an object whose methods model access to external state, such as a filesystem, logger, socket, clock, or random source. ### 2.3 `mut` -`mut` is the only effect modifier in the language. It appears on methods and grants write access to state reachable through `this`; the write lands on the caller's object or on state reachable from it. A value-type `this` is a **borrow** of the caller's slot; a reference-type `this` is an implicit `&` reference to the object (see [`functions.md`](functions.md) §2.4). +`mut` is the only effect modifier in the language. It appears on methods and grants write access to state reachable through `this`; the write lands on the caller's object or on state reachable from it. `this` is a **borrow** of the caller's slot for both kinds: a value-type `this` borrows the value, and a reference-type `this` written bare is an implicit `'T` borrow of the object (see [`functions.md`](functions.md) §2.4). ### 2.4 Parameters are not mutable by default Parameters other than `this` are read-only. Mutation of another object must be expressed by calling a `mut` method on that object as the receiver. A number parameter that resolves to a number value in body positions (see [`generics.md`](generics.md) §3.5) is a value-like binding and is read-only by default; mutating it requires a `mut` declaration. diff --git a/spec/foundations.md b/spec/foundations.md index 7505cb4..f91c1b9 100644 --- a/spec/foundations.md +++ b/spec/foundations.md @@ -89,11 +89,12 @@ Every type is a **value type** unless it is marked with `#`, which makes it a ** A value type is copied on assignment, has no identity, and — the load-bearing restriction — is *transitively* a value: it may contain only other value types, never a reference-type or `&` field. Nothing reachable from a value can be aliased, which is why a value can be copied and shared by snapshot with no bookkeeping, and why a value type cannot recurse (a self-reference would need indirection, and indirection is a reference). A reference type is the opposite in each respect: it has stable identity, may be aliased through `&`, may hold reference-type and `&` fields, and may recurse. -Both kinds are mutated in place through a `mut` method, but the receiver reaches the caller differently: a value-type `this` is a *borrow* of the caller's slot (so a value is mutable without gaining identity), while a reference-type `this` is an implicit `&` to the object. Borrowing is the value world's device; the reference world already has `&`. +Both kinds are mutated in place through a `mut` method, and the receiver reaches the caller the same way in each: `this` is a *borrow* of the caller's slot, so a value is mutable without gaining identity and a reference object is mutable without minting a guest to it. Borrowing serves both worlds; what the reference world adds on top is `&`, for the cases where a callee must keep the object past the call. - **`#` is the only kind modifier**, applied uniformly to any type. See [`types.md`](types.md) §2 and [`adt.md`](adt.md) §2–§3. - **A value type is transitively value** (no reference-type or `&` field, anywhere downstream). This closed value world is specified by [`memory.md`](memory.md) §2.10. - **`&` rides on `#`.** A non-hosting `&` exists only for reference types; a value is shared by copy or by a scoped borrow, never by a stored `&`. See [`memory.md`](memory.md) §2.4. +- **A guest comes from a field, not a symbol.** A new `&` is minted only from a field access or an `&T` parameter; a bare symbol is a place but never a guest source, so a local's own hosting slot has nothing pointing at it. Passing such a symbol into a call is the borrow mode's job. See [`memory.md`](memory.md) §2.8.1 and §2.9. - **Concurrency reads this axis.** A spawned call may mutate only a value-typed receiver, because a value's transitive alias-freedom is exactly what lets the compiler rule out a data race from the signature alone. See [`concurrency.md`](concurrency.md) §4. > **Story:** [`stories/foundations.md`](../stories/foundations.md#identity-is-opt-in-one-axis-for-value-and-reference) — "Identity is opt-in: one axis for value and reference". diff --git a/spec/functions.md b/spec/functions.md index deeb7da..72a67da 100644 --- a/spec/functions.md +++ b/spec/functions.md @@ -56,15 +56,21 @@ A method marked `mut` may write to any state reachable through `this`, whether t A write to `this` lands on the caller's object; how `this` reaches the caller differs by kind (see [`memory.md`](memory.md) §2.9): - For a **value-type** receiver, `this` is a **mutable borrow** of the caller's slot — the actual value, not a copy. The borrow makes the value mutable in place while preserving its value semantics. Because the borrow is scoped and non-escaping, `this` may be read and written but cannot be stored as an `&` or returned as one, since a value type is not `&`-rootable. -- For a **reference-type** receiver, `this` is an implicit **`&` reference** to the object (never swallowed). A `mut` method mutates through it as through any `&`, and `this` composes with the `&` system — it may be passed where an `&T` is expected. +- For a **reference-type** receiver, `this` is a **mutable borrow** too, and for the same reason: the receiver expression at the call site is usually a bare symbol, which is not a guest source ([`memory.md`](memory.md) §2.8.1). Writing `this T` on a reference type therefore means `this 'T`. A `mut` method mutates through the borrow, and the receiver is never swallowed — the caller stays a full host. + +A method that needs to keep the receiver past the call — store it in an `&` field, or return it as `&T` ([`lifetimes.md`](lifetimes.md) §1.7) — declares `this &T` instead. That is a guest receiver, so the call site must supply a guest source. ```zane -Unit setScale(this Node, scale Float) mut { // reference receiver +Unit setScale(this Node, scale Float) mut { // reference receiver: implicit `'Node` borrow this.scale = scale return Unit() } ``` +```zane +&Weapon mainWeapon(this &Player) => this.weapon // guest receiver: may be returned as `&` +``` + ```zane Unit setY(this Vec2, y Float) mut { // value receiver: in-place through the borrow this.y = y @@ -96,10 +102,16 @@ receiver!Pkg$method(arg) → Pkg$method(receiver, arg) ``` ### 2.7 Parameters are read-only -Explicit parameters other than `this` are read-only: they cannot be assigned or marked `mut`. Mutation of another object must be expressed as a `mut` method call on that object as the receiver. How each parameter is passed — a value borrow, or a reference `&`/swallow — is covered in [`memory.md`](memory.md) §2.9. +Explicit parameters other than `this` are read-only: they cannot be assigned or marked `mut`. Mutation of another object must be expressed as a `mut` method call on that object as the receiver. How each parameter is passed — the three reference modes, or a value borrow — is covered in [`memory.md`](memory.md) §2.9. -### 2.8 `&` and swallowing method parameters -A method parameter declared as `&T` is a **reference**: the caller supplies a source that may create a new `&` under [`memory.md`](memory.md) §2.8, and the callee may store it into an `&` field. A parameter declared as a plain reference type `T` **swallows** its argument — it takes the value by hosting access, which the value's call-site scope keeps ([`lifetimes.md`](lifetimes.md) §1.5) — so it cannot be bound into `&` storage, because a swallowed value is hosted at the call site while an `&` field may outlive the call (see [`memory.md`](memory.md) §2.9). A value-type parameter is a read-only borrow. To pass a reference object for reading only, use `&T`. +### 2.8 Swallow, guest, and borrow method parameters +A reference-type method parameter selects one of three passing modes ([`memory.md`](memory.md) §2.9): + +- A parameter declared as `&T` is a **guest**: the caller supplies a guest source under [`memory.md`](memory.md) §2.8, and the callee may store it into an `&` field or return it. +- A parameter declared as `'T` is a **borrow**: the caller may supply any place expression, a bare symbol included, and the callee gets read and `mut` access for the call and nothing more. +- A parameter declared as a plain reference type `T` **swallows** its argument — it takes the value by hosting access, which the value's call-site scope keeps ([`lifetimes.md`](lifetimes.md) §1.5). + +Neither a swallowed nor a borrowed parameter may be bound into `&` storage: a swallowed value is hosted at the call site while an `&` field may outlive the call, and a borrow does not survive the call at all. A value-type parameter is always a read-only borrow. To pass a reference object for reading only, use `'T`. ```zane type Car = #struct { @@ -108,30 +120,33 @@ type Car = #struct { } // `&` parameter: may be stored into an `&` field -Unit setEngine(this Car, engine &Engine) mut { +Unit setEngine(this 'Car, engine &Engine) mut { this.engine = engine // legal return Unit() } -// `&` parameter, read only -Int calculate(this Car, engine &Engine) { - return this._value + engine.speed // legal: reading through the reference +// borrow parameter, read only +Int calculate(this 'Car, engine 'Engine) { + return this._value + engine.speed // legal: reading through the borrow } -// plain reference-type parameter swallows; a swallowed host is not an `&` source -Unit setEngineWrong(this Car, engine Engine) mut { +// plain reference-type parameter swallows; a swallowed host is not a guest source +Unit setEngineWrong(this 'Car, engine Engine) mut { this.engine = engine // ILLEGAL: cannot store a swallowed host into an `&` field return Unit() } ``` -Call syntax is uniform regardless of the parameter mode: +Call syntax is uniform regardless of the parameter mode; only what the caller may supply differs: ```zane engine Engine() -car!setEngine(engine) // legal: engine may create a new `&` -car:calculate(engine) // legal: read-only reference to engine -car!setEngine(Engine()) // ILLEGAL: temporary cannot bind to `&` parameter +garage Garage() + +car:calculate(engine) // legal: a bare symbol may be borrowed +car!setEngine(engine) // ILLEGAL: a bare symbol is not a guest source +car!setEngine(garage.spare) // legal: a field access is a guest source +car!setEngine(Engine()) // ILLEGAL: a temporary is not a place expression ``` ### 2.9 Subscripts are place projections @@ -208,18 +223,21 @@ The return checker does not synthesize a constructor call for `Unit` or any othe ### 4.1 Overload identity is parameter types only Two declarations in the same package conflict when they have the same ordered parameter types. Parameter names, `this`, `mut`, and return type do not distinguish overloads. -Two overloads **MUST NOT** differ only by whether the same parameter position is `T` versus `&T`. Such declarations are illegal and the compiler **MUST** reject them with a compile-time error, for example: "illegal overload set: differs only by `&` on a parameter; rename one declaration or choose a single signature." +Two overloads **MUST NOT** differ only by the **passing mode** at the same parameter position — that is, only by whether that position is `T`, `&T`, or `'T`, the receiver included. Such declarations are illegal and the compiler **MUST** reject them with a compile-time error, for example: "illegal overload set: differs only by the passing mode on a parameter; rename one declaration or choose a single signature." ```zane -Unit consume(this Car, engine Engine) -Unit consume(this Car, engine &Engine) // ERROR +Unit consume(this 'Car, engine Engine) +Unit consume(this 'Car, engine &Engine) // ERROR: differs only by the passing mode +Unit consume(this 'Car, engine 'Engine) // ERROR: same ``` +The mode changes what the caller must supply and what state the call leaves the caller in — not the shape of the call. Overloading on it would make `consume(e)` mean two different things about `e`'s ownership with nothing at the call site to tell them apart. + ### 4.2 Consequences of the overload identity rules Declarations that differ only by return type, parameter names, `this`, or `mut` are compile-time conflicts. ### 4.3 Valid overloads differ by arity or parameter type -Legal overload sets must differ in the number of parameters or in at least one parameter type other than bare `&`-ness at the same position. +Legal overload sets must differ in the number of parameters or in at least one parameter type at the same position, ignoring the passing mode. > **Story:** [`stories/functions.md`](../stories/functions.md#overloading-on-shapes-and-only-shapes) — "Overloading on shapes, and only shapes". @@ -390,14 +408,16 @@ Read-only methods and functions are effect-free with respect to their receiver u | Verb | A callable; its kind is selected by markers, and each marker unlocks a capability | | Capability markers | `this` first → method (private access); name is a type → constructor (`init{ }`, implicit return); symbol name → operator; no name → lambda | | Method | Package-scope verb whose first parameter is `this` | -| `mut` method | Called with `!`; a value-type `this` is a mutable borrow of the caller's slot, a reference-type `this` is an implicit `&` reference; may mutate state reachable through `this` | +| `mut` method | Called with `!`; `this` is a mutable borrow of the caller's slot for both value and reference receivers; may mutate state reachable through `this` | | Read-only method | Called with `:`; may read but not write `this` | | Function | Identifier-named package-scope verb without `this`; no private-field privilege | | Block-bodied return | Every returning path uses `return expr`; `Unit` receives no fallthrough or bare-return exception | -| `&` method parameter | Caller must supply an allowed `&` source; callee may store into `&` fields | -| Plain `T` method parameter | Value-only; caller may supply a temporary; callee **MUST NOT** bind it into `&` storage | +| `&` method parameter | Caller must supply a guest source (never a bare symbol); callee may store it into `&` fields or return it | +| `'T` method parameter | Caller may supply any place expression, bare symbols included; read and `mut` access for the call only; **MUST NOT** be stored, returned, or moved | +| Plain `T` method parameter | Swallows; caller may supply a temporary and downgrades to a guest; callee **MUST NOT** bind it into `&` storage | +| Reference receiver | `this T` is an implicit `'T` borrow; `this &T` is a guest receiver, required to store or return the receiver | | Subscript | Package-scope place projection written `(this T)[...] => placeExpr`; no explicit return type | -| Overload identity | Parameter types only; not names, return type, or `mut`; overloads differing only by `&` at one position are illegal | +| Overload identity | Parameter types only; not names, return type, or `mut`; overloads differing only by the passing mode (`T` / `&T` / `'T`) at one position are illegal | | Overload resolution phases | Direct match, then generic match, then implicit match; ambiguity within any one phase is an error | | Callable reference | Illegal; methods, functions, and operators are call-only and have no value form | | Lambda | Self-typed function value: explicit parameter types, return type, abort type, and `mut`; no capture | diff --git a/spec/glossary.md b/spec/glossary.md index 28b560a..8197d97 100644 --- a/spec/glossary.md +++ b/spec/glossary.md @@ -65,7 +65,7 @@ This file gives short, reusable names to concepts that appear across multiple sp ## 3. Types, Storage, and Binding ### 3.1 place expression -- **Meaning:** A place expression denotes an existing, stable storage location. Some place expressions may create new `&` values, while `[]` expressions remain excluded from that rule. +- **Meaning:** A place expression denotes an existing, stable storage location. Being a place is necessary but not sufficient to mint an `&`: only a field access of a place and an `&T` parameter are guest sources, while bare symbols and `[]` expressions are places that are excluded (§3.36). - **Why this name:** The term names the expressions that refer to a storage "place" rather than to a temporary value. - **Canonical home:** [`memory.md`](memory.md) §2.8 @@ -195,8 +195,8 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`functions.md`](functions.md) §8 ### 3.27 borrow -- **Meaning:** Non-hosting, non-escaping access to a caller's storage for the duration of a call — the passing mode for **value types**, which have no `&` of their own. A value parameter is a read-only borrow and a value-type `mut` receiver is a mutable borrow; a value is copied only when bound into a fresh slot. Reference types are passed as guests or swallowed instead, and a reference-type `this` is an implicit guest. -- **Why this name:** The callee is lent the caller's storage for the call and gives it back at return — it does not host it and cannot keep it. Unlike a guest, a borrow has no anchor or tether and cannot be stored or returned. +- **Meaning:** Non-hosting, non-escaping access to a caller's storage for the duration of a call. Every value type is passed this way — a value parameter is a read-only borrow, a value-type `mut` receiver is a mutable borrow, and a value is copied only when bound into a fresh slot. A reference type may also be borrowed, written `'T`, which is the only non-swallowing way to pass a bare symbol (§3.36); a bare reference-type `this` is an implicit `'T` borrow. +- **Why this name:** The callee is lent the caller's storage for the call and gives it back at return — it does not host it and cannot keep it. Unlike a guest, a borrow has no anchor or tether and cannot be stored, returned, or moved. - **Canonical home:** [`memory.md`](memory.md) §2.9 ### 3.28 coercion site @@ -225,7 +225,7 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`memory.md`](memory.md) §2.1 ### 3.33 guest -- **Meaning:** The source-facing `&T`: access to a hosted reference-type object without storing that object or controlling its lifetime. A guest may be repointed, copied when assigned or passed, stored in an `&` field, or returned as `&T`, but it cannot outlive its host. Internally, a guest is represented by a tether (§3.24) that resolves through an anchor cell (§3.23). +- **Meaning:** The source-facing `&T`: access to a hosted reference-type object without storing that object or controlling its lifetime. A guest may be repointed, copied when assigned or passed, stored in an `&` field, or returned as `&T`, but it cannot outlive its host, and it may be minted only from a field or an `&T` parameter (§3.36). Internally, a guest is represented by a tether (§3.24) that resolves through an anchor cell (§3.23). - **Why this name:** A guest may use what a host provides without owning it, and the guest's stay cannot outlast the host. The pair names the source relationship without exposing its runtime mechanism. - **Canonical home:** [`memory.md`](memory.md) §2.4 @@ -239,6 +239,16 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Why this name:** "Consume" names taking the value for good; "relay" names passing the hosting role through and handing it back out. - **Canonical home:** [`lifetimes.md`](lifetimes.md) §1.8 +### 3.36 guest source restriction +- **Meaning:** A new `&` may be minted only from a field access whose base is a place, or from an `&T` parameter. A **bare symbol** — an identifier standing alone rather than as the base of a field access — is a place expression but never a guest source, so no guest can point at a local's own hosting slot and that slot stays free to be overwritten or moved from. Passing a bare symbol into a call is the borrow mode's job (§3.27). +- **Why this name:** The rule constrains the *source* of a guest — where one may come from — and nothing about what a guest can survive once minted; a guest to a field still follows its host across overwrites and rehosting. +- **Canonical home:** [`memory.md`](memory.md) §2.8.1 + +### 3.37 passing mode +- **Meaning:** Which of three ways a reference-type argument reaches a callee, fixed entirely by the parameter's surface form: `T` **swallows** it (hosting access; the caller downgrades to a guest), `&T` takes a **guest** (storable and returnable; requires a guest source), `'T` **borrows** it (read and `mut` for the call only; accepts any place, bare symbols included). The receiver takes a mode like any parameter, and defaults to the borrow. Two overloads may not differ only by the mode at one position. +- **Why this name:** "Mode" names a choice about *how* the same argument travels rather than *what* it is — the type is unchanged in all three, and only the caller's obligations and resulting state differ. +- **Canonical home:** [`memory.md`](memory.md) §2.9 + --- ## 4. Packages, Operators, and Versioning diff --git a/spec/lexical.md b/spec/lexical.md index 9bf5cd7..6e44780 100644 --- a/spec/lexical.md +++ b/spec/lexical.md @@ -97,7 +97,8 @@ Certain leading characters are reserved and are not ordinary identifier starts: | Sigil | Meaning | Canonical home | |---|---|---| -| `&` | Reference type (`&Node`) | [`memory.md`](memory.md) §2 | +| `&` | Guest type (`&Node`) | [`memory.md`](memory.md) §2 | +| `'` | Borrow type (`'Node`), parameter positions only | [`memory.md`](memory.md) §2.9 | | `@` | Reserved compiler namespace (`@primitives$`, `@concepts$`) | [`syntax.md`](syntax.md) §2.7 | | `$` | Package-member separator (`packageName$member`) | [`packages.md`](packages.md) §1 | @@ -184,6 +185,7 @@ Because the parser always knows whether it is inside a type-expression body or a | Type parameter | An uppercase name (`T`) declared `T Type` (in a type's `<>` header or inline in a verb); referenced bare | | Digits | Legal in a name except as the first character; carry no special meaning | | Leading `_` | A field is private to `this` methods for its type; a named package-scope declaration is private to its package | +| Leading `&` / `'` | `&Node` is a guest type (storage, parameter, and return positions); `'Node` is a borrow type (parameter positions only); mutually exclusive on one type | | `<>` disambiguation | A type (uppercase) on the left means a type argument list; a value (lowercase) means comparison | | Member terminator | `;` terminates every member of a `struct`/`variant` body (marked or unmarked with `#`) and every arm of a `match` block; always trailing, inline or multiline; newlines are insignificant there | | Value separator | `,` separates elements of a value collection (arrays, `enum`, call/constructor args, `init{}` fields, generic args, `match` case groups); never trailing | diff --git a/spec/lifetimes.md b/spec/lifetimes.md index 4eea393..b98c82b 100644 --- a/spec/lifetimes.md +++ b/spec/lifetimes.md @@ -9,17 +9,24 @@ This document specifies Zane's lexical lifetime rules: `&` assignment scope chec ## 1. Scope Rules and Moves ### 1.1 `&` assignment uses host scope -An `&` assignment is legal only when the target's host is declared in the same or a higher lexical scope than the `&` itself. +An `&` assignment is legal only when the source is a guest source ([`memory.md`](memory.md) §2.8) **and** the target's host is declared in the same or a higher lexical scope than the `&` itself. ```zane -outer Node() -r &Node = outer +outerTree Tree() +r &Node = outerTree.root { - innerNode Node() - r = innerNode // ILLEGAL: host's scope is nested relative to the guest + innerTree Tree() + r = innerTree.root // ILLEGAL: the host's scope is nested relative to the guest } ``` +The two conditions are independent, and the second only ever arises for sources the first admits. A bare symbol fails the first condition outright: + +```zane +node Node() +r &Node = node // ILLEGAL: a bare symbol is not a guest source +``` + The compiler compares declaration scopes. It does not perform borrow inference or lifetime annotation solving. > **Story:** [`stories/lifetimes.md`](../stories/lifetimes.md#inheriting-a-debt-safety-without-a-borrow-checker) — "Inheriting a debt: safety without a borrow checker". @@ -34,6 +41,7 @@ A verb that returns a hosting `T` produces a fresh value that no symbol, field, The following are **not** move-sources: - an `&` value, including a verb that returns `&T` (guests are non-hosting and cannot transfer hosting; see [`memory.md`](memory.md) §2.4) +- a `'T` borrow parameter (a borrow neither hosts the object nor outlives the call; see [`memory.md`](memory.md) §2.9) - a field access such as `car.engine` - a container element access such as `cars[1]` - any other access path that projects into an existing host @@ -105,6 +113,8 @@ A parameter's value is exempt. Because a parameter belongs to the call-site scop ### 1.5 Parameters belong to the call site A reference-type parameter is **not part of the callee's body scope**. It behaves as a symbol in the **call-site scope**, one level above the body. Passing a hosting reference-type value to a plain `T` parameter lends it in with hosting access, but the value's lifetime stays with the call site. +This is stated for the swallowing mode because that is the only mode where hosting crosses the call boundary at all. A `&T` guest parameter and a `'T` borrow parameter never take hosting ([`memory.md`](memory.md) §2.9), so nothing about the argument's lifetime changes when either is used; the call-site scope keeps hosting throughout. + This is what makes the passing rule safe. Because the parameter is not part of the body scope, the body draining never destroys the value. The body may read it, move it into a local, or pass it to a nested call; when a local that received it exits, the value is not dropped — the compiler moves it back up to the call site, and the chain repeats outward until the scope that first hosted the value drains. A value passed by hosting access therefore always outlives the call, which is what lets the caller's symbol downgrade to a live guest (§1.8) rather than a dangling one. ```zane @@ -117,7 +127,7 @@ Unit enterMatch(player Player) { `startMatch` puts `player` into the local `island`. Because `player` belongs to the call site, `island` draining does not destroy it; the value lives until `enterMatch`'s own scope drains. Inside `enterMatch`, `player` was passed to `startMatch` by hosting access, so `enterMatch`'s `player` symbol is now a guest to it (§1.8) — and so is the argument symbol in whatever called `enterMatch`. -For `&` fields specifically, the callee must declare the corresponding parameter as `&T` ([`memory.md`](memory.md) §2.9). Attempting to bind a plain `T` parameter into `&` storage is a compile-time error, because a swallowed value is hosted at the call site while an `&` field lives with the object that holds it, which may outlive the call. The callee's signature therefore signals whether an `&`-creating source ([`memory.md`](memory.md) §2.8) is required at the call site. +For `&` fields specifically, the callee must declare the corresponding parameter as `&T` ([`memory.md`](memory.md) §2.9). Binding a plain `T` parameter into `&` storage is a compile-time error, because a swallowed value is hosted at the call site while an `&` field lives with the object that holds it, which may outlive the call. Binding a `'T` parameter into `&` storage is a compile-time error for a stronger reason: a borrow ends with the call. The callee's signature therefore signals which mode applies, and so whether a guest source ([`memory.md`](memory.md) §2.8) is required at the call site. > **Story:** [`stories/lifetimes.md`](../stories/lifetimes.md#consumed-or-borrowed-the-parameter-that-lives-at-the-call-site) — "Consumed or borrowed: the parameter that lives at the call site". @@ -137,24 +147,28 @@ This also applies across calls. Passing a hosting value to a plain `T` parameter A hosting verb result (§1.2) has no symbol to downgrade. The temporary is consumed by the move and cannot be named again, so the double-move question never arises for it. -### 1.7 Returned `&` values must be rooted in a parameter -A function may return an `&T` only when the returned reference is rooted in one of the function's parameters. `this` counts as a parameter for this rule. +### 1.7 Returned `&` values must be rooted in a guest parameter +A function may return an `&T` only when the returned guest is rooted in one of the function's **`&T` parameters** and is itself a guest source ([`memory.md`](memory.md) §2.8) — the parameter used bare, or a field access whose base chain reaches it. `this` counts as a parameter for this rule when it is declared `this &T`. ```zane &Weapon getWeapon(this &Player) => this.weapon ``` +The other two parameter modes are not roots. A `'T` borrow ends with the call, so a guest rooted in one would outlive the access it was granted. A swallowing `T` parameter is a bare symbol in the call-site scope, and a bare symbol is not a guest source at all. + ```zane +&Weapon fromBorrow(this 'Player) => this.weapon // ILLEGAL: a borrow is not a guest root + &Node bad() { value Node() - return value // ILLEGAL: returned `&` is not rooted in a parameter + return value // ILLEGAL: a local is neither a parameter nor a guest source } ``` > **Story:** [`stories/lifetimes.md`](../stories/lifetimes.md#returning-a-ref-without-a-lifetime-to-name-it) — "Returning a ref without a lifetime to name it". ### 1.8 Passing a host to a `T` parameter downgrades it to a guest -A plain reference-type parameter `T` takes its argument by **hosting access**. Passing a hosting value to such a parameter uses that value as a move-source (§1.2), so the caller's symbol downgrades to a guest (§1.6) — **whatever the callee does with the value**. The parameter's declared type is the whole contract: `T` means the caller gives up hosting; `&T` (a guest, [`memory.md`](memory.md) §2.9) means the caller lends the value and stays a full host. Nothing in the callee's body changes the outcome the signature already states. +A plain reference-type parameter `T` takes its argument by **hosting access**. Passing a hosting value to such a parameter uses that value as a move-source (§1.2), so the caller's symbol downgrades to a guest (§1.6) — **whatever the callee does with the value**. The parameter's declared type is the whole contract: `T` means the caller gives up hosting; `&T` and `'T` ([`memory.md`](memory.md) §2.9) both mean the caller stays a full host. Nothing in the callee's body changes the outcome the signature already states. ```zane car Car() @@ -165,13 +179,14 @@ truck Truck(car) // ILLEGAL: car is a guest, not a move-source The value outlives the call (§1.5), so the downgraded guest always resolves to a live object. Where the value comes to rest — moved into another parameter's hosting storage, moved into the return, or held in the call-site scope — the guest follows through the anchor ([`memory.md`](memory.md) §4.5). -A verb treats a reference-type host argument in one of three ways, each fixed by its signature: +A verb treats a reference-type host argument in one of four ways, each fixed by its signature: -- it takes a **guest** — declares the parameter `&T` ([`memory.md`](memory.md) §2.9); the caller stays a full host and lends only a guest for the call. +- it **borrows** the object — declares the parameter `'T` ([`memory.md`](memory.md) §2.9); the caller stays a full host and the callee gets read and `mut` access for the call only. This is the mode for a bare symbol, which no other non-swallowing mode accepts (§2.8.1 of [`memory.md`](memory.md)). +- it takes a **guest** — declares the parameter `&T`; the caller stays a full host, and the callee may keep the guest past the call by storing or returning it. Only a guest source can supply one. - it **relays** the host — declares a swallowing `T` and returns a hosting handle; the caller downgrades to a guest but may bind the return to host the object again (§1.9). - it **consumes** the host — declares a swallowing `T` and returns no host; the caller downgrades to a guest, and the value stays wherever the verb placed it. -Passing a guest leaves the caller as host; relaying and consuming both downgrade it, differing only in whether a hosting handle is handed back. So to keep or recover hosting, either pass `&T` or bind a relayed return: +Borrowing and taking a guest leave the caller as host; relaying and consuming both downgrade it, differing only in whether a hosting handle is handed back. So to keep or recover hosting, pass `'T` or `&T`, or bind a relayed return: ```zane weapon Weapon() @@ -196,7 +211,7 @@ Unit main() { } ``` -A verb that only reads its reference argument may still declare it plain `T`: reading does not change the fact that the signature asked for hosting access, so the caller downgrades all the same. Declaring the parameter `&T` is what keeps the caller as host. Because the signature alone decides the caller's state, there is no interprocedural consumption inference: whether a passed host downgrades never depends on the callee's body or on the build. Using hosting access only to read a value is legal. Leaving a parameter entirely unused is a separate, general matter — a release build rejects an unused parameter whether it hosts a value or not. +A verb that only reads its reference argument may still declare it plain `T`: reading does not change the fact that the signature asked for hosting access, so the caller downgrades all the same. Declaring the parameter `'T` — or `&T`, when the callee needs to keep it — is what keeps the caller as host. Because the signature alone decides the caller's state, there is no interprocedural consumption inference: whether a passed host downgrades never depends on the callee's body or on the build. Using hosting access only to read a value is legal. Leaving a parameter entirely unused is a separate, general matter — a release build rejects an unused parameter whether it hosts a value or not. > **Story:** [`stories/lifetimes.md`](../stories/lifetimes.md#the-signature-is-the-whole-contract-retiring-inferred-consumption) — "The signature is the whole contract: retiring inferred consumption". @@ -249,14 +264,14 @@ Because scope rules (§1.1) prevent guests from outliving their hosts, the runti | Concept | Rule | |---|---| -| `&` return | Returned `&T` must be rooted in a parameter; `this` counts | -| Guest assignment | Only from a place expression whose host is in the same or a higher lexical scope than the guest | -| Move-source | A direct host symbol (local or parameter) or a hosting verb result; not an `&`, field, container element, or other access path | +| `&` return | Returned `&T` must be rooted in an `&T` parameter and be a guest source; `this &T` counts; a `'T` borrow and a swallowing `T` are not roots | +| Guest assignment | Only from a guest source ([`memory.md`](memory.md) §2.8) whose host is in the same or a higher lexical scope than the guest; a bare symbol is never a guest source | +| Move-source | A direct host symbol (local or parameter) or a hosting verb result; not an `&`, a `'T` borrow, a field, a container element, or any other access path | | Move declaration-block restriction | A direct host symbol may only be moved in the exact lexical block where it was declared; parameters may be moved at the body top level | | Move destination scope | Destination host must be in the same or a higher lexical scope than the source host | | Post-move downgrade | After a move, the source symbol downgrades to an `&` and remains readable but is no longer a move-source | | Parameter scope | A reference parameter belongs to the call-site scope, not the body, so a value passed by hosting access outlives the call | -| Hosting argument | A verb takes a **guest** (`&T`, caller keeps it), **relays** the host (`T` and returns a hosting handle, caller may bind it to host again), or **consumes** it (`T`, no host returned, caller keeps a guest); passing to a plain `T` downgrades the caller to a guest whatever the body does | +| Hosting argument | A verb **borrows** it (`'T`, caller keeps it; the only non-swallowing mode a bare symbol may feed), takes a **guest** (`&T`, caller keeps it), **relays** the host (`T` and returns a hosting handle, caller may bind it to host again), or **consumes** it (`T`, no host returned, caller keeps a guest); passing to a plain `T` downgrades the caller to a guest whatever the body does | | Return value | A return need not be bound; an unbound reference-type result floats to the enclosing scope as an anonymous host, while an ignored value-type result is discarded | | Destruction | Deterministic and delayed until the hosting scope drains | diff --git a/spec/memory.md b/spec/memory.md index b1911da..a9b9809 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -12,6 +12,8 @@ Zane eliminates dangling guests by combining single hosting, lexical lifetime ru - **`Overwritable hosts`.** A reference-type host is directly initialized and may later be overwritten. - **`Guests ride on reference types`.** An `&` — a **guest** — is a non-hosting handle to a **reference type** (a `#`-marked type); a value type has no identity to anchor, so it is shared by copy or scoped borrow, never by a stored guest. +- **`Bare symbols are not guest sources`.** A new guest may be minted only from a field access or from an `&T` parameter — never from a bare symbol (§2.8). A local's own hosting slot is therefore never the thing a guest points at. +- **`Three passing modes`.** A reference-type parameter is written `T` to **swallow** it, `&T` to take a **guest**, or `'T` to **borrow** it for the call (§2.9). - **`Repointable guests`.** A guest is non-hosting storage that can point at different hosts over time. - **`Lexical lifetime enforcement`.** Guest assignment and rehosting are checked using declaration scope alone (see [`lifetimes.md`](lifetimes.md) §1). - **`Deterministic destruction`.** Objects are destroyed when their hosting scope drains; there is no tracing garbage collector (see [`lifetimes.md`](lifetimes.md) §2). @@ -20,7 +22,7 @@ Zane eliminates dangling guests by combining single hosting, lexical lifetime ru The source language and runtime use separate terms: an object lives in a **host**, and a **guest** (`&T`) may access it without storing it or controlling its lifetime. Internally, each guest is represented by a **tether** that resolves through an **anchor**. Moving the object updates its terminal anchor or links an older anchor to the destination anchor, so existing tethers — and therefore guests — continue to reach it. -These rules fit together mechanically. Hosts are the only storage that controls destruction. A guest may point only at an existing place, never a temporary. Lexical scope checks ensure the host outlives every guest derived from it. When an object is rehosted or a host is overwritten, guests stay valid. Internally, their tethers follow the host's anchor rather than a fixed object address. +These rules fit together mechanically. Hosts are the only storage that controls destruction. A guest may be minted only from a field or an `&` parameter — never from a temporary, and never from a bare symbol. Lexical scope checks ensure the host outlives every guest derived from it. When an object is rehosted or a host is overwritten, guests stay valid. Internally, their tethers follow the host's anchor rather than a fixed object address. > **Story:** [`stories/memory.md`](../stories/memory.md#safety-without-a-collector-and-without-lifetimes) — "Safety without a collector and without lifetimes". @@ -75,13 +77,15 @@ A guest may be declared as: - a function or constructor parameter - a function return type -An `&` type is legal in storage sites (local symbols, fields, nested storage types), function parameter positions, and function return-type positions. +An `&` type is legal in storage sites (local symbols, fields, nested storage types), function parameter positions, and function return-type positions. The borrow type `'T` (§2.9) is legal in parameter positions only: a borrow is not storage and never escapes its call. + +Declaring an `&` symbol is legal, but the restriction in §2.8 governs what may initialize it: a guest is minted from a field or from an `&T` parameter, not from a bare symbol. > **Story:** [`stories/memory.md`](../stories/memory.md#two-vocabularies-host-and-guest-above-anchor-and-tether) — "Two vocabularies: host and guest above anchor and tether". ### 2.5 Guests are repointable -An `&` symbol or `&` field may be assigned a different target later, as long as the scope rule in [`lifetimes.md`](lifetimes.md) §1.1 is satisfied. +An `&` symbol or `&` field may be assigned a different target later, as long as the new target is a guest source (§2.8) and the scope rule in [`lifetimes.md`](lifetimes.md) §1.1 is satisfied. ### 2.6 Guests are independent @@ -100,61 +104,97 @@ The following are place expressions: - a named local, field-backed, or hosting/`&` storage symbol such as `engine` - a field access whose base is a place, such as `car.engine` or `this.engine` - a subscript expression `list[index]` when `list` is a place expression and `[]` is defined as a place projection for that receiver type -- an `&T` parameter inside the callee body (§2.9) +- an `&T` guest parameter or a `'T` borrow parameter inside the callee body (§2.9) -Only some place expressions may create a new guest. A new `&` binding may be initialized from: +Only some place expressions may mint a new guest. A new `&` value may be minted from: -- a named symbol -- a field access whose base is a place +- a field access whose base is a place **and whose base chain does not pass through a `'T` borrow parameter**, such as `car.engine` or `this.engine` on a guest receiver - an `&T` parameter -A `[]` expression is never a source for creating a new `&`, even when it is a place expression. +Everything else is rejected. In particular: -Temporaries and other value-only expressions are not place expressions. Constructor calls and ordinary function results such as `Engine()` and `makeEngine()` are not places and cannot be bound to an `&`. +- A **bare symbol** is never a guest source, even though it is a place expression (§2.8.1). +- A `[]` expression is never a guest source, even though it is a place expression. +- A field access rooted in a `'T` borrow parameter is never a guest source. A borrow does not escape its call (§2.9), and it would escape just as surely inside a guest minted from one of its fields as it would on its own. +- Temporaries and other value-only expressions are not place expressions at all. Constructor calls and ordinary function results such as `Engine()` and `makeEngine()` are not places. ```zane engine &Engine = Engine() // ILLEGAL: Engine() is a temporary, not a place expression ``` ```zane -engine Engine() -r &Engine = engine // legal: engine is a named, stable storage location +car Car() +r &Engine = car.engine // legal: field access on a place ``` ```zane -weapons List = [Weapon(), Weapon()] -current &Weapon = weapons[1] // ILLEGAL: `[]` cannot create a new `&` +armory Armory() +weapons List<&Weapon> = [armory.primary, armory.backup] +current &Weapon = weapons[1] // legal: reads an `&Weapon` already stored in the list ``` +The last line works because `weapons[1]` reads an `&Weapon` value the list already holds. It does not mint a new `&` from a hosting element. Those stored guests are stable because the language does not let `[]` mint guests from host storage in the first place. + +Non-`&` host bindings may be initialized from any expression, including temporaries. The host materializes the value into stable storage. + ```zane -first Weapon() -second Weapon() -weapons List = [first, second] -current &Weapon = weapons[1] // legal: uses the existing stored `&Weapon` +engine Engine() // legal: plain host binding; Engine() temporary is materialized into engine ``` -This works because `weapons[1]` reads an `&Weapon` value that is already stored in the list. It does not create a new `&` from a hosting element. Those stored guests are stable because the language does not let `[]` create guests from host storage in the first place. +### 2.8.1 A bare symbol is not a guest source -Non-`&` host bindings may be initialized from any expression, including temporaries. The host materializes the value into stable storage. +A **bare symbol** — an identifier naming a local, a parameter, or a package constant, standing alone rather than as the base of a field access — **MUST NOT** be used to mint a new `&`. ```zane -engine Engine() // legal: plain host binding; Engine() temporary is materialized into engine +engine Engine() +r &Engine = engine // ILLEGAL: a bare symbol is not a guest source +inspect(engine) // ILLEGAL if inspect takes `&Engine` ``` +The reason is that a bare symbol's hosting slot is exactly the storage the language lets you overwrite most freely (§2.2, [`lifetimes.md`](lifetimes.md) §1). Without this rule a program can write: + +```zane +main Player() +second Player() +guest &Player = main // ILLEGAL under this rule +second = main +``` + +`second = main` moves the object out of `main`'s slot, and `main` downgrades to guest state ([`lifetimes.md`](lifetimes.md) §1.6). What `guest` should then denote — the object that left, or the slot it left from — has no answer that is right in both directions, and every candidate answer costs either a rule the programmer has to carry or machinery the runtime has to pay for. Removing the source removes the question: line 3 is a compile-time error, so no guest ever depends on a bare symbol's slot. + +Nothing is lost by it. A guest exists to reach an object from storage that does not own it — a field, a container element, a callee. A bare symbol is *already* in scope wherever a guest to it could be declared, so the guest never buys reach that the symbol itself did not already have. What a bare symbol is genuinely needed for is passing an object into a call, and that is what the borrow mode `'T` is for (§2.9): a borrow reads and mutates the caller's object for the duration of the call without minting a guest to it. + +A **field** is a different matter and stays a legal source. A field belongs to an object whose own lifetime the host system already tracks, and a guest to `car.engine` follows that field's host through the anchor path (§4.5) when the field is overwritten or the containing object is rehosted. This is what makes the restriction narrow: it constrains where guests come from, not what they can survive. + > **Story:** [`stories/memory.md`](../stories/memory.md#where-a-new-ref-may-come-from) — "Where a new ref may come from". +> **Story:** [`stories/memory.md`](../stories/memory.md#the-slot-that-could-not-be-pointed-at) — "The slot that could not be pointed at". -### 2.9 Function parameters: borrows and `&` +### 2.9 Function parameters: swallow, guest, and borrow -A **borrow** is non-hosting, non-escaping access to a caller's storage for the duration of a call. Unlike a guest (§2.4), a borrow has no anchor, cannot be stored in a field, and cannot be returned; it exists only while the call runs. Borrowing is the passing mode for **value types**, which have no `&` of their own. A value-type parameter is a **read-only borrow** of the caller's slot, and a value is **copied** only when it is bound into a fresh slot — an assignment, a new declaration, or a field or return store. The one writable borrow is a value-type `mut` receiver (see [`functions.md`](functions.md) §2.4). +A **borrow** is non-hosting, non-escaping access to a caller's storage for the duration of a call. Unlike a guest (§2.4), a borrow has no anchor, cannot be stored in a field, and cannot be returned; it exists only while the call runs. A value type is *always* passed this way: a value-type parameter is a **read-only borrow** of the caller's slot, and a value is **copied** only when it is bound into a fresh slot — an assignment, a new declaration, or a field or return store. -A **reference type** is passed through the hosting/`&` system instead, in one of two modes: +A **reference type** has three passing modes, one per surface form: -- A parameter declared as a plain reference type `T` **swallows** its argument — it takes the value by hosting access. The value belongs to the call-site scope, not the callee body ([`lifetimes.md`](lifetimes.md) §1.5), so it outlives the call. Passing a hosting value to such a parameter downgrades the caller's symbol to a guest ([`lifetimes.md`](lifetimes.md) §1.8), whatever the callee does with it — whether the verb relays the host back through its return or consumes it outright. A swallowing parameter the callee only reads downgrades the caller's host all the same; declaring it `&T` (a guest) is what keeps the caller as host. -- A parameter declared as `&T` is a **guest**: the caller supplies a source that may create a new guest under §2.8 (so `T` is a reference type, §2.4), and inside the callee body it acts as a place expression that may be stored into `&` storage or returned as `&T` under [`lifetimes.md`](lifetimes.md) §1.7. To read a reference-type object *without* taking hosting access, pass it as `&T`. +| Mode | Written | Caller supplies | The callee may | +|---|---|---|---| +| Swallow | `T` | a move-source ([`lifetimes.md`](lifetimes.md) §1.2) | take hosting access; the caller's symbol downgrades to a guest | +| Guest | `&T` | a guest source (§2.8) | store it in `&` storage or return it as `&T` | +| Borrow | `'T` | any place expression, **including a bare symbol** | read and mutate it for the duration of the call only | -A reference-type `mut` receiver is neither of these: `this` is an implicit guest to the object, never swallowed, so it composes with `&T` parameters (see [`functions.md`](functions.md) §2.4). +- A parameter declared as a plain reference type `T` **swallows** its argument — it takes the value by hosting access. The value belongs to the call-site scope, not the callee body ([`lifetimes.md`](lifetimes.md) §1.5), so it outlives the call. Passing a hosting value to such a parameter downgrades the caller's symbol to a guest ([`lifetimes.md`](lifetimes.md) §1.8), whatever the callee does with it — whether the verb relays the host back through its return or consumes it outright. +- A parameter declared as `&T` is a **guest**: the caller supplies a source that may mint a new guest under §2.8 (so `T` is a reference type, §2.4), and inside the callee body it acts as a place expression that may be stored into `&` storage or returned as `&T` under [`lifetimes.md`](lifetimes.md) §1.7. Because a bare symbol is not a guest source, an `&T` parameter can only be fed from a field, a container's stored guest, or another `&T` parameter. +- A parameter declared as `'T` is a **borrow**: the caller may supply any place expression, a bare symbol included, and the callee gets read and `mut` access for the call and nothing more. A `'T` parameter **MUST NOT** be stored in `&` storage, returned as `&T`, or used as a move-source, and neither may a field reached through it (§2.8); `'T` is not a legal storage, field, or return type. Passing a host to a `'T` parameter leaves the caller a full host: nothing downgrades. -Passing a value by borrow is the semantic model; where a read-only borrow is indistinguishable from a copy, the compiler may still pass a small value by copy, the same latitude placement has (§3.5). The distinction becomes observable under concurrent sharing, where a spawned reader sees the borrowed value live (see [`concurrency.md`](concurrency.md) §4.4). +`'T` is the mode that keeps ordinary calls ordinary. Under §2.8.1 a bare local cannot feed an `&T` parameter, so a verb that merely wants to read or mutate a caller's object declares that object `'T`: + +```zane +Float topSpeed(engine 'Engine) => engine.speed + +engine Engine() +s Float = topSpeed(engine) // legal: a bare symbol may be borrowed +``` + +Passing a value by borrow is the semantic model for both worlds; where a read-only borrow is indistinguishable from a copy, the compiler may still pass a small value by copy, the same latitude placement has (§3.5). The distinction becomes observable under concurrent sharing, where a spawned reader sees the borrowed value live (see [`concurrency.md`](concurrency.md) §4.4). ```zane type Car = #struct { @@ -164,33 +204,42 @@ type Car = #struct { } // `&` parameter is a guest; it may be stored into an `&` field -Unit setEngine(this Car, engine &Engine) mut { +Unit setEngine(this 'Car, engine &Engine) mut { this.engine = engine return Unit() } // plain reference-type parameter: taken by hosting access, then moved into a hosting field of this -Unit setSpare(this Car, engine Engine) mut { +Unit setSpare(this 'Car, engine Engine) mut { this.spare = engine return Unit() } -// `&` parameter, read only: a reference-type object passed without consuming it -Int inspect(this Car, engine &Engine) { +// borrow parameter: a reference-type object read without consuming it and without minting a guest +Int inspect(this 'Car, engine 'Engine) { return this._value + engine.speed } ``` -Binding a plain (swallowed) parameter into `&` storage is illegal, because a swallowed value is hosted at the call site while an `&` field lives with the object that holds it — which may outlive the call, leaving the `&` dangling: +A reference-type receiver follows the same three modes and defaults to the borrow: `this T` is an implicit `'T` borrow, and a method that needs to keep or hand back the receiver as a guest writes `this &T` (see [`functions.md`](functions.md) §2.4). + +Binding a swallowed or borrowed parameter into `&` storage is illegal. A swallowed value is hosted at the call site while an `&` field lives with the object that holds it — which may outlive the call. A borrow does not survive the call at all: ```zane -Unit setEngineWrong(this Car, engine Engine) mut { - this.engine = engine // ILLEGAL: a swallowed host is not an `&` source +Unit setEngineSwallowed(this 'Car, engine Engine) mut { + this.engine = engine // ILLEGAL: a swallowed host is not a guest source + return Unit() +} + +Unit setEngineBorrowed(this 'Car, engine 'Engine) mut { + this.engine = engine // ILLEGAL: a borrow is not a guest source and does not escape the call return Unit() } ``` -This rule preserves uniform call syntax. The call site writes `consume(e)` or `inspect(e)` regardless of whether the parameter is `&`. The callee's signature determines whether an `&`-creating source is required from the caller. +This rule preserves uniform call syntax. The call site writes `consume(e)`, `inspect(e)`, or `setEngine(e)` identically; only the callee's signature says which mode applies and therefore what the caller must supply and what state the caller is left in. + +> **Story:** [`stories/memory.md`](../stories/memory.md#three-ways-to-hand-over-an-object) — "Three ways to hand over an object". ### 2.10 Value-downstream enforcement (transitive value-only field restriction) @@ -500,13 +549,15 @@ A single global free stack and frontier require synchronization under concurrent | Value type | Mutable in place through a borrowed `mut` receiver; storage may also be overwritten freely | | `&` (guest) | Guest-only non-hosting storage; stores one tether, may be repointed, copied by value, and returned, but can never directly host a `T` | | Host-capable guest state | After rehosting, the old hosted bytes cease to be live and a slot declared as `T` stores the terminal tether as a guest while retaining enough storage to host another `T` later | -| Place expression | Existing stable storage: a named symbol, a field access of a place, a place-projection subscript of a place, or an `&` parameter | -| New `&` value | May be initialized only from a named symbol, a field access of a place, or an `&` parameter; temporaries and `[]` expressions are rejected | -| `&` parameter | Declares that the caller must supply an `&`-creating source; the parameter is place-like inside the callee | -| Borrow | Non-hosting, non-escaping access to a caller's storage for the duration of a call; the passing mode for value types; no anchor, not storable, not returnable | -| Value-type parameter | A read-only borrow; caller need not supply a place; copied only when bound into a fresh slot (assignment, declaration, field or return store) | -| Reference-type parameter | Plain `T` swallows (hosting access; passing a host downgrades the caller's symbol to a guest whatever the body does — see [`lifetimes.md`](lifetimes.md) §1.8); `&T` is a guest the caller lends while remaining host (may be stored into `&` storage or returned) | -| Reference-type `mut` receiver | `this` is an implicit `&` reference, never swallowed; composes with `&T` parameters | +| Place expression | Existing stable storage: a named symbol, a field access of a place, a place-projection subscript of a place, or an `&`/`'` parameter | +| New `&` value | May be minted only from a field access of a place or an `&` parameter; bare symbols, `[]` expressions, and temporaries are rejected | +| Guest source restriction | A bare symbol is a place but never a guest source (§2.8.1); a guest to a local's own hosting slot cannot be written, so overwriting that slot leaves no guest behind | +| `&` parameter | Declares that the caller must supply a guest source; the parameter is place-like inside the callee and may be stored or returned | +| Borrow | Non-hosting, non-escaping access to a caller's storage for the duration of a call; no anchor, not storable, not returnable, not a move-source | +| Value-type parameter | Always a read-only borrow; caller need not supply a place; copied only when bound into a fresh slot (assignment, declaration, field or return store) | +| Reference-type parameter | `T` swallows (hosting access; passing a host downgrades the caller's symbol to a guest whatever the body does — see [`lifetimes.md`](lifetimes.md) §1.8); `&T` takes a guest, which only a guest source can supply; `'T` borrows any place, bare symbols included, and leaves the caller a full host | +| `'T` position | Parameter positions only; never a storage, field, or return type | +| Reference-type receiver | `this T` is an implicit `'T` borrow; `this &T` is a guest receiver a method may store or return | | Value-downstream enforcement | Value types may contain only primitives and other value types, transitively — never a reference (`#`) or `&` field | | `&` targets reference types | An `&T` requires `T` to be a reference type; a value is shared by copy or scoped borrow, never by a stored `&` | | Symbol declaration | Must be directly initialized | diff --git a/spec/syntax.md b/spec/syntax.md index 3782e61..fc89540 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -145,14 +145,27 @@ TypeName ```zane &TypeName +'TypeName ``` -`&TypeName` is legal in storage sites (local-variable declarations, fields, and nested storage types such as the example below), as well as in function and constructor parameter positions and return-type positions. +`&TypeName` is a **guest** type. It is legal in storage sites (local-variable declarations, fields, and nested storage types such as the example below), as well as in function and constructor parameter positions and return-type positions. ```zane Array<&Node, n> ``` +`'TypeName` is a **borrow** type. It is legal in **parameter positions only** — including the `this` position — and never as a storage, field, element, or return type. + +```zane +Float topSpeed(engine 'Engine) => engine.speed + +held 'Engine = ... // ILLEGAL: a borrow is not storage +'Engine makeEngine() // ILLEGAL: a borrow is not a return type +Array<'Node, n> // ILLEGAL: a borrow is not an element type +``` + +`&` and `'` are mutually exclusive on one type: `&'Node` and `'&Node` are not type forms. See [`memory.md`](memory.md) §2.9 for the semantics of the three passing modes. + ### 2.4 Type expressions A type expression applies arguments to a parameterized type with `<>`. Arguments are positional. @@ -242,11 +255,17 @@ ReturnType?AbortType[this ReceiverType, ParamType, ...] mut The abort type stays attached to the return type, exactly as in a declaration's `ReturnType?AbortType name(...)` header. -Reference-typed parameters and returns use the ordinary type form: +Reference-typed parameters and returns use the ordinary type form. A parameter slot accepts all three passing modes — `ParamType`, `&ParamType`, and `'ParamType` — while a return slot accepts a bare or `&` type only (§2.3): ```zane ReturnType[&ParamType, ...] -&ReturnType[this ReceiverType, &ParamType, ...] +ReturnType['ParamType, ...] +&ReturnType[this &ReceiverType, &ParamType, ...] +ReturnType[this 'ReceiverType, 'ParamType, ...] mut +``` + +```zane +'ReturnType[ParamType] // ILLEGAL: a borrow is not a return type ``` `mut` is legal only when the first parameter is `this`. @@ -276,14 +295,18 @@ type Tree = #variant { leaf Int; node &Tree; } // reference sum type ```zane ReturnType name(param ParamType, ...) { body } ReturnType name(param &ParamType, ...) { body } +ReturnType name(param 'ParamType, ...) { body } ReturnType?AbortType name(param ParamType, ...) { body } ReturnType name(param ParamType, ...) => expr ReturnType name(param &ParamType, ...) => expr +ReturnType name(param 'ParamType, ...) => expr ReturnType?AbortType name(param ParamType, ...) => expr ReturnType name(param T Type, ...) { body } ReturnType name(param Container, ...) { body } ``` +Each parameter independently selects one of the three passing modes (see [`memory.md`](memory.md) §2.9): bare `ParamType` swallows, `&ParamType` takes a guest, `'ParamType` borrows. + A function, method, or constructor has no `<>` parameter header. It introduces a type or number parameter inline within its value parameters, at the parameter's first **marked** occurrence — on a value parameter's type (`param T Type`) or inside a value parameter's nested type (`param Container`) — and references it bare elsewhere, including in positions written earlier such as the return type. Inline parameters are inferred from the value arguments at the call; the same `Type` / `Number` concepts are used as in a type definition's header (§2.5). See [`generics.md`](generics.md) §3 and §5. ### 3.2 Methods @@ -302,10 +325,14 @@ ReturnType name(this ReceiverType, param &ParamType, ...) mut => expr ReturnType?AbortType name(this ReceiverType, param ParamType, ...) => expr ReturnType?AbortType name(this ReceiverType, param ParamType, ...) mut => expr ReturnType name(this ReceiverType, param ParamType, ...) { body } +ReturnType name(this &ReceiverType, param ParamType, ...) { body } +ReturnType name(this 'ReceiverType, param ParamType, ...) { body } ``` `this` is legal only in the first parameter position. A declaration is a method if and only if its first parameter is named `this`. +The receiver takes a passing mode like any other parameter, and every combination above may be written with `&` or `'` on `ReceiverType`. For a reference receiver, bare `this ReceiverType` means `this 'ReceiverType` — the borrow is the default — and `this &ReceiverType` is written when the method stores or returns the receiver as a guest. See [`functions.md`](functions.md) §2.4. + `=> expr` returns `expr`, including when `expr` has type `Unit`. ### 3.3 Positional constructors @@ -425,6 +452,8 @@ A lambda literal is a function declaration with the name removed. It writes its ```zane ReturnType() { body } ReturnType(param ParamType, ...) { body } +ReturnType(param &ParamType, ...) { body } +ReturnType(param 'ParamType, ...) { body } ReturnType() => expr ReturnType(param ParamType, ...) => expr ReturnType?AbortType(param ParamType, ...) { body } @@ -436,7 +465,7 @@ ReturnType(this ReceiverType, param ParamType, ...) => expr ReturnType(this ReceiverType, param ParamType, ...) mut => expr ``` -A lambda literal omits only the function name. `this` is legal only in the first parameter position. `mut` is legal only when the first parameter is `this`. +A lambda literal omits only the function name. `this` is legal only in the first parameter position. `mut` is legal only when the first parameter is `this`. Parameters and the receiver carry the same three passing modes as a named verb (§3.1–§3.2). Examples: diff --git a/spec/types.md b/spec/types.md index b9c9bf7..ce09c02 100644 --- a/spec/types.md +++ b/spec/types.md @@ -281,7 +281,7 @@ Every field of the target type **MUST** be assigned exactly once, either explici Constructors are not methods. They create new values rather than mutating an existing receiver, so `mut` does not apply. ### 3.9 `&` fields require `&` constructor parameters -An `&` field is legal only in a reference type (`#struct`/`#variant`), since a value type is transitively value (§2.2). A constructor that assigns a value to an `&` field must declare the corresponding parameter as `&T`. The caller must then supply a source that may create a new `&` under [`memory.md`](memory.md) §2.8 — not a temporary or `[]` expression. +An `&` field is legal only in a reference type (`#struct`/`#variant`), since a value type is transitively value (§2.2). A constructor that assigns a value to an `&` field must declare the corresponding parameter as `&T` — a `'T` borrow will not do, because a borrow ends with the call while the field outlives it. The caller must then supply a **guest source** under [`memory.md`](memory.md) §2.8: a field access on a place, or an `&T` parameter. A bare symbol, a temporary, and a `[]` expression are all rejected. ```zane package Vehicle @@ -306,14 +306,18 @@ Car(engine Engine) { Call sites: ```zane -engine Engine() -car Car(engine) // legal: engine may create a new `&` +garage Garage() +car Car(garage.spare) // legal: a field access is a guest source ``` ```zane -car Car(Engine()) // ILLEGAL: temporary cannot initialize an `&` field +engine Engine() +car Car(engine) // ILLEGAL: a bare symbol is not a guest source +car Car(Engine()) // ILLEGAL: a temporary cannot initialize an `&` field ``` +The object an `&` field points at therefore has to be hosted somewhere that outlives the bare local — in another object's field, most often. See [`adt.md`](adt.md) §4.1 for the same requirement seen from a recursive type's side. + A reference type whose fields are all plain hosts does not require `&` parameters: ```zane From 00de7e8c6ac9b1651ac87113f270314999f764fe Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 21:18:14 +0000 Subject: [PATCH 17/31] docs: story chapters for the guest-source restriction and the three modes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit stories/memory.md gains "The slot that could not be pointed at" — the five-line aliasing program, the roads tried in order (smarter guests, the name/location/object middle layer, immovable objects, no guests at all) and why each failed, and the narrowing that finally settled it. Then "Three ways to hand over an object": why an `&T`-parameter exemption would have needed interprocedural inference, how the borrow fell out of that, why the receiver became a borrow too, and the defence of `'` over redefining `&`. stories/lifetimes.md gains "Where a guest may be rooted" — what the source rule did to the scope check and the return rule, and why a field reached through a borrow is not a guest source either. Spec sections point into the new chapters; in-prose references are pinned to b10eaed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- README.md | 4 ++-- spec/adt.md | 2 ++ spec/functions.md | 1 + spec/lifetimes.md | 3 +++ spec/types.md | 2 ++ stories/lifetimes.md | 14 ++++++++++++++ stories/memory.md | 43 +++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 67 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 39e82bf..d966e61 100644 --- a/README.md +++ b/README.md @@ -66,8 +66,8 @@ The spec states *what* the language does; the **why** lives in a parallel set of | [`stories/adt.md`](stories/adt.md) | [`spec/adt.md`](spec/adt.md) — splitting `enum` from `variant` against the hype, the shared struct body, escaping the matcher machine with case overloads and the turn to a central `match` block, matching variants rather than patterns, keeping enum data outside the members, reducing a match group to sugar for one arm per case, and building a variant by naming a case rather than calling a constructor | | [`stories/generics.md`](stories/generics.md) | [`spec/generics.md`](spec/generics.md) — the parameter model, the `<>`/`()` split, size-in-the-type, and the deferred features | | [`stories/dependencies.md`](stories/dependencies.md) | [`spec/dependencies.md`](spec/dependencies.md) — URL identity, the manifest/resolution split, prebuilt distribution, symbol-rewriting, the browsable global cache, the package-graph acyclicity rule, opt-in remapping, and why `core` became a bundled implementation package | -| [`stories/memory.md`](stories/memory.md) | [`spec/memory.md`](spec/memory.md) — the no-GC-no-lifetimes goal, the move problem and the anchor, lazy backpointer creation, the indexed heap table, the rooted-guest rules and the host/guest terminology split, the collapse to one value/reference axis with a borrowed receiver, the shift to segmented chunked bump arenas, and the split into fixed-size and dynamic regions with anchors moved to a runtime-global recyclable pool | -| [`stories/lifetimes.md`](stories/lifetimes.md) | [`spec/lifetimes.md`](spec/lifetimes.md) — lexical scope in place of a borrow checker, what may be moved, the declaration-block rule that kills flow analysis, downgrade instead of use-after-move, parameter-rooted returned guests, and why each strict rule is the minimal guard against one specific memory corruption | +| [`stories/memory.md`](stories/memory.md) | [`spec/memory.md`](spec/memory.md) — the no-GC-no-lifetimes goal, the move problem and the anchor, lazy backpointer creation, the indexed heap table, the rooted-guest rules and the host/guest terminology split, the collapse to one value/reference axis with a borrowed receiver, the shift to segmented chunked bump arenas, the split into fixed-size and dynamic regions with anchors moved to a runtime-global recyclable pool, taking the bare symbol away as a guest source, and the three passing modes that split out of it | +| [`stories/lifetimes.md`](stories/lifetimes.md) | [`spec/lifetimes.md`](spec/lifetimes.md) — lexical scope in place of a borrow checker, what may be moved, the declaration-block rule that kills flow analysis, downgrade instead of use-after-move, parameter-rooted returned guests, why each strict rule is the minimal guard against one specific memory corruption, and narrowing a returned guest's root to a guest parameter once borrows arrived | | [`stories/effects.md`](stories/effects.md) | [`spec/effects.md`](spec/effects.md) — inferring effects instead of annotating them, receiver-scoped `mut`, capabilities in place of ambient I/O, the four-level ladder and the Total-Pure/Pure split, what deliberately is not an effect, and mutation through a borrowed receiver | | [`stories/concurrency.md`](stories/concurrency.md) | [`spec/concurrency.md`](spec/concurrency.md) — the parallelism/concurrency split and the refusal of `async` coloring, why `spawn` marks only a call, water-tower lifetimes, signature-based safety without locks, and value-typed mutation closing the aliased-write gap | | [`stories/error-handling.md`](stories/error-handling.md) | [`spec/error-handling.md`](spec/error-handling.md) — the two-doors model and why failure is control flow rather than a `Result` value, `resolve` as expression-substitution rather than assignment, typed abort paths and the deliberately-absent propagate operator, keeping abortability orthogonal to effects, and explicit path values through `Unit` | diff --git a/spec/adt.md b/spec/adt.md index 07e0863..8914aeb 100644 --- a/spec/adt.md +++ b/spec/adt.md @@ -155,6 +155,8 @@ Expr negate(inner &Expr) => Expr.flip(inner) This is the same requirement an `&` field of any `#struct` carries; recursion is not a special case. What it means in practice is that the *root* of a recursive structure lives in a field of the type that owns the structure — which is where a root belongs anyway, since that field is what keeps the whole shape alive. +> **Story:** [`stories/memory.md`](../stories/memory.md#the-slot-that-could-not-be-pointed-at) — "The slot that could not be pointed at". + > **Story:** [`stories/adt.md`](../stories/adt.md#one-body-product-or-sum) — "One body, product or sum". --- diff --git a/spec/functions.md b/spec/functions.md index 72a67da..4218252 100644 --- a/spec/functions.md +++ b/spec/functions.md @@ -91,6 +91,7 @@ node!setScale(Float(3)) Calling a `mut` method with `:` is illegal. Calling a non-`mut` method with `!` is also illegal. > **Story:** [`stories/functions.md`](../stories/functions.md#mutation-you-can-see-at-the-call-site) — "Mutation you can see at the call site". +> **Story:** [`stories/memory.md`](../stories/memory.md#three-ways-to-hand-over-an-object) — "Three ways to hand over an object". ### 2.6 Method desugaring diff --git a/spec/lifetimes.md b/spec/lifetimes.md index b98c82b..06de855 100644 --- a/spec/lifetimes.md +++ b/spec/lifetimes.md @@ -30,6 +30,7 @@ r &Node = node // ILLEGAL: a bare symbol is not a guest source The compiler compares declaration scopes. It does not perform borrow inference or lifetime annotation solving. > **Story:** [`stories/lifetimes.md`](../stories/lifetimes.md#inheriting-a-debt-safety-without-a-borrow-checker) — "Inheriting a debt: safety without a borrow checker". +> **Story:** [`stories/lifetimes.md`](../stories/lifetimes.md#where-a-guest-may-be-rooted) — "Where a guest may be rooted". ### 1.2 Move-sources are host symbols or hosting verb results A move-source must denote a **hosting value the expression is entitled to consume**. Two forms qualify: @@ -166,6 +167,7 @@ The other two parameter modes are not roots. A `'T` borrow ends with the call, s ``` > **Story:** [`stories/lifetimes.md`](../stories/lifetimes.md#returning-a-ref-without-a-lifetime-to-name-it) — "Returning a ref without a lifetime to name it". +> **Story:** [`stories/lifetimes.md`](../stories/lifetimes.md#where-a-guest-may-be-rooted) — "Where a guest may be rooted". ### 1.8 Passing a host to a `T` parameter downgrades it to a guest A plain reference-type parameter `T` takes its argument by **hosting access**. Passing a hosting value to such a parameter uses that value as a move-source (§1.2), so the caller's symbol downgrades to a guest (§1.6) — **whatever the callee does with the value**. The parameter's declared type is the whole contract: `T` means the caller gives up hosting; `&T` and `'T` ([`memory.md`](memory.md) §2.9) both mean the caller stays a full host. Nothing in the callee's body changes the outcome the signature already states. @@ -214,6 +216,7 @@ Unit main() { A verb that only reads its reference argument may still declare it plain `T`: reading does not change the fact that the signature asked for hosting access, so the caller downgrades all the same. Declaring the parameter `'T` — or `&T`, when the callee needs to keep it — is what keeps the caller as host. Because the signature alone decides the caller's state, there is no interprocedural consumption inference: whether a passed host downgrades never depends on the callee's body or on the build. Using hosting access only to read a value is legal. Leaving a parameter entirely unused is a separate, general matter — a release build rejects an unused parameter whether it hosts a value or not. > **Story:** [`stories/lifetimes.md`](../stories/lifetimes.md#the-signature-is-the-whole-contract-retiring-inferred-consumption) — "The signature is the whole contract: retiring inferred consumption". +> **Story:** [`stories/memory.md`](../stories/memory.md#three-ways-to-hand-over-an-object) — "Three ways to hand over an object". ### 1.9 An ignored hosting result floats to the enclosing scope A return value need not be bound. When a call's result is a reference-type host and the call stands as a bare statement, that host is not destroyed at the end of the statement — it **floats**: it becomes an anonymous host in the enclosing scope and lives until that scope drains, like any object hosted by that scope (§2.1). An ignored value-type result, including `Unit()`, is simply discarded. diff --git a/spec/types.md b/spec/types.md index ce09c02..94196fc 100644 --- a/spec/types.md +++ b/spec/types.md @@ -318,6 +318,8 @@ car Car(Engine()) // ILLEGAL: a temporary cannot initialize an `&` field The object an `&` field points at therefore has to be hosted somewhere that outlives the bare local — in another object's field, most often. See [`adt.md`](adt.md) §4.1 for the same requirement seen from a recursive type's side. +> **Story:** [`stories/memory.md`](../stories/memory.md#the-slot-that-could-not-be-pointed-at) — "The slot that could not be pointed at". + A reference type whose fields are all plain hosts does not require `&` parameters: ```zane diff --git a/stories/lifetimes.md b/stories/lifetimes.md index 62bc10f..d6b561d 100644 --- a/stories/lifetimes.md +++ b/stories/lifetimes.md @@ -169,3 +169,17 @@ Move the engine out and `car` is left hosting a gap the type system still swears Laid side by side, the five stop looking like a taste for blunt rules and start looking like what they are: five different doors onto the same room, each the minimal lock on one specific way a host and its value come apart — a guest outliving its host (§1.1, §1.7), a value sinking below a guest that still tracks it (§1.4), a value consumed on some paths but not others (§1.3), a value stolen from a host that still counts it (§1.2). None substitutes for another; each closes a gap the others leave open. That is the answer to the reader who finds them rigid and wonders where the slack is: there is none to give, because loosening any one is not a gentler version of the same safety but a specific, nameable crash let back in. This is [restriction as information](foundations.md#restriction-is-information-and-the-test-of-a-good-one) at the level of a single document — every rule that forbids a program is carrying a fact the compiler would otherwise have to prove — and [that the readable rule and the fast language coincide](foundations.md#strictness-is-the-performance-model) is the standing bet, not a coincidence. The cost of buying safety this way is real, and it is not the false rejections — those the first chapter already owned. It is that the safety arrives as a *list* rather than a principle. A borrow checker derives every one of these cases from a single notion, a lifetime outliving a borrow; learn that idea and you hold all of it at once. Zane asks its reader to carry five separate rules instead, and nothing but a chapter like this one tells them the five are a set with no member removable. We think that is the right trade — five rules you can each check by eye against the code in front of you beat one proof you must trust a solver to have carried — but it is a genuine trade, and the chapter that pretends the rules are self-evidently a system, rather than a hard-won set each earning its place, is the chapter we had before this one. + +## Where a guest may be rooted + +The five rules of the previous chapter were each a lock on one way a host and its value come apart. Narrowing the guest source, over in [the memory story](memory.md#the-slot-that-could-not-be-pointed-at), turned out to change what two of those locks are guarding, and the adjustment is small enough to state precisely and worth stating because the reasoning is easy to get backwards. + +The scope check ([§1.1](https://github.com/zane-lang/spec/blob/b10eaed/spec/lifetimes.md#11--assignment-uses-host-scope)) did not change at all — it still asks only whether the target's host is declared in the same or a higher lexical scope than the guest — but it now sits behind a prior question, because a source that is not a guest source never reaches the scope check in the first place. The visible effect is that the canonical illegal example is no longer the interesting one. `r = innerNode`, rejected for scope, used to be the rule's whole face; now the more common rejection is `r &Node = node`, refused before any scope is compared, and the example that still exercises the scope rule has to reach through a field to get there. That is a small loss of pedagogical clarity in exchange for a large one of exposure: the rule that used to be the only thing standing between a program and a dangling guest is now the second line of defence. + +The rule that did change is the one governing returns ([§1.7](https://github.com/zane-lang/spec/blob/b10eaed/spec/lifetimes.md#17-returned--values-must-be-rooted-in-a-guest-parameter)). It used to say that a returned `&T` must be rooted in *a parameter*, which was the right rule when there was only one kind of reference parameter to be rooted in. With three, "a parameter" is no longer specific enough, and each of the other two fails for its own reason. A swallowing `T` parameter is a bare symbol in the call-site scope, and a bare symbol is not a guest source — so the returned guest could not have been minted in the first place. A `'T` borrow fails harder: the borrow ends when the call does, and a guest rooted in one would outlive the very access it was derived from. So the rule now names the guest parameter specifically. What makes this feel right rather than merely tighter is that the three modes each answer the question the rule is really asking — *may this outlive the call?* — and only one of them answers yes. + +The same reasoning had to be pushed one step further than the rule's own text, and this is the part that is easy to miss. Refusing to return a `'T` parameter is pointless if you may instead return a guest minted from one of its **fields**: `this.weapon` on a borrowed receiver would escape just as surely as `this` would, wrapped in one layer of indirection. So a field access rooted in a borrow is not a guest source either. We debated allowing it — the caller's object does outlive the call, so the guest would in fact be live — and rejected it on the grounds that "in fact live" is not the standard. The compiler would have to reason about the relative scopes of two objects across a call boundary to know it, which is the interprocedural analysis this whole document exists to avoid. A borrow that does not escape, with no exceptions and nothing to check, is worth more than a borrow that escapes safely under an argument only the compiler can follow. + +Nothing else moved. The downgrade rule ([§1.6](https://github.com/zane-lang/spec/blob/b10eaed/spec/lifetimes.md#16-moved-symbols-downgrade-to--values-and-are-no-longer-movable)) still turns a moved-from symbol into a readable guest, and it is worth being clear that this is not in tension with the new source rule: the downgrade is something the language does to a slot, not a guest a program mints, and the reader never writes it. The signature-is-the-whole-contract rule ([§1.8](https://github.com/zane-lang/spec/blob/b10eaed/spec/lifetimes.md#18-passing-a-host-to-a-t-parameter-downgrades-it-to-a-guest)) simply grew a fourth entry — a verb may now *borrow* an argument as well as take a guest, relay, or consume — and the entry it grew is, satisfyingly, the one that leaves the caller in the strongest position: still the host, with the callee unable to keep anything. + +The honest cost here is not a rejected program but a redistribution of where the reader's attention has to go. Before, a reference parameter's mode was visible in one bit — `&` or not — and the question "what happens to my object" had two answers. Now there are three signatures to read and three answers, and the difference between two of them (`&T` and `'T`) is invisible at the call site by design, because the whole point is that the call site should not have to care. The programmer who wants to know whether their host survives a call still reads exactly one thing, the signature; there is just more in it than there used to be. diff --git a/stories/memory.md b/stories/memory.md index 7d91da8..9e7f116 100644 --- a/stories/memory.md +++ b/stories/memory.md @@ -146,3 +146,46 @@ The source pair is now **host** and **guest**. A host is the symbol, field, or c The runtime keeps **anchor** and **tether**. Each guest is represented by a tether that resolves through an anchor; moving or rehosting the object updates the anchor, so existing tethers keep working. That vocabulary remains a natural mechanical picture, but it no longer leaks upward into source semantics. The concise model is: *an object lives in a host; a guest may access it; internally, the guest's tether follows the object through its anchor.* The alternatives each blurred something we wanted to keep sharp. **Owner/tether** named the two halves accurately in isolation but paired source semantics with implementation. **Owner/guest** worked, though “owner” stressed rights and destruction more than residence. **Owner/view** was technically reasonable without being a convincing lived relationship. Proxy, keyholder, delegate, and licensee were variously technical, overloaded, or indirect; “key” also collided with dictionary keys. CC/email language suggested secondary participation, but a CC recipient receives an independent copy rather than live access to one moving object. And keeping **tether** as the name of `&T` remained expressive, but preserved the very overload this split was meant to remove. + +## The slot that could not be pointed at + +The whole of the preceding machinery — hosts, guests, anchors, forwarding cells, retirement stacks — was built to answer one question, and it took a five-line program to show that we had been answering the wrong one: + +```zane +main Player() +second Player() +guest &Player = main +second = main +``` + +Line 4 moves the object out of `main`'s slot and into `second`'s. Line 3 had already handed out a guest. So what does `guest` denote afterwards? The anchor system gives an answer — it follows the object, because that is exactly what anchors are for — and the answer is defensible. But it is not *obviously* right. A reader who wrote `&Player = main` may well have meant "watch that variable," in which case following the object is wrong; a reader who meant "watch that player" is served correctly. The spelling does not distinguish them, and the model had quietly picked one reading and made the other unsayable. Worse, the reader who wanted the other reading had no way to find out except by discovering that their program did something they did not expect. That is the shape of a design flaw, not a documentation gap. + +The first instinct was to make the machinery smarter. If a guest tracked the *name* rather than the object, `guest` would keep denoting `main`'s slot; but `main`'s slot is now empty in every sense that matters, so a guest to it is a guest to nothing and we are back to needing a null state we had spent the whole model avoiding. If it tracked the object, we had what we already had. So we tried a third layer: name → location → object, with the location as a stable middle that either end could be re-pointed at independently. That is a genuine idea and it is also, on inspection, the anchor system with an extra name — it moves the ambiguity from "which does a guest follow" to "which does a location follow," and it charges another indirection for the privilege. We dropped it. + +Then we tried the direction that looks decisive: make the object immovable. If nothing relocates, the question cannot be asked. We built that model out in full and it failed twice over, in ways worth recording because they are not obvious from the outside. It could not express a move whose destination is decided at runtime — `if someIO() { boat.bottom!append(car) } else { boat.top!append(car) }` — and its answer was to forbid the program rather than to place the car. And `append` moves into a list's backing store, which relocates when the list grows; the model was claiming immovability in one section and admitting relocation in another. Anchors exist precisely so that objects *can* move. Removing them to buy an invariant the design cannot actually hold was the wrong turn, and we took it far enough to be sure. + +The opposite extreme — no guests at all, so nothing can be left pointing anywhere — fails for a duller reason: a function that only wants to read an object would have to swallow it, and a language where reading costs you ownership is not one anyone would enjoy writing. + +What finally resolved it was narrowing the question instead of answering it. The trouble is not that guests exist, and not that objects move; it is that a **bare symbol's own hosting slot** is both the thing you may most freely overwrite and, until now, a thing you could point at. Those two properties are what generate the ambiguity, and only one of them is load-bearing. So [`memory.md` §2.8.1](https://github.com/zane-lang/spec/blob/b10eaed/spec/memory.md#281-a-bare-symbol-is-not-a-guest-source) removes the other: a bare symbol stays a place expression — you may read it, mutate it, move from it, borrow it — but it is no longer a **guest source**. A new `&` is minted from a field access or from an `&T` parameter, and from nothing else. Line 3 of the program above is now a compile error, and it is an error at the line that creates the problem rather than a puzzle at the line that reveals it. + +The reason this costs so little is worth stating plainly, because it is the argument that decided it: **a guest to a bare symbol never buys any reach.** A guest exists to let storage that does not own an object nevertheless get at it — a field in another object, an element of a container, a callee's parameter. But a bare symbol is, by construction, already in scope everywhere a guest to it could be declared; you can simply use the symbol. The only thing the removed source was genuinely doing was carrying an object into a call, and that job now belongs to [the borrow mode](#three-ways-to-hand-over-an-object). + +Fields are a different matter, and they stay. A field belongs to an object whose lifetime the host system already tracks, so a guest to `car.engine` has something meaningful to follow when the field is overwritten or the containing object is rehosted — and follow it does, through the anchor path exactly as before. That asymmetry is the whole content of the rule: it constrains where a guest may *come from*, and changes nothing about what a guest can *survive*. The anchor system is untouched. + +The cost is real and lands in one place: a structure that needs guests must be **rooted in a field** rather than in a bare local. A recursive `#variant` boxes its recursive case through `&`, so building one now starts from a node hosted in a field — `Expr.flip(tree.root)` rather than `Expr.flip(leaf)` — as [`adt.md` §4.1](https://github.com/zane-lang/spec/blob/b10eaed/spec/adt.md#41-a-recursive-structure-is-rooted-in-a-field) spells out. We spent some time deciding whether that was a defect. It is not, quite: the root of a recursive structure is what keeps the whole shape alive, and a field of the type that owns the structure is where such a thing belongs anyway. But it *is* a constraint the previous design did not impose, and the programmer who reaches for a bare local to hold their first node will meet it as a surprise before they meet it as a principle. That is the honest ledger: one surprising rejection at the root of a tree, bought with the disappearance of an entire class of question about what a guest means after a move. + +## Three ways to hand over an object + +Taking away the bare-symbol guest source left a hole immediately, and it is the most ordinary hole imaginable: `topSpeed(engine)`, where `engine` is a local and the function only wants to read it. Under the old model that parameter was `&Engine` and the argument was a bare symbol — the exact pairing now forbidden. Swallowing it instead is absurd for a read. So the rule as stated made a common program unwritable, and needed a companion. + +We considered giving `&T` parameters a special exemption: a bare symbol may not initialize `&` *storage*, but may still feed an `&T` *parameter*. That is nearly right, and it is where the thinking sat for a while. What sank it is that an `&T` parameter is not merely a way in — it is a guest, and a callee may store it in a field or return it. Exempting the argument would mean a bare symbol's slot could still end up with something pointing at it, arriving by the one route the rule did not check. The exemption would have to be conditional on what the callee does with the parameter, which is interprocedural inference — the thing [`lifetimes.md` §1.8](https://github.com/zane-lang/spec/blob/b10eaed/spec/lifetimes.md#18-passing-a-host-to-a-t-parameter-downgrades-it-to-a-guest) was written specifically to retire. + +So the passing mode had to split. A parameter that merely reads or mutates the caller's object for the duration of the call is a genuinely different contract from one that keeps a guest past the call, and the old `&T` was carrying both. Separating them gives the three modes in [`memory.md` §2.9](https://github.com/zane-lang/spec/blob/b10eaed/spec/memory.md#29-function-parameters-swallow-guest-and-borrow): `T` **swallows**, taking hosting access and downgrading the caller; `&T` takes a **guest**, which the callee may store or return and which only a guest source can supply; `'T` **borrows**, accepting any place at all — bare symbols included — and granting read and `mut` access that expires with the call. The borrow was not a new concept: value types had always been passed exactly this way, and the reference world had simply never been given the same option. + +That third mode turned out to pay for itself immediately in a place we had not been aiming at. A reference-type receiver had been an implicit guest, which under the new source rule would have made `node!setScale(...)` illegal on a bare local — an absurdity. But a receiver almost never needs to be kept; it needs to be read and written for the duration of the call. So a bare `this T` on a reference type is now an implicit `'T` borrow ([`functions.md` §2.4](https://github.com/zane-lang/spec/blob/b10eaed/spec/functions.md#24-mutating-methods-use-mut)), and `this &T` is what a method writes in the rarer case where it stores or returns the receiver. The pleasing part is that this makes the two type worlds agree: a `mut` receiver is a mutable borrow of the caller's slot whether the type is a value or a reference, and the special-casing that used to sit in that sentence is gone. + +Naming the mode took longer than designing it. The tempting move was to hand the constrained meaning to the bare `&` and mark the escaping one, on the general principle that the marked form should be the restricted form. That principle does not apply here, and noticing why was the turn: **both** forms are marked. The unmarked form is `T`, the swallow. Between `&` and a new sigil there is no asymmetry of markedness to appeal to, so the argument has to be made on continuity instead — and there `&` has a large incumbent claim. It means *guest* in a field type, in a storage declaration, in a return type, in the glossary, and across every chapter above this one. Redefining it in the parameter position alone would make the same character mean two things depending on where it sits, which is precisely the kind of context-dependence [the two-vocabulary chapter](#two-vocabularies-host-and-guest-above-anchor-and-tether) had just finished removing from `tether`. So `&` keeps meaning guest everywhere, and the new concept takes the new mark. + +`'` won it on two counts. The character was unused in Zane's lexis — no character literals, no operator, no identifier start — so it cost nothing to reserve ([`lexical.md` §4.3](https://github.com/zane-lang/spec/blob/b10eaed/spec/lexical.md#43-reserved-sigils)). And its existing association in the reader's mind is with Rust's lifetimes, which is closer to right than to wrong: a Rust lifetime annotates precisely the thing a Zane borrow *is* — access valid for a bounded duration and no longer. The difference is that Zane's version has nothing to name, because the duration is always the call. Borrowing the character while dropping the parameter it usually carries is, if anything, a decent summary of what the language does with lifetimes generally. We weighed a keyword — `borrow Engine` — and rejected it as too heavy for something that appears on most parameters; and we weighed reusing `#`, which is taken by the reference mould and would have been genuinely confusing. + +The cost of three modes is that there are three of them. A programmer now chooses a passing mode per parameter, and the choice is not always forced: a parameter that only reads could be written `'T` or `&T`, and only the question "does the callee keep it?" separates them. We do not think this is a real burden — the modes correspond to three things people already distinguish when they think about a function's contract, and the signature now states which one applies rather than leaving it to the body — but it is one more axis in every signature, and it is fair to count it. Against that, one thing genuinely got simpler: an overload set may not differ only by the passing mode ([`functions.md` §4.1](https://github.com/zane-lang/spec/blob/b10eaed/spec/functions.md#41-overload-identity-is-parameter-types-only)), because the mode changes what the caller must supply and what state the call leaves them in, with nothing at the call site to tell two such overloads apart. That rule already existed for `T` versus `&T`; widening it to all three modes cost nothing and closed the door before anyone tried it. From a9b2a8b5c34d4a0bc478b059bc767941e350b9be Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 21:27:24 +0000 Subject: [PATCH 18/31] docs: address CodeRabbit review on #151 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - CLAUDE.md: the guest-source guard was end-anchored, so it matched no line carrying a `// ILLEGAL: ...` comment — which is every deliberate bad example in the spec. It now tolerates an inline comment and trailing whitespace, and catches all three (memory.md 150/159, lifetimes.md 27) instead of none. - glossary.md §3.1, §3.33, §3.36: mirror memory.md §2.8's exclusion of field accesses whose base chain passes through a `'T` borrow. Without it the glossary made `borrowed.field` a legal guest source while the canonical rule rejected it. - syntax.md §3.1, §3.2: scope the three passing modes to reference types. A value parameter is always a read-only borrow and a value receiver is borrowed, so neither selects a mode; the unqualified wording contradicted effects.md §2. - memory.md §3.1: tag the segmented-offset diagram fence `text`. It was the only untagged opening fence in spec/. - memory.md §4.8: stop promising unchanged reuse order under thread-local anchor caches. Which free slot an allocation receives is unobservable from the source language, so §3.2's LIFO describes the central pool rather than binding an implementation; what a cache may not change is identity uniqueness, the §4.6 retirement events, and the lifetime guarantees resting on them. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- CLAUDE.md | 12 ++++++++---- spec/glossary.md | 6 +++--- spec/memory.md | 4 ++-- spec/syntax.md | 4 ++-- 4 files changed, 15 insertions(+), 11 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 80c34b2..69ae910 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -59,13 +59,17 @@ A second guard covers the memory model. A **bare symbol is not a guest source** bug. Eyeball every hit of: ```sh -grep -RIn -E "&[A-Z][A-Za-z0-9]* *= *[a-z][A-Za-z0-9]*$" spec/ +grep -RIn -E "&[A-Z][A-Za-z0-9]* *= *[a-z][A-Za-z0-9]* *(//.*)?[[:space:]]*$" spec/ ``` Every surviving hit must be a field access (`= car.engine`) or an `&T` -parameter — never a bare local. Run these with `-R` on the directory, not a -`spec/*.md` glob plus a bare directory argument: `grep` prints -`bench/: Is a directory` and silently skips it otherwise. +parameter — never a bare local. The trailing `(//.*)?[[:space:]]*$` is what +makes the guard see the `// ILLEGAL: ...` examples; without it the end anchor +skipped every commented line, which is most of them. + +Run both with `-R` on the directory, not a `spec/*.md` glob plus a bare +directory argument: `grep` prints `bench/: Is a directory` and silently skips +it otherwise. Stories are exempt from both greps: `stories/` records the language as it was at each turn and is never rewritten to match the present spec. diff --git a/spec/glossary.md b/spec/glossary.md index 8197d97..e73c049 100644 --- a/spec/glossary.md +++ b/spec/glossary.md @@ -65,7 +65,7 @@ This file gives short, reusable names to concepts that appear across multiple sp ## 3. Types, Storage, and Binding ### 3.1 place expression -- **Meaning:** A place expression denotes an existing, stable storage location. Being a place is necessary but not sufficient to mint an `&`: only a field access of a place and an `&T` parameter are guest sources, while bare symbols and `[]` expressions are places that are excluded (§3.36). +- **Meaning:** A place expression denotes an existing, stable storage location. Being a place is necessary but not sufficient to mint an `&`: only an `&T` parameter and a field access of a place whose base chain does not pass through a `'T` borrow are guest sources, while bare symbols, `[]` expressions, and anything reached through a borrow are places that are excluded (§3.36). - **Why this name:** The term names the expressions that refer to a storage "place" rather than to a temporary value. - **Canonical home:** [`memory.md`](memory.md) §2.8 @@ -225,7 +225,7 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`memory.md`](memory.md) §2.1 ### 3.33 guest -- **Meaning:** The source-facing `&T`: access to a hosted reference-type object without storing that object or controlling its lifetime. A guest may be repointed, copied when assigned or passed, stored in an `&` field, or returned as `&T`, but it cannot outlive its host, and it may be minted only from a field or an `&T` parameter (§3.36). Internally, a guest is represented by a tether (§3.24) that resolves through an anchor cell (§3.23). +- **Meaning:** The source-facing `&T`: access to a hosted reference-type object without storing that object or controlling its lifetime. A guest may be repointed, copied when assigned or passed, stored in an `&` field, or returned as `&T`, but it cannot outlive its host, and it may be minted only from an `&T` parameter or a field access not rooted in a borrow (§3.36). Internally, a guest is represented by a tether (§3.24) that resolves through an anchor cell (§3.23). - **Why this name:** A guest may use what a host provides without owning it, and the guest's stay cannot outlast the host. The pair names the source relationship without exposing its runtime mechanism. - **Canonical home:** [`memory.md`](memory.md) §2.4 @@ -240,7 +240,7 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`lifetimes.md`](lifetimes.md) §1.8 ### 3.36 guest source restriction -- **Meaning:** A new `&` may be minted only from a field access whose base is a place, or from an `&T` parameter. A **bare symbol** — an identifier standing alone rather than as the base of a field access — is a place expression but never a guest source, so no guest can point at a local's own hosting slot and that slot stays free to be overwritten or moved from. Passing a bare symbol into a call is the borrow mode's job (§3.27). +- **Meaning:** A new `&` may be minted only from an `&T` parameter, or from a field access whose base is a place and whose base chain does not pass through a `'T` borrow parameter. A **bare symbol** — an identifier standing alone rather than as the base of a field access — is a place expression but never a guest source, so no guest can point at a local's own hosting slot and that slot stays free to be overwritten or moved from. Passing a bare symbol into a call is the borrow mode's job (§3.27). The borrow exclusion runs the same way: a guest minted from a borrowed object's field would escape the call just as surely as the borrow itself. - **Why this name:** The rule constrains the *source* of a guest — where one may come from — and nothing about what a guest can survive once minted; a guest to a field still follows its host across overwrites and rehosting. - **Canonical home:** [`memory.md`](memory.md) §2.8.1 diff --git a/spec/memory.md b/spec/memory.md index a9b9809..5267260 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -316,7 +316,7 @@ A dynamic block larger than 1 MiB is an **oversized span**: a dedicated contiguo Scope chunks and global anchor pages draw ids from the same chunk directory, so payload locations, dynamic handles, tethers, backpointers, anchor cells, and size-stack entries all use one **`u32` segmented offset**: -``` +```text u32 segmented offset ┌───────────────┬──────────────────────────┐ │ chunk id │ in-chunk word offset │ @@ -514,7 +514,7 @@ A dangling or misdirected tether would require a guest to outlive the hosted val The segmented encoding adds no meaningful arithmetic cost: the shift and mask that split a `u32` fold into machine addressing once the chunk base is loaded. A terminal tether pays one dependent anchor-cell load beyond direct host access; an older identity pays one additional load per uncompressed forwarding hop. Rehosting never enumerates guests, and path compression makes repeated traversal of a chain approach the terminal case. Physical relocation cost remains proportional to the representation moved. -A single global free stack and frontier require synchronization under concurrent allocation and teardown. Implementations may use thread-local anchor caches backed by the same global pool without changing anchor identity, reuse order semantics, or lifetime guarantees. +A single global free stack and frontier require synchronization under concurrent allocation and teardown. Implementations may use thread-local anchor caches backed by the same global pool. The LIFO discipline of §3.2 describes how the central pool behaves, not a guarantee the language makes: which free slot a given anchor allocation receives is unobservable from the source language, so a cache that hands out slots in another order — or holds a returned slot until it flushes — changes nothing a program can detect. What such a cache **MUST NOT** change is anchor identity uniqueness, the retirement events of §4.6, or the lifetime guarantees that rest on them. --- diff --git a/spec/syntax.md b/spec/syntax.md index fc89540..7ecbaa9 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -305,7 +305,7 @@ ReturnType name(param T Type, ...) { body } ReturnType name(param Container, ...) { body } ``` -Each parameter independently selects one of the three passing modes (see [`memory.md`](memory.md) §2.9): bare `ParamType` swallows, `&ParamType` takes a guest, `'ParamType` borrows. +A **reference-type** parameter independently selects one of the three passing modes (see [`memory.md`](memory.md) §2.9): bare `ParamType` swallows, `&ParamType` takes a guest, `'ParamType` borrows. A **value-type** parameter has no such choice — it is always a read-only borrow — so `&` and `'` are not written on one. A function, method, or constructor has no `<>` parameter header. It introduces a type or number parameter inline within its value parameters, at the parameter's first **marked** occurrence — on a value parameter's type (`param T Type`) or inside a value parameter's nested type (`param Container`) — and references it bare elsewhere, including in positions written earlier such as the return type. Inline parameters are inferred from the value arguments at the call; the same `Type` / `Number` concepts are used as in a type definition's header (§2.5). See [`generics.md`](generics.md) §3 and §5. @@ -331,7 +331,7 @@ ReturnType name(this 'ReceiverType, param ParamType, ...) { body } `this` is legal only in the first parameter position. A declaration is a method if and only if its first parameter is named `this`. -The receiver takes a passing mode like any other parameter, and every combination above may be written with `&` or `'` on `ReceiverType`. For a reference receiver, bare `this ReceiverType` means `this 'ReceiverType` — the borrow is the default — and `this &ReceiverType` is written when the method stores or returns the receiver as a guest. See [`functions.md`](functions.md) §2.4. +A **reference** receiver takes a passing mode like any other reference parameter, and every combination above may be written with `&` or `'` on `ReceiverType`. Bare `this ReceiverType` means `this 'ReceiverType` — the borrow is the default — and `this &ReceiverType` is written when the method stores or returns the receiver as a guest. A **value** receiver has no mode to select: it is a borrow of the caller's slot, mutable when the method is `mut`, and is always written bare. See [`functions.md`](functions.md) §2.4. `=> expr` returns `expr`, including when `expr` has type `Unit`. From fe62f1a0a00be0bead55d0acc36c3b8dd7b91f3c Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 21:35:31 +0000 Subject: [PATCH 19/31] docs: second CodeRabbit round on #151 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - memory.md §2.9: the borrow paragraph said a borrow "cannot be stored in a field, and cannot be returned" and then that a value is copied when bound into "a field or return store" — a flat contradiction. Split the two: the borrow is never storage, but binding *through* one copies the value, and the copy escapes fine. A reference type has no copy, so a `'T` leaves nothing. - glossary.md §3.33: restore the place requirement the shortened wording lost. "A field access not rooted in a borrow" would have admitted `makeCar().engine`; mirror memory.md §2.8 in full. - CLAUDE.md: the guard now accepts a leading-underscore private name (`_engine`) and tab-formatted assignments. Prose rewritten to say what the guard does — it matches a bare-symbol right-hand side only, so every hit is a candidate bug and the legal forms never appear. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- CLAUDE.md | 17 ++++++++++++----- spec/glossary.md | 2 +- spec/memory.md | 4 +++- 3 files changed, 16 insertions(+), 7 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 69ae910..aa58175 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -59,13 +59,20 @@ A second guard covers the memory model. A **bare symbol is not a guest source** bug. Eyeball every hit of: ```sh -grep -RIn -E "&[A-Z][A-Za-z0-9]* *= *[a-z][A-Za-z0-9]* *(//.*)?[[:space:]]*$" spec/ +grep -RIn -E "&[A-Z][A-Za-z0-9]*[[:space:]]*=[[:space:]]*_?[a-z][A-Za-z0-9]*[[:space:]]*(//.*)?[[:space:]]*$" spec/ ``` -Every surviving hit must be a field access (`= car.engine`) or an `&T` -parameter — never a bare local. The trailing `(//.*)?[[:space:]]*$` is what -makes the guard see the `// ILLEGAL: ...` examples; without it the end anchor -skipped every commented line, which is most of them. +The pattern matches a **bare-symbol** right-hand side only, so every hit is a +candidate bug by construction — the legal sources (`= car.engine`, an `&T` +parameter) never match, because `.` is outside the character class. Read each +hit and keep it only if it is a deliberate `// ILLEGAL:` example or a grammar +metavariable; anything else is a real one to fix. + +Two details are load-bearing. The trailing `(//.*)?[[:space:]]*$` is what makes +the guard see the `// ILLEGAL: ...` examples; without it the end anchor skipped +every commented line, which is most of them. The `_?` catches a private +lowercase name (`_engine`) — Zane allows `_` only as a leading character, never +inside a name (`lexical.md` §4.1–4.2), so nothing more is needed there. Run both with `-R` on the directory, not a `spec/*.md` glob plus a bare directory argument: `grep` prints `bench/: Is a directory` and silently skips diff --git a/spec/glossary.md b/spec/glossary.md index e73c049..1907b2f 100644 --- a/spec/glossary.md +++ b/spec/glossary.md @@ -225,7 +225,7 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`memory.md`](memory.md) §2.1 ### 3.33 guest -- **Meaning:** The source-facing `&T`: access to a hosted reference-type object without storing that object or controlling its lifetime. A guest may be repointed, copied when assigned or passed, stored in an `&` field, or returned as `&T`, but it cannot outlive its host, and it may be minted only from an `&T` parameter or a field access not rooted in a borrow (§3.36). Internally, a guest is represented by a tether (§3.24) that resolves through an anchor cell (§3.23). +- **Meaning:** The source-facing `&T`: access to a hosted reference-type object without storing that object or controlling its lifetime. A guest may be repointed, copied when assigned or passed, stored in an `&` field, or returned as `&T`, but it cannot outlive its host, and it may be minted only from an `&T` parameter or a field access whose base is a place and whose base chain does not pass through a `'T` borrow (§3.36). Internally, a guest is represented by a tether (§3.24) that resolves through an anchor cell (§3.23). - **Why this name:** A guest may use what a host provides without owning it, and the guest's stay cannot outlast the host. The pair names the source relationship without exposing its runtime mechanism. - **Canonical home:** [`memory.md`](memory.md) §2.4 diff --git a/spec/memory.md b/spec/memory.md index 5267260..b83fe3c 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -171,7 +171,9 @@ A **field** is a different matter and stays a legal source. A field belongs to a ### 2.9 Function parameters: swallow, guest, and borrow -A **borrow** is non-hosting, non-escaping access to a caller's storage for the duration of a call. Unlike a guest (§2.4), a borrow has no anchor, cannot be stored in a field, and cannot be returned; it exists only while the call runs. A value type is *always* passed this way: a value-type parameter is a **read-only borrow** of the caller's slot, and a value is **copied** only when it is bound into a fresh slot — an assignment, a new declaration, or a field or return store. +A **borrow** is non-hosting, non-escaping access to a caller's storage for the duration of a call. A borrow is never itself storage: unlike a guest (§2.4) it has no anchor, and it **MUST NOT** be stored in a field or returned. It exists only while the call runs. + +That restriction is on the borrow, not on what is read through one. A value type is *always* passed this way — a value-type parameter is a **read-only borrow** of the caller's slot — and binding through that borrow into a fresh slot (an assignment, a new declaration, or a field or return store) **copies** the value. The copy is a new value that outlives the call perfectly well; what does not escape is the borrow. A reference type has no such copy, so a `'T` borrow leaves nothing behind at all. A **reference type** has three passing modes, one per surface form: From 84c350568ab7fbf0bf4a4fdcf76f7412f6a4f087 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 21:45:01 +0000 Subject: [PATCH 20/31] docs: third CodeRabbit round on #151 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - CLAUDE.md: the guard's prose claimed the legal sources "never match". That is true only of a field access, which `.` excludes syntactically. An `&T` parameter is written bare, so `r &Node = source` inside a callee matches and is correct — the reviewing rule now lists it as a keep, checked against the enclosing signature rather than the line. - glossary.md §3.27: the borrow entry still carried the contradiction fixed in memory.md §2.9 — "cannot be stored, returned, or moved" against a value being copied into a fresh slot. Scope it to the borrow itself. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- CLAUDE.md | 17 ++++++++++++----- spec/glossary.md | 2 +- 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index aa58175..47e705b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -62,11 +62,18 @@ bug. Eyeball every hit of: grep -RIn -E "&[A-Z][A-Za-z0-9]*[[:space:]]*=[[:space:]]*_?[a-z][A-Za-z0-9]*[[:space:]]*(//.*)?[[:space:]]*$" spec/ ``` -The pattern matches a **bare-symbol** right-hand side only, so every hit is a -candidate bug by construction — the legal sources (`= car.engine`, an `&T` -parameter) never match, because `.` is outside the character class. Read each -hit and keep it only if it is a deliberate `// ILLEGAL:` example or a grammar -metavariable; anything else is a real one to fix. +The pattern matches a **bare-symbol** right-hand side. Only one legal source is +excluded syntactically: a field access (`= car.engine`) never matches, because +`.` is outside the character class. The other legal source **does** match — an +`&T` parameter is written bare, so `r &Node = source` inside a callee is a hit +even though it is correct. Read every hit and keep it if any of these hold: + +- the right-hand side is an `&T` parameter of the enclosing verb (check the + signature, not the line); +- it is a deliberate `// ILLEGAL:` example; +- it is a grammar metavariable, as in `syntax.md`. + +Anything else is a real one to fix. Two details are load-bearing. The trailing `(//.*)?[[:space:]]*$` is what makes the guard see the `// ILLEGAL: ...` examples; without it the end anchor skipped diff --git a/spec/glossary.md b/spec/glossary.md index 1907b2f..c137c9c 100644 --- a/spec/glossary.md +++ b/spec/glossary.md @@ -196,7 +196,7 @@ This file gives short, reusable names to concepts that appear across multiple sp ### 3.27 borrow - **Meaning:** Non-hosting, non-escaping access to a caller's storage for the duration of a call. Every value type is passed this way — a value parameter is a read-only borrow, a value-type `mut` receiver is a mutable borrow, and a value is copied only when bound into a fresh slot. A reference type may also be borrowed, written `'T`, which is the only non-swallowing way to pass a bare symbol (§3.36); a bare reference-type `this` is an implicit `'T` borrow. -- **Why this name:** The callee is lent the caller's storage for the call and gives it back at return — it does not host it and cannot keep it. Unlike a guest, a borrow has no anchor or tether and cannot be stored, returned, or moved. +- **Why this name:** The callee is lent the caller's storage for the call and gives it back at return — it does not host it and cannot keep it. Unlike a guest, the borrow itself has no anchor or tether and cannot be stored, returned, or used as a move source — a restriction on the borrow, not on the value read through it, which a value type may still copy into a fresh slot. - **Canonical home:** [`memory.md`](memory.md) §2.9 ### 3.28 coercion site From 6b0c9094fa62dac401e1e41541164231ef8fd34f Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 21:46:09 +0000 Subject: [PATCH 21/31] docs(meta): make the glossary a required stop when a rule changes Two findings on #151 were the same defect: a rule was corrected in its canonical home and its glossary entry left paraphrasing the superseded version, so the spec contradicted itself until review caught it. Both times the author missed it, which makes it a habit worth writing down rather than a one-off slip. Adds it to the "Before you edit" list: after editing any normative rule, grep spec/glossary.md for the concept and update the entry in the same commit. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- CLAUDE.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index 47e705b..251a060 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,6 +28,12 @@ next agent gets up to speed — keep it to durable, agent-facing facts. internal contradictions on this codebase before: re-read the *un-updated* spec files and `bench/zane_bench.c` against the new design before opening a PR, not just the file you changed. +3. **A rule correction is not done until `glossary.md` carries it.** The + glossary summarizes rules it does not own, so fixing a rule in its canonical + home and leaving the entry paraphrasing the superseded version produces a + spec that contradicts itself — this has happened twice, both times caught in + review rather than by the author. After editing any normative rule, grep + `spec/glossary.md` for the concept and update the entry in the same commit. ## The `bench/` harness `bench/` is a reference **C** harness for runtime experiments — **not** Zane From 99e143b6697ddd5b3127004de23a3ccd5dc304c8 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 22:24:56 +0000 Subject: [PATCH 22/31] docs: the borrow is the non-swallowing way to pass a bare symbol MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "Passing a bare symbol into a call is the borrow mode's job" overstated it: a bare symbol is a move-source (lifetimes.md §1.2), so a plain `T` parameter swallows one perfectly well. `'T` is the only *non-swallowing* mode that accepts a bare symbol, which is the claim the surrounding rules actually support. Corrected in all three places that carried it — memory.md §2.8.1 (canonical), glossary.md §3.36, and the foundations.md bullet — rather than only where the review pointed, since the imprecision started in the canonical home and propagated from there. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- spec/foundations.md | 2 +- spec/glossary.md | 2 +- spec/memory.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/spec/foundations.md b/spec/foundations.md index f91c1b9..7e1702e 100644 --- a/spec/foundations.md +++ b/spec/foundations.md @@ -94,7 +94,7 @@ Both kinds are mutated in place through a `mut` method, and the receiver reaches - **`#` is the only kind modifier**, applied uniformly to any type. See [`types.md`](types.md) §2 and [`adt.md`](adt.md) §2–§3. - **A value type is transitively value** (no reference-type or `&` field, anywhere downstream). This closed value world is specified by [`memory.md`](memory.md) §2.10. - **`&` rides on `#`.** A non-hosting `&` exists only for reference types; a value is shared by copy or by a scoped borrow, never by a stored `&`. See [`memory.md`](memory.md) §2.4. -- **A guest comes from a field, not a symbol.** A new `&` is minted only from a field access or an `&T` parameter; a bare symbol is a place but never a guest source, so a local's own hosting slot has nothing pointing at it. Passing such a symbol into a call is the borrow mode's job. See [`memory.md`](memory.md) §2.8.1 and §2.9. +- **A guest comes from a field, not a symbol.** A new `&` is minted only from a field access or an `&T` parameter; a bare symbol is a place but never a guest source, so a local's own hosting slot has nothing pointing at it. Such a symbol may still be swallowed by a plain `T` parameter; the borrow mode is the only non-swallowing way to pass one. See [`memory.md`](memory.md) §2.8.1 and §2.9. - **Concurrency reads this axis.** A spawned call may mutate only a value-typed receiver, because a value's transitive alias-freedom is exactly what lets the compiler rule out a data race from the signature alone. See [`concurrency.md`](concurrency.md) §4. > **Story:** [`stories/foundations.md`](../stories/foundations.md#identity-is-opt-in-one-axis-for-value-and-reference) — "Identity is opt-in: one axis for value and reference". diff --git a/spec/glossary.md b/spec/glossary.md index c137c9c..977086c 100644 --- a/spec/glossary.md +++ b/spec/glossary.md @@ -240,7 +240,7 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`lifetimes.md`](lifetimes.md) §1.8 ### 3.36 guest source restriction -- **Meaning:** A new `&` may be minted only from an `&T` parameter, or from a field access whose base is a place and whose base chain does not pass through a `'T` borrow parameter. A **bare symbol** — an identifier standing alone rather than as the base of a field access — is a place expression but never a guest source, so no guest can point at a local's own hosting slot and that slot stays free to be overwritten or moved from. Passing a bare symbol into a call is the borrow mode's job (§3.27). The borrow exclusion runs the same way: a guest minted from a borrowed object's field would escape the call just as surely as the borrow itself. +- **Meaning:** A new `&` may be minted only from an `&T` parameter, or from a field access whose base is a place and whose base chain does not pass through a `'T` borrow parameter. A **bare symbol** — an identifier standing alone rather than as the base of a field access — is a place expression but never a guest source, so no guest can point at a local's own hosting slot and that slot stays free to be overwritten or moved from. A bare symbol may still be swallowed by a plain `T` parameter; `'T` is the only **non-swallowing** mode that accepts one (§3.27, §3.37). The borrow exclusion runs the same way: a guest minted from a borrowed object's field would escape the call just as surely as the borrow itself. - **Why this name:** The rule constrains the *source* of a guest — where one may come from — and nothing about what a guest can survive once minted; a guest to a field still follows its host across overwrites and rehosting. - **Canonical home:** [`memory.md`](memory.md) §2.8.1 diff --git a/spec/memory.md b/spec/memory.md index b83fe3c..6ebe313 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -162,7 +162,7 @@ second = main `second = main` moves the object out of `main`'s slot, and `main` downgrades to guest state ([`lifetimes.md`](lifetimes.md) §1.6). What `guest` should then denote — the object that left, or the slot it left from — has no answer that is right in both directions, and every candidate answer costs either a rule the programmer has to carry or machinery the runtime has to pay for. Removing the source removes the question: line 3 is a compile-time error, so no guest ever depends on a bare symbol's slot. -Nothing is lost by it. A guest exists to reach an object from storage that does not own it — a field, a container element, a callee. A bare symbol is *already* in scope wherever a guest to it could be declared, so the guest never buys reach that the symbol itself did not already have. What a bare symbol is genuinely needed for is passing an object into a call, and that is what the borrow mode `'T` is for (§2.9): a borrow reads and mutates the caller's object for the duration of the call without minting a guest to it. +Nothing is lost by it. A guest exists to reach an object from storage that does not own it — a field, a container element, a callee. A bare symbol is *already* in scope wherever a guest to it could be declared, so the guest never buys reach that the symbol itself did not already have. What a bare symbol is genuinely needed for is passing an object into a call. It may still be swallowed by a plain `T` parameter, which takes hosting outright; where the call must *not* take hosting, the borrow mode `'T` is what carries it (§2.9), reading and mutating the caller's object for the duration of the call without minting a guest to it. A **field** is a different matter and stays a legal source. A field belongs to an object whose own lifetime the host system already tracks, and a guest to `car.engine` follows that field's host through the anchor path (§4.5) when the field is overwritten or the containing object is rehosted. This is what makes the restriction narrow: it constrains where guests come from, not what they can survive. From 1e4dcd5aca3c15dcb25e7ec667dcef95be9cbf46 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Jul 2026 22:35:30 +0000 Subject: [PATCH 23/31] docs: tighten two guest-source summaries MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - foundations.md §7: the bullet said a guest is minted "from a field access or an `&T` parameter", which over-approximates memory.md §2.8.1 — the base must be a place and must not be reached through a `'T` borrow. Named both qualifiers compactly rather than reproducing the rule; foundations.md is a map that defers to canonical homes. - memory.md §2.8.1: "a field, a container element, a callee" listed where a guest may live, not what may mint one, and was read as the latter. Says it in the precise terms instead — an `&` field, a container element, an `&T` parameter inside a callee. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- spec/foundations.md | 2 +- spec/memory.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/foundations.md b/spec/foundations.md index 7e1702e..0e8e72a 100644 --- a/spec/foundations.md +++ b/spec/foundations.md @@ -94,7 +94,7 @@ Both kinds are mutated in place through a `mut` method, and the receiver reaches - **`#` is the only kind modifier**, applied uniformly to any type. See [`types.md`](types.md) §2 and [`adt.md`](adt.md) §2–§3. - **A value type is transitively value** (no reference-type or `&` field, anywhere downstream). This closed value world is specified by [`memory.md`](memory.md) §2.10. - **`&` rides on `#`.** A non-hosting `&` exists only for reference types; a value is shared by copy or by a scoped borrow, never by a stored `&`. See [`memory.md`](memory.md) §2.4. -- **A guest comes from a field, not a symbol.** A new `&` is minted only from a field access or an `&T` parameter; a bare symbol is a place but never a guest source, so a local's own hosting slot has nothing pointing at it. Such a symbol may still be swallowed by a plain `T` parameter; the borrow mode is the only non-swallowing way to pass one. See [`memory.md`](memory.md) §2.8.1 and §2.9. +- **A guest comes from a field, not a symbol.** A new `&` is minted only from a qualifying field access — base a place, not reached through a `'T` borrow — or from an `&T` parameter; a bare symbol is a place but never a guest source, so a local's own hosting slot has nothing pointing at it. Such a symbol may still be swallowed by a plain `T` parameter; the borrow mode is the only non-swallowing way to pass one. See [`memory.md`](memory.md) §2.8.1 and §2.9. - **Concurrency reads this axis.** A spawned call may mutate only a value-typed receiver, because a value's transitive alias-freedom is exactly what lets the compiler rule out a data race from the signature alone. See [`concurrency.md`](concurrency.md) §4. > **Story:** [`stories/foundations.md`](../stories/foundations.md#identity-is-opt-in-one-axis-for-value-and-reference) — "Identity is opt-in: one axis for value and reference". diff --git a/spec/memory.md b/spec/memory.md index 6ebe313..faa58ce 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -162,7 +162,7 @@ second = main `second = main` moves the object out of `main`'s slot, and `main` downgrades to guest state ([`lifetimes.md`](lifetimes.md) §1.6). What `guest` should then denote — the object that left, or the slot it left from — has no answer that is right in both directions, and every candidate answer costs either a rule the programmer has to carry or machinery the runtime has to pay for. Removing the source removes the question: line 3 is a compile-time error, so no guest ever depends on a bare symbol's slot. -Nothing is lost by it. A guest exists to reach an object from storage that does not own it — a field, a container element, a callee. A bare symbol is *already* in scope wherever a guest to it could be declared, so the guest never buys reach that the symbol itself did not already have. What a bare symbol is genuinely needed for is passing an object into a call. It may still be swallowed by a plain `T` parameter, which takes hosting outright; where the call must *not* take hosting, the borrow mode `'T` is what carries it (§2.9), reading and mutating the caller's object for the duration of the call without minting a guest to it. +Nothing is lost by it. A guest exists so that storage which does not own an object can still reach it — an `&` field, a container element, an `&T` parameter inside a callee. A bare symbol is *already* in scope wherever a guest to it could be declared, so the guest never buys reach that the symbol itself did not already have. What a bare symbol is genuinely needed for is passing an object into a call. It may still be swallowed by a plain `T` parameter, which takes hosting outright; where the call must *not* take hosting, the borrow mode `'T` is what carries it (§2.9), reading and mutating the caller's object for the duration of the call without minting a guest to it. A **field** is a different matter and stays a legal source. A field belongs to an object whose own lifetime the host system already tracks, and a guest to `car.engine` follows that field's host through the anchor path (§4.5) when the field is overwritten or the containing object is rehosted. This is what makes the restriction narrow: it constrains where guests come from, not what they can survive. From dba89badee43dcdc77f7a1958c10b54d3fc4c1d6 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 10:00:44 +0000 Subject: [PATCH 24/31] docs: never write ' on this; add a forward pointer in the anchor story MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Maintainer review on #151. The receiver was specified two ways at once: memory.md §2.9 said a bare `this T` on a reference type is an implicit `'T` borrow, and then every example in that same section wrote `this 'Car`. Decided in favour of the implicit form — `'` is never written on `this`. The rule is forced rather than chosen. If the marker were required on the receiver, a bare `this T` would have to mean the method swallows its own receiver, which no method wants; so the borrow has to be the bare meaning, and `this 'T` is then a second spelling for the default. It also makes the two type worlds agree: value and reference receivers are both borrows and both written bare, with `&` the single marker `this` may carry. Applied to memory.md, functions.md, lifetimes.md, syntax.md, glossary.md, and effects.md — including dropping the `this 'ReceiverType` production from syntax.md §3.2 and the function-type form in §2.9. stories/memory.md: the segmented-offset chapter still ends on the one-cell promotion model, which the later global-pool chapter superseded. Stories are append-only, so rather than rewrite it, that chapter now carries a forward pointer naming what did not survive and why — the cell leaving the payload stream, and one-cell promotion failing once both sides of a move are anchored. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- spec/effects.md | 2 +- spec/functions.md | 16 ++++++++-------- spec/glossary.md | 4 ++-- spec/lifetimes.md | 2 +- spec/memory.md | 14 +++++++------- spec/syntax.md | 5 ++--- stories/memory.md | 2 ++ 7 files changed, 23 insertions(+), 22 deletions(-) diff --git a/spec/effects.md b/spec/effects.md index bc5b4ac..7ae0b1c 100644 --- a/spec/effects.md +++ b/spec/effects.md @@ -31,7 +31,7 @@ A side effect is any observable interaction beyond returning a value, including: A capability is an object whose methods model access to external state, such as a filesystem, logger, socket, clock, or random source. ### 2.3 `mut` -`mut` is the only effect modifier in the language. It appears on methods and grants write access to state reachable through `this`; the write lands on the caller's object or on state reachable from it. `this` is a **borrow** of the caller's slot for both kinds: a value-type `this` borrows the value, and a reference-type `this` written bare is an implicit `'T` borrow of the object (see [`functions.md`](functions.md) §2.4). +`mut` is the only effect modifier in the language. It appears on methods and grants write access to state reachable through `this`; the write lands on the caller's object or on state reachable from it. `this` is a **borrow** of the caller's slot for both kinds: a value-type `this` borrows the value, and a reference-type `this` written bare is a borrow of the object, `'` never being written on `this` (see [`functions.md`](functions.md) §2.4). ### 2.4 Parameters are not mutable by default Parameters other than `this` are read-only. Mutation of another object must be expressed by calling a `mut` method on that object as the receiver. A number parameter that resolves to a number value in body positions (see [`generics.md`](generics.md) §3.5) is a value-like binding and is read-only by default; mutating it requires a `mut` declaration. diff --git a/spec/functions.md b/spec/functions.md index 4218252..95f228c 100644 --- a/spec/functions.md +++ b/spec/functions.md @@ -56,7 +56,7 @@ A method marked `mut` may write to any state reachable through `this`, whether t A write to `this` lands on the caller's object; how `this` reaches the caller differs by kind (see [`memory.md`](memory.md) §2.9): - For a **value-type** receiver, `this` is a **mutable borrow** of the caller's slot — the actual value, not a copy. The borrow makes the value mutable in place while preserving its value semantics. Because the borrow is scoped and non-escaping, `this` may be read and written but cannot be stored as an `&` or returned as one, since a value type is not `&`-rootable. -- For a **reference-type** receiver, `this` is a **mutable borrow** too, and for the same reason: the receiver expression at the call site is usually a bare symbol, which is not a guest source ([`memory.md`](memory.md) §2.8.1). Writing `this T` on a reference type therefore means `this 'T`. A `mut` method mutates through the borrow, and the receiver is never swallowed — the caller stays a full host. +- For a **reference-type** receiver, `this` is a **mutable borrow** too, and for the same reason: the receiver expression at the call site is usually a bare symbol, which is not a guest source ([`memory.md`](memory.md) §2.8.1). A bare `this T` on a reference type *is* that borrow — **`'` is never written on `this`** — so the receiver is never swallowed and the caller stays a full host. A method that needs to keep the receiver past the call — store it in an `&` field, or return it as `&T` ([`lifetimes.md`](lifetimes.md) §1.7) — declares `this &T` instead. That is a guest receiver, so the call site must supply a guest source. @@ -121,18 +121,18 @@ type Car = #struct { } // `&` parameter: may be stored into an `&` field -Unit setEngine(this 'Car, engine &Engine) mut { +Unit setEngine(this Car, engine &Engine) mut { this.engine = engine // legal return Unit() } // borrow parameter, read only -Int calculate(this 'Car, engine 'Engine) { +Int calculate(this Car, engine 'Engine) { return this._value + engine.speed // legal: reading through the borrow } // plain reference-type parameter swallows; a swallowed host is not a guest source -Unit setEngineWrong(this 'Car, engine Engine) mut { +Unit setEngineWrong(this Car, engine Engine) mut { this.engine = engine // ILLEGAL: cannot store a swallowed host into an `&` field return Unit() } @@ -227,9 +227,9 @@ Two declarations in the same package conflict when they have the same ordered pa Two overloads **MUST NOT** differ only by the **passing mode** at the same parameter position — that is, only by whether that position is `T`, `&T`, or `'T`, the receiver included. Such declarations are illegal and the compiler **MUST** reject them with a compile-time error, for example: "illegal overload set: differs only by the passing mode on a parameter; rename one declaration or choose a single signature." ```zane -Unit consume(this 'Car, engine Engine) -Unit consume(this 'Car, engine &Engine) // ERROR: differs only by the passing mode -Unit consume(this 'Car, engine 'Engine) // ERROR: same +Unit consume(this Car, engine Engine) +Unit consume(this Car, engine &Engine) // ERROR: differs only by the passing mode +Unit consume(this Car, engine 'Engine) // ERROR: same ``` The mode changes what the caller must supply and what state the call leaves the caller in — not the shape of the call. Overloading on it would make `consume(e)` mean two different things about `e`'s ownership with nothing at the call site to tell them apart. @@ -416,7 +416,7 @@ Read-only methods and functions are effect-free with respect to their receiver u | `&` method parameter | Caller must supply a guest source (never a bare symbol); callee may store it into `&` fields or return it | | `'T` method parameter | Caller may supply any place expression, bare symbols included; read and `mut` access for the call only; **MUST NOT** be stored, returned, or moved | | Plain `T` method parameter | Swallows; caller may supply a temporary and downgrades to a guest; callee **MUST NOT** bind it into `&` storage | -| Reference receiver | `this T` is an implicit `'T` borrow; `this &T` is a guest receiver, required to store or return the receiver | +| Reference receiver | Bare `this T` is the borrow receiver — `'` is never written on `this`; `this &T` is a guest receiver, required to store or return the receiver | | Subscript | Package-scope place projection written `(this T)[...] => placeExpr`; no explicit return type | | Overload identity | Parameter types only; not names, return type, or `mut`; overloads differing only by the passing mode (`T` / `&T` / `'T`) at one position are illegal | | Overload resolution phases | Direct match, then generic match, then implicit match; ambiguity within any one phase is an error | diff --git a/spec/glossary.md b/spec/glossary.md index 977086c..5334447 100644 --- a/spec/glossary.md +++ b/spec/glossary.md @@ -195,7 +195,7 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`functions.md`](functions.md) §8 ### 3.27 borrow -- **Meaning:** Non-hosting, non-escaping access to a caller's storage for the duration of a call. Every value type is passed this way — a value parameter is a read-only borrow, a value-type `mut` receiver is a mutable borrow, and a value is copied only when bound into a fresh slot. A reference type may also be borrowed, written `'T`, which is the only non-swallowing way to pass a bare symbol (§3.36); a bare reference-type `this` is an implicit `'T` borrow. +- **Meaning:** Non-hosting, non-escaping access to a caller's storage for the duration of a call. Every value type is passed this way — a value parameter is a read-only borrow, a value-type `mut` receiver is a mutable borrow, and a value is copied only when bound into a fresh slot. A reference type may also be borrowed, written `'T`, which is the only non-swallowing way to pass a bare symbol (§3.36); a bare reference-type `this` is that borrow — `'` is never written on `this`. - **Why this name:** The callee is lent the caller's storage for the call and gives it back at return — it does not host it and cannot keep it. Unlike a guest, the borrow itself has no anchor or tether and cannot be stored, returned, or used as a move source — a restriction on the borrow, not on the value read through it, which a value type may still copy into a fresh slot. - **Canonical home:** [`memory.md`](memory.md) §2.9 @@ -245,7 +245,7 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`memory.md`](memory.md) §2.8.1 ### 3.37 passing mode -- **Meaning:** Which of three ways a reference-type argument reaches a callee, fixed entirely by the parameter's surface form: `T` **swallows** it (hosting access; the caller downgrades to a guest), `&T` takes a **guest** (storable and returnable; requires a guest source), `'T` **borrows** it (read and `mut` for the call only; accepts any place, bare symbols included). The receiver takes a mode like any parameter, and defaults to the borrow. Two overloads may not differ only by the mode at one position. +- **Meaning:** Which of three ways a reference-type argument reaches a callee, fixed entirely by the parameter's surface form: `T` **swallows** it (hosting access; the caller downgrades to a guest), `&T` takes a **guest** (storable and returnable; requires a guest source), `'T` **borrows** it (read and `mut` for the call only; accepts any place, bare symbols included). The receiver selects between the borrow and `&T` only: a bare `this T` is the borrow and `'` is never written on `this`. Two overloads may not differ only by the mode at one position. - **Why this name:** "Mode" names a choice about *how* the same argument travels rather than *what* it is — the type is unchanged in all three, and only the caller's obligations and resulting state differ. - **Canonical home:** [`memory.md`](memory.md) §2.9 diff --git a/spec/lifetimes.md b/spec/lifetimes.md index 06de855..f7d83d0 100644 --- a/spec/lifetimes.md +++ b/spec/lifetimes.md @@ -158,7 +158,7 @@ A function may return an `&T` only when the returned guest is rooted in one of t The other two parameter modes are not roots. A `'T` borrow ends with the call, so a guest rooted in one would outlive the access it was granted. A swallowing `T` parameter is a bare symbol in the call-site scope, and a bare symbol is not a guest source at all. ```zane -&Weapon fromBorrow(this 'Player) => this.weapon // ILLEGAL: a borrow is not a guest root +&Weapon fromBorrow(this Player) => this.weapon // ILLEGAL: a borrow is not a guest root &Node bad() { value Node() diff --git a/spec/memory.md b/spec/memory.md index faa58ce..62785ef 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -206,34 +206,34 @@ type Car = #struct { } // `&` parameter is a guest; it may be stored into an `&` field -Unit setEngine(this 'Car, engine &Engine) mut { +Unit setEngine(this Car, engine &Engine) mut { this.engine = engine return Unit() } // plain reference-type parameter: taken by hosting access, then moved into a hosting field of this -Unit setSpare(this 'Car, engine Engine) mut { +Unit setSpare(this Car, engine Engine) mut { this.spare = engine return Unit() } // borrow parameter: a reference-type object read without consuming it and without minting a guest -Int inspect(this 'Car, engine 'Engine) { +Int inspect(this Car, engine 'Engine) { return this._value + engine.speed } ``` -A reference-type receiver follows the same three modes and defaults to the borrow: `this T` is an implicit `'T` borrow, and a method that needs to keep or hand back the receiver as a guest writes `this &T` (see [`functions.md`](functions.md) §2.4). +A reference-type receiver borrows by default, and **`'` is never written on `this`**: a bare `this T` on a reference type *is* the borrow. It has to be — if the marker were required here, bare `this T` would mean the method swallows its own receiver, which is not something a method wants. A method that needs to keep or hand back the receiver writes `this &T`, the guest receiver (see [`functions.md`](functions.md) §2.4). So `this` carries at most one marker, `&`, and its absence means borrow for value and reference receivers alike. Binding a swallowed or borrowed parameter into `&` storage is illegal. A swallowed value is hosted at the call site while an `&` field lives with the object that holds it — which may outlive the call. A borrow does not survive the call at all: ```zane -Unit setEngineSwallowed(this 'Car, engine Engine) mut { +Unit setEngineSwallowed(this Car, engine Engine) mut { this.engine = engine // ILLEGAL: a swallowed host is not a guest source return Unit() } -Unit setEngineBorrowed(this 'Car, engine 'Engine) mut { +Unit setEngineBorrowed(this Car, engine 'Engine) mut { this.engine = engine // ILLEGAL: a borrow is not a guest source and does not escape the call return Unit() } @@ -559,7 +559,7 @@ A single global free stack and frontier require synchronization under concurrent | Value-type parameter | Always a read-only borrow; caller need not supply a place; copied only when bound into a fresh slot (assignment, declaration, field or return store) | | Reference-type parameter | `T` swallows (hosting access; passing a host downgrades the caller's symbol to a guest whatever the body does — see [`lifetimes.md`](lifetimes.md) §1.8); `&T` takes a guest, which only a guest source can supply; `'T` borrows any place, bare symbols included, and leaves the caller a full host | | `'T` position | Parameter positions only; never a storage, field, or return type | -| Reference-type receiver | `this T` is an implicit `'T` borrow; `this &T` is a guest receiver a method may store or return | +| Reference-type receiver | Bare `this T` is the borrow receiver — `'` is never written on `this`; `this &T` is a guest receiver a method may store or return | | Value-downstream enforcement | Value types may contain only primitives and other value types, transitively — never a reference (`#`) or `&` field | | `&` targets reference types | An `&T` requires `T` to be a reference type; a value is shared by copy or scoped borrow, never by a stored `&` | | Symbol declaration | Must be directly initialized | diff --git a/spec/syntax.md b/spec/syntax.md index 7ecbaa9..cdde99f 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -261,7 +261,7 @@ Reference-typed parameters and returns use the ordinary type form. A parameter s ReturnType[&ParamType, ...] ReturnType['ParamType, ...] &ReturnType[this &ReceiverType, &ParamType, ...] -ReturnType[this 'ReceiverType, 'ParamType, ...] mut +ReturnType[this ReceiverType, 'ParamType, ...] mut ``` ```zane @@ -326,12 +326,11 @@ ReturnType?AbortType name(this ReceiverType, param ParamType, ...) => expr ReturnType?AbortType name(this ReceiverType, param ParamType, ...) mut => expr ReturnType name(this ReceiverType, param ParamType, ...) { body } ReturnType name(this &ReceiverType, param ParamType, ...) { body } -ReturnType name(this 'ReceiverType, param ParamType, ...) { body } ``` `this` is legal only in the first parameter position. A declaration is a method if and only if its first parameter is named `this`. -A **reference** receiver takes a passing mode like any other reference parameter, and every combination above may be written with `&` or `'` on `ReceiverType`. Bare `this ReceiverType` means `this 'ReceiverType` — the borrow is the default — and `this &ReceiverType` is written when the method stores or returns the receiver as a guest. A **value** receiver has no mode to select: it is a borrow of the caller's slot, mutable when the method is `mut`, and is always written bare. See [`functions.md`](functions.md) §2.4. +The receiver takes at most one marker, `&`. A bare `this ReceiverType` is the **borrow** receiver, and `this &ReceiverType` is written when the method stores or returns the receiver as a guest; `'` is **never** written on `this`, for either kind of type. A value receiver is likewise a borrow of the caller's slot, mutable when the method is `mut`, and always written bare. See [`functions.md`](functions.md) §2.4. `=> expr` returns `expr`, including when `expr` has type `Unit`. diff --git a/stories/memory.md b/stories/memory.md index 9e7f116..342f6c4 100644 --- a/stories/memory.md +++ b/stories/memory.md @@ -89,6 +89,8 @@ The one motion this has to survive is escape. A value that outlives its scope is The cost is a ceiling, and a lower one than the flat region carried its own version of. Splitting a `u32` into a chunk id and an offset spends bits on structure that a flat offset spent on range: with 1 MiB chunks and 8-byte-aligned slots the arithmetic lands at 32 GiB of live arena across up to 32768 chunks — roomy, but a hard cap, and a program that genuinely needs more cannot have it without widening the reference and paying back the cache density we just secured. The chunk directory is a hop the flat "region base plus offset" did without, too: one more small, hot table on the resolve path. We were content to trade a fixed horizon and a register-resident directory for an allocator that never fragments, never resizes an anchor table, and vaporizes an entire scope's memory in a single unmap. +Two things in this chapter did not survive. The cell does not live beside its payload any more — [the next chapter](#where-the-cells-live-and-the-scan-that-pays-for-them) pulls on the loose end just named and moves cells into their own region. And the promotion story above, where the runtime follows the backpointer to *the* one cell and rewrites it, holds only while a payload has exactly one cell to rewrite. It stops holding as soon as both sides of a move are already anchored, which is the pressure that eventually takes anchors out of scopes altogether; [the chapter on two payload streams](#two-payload-streams-and-the-anchor-that-leaves-the-scope) tells that turn. The twelve bytes quoted here become sixteen there, for the same reason. + ## Where the cells live, and the scan that pays for them The [previous chapter](#the-last-table-problem-and-the-segmented-offset) left a cell sitting beside every payload and called the shared cache line a win. It is a win — for the deref. What we had not yet measured was the other side of the same coin, and when we built the layout and ran it on real hardware the coin landed. Two workloads slid backwards. A sweep that reads only payloads — the common case of walking a collection and touching each object's fields — slowed by around a fifth, because the interleaved cells sit between the payloads and a scan that wants only payloads drags the cells through cache anyway; the same objects packed tight would have touched fewer lines. And a write-heavy growth buffer regressed harder still, because once cells share the payload stream the stream's alignment stops being the compiler's to control — a buffer's base now depends on how many cells were minted ahead of it, and a run of entities that should have sat one-per-line ended up straddling two. The shared-line trick had quietly made the *arena's* geometry a function of tether history, and iteration paid for it. From 19806f0d680a75d55e7701dd5a531da2dd9f003a Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 10:12:44 +0000 Subject: [PATCH 25/31] docs(stories): keep stories/memory.md strictly append-only against main MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Maintainer ruling on #151: the story is append-only, and that is verified by diffing against main — an existing chapter is not edited, and a new chapter goes at the end rather than being slotted into the middle. Two violations, both now gone: - One sentence in "The last table problem, and the segmented offset" had been rewritten (it came in with the PR 147 commits, and I then appended a forward-pointer paragraph to the same chapter, which is the same violation again). Both reverted; that chapter is byte-identical to main. - "Two payload streams, and the anchor that leaves the scope" had been inserted between two chapters that already existed on main. Moved to the append position, after "Two vocabularies" and before the chapters this PR adds — which is also the right chronology, since the arena and anchor work came after the host/guest rename. The correction the forward pointer was carrying now lives where it belongs, in the appended chapter: it names the one-cell promotion claim from the segmented-offset chapter and says what retires it — that the claim holds only while a payload has one cell to rewrite, and fails once both sides of a move are anchored. `git diff origin/main -- stories/memory.md` is now additions only. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- stories/memory.md | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/stories/memory.md b/stories/memory.md index 342f6c4..4b90e61 100644 --- a/stories/memory.md +++ b/stories/memory.md @@ -85,12 +85,10 @@ The two dead ends pointed at the same missing idea: we needed cells that could b What falls out is a memory model that is uniformly 32-bit and, per tethered object, exactly twelve bytes of machinery: the four-byte tether wherever it is stored, the four-byte anchor cell, and the four-byte backpointer the payload carries home to that cell. The double indirection a tether walks — tether to cell, cell to payload — looks like it should cost two cache misses, and the arena is what makes it cost closer to zero: the cell read is a load into arena memory that is almost always already warm. The first thing we tried for that warmth was to drop each cell right beside the payload that mints it, so the two share a cache line and the second hop is paid for by the first. It worked for the deref, and it opened a loose end the next chapter has to pull on — a cell sitting in the payload stream is a cell a payload-only scan has to step over, and a payload's position starts to depend on how many of its neighbours were tethered before it. -The one motion this has to survive is escape. A value that outlives its scope is moved into a parent, and under arenas that means its payload is *copied* into the parent's arena — the child arena is about to be unmapped and cannot keep it. The backpointer is what makes that copy invisible: the runtime follows it to the payload's one anchor cell and rewrites the cell with the payload's new location. Every tether still points at that same cell and never learns the payload moved ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)). The one-cell update keeps anchor bookkeeping O(1) in the number of tethers; physical promotion still costs proportionally to the representation copied. When the scope finally drains, its chunks are unmapped and its cells vanish with the objects they served, and the scope rules that governed tethers all along guarantee nothing still pointing could point into them. +The one motion this has to survive is escape. A value that outlives its scope is moved into a parent, and under arenas that means its payload is *copied* into the parent's arena — the child arena is about to be unmapped and cannot keep it. The backpointer is what makes that copy invisible: the runtime follows it to the payload's one anchor cell and rewrites the cell with the payload's new location. Every tether still points at that same cell and never learns the payload moved ([`memory.md` §4.5](https://github.com/zane-lang/spec/blob/dc33d1eae46bfc71bfb7e8e3b2f575926aa00059/spec/memory.md#45-moves-overwrites-and-promotion-update-one-cell-not-all-tethers)) — the same one-cell-update that made an in-place move O(1) makes a cross-arena promotion O(1) too. When the scope finally drains, its chunks are unmapped and its cells vanish with the objects they served, and the scope rules that governed tethers all along guarantee nothing still pointing could point into them. The cost is a ceiling, and a lower one than the flat region carried its own version of. Splitting a `u32` into a chunk id and an offset spends bits on structure that a flat offset spent on range: with 1 MiB chunks and 8-byte-aligned slots the arithmetic lands at 32 GiB of live arena across up to 32768 chunks — roomy, but a hard cap, and a program that genuinely needs more cannot have it without widening the reference and paying back the cache density we just secured. The chunk directory is a hop the flat "region base plus offset" did without, too: one more small, hot table on the resolve path. We were content to trade a fixed horizon and a register-resident directory for an allocator that never fragments, never resizes an anchor table, and vaporizes an entire scope's memory in a single unmap. -Two things in this chapter did not survive. The cell does not live beside its payload any more — [the next chapter](#where-the-cells-live-and-the-scan-that-pays-for-them) pulls on the loose end just named and moves cells into their own region. And the promotion story above, where the runtime follows the backpointer to *the* one cell and rewrites it, holds only while a payload has exactly one cell to rewrite. It stops holding as soon as both sides of a move are already anchored, which is the pressure that eventually takes anchors out of scopes altogether; [the chapter on two payload streams](#two-payload-streams-and-the-anchor-that-leaves-the-scope) tells that turn. The twelve bytes quoted here become sixteen there, for the same reason. - ## Where the cells live, and the scan that pays for them The [previous chapter](#the-last-table-problem-and-the-segmented-offset) left a cell sitting beside every payload and called the shared cache line a win. It is a win — for the deref. What we had not yet measured was the other side of the same coin, and when we built the layout and ran it on real hardware the coin landed. Two workloads slid backwards. A sweep that reads only payloads — the common case of walking a collection and touching each object's fields — slowed by around a fifth, because the interleaved cells sit between the payloads and a scan that wants only payloads drags the cells through cache anyway; the same objects packed tight would have touched fewer lines. And a write-heavy growth buffer regressed harder still, because once cells share the payload stream the stream's alignment stops being the compiler's to control — a buffer's base now depends on how many cells were minted ahead of it, and a run of entities that should have sat one-per-line ended up straddling two. The shared-line trick had quietly made the *arena's* geometry a function of tether history, and iteration paid for it. @@ -119,13 +117,23 @@ There was a larger temptation in the same corner, and we turned it down. If a te The cost of what we kept is a rounding: up to a cache line of padding before each backing store, unmeasurable against the store it precedes, and the standing discipline that the payload frontier and the cell region stay separate so that zero keeps its meaning for free. Cheap insurance, for a sentinel that now costs nothing and a fill that no longer straddles. +## Two vocabularies: host and guest above anchor and tether + +With the arena layout settled, one vocabulary problem remained. Calling `&T` a tether had solved the collision with “reference type,” but it left the source language and the runtime sharing one word. The problem is that those are different layers. Source code needs names for the lifetime relationship a programmer reasons about; the memory model needs names for the indirection that keeps that relationship working when an object moves. Using `tether` for both made an implementation choice sound like the meaning of `&T` itself. + +The source pair is now **host** and **guest**. A host is the symbol, field, or container slot that stores a reference-type object — or its hosting handle — and governs the object's lifetime. A guest is an `&T`: it may access the hosted object, but it neither stores that object nor controls how long it lives. When the object moves, it is rehosted, and its guests continue reaching it. The ordinary relationship does useful work here: a host provides both accommodation and the duration of a guest's stay, while a guest may use what is provided but cannot outlast the host. + +The runtime keeps **anchor** and **tether**. Each guest is represented by a tether that resolves through an anchor; moving or rehosting the object updates the anchor, so existing tethers keep working. That vocabulary remains a natural mechanical picture, but it no longer leaks upward into source semantics. The concise model is: *an object lives in a host; a guest may access it; internally, the guest's tether follows the object through its anchor.* + +The alternatives each blurred something we wanted to keep sharp. **Owner/tether** named the two halves accurately in isolation but paired source semantics with implementation. **Owner/guest** worked, though “owner” stressed rights and destruction more than residence. **Owner/view** was technically reasonable without being a convincing lived relationship. Proxy, keyholder, delegate, and licensee were variously technical, overloaded, or indirect; “key” also collided with dictionary keys. CC/email language suggested secondary participation, but a CC recipient receives an independent copy rather than live access to one moving object. And keeping **tether** as the name of `&T` remained expressive, but preserved the very overload this split was meant to remove. + ## Two payload streams, and the anchor that leaves the scope -The scope arena survived, but the pure-bump conclusion did not survive unchanged. Fixed-size values, reference-type hosts, and dynamic handles still fit the original rule: append them densely and reclaim their chunks when the scope drains. Resizable backing stores do not. A list can abandon several buffers while its scope remains alive, so treating those buffers like ordinary fixed-size payloads strands exactly the kind of reusable holes that matter. The arena therefore split into two lazy chunk chains per scope: one fixed-size region that remains a pure bump allocator, and one dynamic region that may reuse backing-store blocks. A chunk belongs to exactly one region, and a scope that never allocates a backing store never maps a dynamic chunk. +With the vocabulary settled, the allocator came back open. The scope arena survived, but the pure-bump conclusion did not survive unchanged. Fixed-size values, reference-type hosts, and dynamic handles still fit the original rule: append them densely and reclaim their chunks when the scope drains. Resizable backing stores do not. A list can abandon several buffers while its scope remains alive, so treating those buffers like ordinary fixed-size payloads strands exactly the kind of reusable holes that matter. The arena therefore split into two lazy chunk chains per scope: one fixed-size region that remains a pure bump allocator, and one dynamic region that may reuse backing-store blocks. A chunk belongs to exactly one region, and a scope that never allocates a backing store never maps a dynamic chunk. The dynamic region brings back free stacks in the one place where their fragmentation is controlled rather than global. Blocks use shared power-of-two byte classes beginning at 128 bytes, independent of element type. Allocation checks the exact-size LIFO stack first and bumps the frontier only when that stack is empty. A full list requests exactly twice its current byte size; it grows in place only when it is the frontier allocation and the added bytes fit before the chunk boundary. Otherwise its elements relocate into a reusable doubled block or a newly bumped one, and the old block enters its exact-size stack. Blocks above 1 MiB become dedicated contiguous oversized spans, addressed by one base segmented offset and reused through the same exact-size rule. The cost is dead space between size classes and until scope teardown, but reuse is confined to the buffers whose repeated growth creates it. -The scope-local anchor region turned out to have a deeper problem than teardown. Promotion could leave source guests naming the old cell while destination guests named a new one; a later move had only one payload backpointer and could update only one path. Moving either cell would merely force every existing guest on that side to be repointed. The final design therefore takes anchors out of scopes entirely. One runtime-global pool grows through anchor-only pages, and an anchor cell may target either a payload or another anchor. +The scope-local anchor region turned out to have a deeper problem than teardown, and it retires a claim two chapters back. [The segmented-offset chapter](#the-last-table-problem-and-the-segmented-offset) had promotion follow the backpointer to *the* one anchor cell and rewrite it, so no tether ever learns the payload moved. That holds only while a payload has exactly one cell to rewrite. It stops the moment both sides of a move are already anchored: promotion could leave source guests naming the old cell while destination guests named a new one, and a later move had only one payload backpointer and could update only one path. Moving either cell would merely force every existing guest on that side to be repointed — the work anchors exist to avoid. The final design therefore takes anchors out of scopes entirely. One runtime-global pool grows through anchor-only pages, and an anchor cell may target either a payload or another anchor. That second target kind makes identity merging mechanical. A move into an already-anchored destination destroys its old occupant but preserves the destination host identity, so the destination anchor remains the terminal payload anchor. A distinct source anchor changes into a forwarding cell that targets it. Old source guests walk source anchor to destination anchor to payload; destination guests and newly created guests go directly to the destination anchor. Nothing in the source language exposes the chain, and no guest is enumerated or rewritten. @@ -139,16 +147,6 @@ That gives the two cell kinds different retirement events. A terminal payload an The sentinel changes by one small accounting detail. Segmented offset zero remains a valid payload location, but the global pool never issues anchor identity zero. The sentinel therefore costs one unusable anchor-slot identity rather than forcing either payload region away from its naturally aligned chunk base. Dynamic blocks begin at 128 bytes and preserve cache-line alignment through doubling, reuse, and oversized spans. -## Two vocabularies: host and guest above anchor and tether - -With the arena layout settled, one vocabulary problem remained. Calling `&T` a tether had solved the collision with “reference type,” but it left the source language and the runtime sharing one word. The problem is that those are different layers. Source code needs names for the lifetime relationship a programmer reasons about; the memory model needs names for the indirection that keeps that relationship working when an object moves. Using `tether` for both made an implementation choice sound like the meaning of `&T` itself. - -The source pair is now **host** and **guest**. A host is the symbol, field, or container slot that stores a reference-type object — or its hosting handle — and governs the object's lifetime. A guest is an `&T`: it may access the hosted object, but it neither stores that object nor controls how long it lives. When the object moves, it is rehosted, and its guests continue reaching it. The ordinary relationship does useful work here: a host provides both accommodation and the duration of a guest's stay, while a guest may use what is provided but cannot outlast the host. - -The runtime keeps **anchor** and **tether**. Each guest is represented by a tether that resolves through an anchor; moving or rehosting the object updates the anchor, so existing tethers keep working. That vocabulary remains a natural mechanical picture, but it no longer leaks upward into source semantics. The concise model is: *an object lives in a host; a guest may access it; internally, the guest's tether follows the object through its anchor.* - -The alternatives each blurred something we wanted to keep sharp. **Owner/tether** named the two halves accurately in isolation but paired source semantics with implementation. **Owner/guest** worked, though “owner” stressed rights and destruction more than residence. **Owner/view** was technically reasonable without being a convincing lived relationship. Proxy, keyholder, delegate, and licensee were variously technical, overloaded, or indirect; “key” also collided with dictionary keys. CC/email language suggested secondary participation, but a CC recipient receives an independent copy rather than live access to one moving object. And keeping **tether** as the name of `&T` remained expressive, but preserved the very overload this split was meant to remove. - ## The slot that could not be pointed at The whole of the preceding machinery — hosts, guests, anchors, forwarding cells, retirement stacks — was built to answer one question, and it took a five-line program to show that we had been answering the wrong one: From 2905461611d7a47cd996586511b9b75b2aabf58e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 10:13:34 +0000 Subject: [PATCH 26/31] docs(meta): state the append-only story rule literally, with a check MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The story guide licensed exactly what review just rejected: §5 said to "open a new chapter (or extend the relevant one)", which reads as permission to edit an existing chapter, and it said nothing about where a new chapter goes. Both gaps were used in this PR — a chapter was edited and another was inserted between two that already existed. §5 now spells out the two teeth: an existing chapter is not touched at all, not even to bolt a forward pointer onto its end, and a new chapter goes at the end of the file rather than into the middle. The correction a forward pointer would carry belongs in the new chapter, naming the older chapter's claim. Both are mechanically checkable, so the guide and CLAUDE.md now carry the check — `git diff origin/main -- stories/.md | grep -E "^-[^-]"` must print nothing. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- CLAUDE.md | 16 ++++++++++++++++ contributing/writing-stories-docs.md | 15 ++++++++++++++- 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 251a060..a1109d3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -110,6 +110,22 @@ quality bar — dense, opinionated, long-form prose. Writing a story is two halves: write the narrative, then integrate it into the spec. Don't skip the second half. +### Append-only is literal, and it is checked by diffing +A story chapter is never edited once written — not to correct a claim the +design has since retired, and **not** to append a forward pointer to it. A new +chapter goes at the **end of the file**, never between chapters that already +exist. Say what stopped being true from the *new* chapter, naming the older +chapter's claim. Both of these were caught in review on this repo rather than +by the author, so verify before committing: + +```sh +git diff origin/main -- stories/.md | grep -E "^-[^-]" +``` + +Additions only is the passing result. Any output means an existing chapter was +edited, or a chapter was inserted instead of appended. Story guide §5 has the +reasoning. + ### Interview the maintainer — you cannot reconstruct the real reasoning The actual thread — which roads were tried and rejected, in what order the realizations came, what pressure forced each turn — lives only in the diff --git a/contributing/writing-stories-docs.md b/contributing/writing-stories-docs.md index ec4e5f0..3a01196 100644 --- a/contributing/writing-stories-docs.md +++ b/contributing/writing-stories-docs.md @@ -120,7 +120,20 @@ The href ends in the chapter's heading **anchor** so the link scrolls straight t This is the discipline that makes the folder a *history* rather than a stale snapshot. -**Append, don't overwrite.** When the design changes, the old reasoning did not become false — it became *the previous chapter*. So when the spec moves, add to the story: open a new chapter (or extend the relevant one) that names the cause and what it forced — *"The shift to X meant the old Y no longer held, so we…"* — and pin its spec references to the new commit (§4.2). The discarded path stays on the page as the record of why the design used to be one way and is now another; that causal trail is often the most illuminating thing in the file, and rewriting it away destroys it. +**Append, don't overwrite.** When the design changes, the old reasoning did not become false — it became *the previous chapter*. So when the spec moves, add to the story: open a **new chapter at the end of the file** that names the cause and what it forced — *"The shift to X meant the old Y no longer held, so we…"* — and pin its spec references to the new commit (§4.2). The discarded path stays on the page as the record of why the design used to be one way and is now another; that causal trail is often the most illuminating thing in the file, and rewriting it away destroys it. + +"Append" is meant literally, and it has two teeth: + +- **Do not touch an existing chapter — at all.** Not to correct a claim the design has since retired, and not to bolt a forward pointer onto the end of it. A chapter records what was true when it was written; a later chapter is where you say what stopped being true and why. Naming the superseded claim explicitly *from the new chapter* — "the segmented-offset chapter had promotion rewrite the one anchor cell; that holds only while…" — does the same job for the reader without editing history. +- **A new chapter goes at the end**, never slotted between chapters that already exist. Chapter order is the order the thinking moved, and the file's tail is the present. Inserting into the middle rewrites the sequence even when no existing character changes. + +**Verify it by diffing.** Before committing a story change, check it against the branch you are merging into: + +```sh +git diff origin/main -- stories/.md | grep -E "^-[^-]" +``` + +Any output is a violation: a removed or rewritten line means an existing chapter was edited, and a `-` next to a chapter heading means a chapter was inserted rather than appended. The clean result is additions only. **Consolidate dead threads, sparingly.** Appending forever would bury the present under history. So a chapter *may* be rewritten or folded down — but only when its narrative has become pure dead weight: it no longer illuminates the present design *and* is not interesting as history. That is a high bar. The default is to append; consolidation is the rare exception, not routine cleanup, and when in doubt you keep the history. From 5b7616a7fd97906fd985571abbcfb9de7f1b4c74 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 13:44:22 +0000 Subject: [PATCH 27/31] docs: the receiver is never a swallow position, and say why properly MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Maintainer asked why bare `T` borrows on `this` when the default for `T` is swallowing. Fair — the justification I had written was wrong. It said the borrow "has to be" the bare meaning because otherwise a method would swallow its own receiver, which implies the swallow reading was ever on the table. It never was. main already says a reference receiver is an implicit `&` reference "(never swallowed)", and memory.md §2.9 on main says the same. So the receiver has never been a swallow position, and bare `T` meaning swallow is a rule about ordinary parameters. What this PR changes is only *which* non-swallowing mode a bare receiver is — guest before, borrow now — because the call-site receiver is usually a bare symbol, which §2.8.1 no longer admits as a guest source. §2.9 now states that, says plainly that bare `T` does not mean the same thing in both positions, and names the pre-existing rule that makes it consistent rather than arguing from convenience. The §2.9 mode table is scoped to parameters, and functions.md §2.4 and both summary rows match. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_017AYiJXjzCPEDfYRW1ZtxH7 --- spec/functions.md | 6 +++--- spec/memory.md | 8 +++++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/spec/functions.md b/spec/functions.md index 95f228c..17fe14a 100644 --- a/spec/functions.md +++ b/spec/functions.md @@ -56,12 +56,12 @@ A method marked `mut` may write to any state reachable through `this`, whether t A write to `this` lands on the caller's object; how `this` reaches the caller differs by kind (see [`memory.md`](memory.md) §2.9): - For a **value-type** receiver, `this` is a **mutable borrow** of the caller's slot — the actual value, not a copy. The borrow makes the value mutable in place while preserving its value semantics. Because the borrow is scoped and non-escaping, `this` may be read and written but cannot be stored as an `&` or returned as one, since a value type is not `&`-rootable. -- For a **reference-type** receiver, `this` is a **mutable borrow** too, and for the same reason: the receiver expression at the call site is usually a bare symbol, which is not a guest source ([`memory.md`](memory.md) §2.8.1). A bare `this T` on a reference type *is* that borrow — **`'` is never written on `this`** — so the receiver is never swallowed and the caller stays a full host. +- For a **reference-type** receiver, `this` is a **mutable borrow** too. The receiver is never a swallow position — a method does not consume its own receiver — so bare `this T` here is the borrow rather than the swallow it would be on an ordinary parameter, and **`'` is never written on `this`**. This is where a reference receiver's implicit `&` went: the call-site receiver is usually a bare symbol, which is not a guest source ([`memory.md`](memory.md) §2.8.1), so the implicit mode became the borrow. Either way the caller stays a full host. A method that needs to keep the receiver past the call — store it in an `&` field, or return it as `&T` ([`lifetimes.md`](lifetimes.md) §1.7) — declares `this &T` instead. That is a guest receiver, so the call site must supply a guest source. ```zane -Unit setScale(this Node, scale Float) mut { // reference receiver: implicit `'Node` borrow +Unit setScale(this Node, scale Float) mut { // reference receiver: the implicit borrow this.scale = scale return Unit() } @@ -416,7 +416,7 @@ Read-only methods and functions are effect-free with respect to their receiver u | `&` method parameter | Caller must supply a guest source (never a bare symbol); callee may store it into `&` fields or return it | | `'T` method parameter | Caller may supply any place expression, bare symbols included; read and `mut` access for the call only; **MUST NOT** be stored, returned, or moved | | Plain `T` method parameter | Swallows; caller may supply a temporary and downgrades to a guest; callee **MUST NOT** bind it into `&` storage | -| Reference receiver | Bare `this T` is the borrow receiver — `'` is never written on `this`; `this &T` is a guest receiver, required to store or return the receiver | +| Reference receiver | Never a swallow position: bare `this T` is the borrow receiver — `'` is never written on `this` — and `this &T` is a guest receiver, required to store or return the receiver | | Subscript | Package-scope place projection written `(this T)[...] => placeExpr`; no explicit return type | | Overload identity | Parameter types only; not names, return type, or `mut`; overloads differing only by the passing mode (`T` / `&T` / `'T`) at one position are illegal | | Overload resolution phases | Direct match, then generic match, then implicit match; ambiguity within any one phase is an error | diff --git a/spec/memory.md b/spec/memory.md index 62785ef..4e85d92 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -175,7 +175,7 @@ A **borrow** is non-hosting, non-escaping access to a caller's storage for the d That restriction is on the borrow, not on what is read through one. A value type is *always* passed this way — a value-type parameter is a **read-only borrow** of the caller's slot — and binding through that borrow into a fresh slot (an assignment, a new declaration, or a field or return store) **copies** the value. The copy is a new value that outlives the call perfectly well; what does not escape is the borrow. A reference type has no such copy, so a `'T` borrow leaves nothing behind at all. -A **reference type** has three passing modes, one per surface form: +A **reference type** parameter has three passing modes, one per surface form. The receiver is not one of these positions and has its own rule, below: | Mode | Written | Caller supplies | The callee may | |---|---|---|---| @@ -223,7 +223,9 @@ Int inspect(this Car, engine 'Engine) { } ``` -A reference-type receiver borrows by default, and **`'` is never written on `this`**: a bare `this T` on a reference type *is* the borrow. It has to be — if the marker were required here, bare `this T` would mean the method swallows its own receiver, which is not something a method wants. A method that needs to keep or hand back the receiver writes `this &T`, the guest receiver (see [`functions.md`](functions.md) §2.4). So `this` carries at most one marker, `&`, and its absence means borrow for value and reference receivers alike. +**The receiver is never a swallow position.** A method does not consume its own receiver, so `this` chooses between only two of the three modes: it is a **borrow** written bare, or a **guest** written `this &T` when the method stores or returns the receiver (see [`functions.md`](functions.md) §2.4). `'` is **never** written on `this`. + +So bare `T` does not mean the same thing in both positions — on an ordinary parameter it swallows, on `this` it borrows — because the receiver was never a swallow position to begin with. That much predates the borrow mode: a reference receiver used to be an implicit **guest**, likewise never swallowed. What changed is only *which* non-swallowing mode a bare receiver is, and it moved to the borrow because the receiver expression at a call site is usually a bare symbol, which §2.8.1 no longer admits as a guest source. Value and reference receivers now agree: `this` carries at most one marker, `&`, and its absence means borrow. Binding a swallowed or borrowed parameter into `&` storage is illegal. A swallowed value is hosted at the call site while an `&` field lives with the object that holds it — which may outlive the call. A borrow does not survive the call at all: @@ -559,7 +561,7 @@ A single global free stack and frontier require synchronization under concurrent | Value-type parameter | Always a read-only borrow; caller need not supply a place; copied only when bound into a fresh slot (assignment, declaration, field or return store) | | Reference-type parameter | `T` swallows (hosting access; passing a host downgrades the caller's symbol to a guest whatever the body does — see [`lifetimes.md`](lifetimes.md) §1.8); `&T` takes a guest, which only a guest source can supply; `'T` borrows any place, bare symbols included, and leaves the caller a full host | | `'T` position | Parameter positions only; never a storage, field, or return type | -| Reference-type receiver | Bare `this T` is the borrow receiver — `'` is never written on `this`; `this &T` is a guest receiver a method may store or return | +| Reference-type receiver | Never a swallow position: bare `this T` is the borrow receiver — `'` is never written on `this` — and `this &T` is a guest receiver a method may store or return | | Value-downstream enforcement | Value types may contain only primitives and other value types, transitively — never a reference (`#`) or `&` field | | `&` targets reference types | An `&T` requires `T` to be a reference type; a value is shared by copy or scoped borrow, never by a stored `&` | | Symbol declaration | Must be directly initialized | From ad392248c0ed54ed2b08ad8e7d551835de5cc1c3 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 16:24:44 +0000 Subject: [PATCH 28/31] docs: say which sense of "receiver" each rule means MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "Receiver" was doing three jobs at once — the object a method is called on, the `this` parameter that fixes the passing mode, and the call-site expression that supplies the object — and the rule that it is never a swallow position is about the second of those, not the first. Define the three in functions.md §2.1, add a glossary entry, and use the precise term wherever the distinction carries weight. --- spec/functions.md | 6 ++++-- spec/glossary.md | 7 ++++++- spec/memory.md | 10 +++++----- 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/spec/functions.md b/spec/functions.md index 17fe14a..c06c75f 100644 --- a/spec/functions.md +++ b/spec/functions.md @@ -26,6 +26,8 @@ Zane unifies methods, functions, and lambdas under one model: a callable is a pa ### 2.1 Methods are verbs whose first parameter is `this` A method is any package-scope verb whose first parameter is named `this`. `this` **MUST** be the first parameter and **MUST NOT** appear in any other parameter position. +The **receiver** is the object a method is called on. Two things are named after it and are not interchangeable: `this` is the **receiver parameter** — the declaration's first parameter, whose surface form fixes how the object reaches the body ([`memory.md`](memory.md) §2.9) — and the expression to the left of `:` or `!` at a call site is the **receiver expression**, which supplies the object and must satisfy whatever that form requires. Unqualified, "the receiver" means the object itself. + ```zane Int scaledId(this Node, factor Int) { return this._id * factor @@ -56,7 +58,7 @@ A method marked `mut` may write to any state reachable through `this`, whether t A write to `this` lands on the caller's object; how `this` reaches the caller differs by kind (see [`memory.md`](memory.md) §2.9): - For a **value-type** receiver, `this` is a **mutable borrow** of the caller's slot — the actual value, not a copy. The borrow makes the value mutable in place while preserving its value semantics. Because the borrow is scoped and non-escaping, `this` may be read and written but cannot be stored as an `&` or returned as one, since a value type is not `&`-rootable. -- For a **reference-type** receiver, `this` is a **mutable borrow** too. The receiver is never a swallow position — a method does not consume its own receiver — so bare `this T` here is the borrow rather than the swallow it would be on an ordinary parameter, and **`'` is never written on `this`**. This is where a reference receiver's implicit `&` went: the call-site receiver is usually a bare symbol, which is not a guest source ([`memory.md`](memory.md) §2.8.1), so the implicit mode became the borrow. Either way the caller stays a full host. +- For a **reference-type** receiver, `this` is a **mutable borrow** too. The receiver parameter is never a swallow position — a method does not consume the object it is called on — so bare `this T` here is the borrow rather than the swallow it would be on an ordinary parameter, and **`'` is never written on `this`**. This is where a bare reference-type `this`'s implicit `&` went: the receiver expression is usually a bare symbol, which is not a guest source ([`memory.md`](memory.md) §2.8.1), so the implicit mode became the borrow. Either way the caller stays a full host. A method that needs to keep the receiver past the call — store it in an `&` field, or return it as `&T` ([`lifetimes.md`](lifetimes.md) §1.7) — declares `this &T` instead. That is a guest receiver, so the call site must supply a guest source. @@ -416,7 +418,7 @@ Read-only methods and functions are effect-free with respect to their receiver u | `&` method parameter | Caller must supply a guest source (never a bare symbol); callee may store it into `&` fields or return it | | `'T` method parameter | Caller may supply any place expression, bare symbols included; read and `mut` access for the call only; **MUST NOT** be stored, returned, or moved | | Plain `T` method parameter | Swallows; caller may supply a temporary and downgrades to a guest; callee **MUST NOT** bind it into `&` storage | -| Reference receiver | Never a swallow position: bare `this T` is the borrow receiver — `'` is never written on `this` — and `this &T` is a guest receiver, required to store or return the receiver | +| Reference-type `this` | Never a swallow position: bare `this T` is the borrow receiver — `'` is never written on `this` — and `this &T` is a guest receiver, required to store or return the receiver | | Subscript | Package-scope place projection written `(this T)[...] => placeExpr`; no explicit return type | | Overload identity | Parameter types only; not names, return type, or `mut`; overloads differing only by the passing mode (`T` / `&T` / `'T`) at one position are illegal | | Overload resolution phases | Direct match, then generic match, then implicit match; ambiguity within any one phase is an error | diff --git a/spec/glossary.md b/spec/glossary.md index 5334447..39a8aa2 100644 --- a/spec/glossary.md +++ b/spec/glossary.md @@ -245,10 +245,15 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`memory.md`](memory.md) §2.8.1 ### 3.37 passing mode -- **Meaning:** Which of three ways a reference-type argument reaches a callee, fixed entirely by the parameter's surface form: `T` **swallows** it (hosting access; the caller downgrades to a guest), `&T` takes a **guest** (storable and returnable; requires a guest source), `'T` **borrows** it (read and `mut` for the call only; accepts any place, bare symbols included). The receiver selects between the borrow and `&T` only: a bare `this T` is the borrow and `'` is never written on `this`. Two overloads may not differ only by the mode at one position. +- **Meaning:** Which of three ways a reference-type argument reaches a callee, fixed entirely by the parameter's surface form: `T` **swallows** it (hosting access; the caller downgrades to a guest), `&T` takes a **guest** (storable and returnable; requires a guest source), `'T` **borrows** it (read and `mut` for the call only; accepts any place, bare symbols included). The receiver parameter (§3.38) selects between the borrow and `&T` only: a bare `this T` is the borrow and `'` is never written on `this`. Two overloads may not differ only by the mode at one position. - **Why this name:** "Mode" names a choice about *how* the same argument travels rather than *what* it is — the type is unchanged in all three, and only the caller's obligations and resulting state differ. - **Canonical home:** [`memory.md`](memory.md) §2.9 +### 3.38 receiver / receiver parameter / receiver expression +- **Meaning:** The **receiver** is the object a method is called on. The **receiver parameter** is `this`, the declaration's first parameter, whose surface form fixes the passing mode (§3.37) — bare for the borrow, `this &T` for the guest, never `'`. The **receiver expression** is what stands left of `:` or `!` at the call site and supplies the object; it must satisfy what that form requires, which is why a bare symbol works for a bare `this` but not for `this &T` (§3.36). +- **Why this name:** The three are one word in ordinary use because they usually coincide; the spec separates them where a rule holds of the declaration but not the object, or the other way round. +- **Canonical home:** [`functions.md`](functions.md) §2.1 + --- ## 4. Packages, Operators, and Versioning diff --git a/spec/memory.md b/spec/memory.md index 4e85d92..bbe46c0 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -55,7 +55,7 @@ Rewriting `hosts[1]` replaces the hosted reference-type instance in that slot. G ### 2.3 Value types are mutable in place and freely overwritable -Value types have no anchor and no heap identity. A value is mutated in place through a `mut` method whose receiver is a borrow of the value's storage (see [`effects.md`](effects.md) §2.3, [`functions.md`](functions.md) §2.4), and its storage slot may also be reassigned wholesale. Neither operation goes through the anchor system, because a value has no identity to track. +Value types have no anchor and no heap identity. A value is mutated in place through a `mut` method whose `this` is a borrow of the value's storage (see [`effects.md`](effects.md) §2.3, [`functions.md`](functions.md) §2.4), and its storage slot may also be reassigned wholesale. Neither operation goes through the anchor system, because a value has no identity to track. ```zane pos Vec2(1, 2) @@ -175,7 +175,7 @@ A **borrow** is non-hosting, non-escaping access to a caller's storage for the d That restriction is on the borrow, not on what is read through one. A value type is *always* passed this way — a value-type parameter is a **read-only borrow** of the caller's slot — and binding through that borrow into a fresh slot (an assignment, a new declaration, or a field or return store) **copies** the value. The copy is a new value that outlives the call perfectly well; what does not escape is the borrow. A reference type has no such copy, so a `'T` borrow leaves nothing behind at all. -A **reference type** parameter has three passing modes, one per surface form. The receiver is not one of these positions and has its own rule, below: +A **reference type** parameter has three passing modes, one per surface form. The receiver parameter `this` is not one of these positions and has its own rule, below: | Mode | Written | Caller supplies | The callee may | |---|---|---|---| @@ -223,9 +223,9 @@ Int inspect(this Car, engine 'Engine) { } ``` -**The receiver is never a swallow position.** A method does not consume its own receiver, so `this` chooses between only two of the three modes: it is a **borrow** written bare, or a **guest** written `this &T` when the method stores or returns the receiver (see [`functions.md`](functions.md) §2.4). `'` is **never** written on `this`. +**The receiver parameter is never a swallow position.** A method does not consume the object it is called on, so `this` — the first parameter, and only it ([`functions.md`](functions.md) §2.1) — chooses between two of the three modes rather than all three: it is a **borrow** written bare, or a **guest** written `this &T` when the method stores the receiver past the call or returns it as `&T` (see [`functions.md`](functions.md) §2.4). `'` is **never** written on `this`. -So bare `T` does not mean the same thing in both positions — on an ordinary parameter it swallows, on `this` it borrows — because the receiver was never a swallow position to begin with. That much predates the borrow mode: a reference receiver used to be an implicit **guest**, likewise never swallowed. What changed is only *which* non-swallowing mode a bare receiver is, and it moved to the borrow because the receiver expression at a call site is usually a bare symbol, which §2.8.1 no longer admits as a guest source. Value and reference receivers now agree: `this` carries at most one marker, `&`, and its absence means borrow. +So bare `T` does not mean the same thing in both positions — on an ordinary parameter it swallows, on `this` it borrows — because `this` was never a swallow position to begin with. That much predates the borrow mode: a bare reference-type `this` used to be an implicit **guest**, likewise never swallowed. What changed is only *which* non-swallowing mode it is, and it moved to the borrow because the receiver expression at a call site is usually a bare symbol, which §2.8.1 no longer admits as a guest source. Value and reference receivers now agree: `this` carries at most one marker, `&`, and its absence means borrow. Binding a swallowed or borrowed parameter into `&` storage is illegal. A swallowed value is hosted at the call site while an `&` field lives with the object that holds it — which may outlive the call. A borrow does not survive the call at all: @@ -561,7 +561,7 @@ A single global free stack and frontier require synchronization under concurrent | Value-type parameter | Always a read-only borrow; caller need not supply a place; copied only when bound into a fresh slot (assignment, declaration, field or return store) | | Reference-type parameter | `T` swallows (hosting access; passing a host downgrades the caller's symbol to a guest whatever the body does — see [`lifetimes.md`](lifetimes.md) §1.8); `&T` takes a guest, which only a guest source can supply; `'T` borrows any place, bare symbols included, and leaves the caller a full host | | `'T` position | Parameter positions only; never a storage, field, or return type | -| Reference-type receiver | Never a swallow position: bare `this T` is the borrow receiver — `'` is never written on `this` — and `this &T` is a guest receiver a method may store or return | +| Reference-type `this` | Never a swallow position: bare `this T` is the borrow receiver — `'` is never written on `this` — and `this &T` is a guest receiver a method may store or return | | Value-downstream enforcement | Value types may contain only primitives and other value types, transitively — never a reference (`#`) or `&` field | | `&` targets reference types | An `&T` requires `T` to be a reference type; a value is shared by copy or scoped borrow, never by a stored `&` | | Symbol declaration | Must be directly initialized | From 97e5bf285d917a00a3705df2e2b3ae3389cd2f17 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 17:17:33 +0000 Subject: [PATCH 29/31] docs: rename receiver to subject MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "Receiver" is Smalltalk residue — a call was a message and the object received it. Zane has no messages, so the word named nothing: it neither fought its old meaning nor fed the new one, it was simply dead metaphor. "Subject" puts the concept in the register the spec already chose when it named callables verbs. A call reads subject-verb-object, and the subject is what the verb acts from. Merged story chapters keep the old word — they record the language as it was — so the term is split across the two trees by construction. Also records that append-only is per pull request, not per commit: chapters a branch has not merged yet are still drafts. --- CLAUDE.md | 24 +++++--- README.md | 4 +- contributing/naming-terms.md | 7 ++- contributing/writing-stories-docs.md | 8 ++- spec/concurrency.md | 12 ++-- spec/effects.md | 20 +++---- spec/foundations.md | 4 +- spec/functions.md | 62 ++++++++++---------- spec/generics.md | 6 +- spec/glossary.md | 16 ++--- spec/memory.md | 14 ++--- spec/operators.md | 2 +- spec/packages.md | 4 +- spec/syntax.md | 88 ++++++++++++++-------------- spec/types.md | 20 +++---- stories/lifetimes.md | 2 +- stories/memory.md | 2 +- 17 files changed, 154 insertions(+), 141 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index a1109d3..3c953c5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -111,21 +111,29 @@ halves: write the narrative, then integrate it into the spec. Don't skip the second half. ### Append-only is literal, and it is checked by diffing -A story chapter is never edited once written — not to correct a claim the -design has since retired, and **not** to append a forward pointer to it. A new -chapter goes at the **end of the file**, never between chapters that already -exist. Say what stopped being true from the *new* chapter, naming the older -chapter's claim. Both of these were caught in review on this repo rather than -by the author, so verify before committing: +A **merged** story chapter is never edited — not to correct a claim the design +has since retired, and **not** to append a forward pointer to it. A new chapter +goes after every merged one, never between chapters already on `main`. Say what +stopped being true from the *new* chapter, naming the older chapter's claim. +Both of these were caught in review on this repo rather than by the author, so +verify before committing: ```sh git diff origin/main -- stories/.md | grep -E "^-[^-]" ``` -Additions only is the passing result. Any output means an existing chapter was -edited, or a chapter was inserted instead of appended. Story guide §5 has the +Additions only is the passing result. Any output means a merged chapter was +edited, or a chapter was inserted ahead of one. Story guide §5 has the reasoning. +**The frozen unit is the PR, not the commit.** Chapters your own branch adds are +drafts until it merges: rewrite them, reorder them, insert a new chapter among +them freely — a decision reached late in review often belongs *before* the +chapters already drafted. The grep above is exactly the right check because it +diffs against `main`, so it stays quiet through all of that and fires only when +something merged moves. Don't over-apply the rule to your own unmerged work; a +previous session did, and had to be corrected by the maintainer. + ### Interview the maintainer — you cannot reconstruct the real reasoning The actual thread — which roads were tried and rejected, in what order the realizations came, what pressure forced each turn — lives only in the diff --git a/README.md b/README.md index d966e61..ac81053 100644 --- a/README.md +++ b/README.md @@ -66,9 +66,9 @@ The spec states *what* the language does; the **why** lives in a parallel set of | [`stories/adt.md`](stories/adt.md) | [`spec/adt.md`](spec/adt.md) — splitting `enum` from `variant` against the hype, the shared struct body, escaping the matcher machine with case overloads and the turn to a central `match` block, matching variants rather than patterns, keeping enum data outside the members, reducing a match group to sugar for one arm per case, and building a variant by naming a case rather than calling a constructor | | [`stories/generics.md`](stories/generics.md) | [`spec/generics.md`](spec/generics.md) — the parameter model, the `<>`/`()` split, size-in-the-type, and the deferred features | | [`stories/dependencies.md`](stories/dependencies.md) | [`spec/dependencies.md`](spec/dependencies.md) — URL identity, the manifest/resolution split, prebuilt distribution, symbol-rewriting, the browsable global cache, the package-graph acyclicity rule, opt-in remapping, and why `core` became a bundled implementation package | -| [`stories/memory.md`](stories/memory.md) | [`spec/memory.md`](spec/memory.md) — the no-GC-no-lifetimes goal, the move problem and the anchor, lazy backpointer creation, the indexed heap table, the rooted-guest rules and the host/guest terminology split, the collapse to one value/reference axis with a borrowed receiver, the shift to segmented chunked bump arenas, the split into fixed-size and dynamic regions with anchors moved to a runtime-global recyclable pool, taking the bare symbol away as a guest source, and the three passing modes that split out of it | +| [`stories/memory.md`](stories/memory.md) | [`spec/memory.md`](spec/memory.md) — the no-GC-no-lifetimes goal, the move problem and the anchor, lazy backpointer creation, the indexed heap table, the rooted-guest rules and the host/guest terminology split, the collapse to one value/reference axis with a borrowed subject, the shift to segmented chunked bump arenas, the split into fixed-size and dynamic regions with anchors moved to a runtime-global recyclable pool, taking the bare symbol away as a guest source, and the three passing modes that split out of it | | [`stories/lifetimes.md`](stories/lifetimes.md) | [`spec/lifetimes.md`](spec/lifetimes.md) — lexical scope in place of a borrow checker, what may be moved, the declaration-block rule that kills flow analysis, downgrade instead of use-after-move, parameter-rooted returned guests, why each strict rule is the minimal guard against one specific memory corruption, and narrowing a returned guest's root to a guest parameter once borrows arrived | -| [`stories/effects.md`](stories/effects.md) | [`spec/effects.md`](spec/effects.md) — inferring effects instead of annotating them, receiver-scoped `mut`, capabilities in place of ambient I/O, the four-level ladder and the Total-Pure/Pure split, what deliberately is not an effect, and mutation through a borrowed receiver | +| [`stories/effects.md`](stories/effects.md) | [`spec/effects.md`](spec/effects.md) — inferring effects instead of annotating them, subject-scoped `mut`, capabilities in place of ambient I/O, the four-level ladder and the Total-Pure/Pure split, what deliberately is not an effect, and mutation through a borrowed subject | | [`stories/concurrency.md`](stories/concurrency.md) | [`spec/concurrency.md`](spec/concurrency.md) — the parallelism/concurrency split and the refusal of `async` coloring, why `spawn` marks only a call, water-tower lifetimes, signature-based safety without locks, and value-typed mutation closing the aliased-write gap | | [`stories/error-handling.md`](stories/error-handling.md) | [`spec/error-handling.md`](spec/error-handling.md) — the two-doors model and why failure is control flow rather than a `Result` value, `resolve` as expression-substitution rather than assignment, typed abort paths and the deliberately-absent propagate operator, keeping abortability orthogonal to effects, and explicit path values through `Unit` | | [`stories/control-flow.md`](stories/control-flow.md) | [`spec/control-flow.md`](spec/control-flow.md) — `guard` as an active exit that opens no scope of its own, doing without `while` behind a written loop bound, one-based counting after the loop that forced the question, and why control-flow contracts use fundamental semantic types | diff --git a/contributing/naming-terms.md b/contributing/naming-terms.md index 878f965..ff71ff5 100644 --- a/contributing/naming-terms.md +++ b/contributing/naming-terms.md @@ -2,7 +2,7 @@ This guide describes how the spec chooses the coined terms it reuses — the named concepts recorded in [`glossary.md`](../spec/glossary.md), such as `verb`, -`mould`, `borrow`, `host`, `guest`, `anchor`, and `tether`. It governs the *terms of art* the +`subject`, `mould`, `borrow`, `host`, `guest`, `anchor`, and `tether`. It governs the *terms of art* the documentation leans on, not the surface keywords of the language itself. Terminology is worth naming deliberately because a good term is used on nearly @@ -20,6 +20,9 @@ does the teaching before the definition is even read. - **`verb`** — a function, method, operator, constructor, or lambda. In grammar a verb is the word that *acts*; a callable is the construct that *does* work. +- **`subject`** — the object a method is called on. Grammar again, and the same + sentence: the subject is what the verb acts from, so `player!setScale(...)` + reads subject–verb–object down the line. - **`mould`** — a `struct`/`variant`/`enum` form. A mould gives shapeless material a fixed form; these forms give a type its shape, and the type is what is cast from them. @@ -99,7 +102,7 @@ seen rarely and gains its meaning slowly, so an oblique reference like *Ariadne* (the thread through the labyrinth) is a strength. A **term** is the opposite case: read constantly, and needed to teach on contact. -Terms therefore lean plain and everyday — `verb`, `mould`, `borrow`, `host`, +Terms therefore lean plain and everyday — `verb`, `subject`, `mould`, `borrow`, `host`, `guest`, `anchor`, `tether` — even when the underlying instinct (name by metaphor, keep the link oblique) is the same. When in doubt for a term, choose the ordinary word over the exotic one. diff --git a/contributing/writing-stories-docs.md b/contributing/writing-stories-docs.md index 3a01196..b8b48ed 100644 --- a/contributing/writing-stories-docs.md +++ b/contributing/writing-stories-docs.md @@ -124,8 +124,10 @@ This is the discipline that makes the folder a *history* rather than a stale sna "Append" is meant literally, and it has two teeth: -- **Do not touch an existing chapter — at all.** Not to correct a claim the design has since retired, and not to bolt a forward pointer onto the end of it. A chapter records what was true when it was written; a later chapter is where you say what stopped being true and why. Naming the superseded claim explicitly *from the new chapter* — "the segmented-offset chapter had promotion rewrite the one anchor cell; that holds only while…" — does the same job for the reader without editing history. -- **A new chapter goes at the end**, never slotted between chapters that already exist. Chapter order is the order the thinking moved, and the file's tail is the present. Inserting into the middle rewrites the sequence even when no existing character changes. +- **Do not touch a published chapter — at all.** Not to correct a claim the design has since retired, and not to bolt a forward pointer onto the end of it. A chapter records what was true when it was written; a later chapter is where you say what stopped being true and why. Naming the superseded claim explicitly *from the new chapter* — "the segmented-offset chapter had promotion rewrite the one anchor cell; that holds only while…" — does the same job for the reader without editing history. +- **A new chapter goes after every published one**, never slotted between chapters that already exist. Chapter order is the order the thinking moved, and the file's tail is the present. Inserting into the middle rewrites the sequence even when no existing character changes. + +**The unit of publication is the pull request, not the commit.** "Published" means merged — what is on the default branch. The chapters a PR is *itself* adding are still draft until it lands, so within that PR they may be rewritten, reordered, or have a new chapter inserted among them, however many commits it takes. A design decision reached late in review often belongs *before* the chapters already drafted on the branch, and putting it there is not a violation. What must not move is anything that was already merged. **Verify it by diffing.** Before committing a story change, check it against the branch you are merging into: @@ -133,7 +135,7 @@ This is the discipline that makes the folder a *history* rather than a stale sna git diff origin/main -- stories/.md | grep -E "^-[^-]" ``` -Any output is a violation: a removed or rewritten line means an existing chapter was edited, and a `-` next to a chapter heading means a chapter was inserted rather than appended. The clean result is additions only. +Any output is a violation: a removed or rewritten line means a published chapter was edited, and a `-` next to a chapter heading means a chapter was inserted ahead of one that had already merged. The clean result is additions only — which is also why the check is the right one to run: it compares against what is published, so it stays silent while you rearrange your own branch's new chapters and speaks up the moment you disturb a merged one. **Consolidate dead threads, sparingly.** Appending forever would bury the present under history. So a chapter *may* be rewritten or folded down — but only when its narrative has become pure dead weight: it no longer illuminates the present design *and* is not interesting as history. That is a high bar. The default is to append; consolidation is the rare exception, not routine cleanup, and when in doubt you keep the history. diff --git a/spec/concurrency.md b/spec/concurrency.md index f3b2620..5c354e2 100644 --- a/spec/concurrency.md +++ b/spec/concurrency.md @@ -13,7 +13,7 @@ Zane separates **parallelism** (compiler-managed, unobservable) from **concurren - **`Implicit parallelism`.** The compiler may run provably independent work in parallel when it cannot change program results. - **`Explicit concurrency`.** `spawn` starts a concurrent function or method call; ordering is the programmer’s responsibility. - **`Water-tower lifetimes`.** A scope’s hosted objects live until all spawned work in that scope completes. -- **`Mutation needs a value receiver`.** A spawned call may mutate only a value-typed receiver; a value type's transitive alias-freedom lets the compiler rule out a data race from the receiver's type, and at most one spawn may mutably borrow a given location. +- **`Mutation needs a value subject`.** A spawned call may mutate only a value-typed subject; a value type's transitive alias-freedom lets the compiler rule out a data race from the subject's type, and at most one spawn may mutably borrow a given location. - **`No async coloring`.** Concurrency is chosen at the call site rather than encoded into function signatures. --- @@ -115,13 +115,13 @@ Each time one spawned call finishes, one plate is removed. The water level drops > **Story:** [`stories/concurrency.md`](../stories/concurrency.md#the-water-tower-lifetimes-that-survive-the-spawn) — "The water tower: lifetimes that survive the spawn". -### 4.2 Concurrent mutation requires a value-typed receiver -A spawned call may **mutate** state only through a value-typed receiver. A spawned `mut` call whose receiver is a reference type (a `#`-marked type) is a compile-time error. The rule is sound because a value type is transitively alias-free — it contains no reference-type or `&` field anywhere downstream (see [`memory.md`](memory.md) §2.10) — so no two names can reach the same mutated object by different paths. The compiler therefore rules out an aliased data race from the receiver's *type* alone, with no whole-program alias analysis. +### 4.2 Concurrent mutation requires a value-typed subject +A spawned call may **mutate** state only through a value-typed subject. A spawned `mut` call whose subject is a reference type (a `#`-marked type) is a compile-time error. The rule is sound because a value type is transitively alias-free — it contains no reference-type or `&` field anywhere downstream (see [`memory.md`](memory.md) §2.10) — so no two names can reach the same mutated object by different paths. The compiler therefore rules out an aliased data race from the subject's *type* alone, with no whole-program alias analysis. A direct consequence is that reference types are never mutated by spawned work, so every concurrent **read** of the reference-typed object graph is safe by construction. ### 4.3 Single writer per storage location -For any one storage location, at most one live spawned call may hold a **mutable borrow** — the `!` receiver of a spawned `mut` call. By §4.2 that receiver is always value-typed, so the borrows this rule counts are value borrows; a reference-type `'T` borrow ([`memory.md`](memory.md) §2.9) never reaches a spawned `mut` receiver. Two spawned calls that mutably borrow the same location are a compile-time error. Because value types carry no `&`, a location's identity is unambiguous — there is no hidden alias to obscure that two receivers denote the same slot — so this disjointness is checked at the spawn site by inspecting the receivers, not by tracing the program. The hosting scope may not access a location while a live spawn holds its mutable borrow; the borrow is released when that spawn completes (§4.1). +For any one storage location, at most one live spawned call may hold a **mutable borrow** — the `!` subject of a spawned `mut` call. By §4.2 that subject is always value-typed, so the borrows this rule counts are value borrows; a reference-type `'T` borrow ([`memory.md`](memory.md) §2.9) never reaches a spawned `mut` subject. Two spawned calls that mutably borrow the same location are a compile-time error. Because value types carry no `&`, a location's identity is unambiguous — there is no hidden alias to obscure that two subjects denote the same slot — so this disjointness is checked at the spawn site by inspecting the subjects, not by tracing the program. The hosting scope may not access a location while a live spawn holds its mutable borrow; the borrow is released when that spawn completes (§4.1). ### 4.4 Reads take a coherent snapshot A spawned call may read a value that another live spawn is mutating; the read observes a **coherent snapshot** of the value rather than blocking. Reading a shared value into a fresh binding — `snap VarType = shared` — is what takes the snapshot, and the copy is tear-free even when the writer is mid-update. This replaces lock-based serialization for in-memory value state, so a real-time reader never waits on a writer. Serialization still applies to external, capability-backed resources (§4.5). @@ -152,7 +152,7 @@ The language does not provide cancellation, kill groups, or shutdown ordering. A > **Story:** [`stories/concurrency.md`](../stories/concurrency.md#what-the-core-deliberately-leaves-out) — "What the core deliberately leaves out". ### 5.2 Lambdas do not capture -Lambdas (and blocks used as values) **MUST NOT** capture outer variables. All dependencies must be passed explicitly. This keeps effect tracking and the value-receiver check (§4.2) tractable. +Lambdas (and blocks used as values) **MUST NOT** capture outer variables. All dependencies must be passed explicitly. This keeps effect tracking and the value-subject check (§4.2) tractable. > **Story:** [`stories/concurrency.md`](../stories/concurrency.md#safety-the-compiler-proves-from-signatures-not-locks) — "Safety the compiler proves from signatures, not locks". @@ -176,4 +176,4 @@ Zane does not define a dedicated `Process` type, actor primitive, or channel pri | `spawn` | Starts a concurrent function or method call; blocks only when results are read | | Abortable `spawn` | Must attach `?` or `??` directly to the spawn expression | | Water tower | A scope exits only after all spawned work completes | -| Mutation | A spawned mutating call requires a value-typed receiver; at most one mutable borrow per storage location; concurrent reads take a coherent snapshot | +| Mutation | A spawned mutating call requires a value-typed subject; at most one mutable borrow per storage location; concurrent reads take a coherent snapshot | diff --git a/spec/effects.md b/spec/effects.md index 7ae0b1c..9414015 100644 --- a/spec/effects.md +++ b/spec/effects.md @@ -11,7 +11,7 @@ This document specifies Zane's effect model: `mut`, inferred effect levels, capa Zane uses a structural effect model with a single user-facing effect modifier: `mut`. - **`No purity keywords`.** Users do not write `pure`, `readonly`, or capability qualifiers. -- **`Receiver-local mutation`.** `mut` grants write access to state reachable through `this`, including through guests. +- **`Subject-local mutation`.** `mut` grants write access to state reachable through `this`, including through guests. - **`Compiler-inferred effect levels`.** The compiler classifies code by what state it can read or write. - **`Capability-based external effects`.** I/O and external state remain explicit because capability objects must be passed or stored. @@ -34,7 +34,7 @@ A capability is an object whose methods model access to external state, such as `mut` is the only effect modifier in the language. It appears on methods and grants write access to state reachable through `this`; the write lands on the caller's object or on state reachable from it. `this` is a **borrow** of the caller's slot for both kinds: a value-type `this` borrows the value, and a reference-type `this` written bare is a borrow of the object, `'` never being written on `this` (see [`functions.md`](functions.md) §2.4). ### 2.4 Parameters are not mutable by default -Parameters other than `this` are read-only. Mutation of another object must be expressed by calling a `mut` method on that object as the receiver. A number parameter that resolves to a number value in body positions (see [`generics.md`](generics.md) §3.5) is a value-like binding and is read-only by default; mutating it requires a `mut` declaration. +Parameters other than `this` are read-only. Mutation of another object must be expressed by calling a `mut` method on that object as the subject. A number parameter that resolves to a number value in body positions (see [`generics.md`](generics.md) §3.5) is a value-like binding and is read-only by default; mutating it requires a `mut` declaration. > **Story:** [`stories/effects.md`](../stories/effects.md#where-mutation-is-allowed-to-reach) — "Where mutation is allowed to reach". @@ -42,7 +42,7 @@ Parameters other than `this` are read-only. Mutation of another object must be e ## 3. Inferred Effect Levels -The compiler assigns a function to the strongest effect level required by any operation in its body or any function it calls transitively. Reading capability-backed state raises a function out of the pure levels; writes through a receiver or to external state raise it to Write Impure. +The compiler assigns a function to the strongest effect level required by any operation in its body or any function it calls transitively. Reading capability-backed state raises a function out of the pure levels; writes through a subject or to external state raise it to Write Impure. ### 3.1 Level 1 — Total Pure Total Pure functions depend only on explicit parameters and immutable package constants. They have no side effects and are guaranteed to terminate for all inputs. @@ -66,16 +66,16 @@ Write Impure functions mutate `this`, mutate capability-backed state, or otherwi A method without `mut` may not assign through `this` or call `mut` methods on state reached through `this`. ### 4.2 `mut` does not authorize arbitrary writes -Even a `mut` method may write only through `this`. It does not gain permission to mutate unrelated parameters. This applies whether the receiver is a value type or a reference type: a value receiver is mutated in place through its borrow (see [`functions.md`](functions.md) §2.4), not by returning a replacement. +Even a `mut` method may write only through `this`. It does not gain permission to mutate unrelated parameters. This applies whether the subject is a value type or a reference type: a value subject is mutated in place through its borrow (see [`functions.md`](functions.md) §2.4), not by returning a replacement. ### 4.3 `&` use sites follow ordinary call rules -Reading through a guest is not a side effect by itself. At use sites, guests follow the same field-access and method-call rules as hosts. Mutation of the hosted object's state must still be expressed through a `mut` method call with that object as the receiver. +Reading through a guest is not a side effect by itself. At use sites, guests follow the same field-access and method-call rules as hosts. Mutation of the hosted object's state must still be expressed through a `mut` method call with that object as the subject. --- ## 5. Structural Inference -### 5.1 Receiver reachability drives effects +### 5.1 Subject reachability drives effects The compiler uses reachability from `this` to determine which state is writable in a `mut` method and readable in any method. ### 5.2 Call-graph propagation @@ -115,10 +115,10 @@ Passing capabilities through constructors and methods is part of the design. It ## 7. Constructors, Allocation, and Abortability ### 7.1 Constructors may allocate but are not `mut` -Constructors create values and therefore participate in allocation, but they do not mutate an existing receiver. +Constructors create values and therefore participate in allocation, but they do not mutate an existing subject. ### 7.2 Allocation and destruction do not by themselves raise effect level -Heap allocation and destruction are runtime implementation events, but they are not side effects by themselves for effect classification. A function stays in the pure levels unless it also mutates receiver-reachable state or reads/writes through capabilities. +Heap allocation and destruction are runtime implementation events, but they are not side effects by themselves for effect classification. A function stays in the pure levels unless it also mutates subject-reachable state or reads/writes through capabilities. ### 7.3 Abortability is orthogonal A function's abort type and effect level are independent. An abortable function may be Total Pure, Read-Only Impure, or Write Impure depending on what else it does. @@ -136,13 +136,13 @@ Because they do not write mutable state, they can be reordered and parallelized Multiple concurrent reads are legal. For external, capability-backed state a read that conflicts with a concurrent write is serialized by the compiler/runtime. For in-memory value state, a concurrent read instead takes a coherent snapshot rather than blocking (see [`concurrency.md`](concurrency.md) §4.4). ### 8.3 Concurrent mutation is governed by the spawn rules -Concurrent mutation is not a per-`mut`-call property; it is governed by the spawn rules in [`concurrency.md`](concurrency.md) §4. A spawned mutating call's receiver **MUST** be a value type, and no two concurrent spawns may mutably borrow the same storage. A value type's transitive alias-freedom (see [`memory.md`](memory.md) §2.10) is what lets the compiler settle the absence of a data race from the receiver's type alone. +Concurrent mutation is not a per-`mut`-call property; it is governed by the spawn rules in [`concurrency.md`](concurrency.md) §4. A spawned mutating call's subject **MUST** be a value type, and no two concurrent spawns may mutably borrow the same storage. A value type's transitive alias-freedom (see [`memory.md`](memory.md) §2.10) is what lets the compiler settle the absence of a data race from the subject's type alone. --- ## 9. Effect Level Matrix -| Level | Reads capability-backed state | Writes receiver-reachable state | May write external state | Compile-time evaluation | +| Level | Reads capability-backed state | Writes subject-reachable state | May write external state | Compile-time evaluation | |---|---|---|---|---| | Total Pure | ❌ | ❌ | ❌ | ✅ | | Pure | ❌ | ❌ | ❌ | ❌ | diff --git a/spec/foundations.md b/spec/foundations.md index 0e8e72a..b36c300 100644 --- a/spec/foundations.md +++ b/spec/foundations.md @@ -89,13 +89,13 @@ Every type is a **value type** unless it is marked with `#`, which makes it a ** A value type is copied on assignment, has no identity, and — the load-bearing restriction — is *transitively* a value: it may contain only other value types, never a reference-type or `&` field. Nothing reachable from a value can be aliased, which is why a value can be copied and shared by snapshot with no bookkeeping, and why a value type cannot recurse (a self-reference would need indirection, and indirection is a reference). A reference type is the opposite in each respect: it has stable identity, may be aliased through `&`, may hold reference-type and `&` fields, and may recurse. -Both kinds are mutated in place through a `mut` method, and the receiver reaches the caller the same way in each: `this` is a *borrow* of the caller's slot, so a value is mutable without gaining identity and a reference object is mutable without minting a guest to it. Borrowing serves both worlds; what the reference world adds on top is `&`, for the cases where a callee must keep the object past the call. +Both kinds are mutated in place through a `mut` method, and the subject reaches the caller the same way in each: `this` is a *borrow* of the caller's slot, so a value is mutable without gaining identity and a reference object is mutable without minting a guest to it. Borrowing serves both worlds; what the reference world adds on top is `&`, for the cases where a callee must keep the object past the call. - **`#` is the only kind modifier**, applied uniformly to any type. See [`types.md`](types.md) §2 and [`adt.md`](adt.md) §2–§3. - **A value type is transitively value** (no reference-type or `&` field, anywhere downstream). This closed value world is specified by [`memory.md`](memory.md) §2.10. - **`&` rides on `#`.** A non-hosting `&` exists only for reference types; a value is shared by copy or by a scoped borrow, never by a stored `&`. See [`memory.md`](memory.md) §2.4. - **A guest comes from a field, not a symbol.** A new `&` is minted only from a qualifying field access — base a place, not reached through a `'T` borrow — or from an `&T` parameter; a bare symbol is a place but never a guest source, so a local's own hosting slot has nothing pointing at it. Such a symbol may still be swallowed by a plain `T` parameter; the borrow mode is the only non-swallowing way to pass one. See [`memory.md`](memory.md) §2.8.1 and §2.9. -- **Concurrency reads this axis.** A spawned call may mutate only a value-typed receiver, because a value's transitive alias-freedom is exactly what lets the compiler rule out a data race from the signature alone. See [`concurrency.md`](concurrency.md) §4. +- **Concurrency reads this axis.** A spawned call may mutate only a value-typed subject, because a value's transitive alias-freedom is exactly what lets the compiler rule out a data race from the signature alone. See [`concurrency.md`](concurrency.md) §4. > **Story:** [`stories/foundations.md`](../stories/foundations.md#identity-is-opt-in-one-axis-for-value-and-reference) — "Identity is opt-in: one axis for value and reference". diff --git a/spec/functions.md b/spec/functions.md index c06c75f..bbc6f67 100644 --- a/spec/functions.md +++ b/spec/functions.md @@ -12,7 +12,7 @@ Zane unifies methods, functions, and lambdas under one model: a callable is a pa - **`Verb`.** A **verb** is a callable whose body is a sequence of statements that executes to do work: functions, methods, operators, constructors, and lambdas (a lambda being an anonymous verb). The spec uses "verb" whenever a rule applies to all of these as a group, and reserves "function" for the narrow form — an ordinary identifier-named verb with no `this`. A subscript is not a verb — its body must be a place expression that projects a place rather than running computation (§2.9). - **`Package-scope behavior`.** All methods, functions, and constructors are declared at package scope; type bodies never contain behavior. -- **`Methods as verbs`.** A method is a verb whose first parameter is `this`, so methods and functions share one model and differ only by the receiver. +- **`Methods as verbs`.** A method is a verb whose first parameter is `this`, so methods and functions share one model and differ only by the subject. - **`Capability markers`.** A verb's kind is selected by surface markers, and each marker unlocks a capability: naming the first parameter `this` grants private-field access (a method); naming the verb after a type grants `init{ }` and an implicit return type (a constructor). See §8. - **`Explicit mutation at the call site`.** `:` calls are read-only; `!` calls invoke `mut` methods. - **`Overload identity is parameter types only`.** Names, return type, and `mut` do not distinguish overloads. @@ -26,7 +26,7 @@ Zane unifies methods, functions, and lambdas under one model: a callable is a pa ### 2.1 Methods are verbs whose first parameter is `this` A method is any package-scope verb whose first parameter is named `this`. `this` **MUST** be the first parameter and **MUST NOT** appear in any other parameter position. -The **receiver** is the object a method is called on. Two things are named after it and are not interchangeable: `this` is the **receiver parameter** — the declaration's first parameter, whose surface form fixes how the object reaches the body ([`memory.md`](memory.md) §2.9) — and the expression to the left of `:` or `!` at a call site is the **receiver expression**, which supplies the object and must satisfy whatever that form requires. Unqualified, "the receiver" means the object itself. +The **subject** is the object a method is called on. Two things are named after it and are not interchangeable: `this` is the **subject parameter** — the declaration's first parameter, whose surface form fixes how the object reaches the body ([`memory.md`](memory.md) §2.9) — and the expression to the left of `:` or `!` at a call site is the **subject expression**, which supplies the object and must satisfy whatever that form requires. Unqualified, "the subject" means the object itself. ```zane Int scaledId(this Node, factor Int) { @@ -35,7 +35,7 @@ Int scaledId(this Node, factor Int) { ``` ### 2.2 `this` grants private-field access -Naming the first parameter `this` is the only thing that makes a declaration a method. That token grants access to `_`-prefixed fields on the receiver type regardless of which package declares the method; home-package status does not matter. The same parameter type written with another name is a function and does not grant private-field access. +Naming the first parameter `this` is the only thing that makes a declaration a method. That token grants access to `_`-prefixed fields on the subject type regardless of which package declares the method; home-package status does not matter. The same parameter type written with another name is a function and does not grant private-field access. ```zane Int scaledId(this Node, factor Int) { @@ -57,24 +57,24 @@ A method marked `mut` may write to any state reachable through `this`, whether t A write to `this` lands on the caller's object; how `this` reaches the caller differs by kind (see [`memory.md`](memory.md) §2.9): -- For a **value-type** receiver, `this` is a **mutable borrow** of the caller's slot — the actual value, not a copy. The borrow makes the value mutable in place while preserving its value semantics. Because the borrow is scoped and non-escaping, `this` may be read and written but cannot be stored as an `&` or returned as one, since a value type is not `&`-rootable. -- For a **reference-type** receiver, `this` is a **mutable borrow** too. The receiver parameter is never a swallow position — a method does not consume the object it is called on — so bare `this T` here is the borrow rather than the swallow it would be on an ordinary parameter, and **`'` is never written on `this`**. This is where a bare reference-type `this`'s implicit `&` went: the receiver expression is usually a bare symbol, which is not a guest source ([`memory.md`](memory.md) §2.8.1), so the implicit mode became the borrow. Either way the caller stays a full host. +- For a **value-type** subject, `this` is a **mutable borrow** of the caller's slot — the actual value, not a copy. The borrow makes the value mutable in place while preserving its value semantics. Because the borrow is scoped and non-escaping, `this` may be read and written but cannot be stored as an `&` or returned as one, since a value type is not `&`-rootable. +- For a **reference-type** subject, `this` is a **mutable borrow** too. The subject parameter is never a swallow position — a method does not consume the object it is called on — so bare `this T` here is the borrow rather than the swallow it would be on an ordinary parameter, and **`'` is never written on `this`**. This is where a bare reference-type `this`'s implicit `&` went: the subject expression is usually a bare symbol, which is not a guest source ([`memory.md`](memory.md) §2.8.1), so the implicit mode became the borrow. Either way the caller stays a full host. -A method that needs to keep the receiver past the call — store it in an `&` field, or return it as `&T` ([`lifetimes.md`](lifetimes.md) §1.7) — declares `this &T` instead. That is a guest receiver, so the call site must supply a guest source. +A method that needs to keep the subject past the call — store it in an `&` field, or return it as `&T` ([`lifetimes.md`](lifetimes.md) §1.7) — declares `this &T` instead. That is a guest subject, so the call site must supply a guest source. ```zane -Unit setScale(this Node, scale Float) mut { // reference receiver: the implicit borrow +Unit setScale(this Node, scale Float) mut { // reference subject: the implicit borrow this.scale = scale return Unit() } ``` ```zane -&Weapon mainWeapon(this &Player) => this.weapon // guest receiver: may be returned as `&` +&Weapon mainWeapon(this &Player) => this.weapon // guest subject: may be returned as `&` ``` ```zane -Unit setY(this Vec2, y Float) mut { // value receiver: in-place through the borrow +Unit setY(this Vec2, y Float) mut { // value subject: in-place through the borrow this.y = y return Unit() } @@ -98,14 +98,14 @@ Calling a `mut` method with `:` is illegal. Calling a non-`mut` method with `!` ### 2.6 Method desugaring ```zane -receiver:method(arg) → ResolvedPkg$method(receiver, arg) -receiver!method(arg) → ResolvedPkg$method(receiver, arg) -receiver:Pkg$method(arg) → Pkg$method(receiver, arg) -receiver!Pkg$method(arg) → Pkg$method(receiver, arg) +subject:method(arg) → ResolvedPkg$method(subject, arg) +subject!method(arg) → ResolvedPkg$method(subject, arg) +subject:Pkg$method(arg) → Pkg$method(subject, arg) +subject!Pkg$method(arg) → Pkg$method(subject, arg) ``` ### 2.7 Parameters are read-only -Explicit parameters other than `this` are read-only: they cannot be assigned or marked `mut`. Mutation of another object must be expressed as a `mut` method call on that object as the receiver. How each parameter is passed — the three reference modes, or a value borrow — is covered in [`memory.md`](memory.md) §2.9. +Explicit parameters other than `this` are read-only: they cannot be assigned or marked `mut`. Mutation of another object must be expressed as a `mut` method call on that object as the subject. How each parameter is passed — the three reference modes, or a value borrow — is covered in [`memory.md`](memory.md) §2.9. ### 2.8 Swallow, guest, and borrow method parameters A reference-type method parameter selects one of three passing modes ([`memory.md`](memory.md) §2.9): @@ -153,7 +153,7 @@ car!setEngine(Engine()) // ILLEGAL: a temporary is not a place expression ``` ### 2.9 Subscripts are place projections -Subscripts are package-scope declarations with the receiver first: +Subscripts are package-scope declarations with the subject first: ```zane (this CustomList)[index Int] => this._data[index] @@ -162,7 +162,7 @@ Subscripts are package-scope declarations with the receiver first: The body of a subscript definition **MUST** be a place expression. `[]` is not a general function call and cannot return a computed value. Its result is always inferred from the projected place, so subscripts have no explicit return type annotation. A subscript may declare any number of comma-separated parameters inside `[]`; it is not limited to one or two. -When a receiver interprets an `Int` subscript as an ordinal position in an ordered sequence, that position is 1-based. The first element is at `1`, and a sequence with `n` elements uses `1` through `n` as its positional range. +When a subject interprets an `Int` subscript as an ordinal position in an ordered sequence, that position is 1-based. The first element is at `1`, and a sequence with `n` elements uses `1` through `n` as its positional range. > **See also:** [`memory.md`](memory.md) §2.8 for when a place expression may create a new `&`. @@ -226,7 +226,7 @@ The return checker does not synthesize a constructor call for `Unit` or any othe ### 4.1 Overload identity is parameter types only Two declarations in the same package conflict when they have the same ordered parameter types. Parameter names, `this`, `mut`, and return type do not distinguish overloads. -Two overloads **MUST NOT** differ only by the **passing mode** at the same parameter position — that is, only by whether that position is `T`, `&T`, or `'T`, the receiver included. Such declarations are illegal and the compiler **MUST** reject them with a compile-time error, for example: "illegal overload set: differs only by the passing mode on a parameter; rename one declaration or choose a single signature." +Two overloads **MUST NOT** differ only by the **passing mode** at the same parameter position — that is, only by whether that position is `T`, `&T`, or `'T`, the subject included. Such declarations are illegal and the compiler **MUST** reject them with a compile-time error, for example: "illegal overload set: differs only by the passing mode on a parameter; rename one declaration or choose a single signature." ```zane Unit consume(this Car, engine Engine) @@ -269,12 +269,12 @@ These phases describe **static** overload resolution. Matching a `variant` on it ## 6. Method Name Resolution and Extension Methods ### 6.1 Unqualified method lookup -For `receiver:methodName(...)` or `receiver!methodName(...)`, the compiler resolves candidates in this order: +For `subject:methodName(...)` or `subject!methodName(...)`, the compiler resolves candidates in this order: -1. the receiver type's home package; for a fundamental type, the bundled `core` implementation package fills this role +1. the subject type's home package; for a fundamental type, the bundled `core` implementation package fills this role 2. the current package -If no candidate matches, the call is a compile-time error. If multiple candidates remain after overload resolution, the call is a compile-time error and must be written with an explicit package qualifier. Searching the receiver type's defining declarations first makes an unqualified call resolve the same way wherever it is written, independent of which packages the caller has imported. +If no candidate matches, the call is a compile-time error. If multiple candidates remain after overload resolution, the call is a compile-time error and must be written with an explicit package qualifier. Searching the subject type's defining declarations first makes an unqualified call resolve the same way wherever it is written, independent of which packages the caller has imported. ### 6.2 Qualified method calls Cross-package extension methods are written explicitly: @@ -284,7 +284,7 @@ vec:Physics$kineticEnergy() ``` ### 6.3 Extension methods may be declared in any package -Because methods are package-scope verbs, any package may define methods on imported types. This follows the same rule as [`types.md`](types.md) §2.3 and §2.2 above: if the first parameter is `this`, the declaration is a method and gets the same private-field access as any other method on that receiver type. +Because methods are package-scope verbs, any package may define methods on imported types. This follows the same rule as [`types.md`](types.md) §2.3 and §2.2 above: if the first parameter is `this`, the declaration is a method and gets the same private-field access as any other method on that subject type. > **Story:** [`stories/functions.md`](../stories/functions.md#pulling-methods-out-of-the-type-body) — "Pulling methods out of the type body". @@ -310,7 +310,7 @@ The reason is the same one that makes operators safe to overload. An overloaded A lambda literal is a function declaration with the name removed. It writes its own parameter types, return type, abort type, and `mut` (see [`syntax.md`](syntax.md) §3.8). Nothing is inferred from context. ```zane -receiver(Float(x Int) { +callee(Float(x Int) { if x < Int(10) { return Float(0) } else { @@ -319,7 +319,7 @@ receiver(Float(x Int) { }) ``` -Because a lambda carries its complete type, it is a single value with one exact type. It can therefore be passed to an **overloaded** receiver without ambiguity: the lambda fixes its own type, so overload resolution on the receiver proceeds with ordinary argument types and no circularity. Its complete written type also allows it to be defined and passed directly in the same expression without depending on surrounding context. +Because a lambda carries its complete type, it is a single value with one exact type. It can therefore be passed to an **overloaded** callee without ambiguity: the lambda fixes its own type, so overload resolution on that callee proceeds with ordinary argument types and no circularity. Its complete written type also allows it to be defined and passed directly in the same expression without depending on surrounding context. `mut` is part of the lambda's written type. A lambda that does not declare `mut` may still be assigned to a `mut` function type — it simply does not use the mutation permission — but a `mut` lambda may not be assigned to a non-`mut` function type: @@ -343,7 +343,7 @@ Float callback(x Int) { ... } // function declaration callback Float(x Int) { ... } // lambda-variable declaration ``` -A lambda-variable is an ordinary symbol with a single function type. Because a symbol cannot be redeclared with a different type, a lambda-variable name can never accumulate an overload set, so it is always unambiguous in value position. This is what makes `receiver(callback)` well-defined where referencing an overloaded callable would not be. +A lambda-variable is an ordinary symbol with a single function type. Because a symbol cannot be redeclared with a different type, a lambda-variable name can never accumulate an overload set, so it is always unambiguous in value position. This is what makes `callee(callback)` well-defined where referencing an overloaded callable would not be. > **See also:** [`syntax.md`](syntax.md) §2.9 for function types and §3.8 for lambda literals and lambda-variable declarations. @@ -353,7 +353,7 @@ Lambdas **MUST NOT** capture outer variables. Every dependency must be passed as > **Story:** [`stories/functions.md`](../stories/functions.md#names-that-are-not-values) — "Names that are not values". ### 7.5 No bound method references -Zane does not provide bound method references as a separate feature. Because lambdas do not capture, there is no syntax that implicitly stores a receiver inside a function value. Code that needs a receiver later must keep that receiver in ordinary storage and pass it explicitly when the function value is invoked. +Zane does not provide bound method references as a separate feature. Because lambdas do not capture, there is no syntax that implicitly stores a subject inside a function value. Code that needs a subject later must keep that subject in ordinary storage and pass it explicitly when the function value is invoked. ### 7.6 Generics are orthogonal to overloading for function values A lambda is a single value with one exact type, even when that type is a function type (§7.2). Overload identity is parameter types only (§4.1), so a function type is a single, unique parameter shape. Passing a lambda to an overloaded callable is therefore an exact shape match at that parameter position, not a contest the lambda must win. @@ -372,13 +372,13 @@ Every callable in Zane is a verb (§1). What *kind* of verb a declaration is — | Marker | Verb kind | Capability unlocked | |---|---|---| -| First parameter named `this` | Method | Private-field access on the receiver; `:` / `!` call syntax | +| First parameter named `this` | Method | Private-field access on the subject; `:` / `!` call syntax | | Name is a type | Constructor | Return type is the named type (no return annotation); `init{ }` for field **initialization** | | Symbol name (operator token) | Operator | Operator-position calls | | No name | Lambda | Anonymous function value | | Plain identifier, none of the above | Function | No special capability | -The markers are largely independent — a lambda may still declare a `this` receiver (§7.2), for example — but the kinds above are distinguished by which markers are present. A constructor body and a method body are otherwise ordinary verb bodies (§2, §3). +The markers are largely independent — a lambda may still declare a `this` subject (§7.2), for example — but the kinds above are distinguished by which markers are present. A constructor body and a method body are otherwise ordinary verb bodies (§2, §3). ### 8.2 `init{ }` is to constructors what `this` is to methods @@ -398,7 +398,7 @@ All verbs share one parameter system (see [`generics.md`](generics.md) §3), one ## 9. Connection to the Effect Model -Read-only methods and functions are effect-free with respect to their receiver unless they touch guests or capabilities. `mut` marks the path for writing state reachable through `this`. This is why overload identity ignores `mut`: the call contract is structurally the same even though the behavioral permissions differ. +Read-only methods and functions are effect-free with respect to their subject unless they touch guests or capabilities. `mut` marks the path for writing state reachable through `this`. This is why overload identity ignores `mut`: the call contract is structurally the same even though the behavioral permissions differ. > **See also:** [`effects.md`](effects.md) for the complete effect model and concurrency implications. @@ -411,14 +411,14 @@ Read-only methods and functions are effect-free with respect to their receiver u | Verb | A callable; its kind is selected by markers, and each marker unlocks a capability | | Capability markers | `this` first → method (private access); name is a type → constructor (`init{ }`, implicit return); symbol name → operator; no name → lambda | | Method | Package-scope verb whose first parameter is `this` | -| `mut` method | Called with `!`; `this` is a mutable borrow of the caller's slot for both value and reference receivers; may mutate state reachable through `this` | +| `mut` method | Called with `!`; `this` is a mutable borrow of the caller's slot for both value and reference subjects; may mutate state reachable through `this` | | Read-only method | Called with `:`; may read but not write `this` | | Function | Identifier-named package-scope verb without `this`; no private-field privilege | | Block-bodied return | Every returning path uses `return expr`; `Unit` receives no fallthrough or bare-return exception | | `&` method parameter | Caller must supply a guest source (never a bare symbol); callee may store it into `&` fields or return it | | `'T` method parameter | Caller may supply any place expression, bare symbols included; read and `mut` access for the call only; **MUST NOT** be stored, returned, or moved | | Plain `T` method parameter | Swallows; caller may supply a temporary and downgrades to a guest; callee **MUST NOT** bind it into `&` storage | -| Reference-type `this` | Never a swallow position: bare `this T` is the borrow receiver — `'` is never written on `this` — and `this &T` is a guest receiver, required to store or return the receiver | +| Reference-type `this` | Never a swallow position: bare `this T` is the borrow subject — `'` is never written on `this` — and `this &T` is a guest subject, required to store or return the subject | | Subscript | Package-scope place projection written `(this T)[...] => placeExpr`; no explicit return type | | Overload identity | Parameter types only; not names, return type, or `mut`; overloads differing only by the passing mode (`T` / `&T` / `'T`) at one position are illegal | | Overload resolution phases | Direct match, then generic match, then implicit match; ambiguity within any one phase is an error | @@ -426,5 +426,5 @@ Read-only methods and functions are effect-free with respect to their receiver u | Lambda | Self-typed function value: explicit parameter types, return type, abort type, and `mut`; no capture | | Lambda-variable | Symbol bound to a lambda literal; has one function type; the only way to hold a function value | | Generic function value | Not specified in this version; deferred on runtime-representation grounds, not overloading (see [`generics.md`](generics.md) §9) | -| Unqualified method lookup | Searches the receiver's home package (the bundled `core` implementation for a fundamental type), then the current package | +| Unqualified method lookup | Searches the subject's home package (the bundled `core` implementation for a fundamental type), then the current package | | Extension methods | Any package may declare methods on imported types by naming the first parameter `this` | diff --git a/spec/generics.md b/spec/generics.md index 1060aef..ac8aed3 100644 --- a/spec/generics.md +++ b/spec/generics.md @@ -117,7 +117,7 @@ A **verb** — a function, method, or constructor — has no header. It *introdu ```zane Vector(x T Type, y T Type) { ... } // T introduced on a value parameter T head(arr Array) { ... } // T, n introduced inside a nested type -Int size(this Buffer) { ... } // T, n introduced on the receiver +Int size(this Buffer) { ... } // T, n introduced on the subject ``` A verb has no header because it never needs one: its parameters are always inferred (§5) and never applied positionally, so there is no order to fix and nothing for a header to declare. @@ -153,7 +153,7 @@ Int size(this Buffer) { } ``` -Here `n` in the return position is the number the use site supplied for that parameter. The receiver type `Buffer` introduces `T` and `n` inline; the return position references `n`. The `Array` layout inside `Buffer` uses the same `n` to fix the storage size. +Here `n` in the return position is the number the use site supplied for that parameter. The subject type `Buffer` introduces `T` and `n` inline; the return position references `n`. The `Array` layout inside `Buffer` uses the same `n` to fix the storage size. > **See also:** [`effects.md`](effects.md) §2 — a number parameter read in a body position is a read-only value-like binding. @@ -355,7 +355,7 @@ The following are intentionally not specified in this version: - dynamic container types such as lists and maps - bounds-checking rules for element access APIs - named lane access (`.x`, `.y`, `.z`, `.w`) -- phantom type parameters — an introduced parameter (a type's header parameter, or a verb's inline parameter) with no path from any value argument, receiver, or literal that fixes it +- phantom type parameters — an introduced parameter (a type's header parameter, or a verb's inline parameter) with no path from any value argument, subject, or literal that fixes it - generic function values — a function *value* that is itself polymorphic over type or number parameters; the open question is runtime representation (monomorphization versus dictionary passing), not overload resolution or type checking, since a generic function type is a unique parameter shape (see [`functions.md`](functions.md) §7.6) > **Story:** [`stories/generics.md`](../stories/generics.md#deferred-what-the-model-promises-but-does-not-yet-deliver) — "Deferred: what the model promises but does not yet deliver" records why each item is open, including the constraints/bounds gap and the type-level equality problem. diff --git a/spec/glossary.md b/spec/glossary.md index 39a8aa2..16766dc 100644 --- a/spec/glossary.md +++ b/spec/glossary.md @@ -36,8 +36,8 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`control-flow.md`](control-flow.md) §3 ### 2.4 value-typed mutation rule -- **Meaning:** A spawned call may mutate only a value-typed receiver, and at most one live spawn may mutably borrow a given storage location. A value type is transitively alias-free, so the rule rules out an aliased data race from the receiver's type alone; concurrent reads take a coherent snapshot instead of serializing. -- **Why this name:** Concurrent mutation is gated on the receiver being a value type — the property that makes race-freedom checkable without whole-program alias analysis. +- **Meaning:** A spawned call may mutate only a value-typed subject, and at most one live spawn may mutably borrow a given storage location. A value type is transitively alias-free, so the rule rules out an aliased data race from the subject's type alone; concurrent reads take a coherent snapshot instead of serializing. +- **Why this name:** Concurrent mutation is gated on the subject being a value type — the property that makes race-freedom checkable without whole-program alias analysis. - **Canonical home:** [`concurrency.md`](concurrency.md) §4.2 and §4.3 ### 2.5 water-tower lifetimes @@ -91,7 +91,7 @@ This file gives short, reusable names to concepts that appear across multiple sp ### 3.6 method-based privacy - **Meaning:** `_` fields are private to methods whose first parameter is `this` for that type, rather than to a package boundary. -- **Why this name:** Privacy is granted by the method/receiver relationship, not by where the function is declared. +- **Why this name:** Privacy is granted by the method/subject relationship, not by where the function is declared. - **Canonical home:** [`types.md`](types.md) §2.3 ### 3.7 direct initialization @@ -195,7 +195,7 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`functions.md`](functions.md) §8 ### 3.27 borrow -- **Meaning:** Non-hosting, non-escaping access to a caller's storage for the duration of a call. Every value type is passed this way — a value parameter is a read-only borrow, a value-type `mut` receiver is a mutable borrow, and a value is copied only when bound into a fresh slot. A reference type may also be borrowed, written `'T`, which is the only non-swallowing way to pass a bare symbol (§3.36); a bare reference-type `this` is that borrow — `'` is never written on `this`. +- **Meaning:** Non-hosting, non-escaping access to a caller's storage for the duration of a call. Every value type is passed this way — a value parameter is a read-only borrow, a value-type `mut` subject is a mutable borrow, and a value is copied only when bound into a fresh slot. A reference type may also be borrowed, written `'T`, which is the only non-swallowing way to pass a bare symbol (§3.36); a bare reference-type `this` is that borrow — `'` is never written on `this`. - **Why this name:** The callee is lent the caller's storage for the call and gives it back at return — it does not host it and cannot keep it. Unlike a guest, the borrow itself has no anchor or tether and cannot be stored, returned, or used as a move source — a restriction on the borrow, not on the value read through it, which a value type may still copy into a fresh slot. - **Canonical home:** [`memory.md`](memory.md) §2.9 @@ -245,13 +245,13 @@ This file gives short, reusable names to concepts that appear across multiple sp - **Canonical home:** [`memory.md`](memory.md) §2.8.1 ### 3.37 passing mode -- **Meaning:** Which of three ways a reference-type argument reaches a callee, fixed entirely by the parameter's surface form: `T` **swallows** it (hosting access; the caller downgrades to a guest), `&T` takes a **guest** (storable and returnable; requires a guest source), `'T` **borrows** it (read and `mut` for the call only; accepts any place, bare symbols included). The receiver parameter (§3.38) selects between the borrow and `&T` only: a bare `this T` is the borrow and `'` is never written on `this`. Two overloads may not differ only by the mode at one position. +- **Meaning:** Which of three ways a reference-type argument reaches a callee, fixed entirely by the parameter's surface form: `T` **swallows** it (hosting access; the caller downgrades to a guest), `&T` takes a **guest** (storable and returnable; requires a guest source), `'T` **borrows** it (read and `mut` for the call only; accepts any place, bare symbols included). The subject parameter (§3.38) selects between the borrow and `&T` only: a bare `this T` is the borrow and `'` is never written on `this`. Two overloads may not differ only by the mode at one position. - **Why this name:** "Mode" names a choice about *how* the same argument travels rather than *what* it is — the type is unchanged in all three, and only the caller's obligations and resulting state differ. - **Canonical home:** [`memory.md`](memory.md) §2.9 -### 3.38 receiver / receiver parameter / receiver expression -- **Meaning:** The **receiver** is the object a method is called on. The **receiver parameter** is `this`, the declaration's first parameter, whose surface form fixes the passing mode (§3.37) — bare for the borrow, `this &T` for the guest, never `'`. The **receiver expression** is what stands left of `:` or `!` at the call site and supplies the object; it must satisfy what that form requires, which is why a bare symbol works for a bare `this` but not for `this &T` (§3.36). -- **Why this name:** The three are one word in ordinary use because they usually coincide; the spec separates them where a rule holds of the declaration but not the object, or the other way round. +### 3.38 subject / subject parameter / subject expression +- **Meaning:** The **subject** is the object a method is called on. The **subject parameter** is `this`, the declaration's first parameter, whose surface form fixes the passing mode (§3.37) — bare for the borrow, `this &T` for the guest, never `'`. The **subject expression** is what stands left of `:` or `!` at the call site and supplies the object; it must satisfy what that form requires, which is why a bare symbol works for a bare `this` but not for `this &T` (§3.36). +- **Why this name:** Grammar, matching `verb` (§3.22): a call reads *subject–verb–object*, and the subject is what the verb acts from. The three senses are one word in ordinary use because they usually coincide; the spec separates them where a rule holds of the declaration but not the object, or the other way round. - **Canonical home:** [`functions.md`](functions.md) §2.1 --- diff --git a/spec/memory.md b/spec/memory.md index bbe46c0..f2ea506 100644 --- a/spec/memory.md +++ b/spec/memory.md @@ -103,12 +103,12 @@ The following are place expressions: - a named local, field-backed, or hosting/`&` storage symbol such as `engine` - a field access whose base is a place, such as `car.engine` or `this.engine` -- a subscript expression `list[index]` when `list` is a place expression and `[]` is defined as a place projection for that receiver type +- a subscript expression `list[index]` when `list` is a place expression and `[]` is defined as a place projection for that subject type - an `&T` guest parameter or a `'T` borrow parameter inside the callee body (§2.9) Only some place expressions may mint a new guest. A new `&` value may be minted from: -- a field access whose base is a place **and whose base chain does not pass through a `'T` borrow parameter**, such as `car.engine` or `this.engine` on a guest receiver +- a field access whose base is a place **and whose base chain does not pass through a `'T` borrow parameter**, such as `car.engine` or `this.engine` on a guest subject - an `&T` parameter Everything else is rejected. In particular: @@ -175,7 +175,7 @@ A **borrow** is non-hosting, non-escaping access to a caller's storage for the d That restriction is on the borrow, not on what is read through one. A value type is *always* passed this way — a value-type parameter is a **read-only borrow** of the caller's slot — and binding through that borrow into a fresh slot (an assignment, a new declaration, or a field or return store) **copies** the value. The copy is a new value that outlives the call perfectly well; what does not escape is the borrow. A reference type has no such copy, so a `'T` borrow leaves nothing behind at all. -A **reference type** parameter has three passing modes, one per surface form. The receiver parameter `this` is not one of these positions and has its own rule, below: +A **reference type** parameter has three passing modes, one per surface form. The subject parameter `this` is not one of these positions and has its own rule, below: | Mode | Written | Caller supplies | The callee may | |---|---|---|---| @@ -223,9 +223,9 @@ Int inspect(this Car, engine 'Engine) { } ``` -**The receiver parameter is never a swallow position.** A method does not consume the object it is called on, so `this` — the first parameter, and only it ([`functions.md`](functions.md) §2.1) — chooses between two of the three modes rather than all three: it is a **borrow** written bare, or a **guest** written `this &T` when the method stores the receiver past the call or returns it as `&T` (see [`functions.md`](functions.md) §2.4). `'` is **never** written on `this`. +**The subject parameter is never a swallow position.** A method does not consume the object it is called on, so `this` — the first parameter, and only it ([`functions.md`](functions.md) §2.1) — chooses between two of the three modes rather than all three: it is a **borrow** written bare, or a **guest** written `this &T` when the method stores the subject past the call or returns it as `&T` (see [`functions.md`](functions.md) §2.4). `'` is **never** written on `this`. -So bare `T` does not mean the same thing in both positions — on an ordinary parameter it swallows, on `this` it borrows — because `this` was never a swallow position to begin with. That much predates the borrow mode: a bare reference-type `this` used to be an implicit **guest**, likewise never swallowed. What changed is only *which* non-swallowing mode it is, and it moved to the borrow because the receiver expression at a call site is usually a bare symbol, which §2.8.1 no longer admits as a guest source. Value and reference receivers now agree: `this` carries at most one marker, `&`, and its absence means borrow. +So bare `T` does not mean the same thing in both positions — on an ordinary parameter it swallows, on `this` it borrows — because `this` was never a swallow position to begin with. That much predates the borrow mode: a bare reference-type `this` used to be an implicit **guest**, likewise never swallowed. What changed is only *which* non-swallowing mode it is, and it moved to the borrow because the subject expression at a call site is usually a bare symbol, which §2.8.1 no longer admits as a guest source. Value and reference subjects now agree: `this` carries at most one marker, `&`, and its absence means borrow. Binding a swallowed or borrowed parameter into `&` storage is illegal. A swallowed value is hosted at the call site while an `&` field lives with the object that holds it — which may outlive the call. A borrow does not survive the call at all: @@ -550,7 +550,7 @@ A single global free stack and frontier require synchronization under concurrent | Concept | Rule | |---|---| | Hosting storage | Reference-typed symbols, fields, and container elements are directly initialized and may later be overwritten | -| Value type | Mutable in place through a borrowed `mut` receiver; storage may also be overwritten freely | +| Value type | Mutable in place through a borrowed `mut` subject; storage may also be overwritten freely | | `&` (guest) | Guest-only non-hosting storage; stores one tether, may be repointed, copied by value, and returned, but can never directly host a `T` | | Host-capable guest state | After rehosting, the old hosted bytes cease to be live and a slot declared as `T` stores the terminal tether as a guest while retaining enough storage to host another `T` later | | Place expression | Existing stable storage: a named symbol, a field access of a place, a place-projection subscript of a place, or an `&`/`'` parameter | @@ -561,7 +561,7 @@ A single global free stack and frontier require synchronization under concurrent | Value-type parameter | Always a read-only borrow; caller need not supply a place; copied only when bound into a fresh slot (assignment, declaration, field or return store) | | Reference-type parameter | `T` swallows (hosting access; passing a host downgrades the caller's symbol to a guest whatever the body does — see [`lifetimes.md`](lifetimes.md) §1.8); `&T` takes a guest, which only a guest source can supply; `'T` borrows any place, bare symbols included, and leaves the caller a full host | | `'T` position | Parameter positions only; never a storage, field, or return type | -| Reference-type `this` | Never a swallow position: bare `this T` is the borrow receiver — `'` is never written on `this` — and `this &T` is a guest receiver a method may store or return | +| Reference-type `this` | Never a swallow position: bare `this T` is the borrow subject — `'` is never written on `this` — and `this &T` is a guest subject a method may store or return | | Value-downstream enforcement | Value types may contain only primitives and other value types, transitively — never a reference (`#`) or `&` field | | `&` targets reference types | An `&T` requires `T` to be a reference type; a value is shared by copy or scoped borrow, never by a stored `&` | | Symbol declaration | Must be directly initialized | diff --git a/spec/operators.md b/spec/operators.md index be003ee..57bd534 100644 --- a/spec/operators.md +++ b/spec/operators.md @@ -34,7 +34,7 @@ Primitive operators are implementable and define the operator surface area: | `<` | binary | `Bool <(left T, right T)` | ### 2.2 Where operators may be defined -Operator implementations are package-scope verb declarations whose names are operator tokens. They are ordinary non-`mut` verbs with special names, not methods: an operator declaration never has a `this` receiver parameter. +Operator implementations are package-scope verb declarations whose names are operator tokens. They are ordinary non-`mut` verbs with special names, not methods: an operator declaration never has a `this` subject parameter. A unary operator is legal only in the home package of its operand type. A binary operator `(left T, right U)` is legal only in the home package of `T` or `U`. The bundled `core` implementation is the home package of fundamental types, but source packages cannot add declarations to it; a fundamental operand therefore does not by itself grant a source package permission to declare an operator. See [`functions.md`](functions.md) §6.1 for the corresponding method-resolution rule. diff --git a/spec/packages.md b/spec/packages.md index e379e52..06a4a1d 100644 --- a/spec/packages.md +++ b/spec/packages.md @@ -69,7 +69,7 @@ import math result Float = math$sqrt(value) ``` -The method-call lookup rules in [`functions.md`](functions.md) §6 are a distinct resolution mechanism. A qualified extension-method call writes the package name explicitly as `receiver:packageName$method(...)`. +The method-call lookup rules in [`functions.md`](functions.md) §6 are a distinct resolution mechanism. A qualified extension-method call writes the package name explicitly as `subject:packageName$method(...)`. ### 3.4 `$` separates a package namespace from its member @@ -101,7 +101,7 @@ Operators are symbol-named rather than identifier-named and cannot carry a leadi Package scope may contain immutable constants and verbs. It **MUST NOT** contain mutable variables or any other time-varying package state. -State that changes over time must live in a value, such as a `struct` or reference-typed object, and reach operations through ordinary parameters, receivers, or capability wiring. This keeps mutation visible to the effect model in [`effects.md`](effects.md). +State that changes over time must live in a value, such as a `struct` or reference-typed object, and reach operations through ordinary parameters, subjects, or capability wiring. This keeps mutation visible to the effect model in [`effects.md`](effects.md). > **Story:** [`stories/packages.md`](../stories/packages.md#state-has-to-be-a-value) — "State has to be a value". diff --git a/spec/syntax.md b/spec/syntax.md index cdde99f..186d52e 100644 --- a/spec/syntax.md +++ b/spec/syntax.md @@ -246,11 +246,11 @@ A function type leads with its return type, then lists parameter types inside `[ ```zane ReturnType[ParamType, ...] ReturnType?AbortType[ParamType, ...] -ReturnType[this ReceiverType, ParamType, ...] -ReturnType[this ReceiverType, ParamType, ...] mut -&ReturnType[this ReceiverType, ParamType, ...] -ReturnType?AbortType[this ReceiverType, ParamType, ...] -ReturnType?AbortType[this ReceiverType, ParamType, ...] mut +ReturnType[this SubjectType, ParamType, ...] +ReturnType[this SubjectType, ParamType, ...] mut +&ReturnType[this SubjectType, ParamType, ...] +ReturnType?AbortType[this SubjectType, ParamType, ...] +ReturnType?AbortType[this SubjectType, ParamType, ...] mut ``` The abort type stays attached to the return type, exactly as in a declaration's `ReturnType?AbortType name(...)` header. @@ -260,8 +260,8 @@ Reference-typed parameters and returns use the ordinary type form. A parameter s ```zane ReturnType[&ParamType, ...] ReturnType['ParamType, ...] -&ReturnType[this &ReceiverType, &ParamType, ...] -ReturnType[this ReceiverType, 'ParamType, ...] mut +&ReturnType[this &SubjectType, &ParamType, ...] +ReturnType[this SubjectType, 'ParamType, ...] mut ``` ```zane @@ -312,25 +312,25 @@ A function, method, or constructor has no `<>` parameter header. It introduces a ### 3.2 Methods ```zane -ReturnType name(this ReceiverType, param ParamType, ...) { body } -ReturnType name(this ReceiverType, param &ParamType, ...) { body } -ReturnType name(this ReceiverType, param ParamType, ...) mut { body } -ReturnType name(this ReceiverType, param &ParamType, ...) mut { body } -ReturnType?AbortType name(this ReceiverType, param ParamType, ...) { body } -ReturnType?AbortType name(this ReceiverType, param ParamType, ...) mut { body } -ReturnType name(this ReceiverType, param ParamType, ...) => expr -ReturnType name(this ReceiverType, param &ParamType, ...) => expr -ReturnType name(this ReceiverType, param ParamType, ...) mut => expr -ReturnType name(this ReceiverType, param &ParamType, ...) mut => expr -ReturnType?AbortType name(this ReceiverType, param ParamType, ...) => expr -ReturnType?AbortType name(this ReceiverType, param ParamType, ...) mut => expr -ReturnType name(this ReceiverType, param ParamType, ...) { body } -ReturnType name(this &ReceiverType, param ParamType, ...) { body } +ReturnType name(this SubjectType, param ParamType, ...) { body } +ReturnType name(this SubjectType, param &ParamType, ...) { body } +ReturnType name(this SubjectType, param ParamType, ...) mut { body } +ReturnType name(this SubjectType, param &ParamType, ...) mut { body } +ReturnType?AbortType name(this SubjectType, param ParamType, ...) { body } +ReturnType?AbortType name(this SubjectType, param ParamType, ...) mut { body } +ReturnType name(this SubjectType, param ParamType, ...) => expr +ReturnType name(this SubjectType, param &ParamType, ...) => expr +ReturnType name(this SubjectType, param ParamType, ...) mut => expr +ReturnType name(this SubjectType, param &ParamType, ...) mut => expr +ReturnType?AbortType name(this SubjectType, param ParamType, ...) => expr +ReturnType?AbortType name(this SubjectType, param ParamType, ...) mut => expr +ReturnType name(this SubjectType, param ParamType, ...) { body } +ReturnType name(this &SubjectType, param ParamType, ...) { body } ``` `this` is legal only in the first parameter position. A declaration is a method if and only if its first parameter is named `this`. -The receiver takes at most one marker, `&`. A bare `this ReceiverType` is the **borrow** receiver, and `this &ReceiverType` is written when the method stores or returns the receiver as a guest; `'` is **never** written on `this`, for either kind of type. A value receiver is likewise a borrow of the caller's slot, mutable when the method is `mut`, and always written bare. See [`functions.md`](functions.md) §2.4. +The subject takes at most one marker, `&`. A bare `this SubjectType` is the **borrow** subject, and `this &SubjectType` is written when the method stores or returns the subject as a guest; `'` is **never** written on `this`, for either kind of type. A value subject is likewise a borrow of the caller's slot, mutable when the method is `mut`, and always written bare. See [`functions.md`](functions.md) §2.4. `=> expr` returns `expr`, including when `expr` has type `Unit`. @@ -417,7 +417,7 @@ implicit TypeName{field FieldType} { ... } // ILLEGAL: field-constructor form is ### 3.6 Subscript definitions ```zane -(this ReceiverType)[param ParamType, ...] => placeExpr +(this SubjectType)[param ParamType, ...] => placeExpr ``` Subscript definitions have no explicit return type annotation. The body **MUST** be a place expression. If the body is not a place expression, the declaration is a compile-time error. @@ -427,10 +427,10 @@ A subscript definition may declare any number of comma-separated parameters insi The following forms are not part of the grammar: ```zane -ReturnType (this ReceiverType)[index ParamType] => expr +ReturnType (this SubjectType)[index ParamType] => expr ``` -`[]` is not a general function call form. A subscript definition always declares a place projection that references existing storage within the receiver. +`[]` is not a general function call form. A subscript definition always declares a place projection that references existing storage within the subject. ### 3.7 `init{ }` @@ -456,15 +456,15 @@ ReturnType(param 'ParamType, ...) { body } ReturnType() => expr ReturnType(param ParamType, ...) => expr ReturnType?AbortType(param ParamType, ...) { body } -ReturnType(this ReceiverType) { body } -ReturnType(this ReceiverType) mut { body } -ReturnType(this ReceiverType, param ParamType, ...) { body } -ReturnType(this ReceiverType, param ParamType, ...) mut { body } -ReturnType(this ReceiverType, param ParamType, ...) => expr -ReturnType(this ReceiverType, param ParamType, ...) mut => expr +ReturnType(this SubjectType) { body } +ReturnType(this SubjectType) mut { body } +ReturnType(this SubjectType, param ParamType, ...) { body } +ReturnType(this SubjectType, param ParamType, ...) mut { body } +ReturnType(this SubjectType, param ParamType, ...) => expr +ReturnType(this SubjectType, param ParamType, ...) mut => expr ``` -A lambda literal omits only the function name. `this` is legal only in the first parameter position. `mut` is legal only when the first parameter is `this`. Parameters and the receiver carry the same three passing modes as a named verb (§3.1–§3.2). +A lambda literal omits only the function name. `this` is legal only in the first parameter position. `mut` is legal only when the first parameter is `this`. Parameters and the subject carry the same three passing modes as a named verb (§3.1–§3.2). Examples: @@ -486,7 +486,7 @@ A lambda-variable declaration binds a lambda literal to a symbol. The shorthand name ReturnType(param ParamType, ...) { body } name ReturnType(param ParamType, ...) => expr name ReturnType?AbortType(param ParamType, ...) { body } -name ReturnType(this ReceiverType, param ParamType, ...) mut { body } +name ReturnType(this SubjectType, param ParamType, ...) mut { body } ``` The shorthand expands to a symbol declaration whose type is the function type (§2.9) and whose value is the lambda literal: @@ -538,10 +538,10 @@ packageName$name(args...) ### 4.2 Method calls ```zane -receiver:method(args...) -receiver!method(args...) -receiver:packageName$method(args...) -receiver!packageName$method(args...) +subject:method(args...) +subject!method(args...) +subject:packageName$method(args...) +subject!packageName$method(args...) ``` ### 4.3 Callables are call-only @@ -576,14 +576,14 @@ Vec2(2)|100 // groups as Vec2(2)|100 ```zane spawn functionName(args...) -spawn receiver:methodName(args...) -spawn receiver!methodName(args...) +spawn subject:methodName(args...) +spawn subject!methodName(args...) spawn functionName(args...) ? binder { ... } -spawn receiver:methodName(args...) ? binder { ... } -spawn receiver!methodName(args...) ?? fallbackExpr +spawn subject:methodName(args...) ? binder { ... } +spawn subject!methodName(args...) ?? fallbackExpr name VarType = spawn functionName(args...) -name VarType = spawn receiver:methodName(args...) ? binder { ... } -name VarType = spawn receiver!methodName(args...) ? binder { ... } +name VarType = spawn subject:methodName(args...) ? binder { ... } +name VarType = spawn subject!methodName(args...) ? binder { ... } name VarType = spawn functionName(args...) ?? fallbackExpr ``` @@ -595,7 +595,7 @@ name VarType = spawn functionName(args...) ?? fallbackExpr placeExpr[argExpr, ...] ``` -`[]` is legal only when the receiver type defines a subscript declaration. A subscript expression is a place projection, not a general function call, so it is legal only when its base is a place expression. +`[]` is legal only when the subject type defines a subscript declaration. A subscript expression is a place projection, not a general function call, so it is legal only when its base is a place expression. Examples: diff --git a/spec/types.md b/spec/types.md index 94196fc..5bc8264 100644 --- a/spec/types.md +++ b/spec/types.md @@ -41,7 +41,7 @@ type Node = #struct { // reference type: identity, may hold `&`, may recurs ### 2.2 Value types are transitive and mutable in place A value-type body contains only field declarations, stored inline. A value type **MUST NOT** contain a reference-type or `&` field, and this holds transitively: a value type reachable through a value type must itself be a value type (see [`memory.md`](memory.md) §2.10). The restriction is what makes a value copyable and shareable-by-snapshot with no hosting or anchor bookkeeping. -A value is **mutable in place**: a `mut` method may write its fields, because the receiver is a *borrow* of the caller's storage rather than a copy (see [`effects.md`](effects.md) §2.3 and [`functions.md`](functions.md) §2.4). A value's storage slot may also be overwritten wholesale. +A value is **mutable in place**: a `mut` method may write its fields, because the subject is a *borrow* of the caller's storage rather than a copy (see [`effects.md`](effects.md) §2.3 and [`functions.md`](functions.md) §2.4). A value's storage slot may also be overwritten wholesale. ```zane package Math @@ -59,7 +59,7 @@ pos = Vec2(3, 4) // legal: overwrites the whole value ### 2.3 Field visibility is name-based Fields whose names begin with `_` are private to methods whose first parameter is `this` for that type, regardless of which package declares the method. -The same receiver type written under any other parameter name is a non-receiver parameter and does not gain private-field access. +The same subject type written under any other parameter name is a non-subject parameter and does not gain private-field access. All fields whose names do not begin with `_` are public. @@ -223,7 +223,7 @@ A named constructor is an ordinary constructor in every other respect. Naming a - overloads by parameter types, alongside the anonymous constructor and the other named ones; - is called by its qualified name and yields the **base type** — `Vector2.zeros()` is a `Vector2`, never a `Vector2.zeros` type. -The casing rule (see [`lexical.md`](lexical.md) §3) keeps the call unambiguous: `Vector2.zeros()` has an uppercase receiver, so `.zeros` is a member of the *type* — a constructor — while `v.zeros` has a lowercase receiver, so `.zeros` is a field or method of a *value*. The two never collide. +The casing rule (see [`lexical.md`](lexical.md) §3) keeps the call unambiguous: `Vector2.zeros()` has an uppercase subject, so `.zeros` is a member of the *type* — a constructor — while `v.zeros` has a lowercase subject, so `.zeros` is a field or method of a *value*. The two never collide. A named constructor **MUST NOT** be marked `implicit`: an implicit constructor is an anonymous single-argument conversion the compiler inserts at a coercion site (§4), and a name has nothing to insert. @@ -278,7 +278,7 @@ Vector{x Int, y Int} { Every field of the target type **MUST** be assigned exactly once, either explicitly or through implicit field access shorthand. ### 3.8 Constructors do not use `mut` -Constructors are not methods. They create new values rather than mutating an existing receiver, so `mut` does not apply. +Constructors are not methods. They create new values rather than mutating an existing subject, so `mut` does not apply. ### 3.9 `&` fields require `&` constructor parameters An `&` field is legal only in a reference type (`#struct`/`#variant`), since a value type is transitively value (§2.2). A constructor that assigns a value to an `&` field must declare the corresponding parameter as `&T` — a `'T` borrow will not do, because a borrow ends with the call while the field outlives it. The caller must then supply a **guest source** under [`memory.md`](memory.md) §2.8: a field access on a place, or an `&T` parameter. A bare symbol, a temporary, and a `[]` expression are all rejected. @@ -412,7 +412,7 @@ distance Meters = Meters(Feet(Float(10))) // legal: explicit conversion A coercion site is a position that passes a value into a contract whose destination type is fixed by a callable or language construct. These are the only positions where the compiler inserts an implicit constructor: - Positional arguments of a function call -- Positional arguments of a method call (the receiver is excluded; see §4.6) +- Positional arguments of a method call (the subject is excluded; see §4.6) - Positional arguments of a positional constructor call `Type(...)` - Positional arguments of a named-constructor call `Type.name(...)` - Named field entries of a field-constructor call `Type{ field = expr }` @@ -498,8 +498,8 @@ import Units implicit Units$Meters(feet Units$Feet) => init{value = feet.value * Float(0.3048)} ``` -### 4.6 Method receivers are never implicitly converted -The receiver expression (`this`) in a method call is never subject to implicit conversion. This remains true even though method calls desugar to ordinary function calls. If the receiver type does not match, the call is a type error. +### 4.6 Method subjects are never implicitly converted +The subject expression (`this`) in a method call is never subject to implicit conversion. This remains true even though method calls desugar to ordinary function calls. If the subject type does not match, the call is a type error. ```zane Unit logDistance(this Meters) { @@ -508,7 +508,7 @@ Unit logDistance(this Meters) { } feet Feet(Float(10)) -feet:logDistance() // ILLEGAL: receiver type is Feet, not Meters +feet:logDistance() // ILLEGAL: subject type is Feet, not Meters ``` > **See also:** [`functions.md`](functions.md) §5 for how implicit constructors interact with overload resolution. @@ -568,11 +568,11 @@ Intent lives entirely in the keyword — `type` versus `alias` — not in the pu | Value/reference axis | A type is a value type unless marked `#`; `#` marks only a mould — `#struct`/`#variant`/`#enum` (declared and named), each a distinct reference type with identity, `&`-aliasing, and recursion; the unmarked moulds declare value types | | Mould | One of the three type-shaping forms — `struct`, `variant`, or `enum`; each has a value form and a `#` reference form; appears only as a `type`/`alias` right-hand side, so every constructible type is named | | Use-site types | A field, parameter, or return type names a declared type or an instantiation (`Weapon`, `Vector`, `&Node`); a mould appears only as a `type`/`alias` right-hand side | -| Value type | Copied on assignment; transitively value (no reference-type or `&` field, anywhere downstream); mutable in place through a borrowed `mut` receiver; storage may also be overwritten wholesale | +| Value type | Copied on assignment; transitively value (no reference-type or `&` field, anywhere downstream); mutable in place through a borrowed `mut` subject; storage may also be overwritten wholesale | | Reference type (`#`) | Single hosting and stable identity; may hold reference-type and `&` fields; may recurse; placement is unobservable | | Fundamental type | `Int`, `Float`, `Bool`, `String`, or `Unit`; declared by the bundled `core` implementation and available unqualified | | `Unit` | Empty `core` value type; `Unit()` constructs its sole value, which may be stored or used as a generic argument | -| Field visibility | Names starting with `_` are private to `this`-parameter methods on the receiver type; all other names are public | +| Field visibility | Names starting with `_` are private to `this`-parameter methods on the subject type; all other names are public | | Constructor | Package-scope verb named after the type; the written type name is the return type; no `this`; may use block or `=> init{...}` form | | Field constructor | Declares field parameters directly, may assign default values, and may use `init{field}` shorthand | | Implicit constructor | Single-parameter constructor marked `implicit`; inserted at callable arguments, named field-constructor entries, conditions, and counted-loop bounds — never at declarations, assignments, stores, `return`, or the `init{field = value}` inside a constructor body; no field-constructor form; source type must be a value type or compiler concept; orphan rule applies | diff --git a/stories/lifetimes.md b/stories/lifetimes.md index d6b561d..1e0d235 100644 --- a/stories/lifetimes.md +++ b/stories/lifetimes.md @@ -178,7 +178,7 @@ The scope check ([§1.1](https://github.com/zane-lang/spec/blob/b10eaed/spec/lif The rule that did change is the one governing returns ([§1.7](https://github.com/zane-lang/spec/blob/b10eaed/spec/lifetimes.md#17-returned--values-must-be-rooted-in-a-guest-parameter)). It used to say that a returned `&T` must be rooted in *a parameter*, which was the right rule when there was only one kind of reference parameter to be rooted in. With three, "a parameter" is no longer specific enough, and each of the other two fails for its own reason. A swallowing `T` parameter is a bare symbol in the call-site scope, and a bare symbol is not a guest source — so the returned guest could not have been minted in the first place. A `'T` borrow fails harder: the borrow ends when the call does, and a guest rooted in one would outlive the very access it was derived from. So the rule now names the guest parameter specifically. What makes this feel right rather than merely tighter is that the three modes each answer the question the rule is really asking — *may this outlive the call?* — and only one of them answers yes. -The same reasoning had to be pushed one step further than the rule's own text, and this is the part that is easy to miss. Refusing to return a `'T` parameter is pointless if you may instead return a guest minted from one of its **fields**: `this.weapon` on a borrowed receiver would escape just as surely as `this` would, wrapped in one layer of indirection. So a field access rooted in a borrow is not a guest source either. We debated allowing it — the caller's object does outlive the call, so the guest would in fact be live — and rejected it on the grounds that "in fact live" is not the standard. The compiler would have to reason about the relative scopes of two objects across a call boundary to know it, which is the interprocedural analysis this whole document exists to avoid. A borrow that does not escape, with no exceptions and nothing to check, is worth more than a borrow that escapes safely under an argument only the compiler can follow. +The same reasoning had to be pushed one step further than the rule's own text, and this is the part that is easy to miss. Refusing to return a `'T` parameter is pointless if you may instead return a guest minted from one of its **fields**: `this.weapon` on a borrowed subject would escape just as surely as `this` would, wrapped in one layer of indirection. So a field access rooted in a borrow is not a guest source either. We debated allowing it — the caller's object does outlive the call, so the guest would in fact be live — and rejected it on the grounds that "in fact live" is not the standard. The compiler would have to reason about the relative scopes of two objects across a call boundary to know it, which is the interprocedural analysis this whole document exists to avoid. A borrow that does not escape, with no exceptions and nothing to check, is worth more than a borrow that escapes safely under an argument only the compiler can follow. Nothing else moved. The downgrade rule ([§1.6](https://github.com/zane-lang/spec/blob/b10eaed/spec/lifetimes.md#16-moved-symbols-downgrade-to--values-and-are-no-longer-movable)) still turns a moved-from symbol into a readable guest, and it is worth being clear that this is not in tension with the new source rule: the downgrade is something the language does to a slot, not a guest a program mints, and the reader never writes it. The signature-is-the-whole-contract rule ([§1.8](https://github.com/zane-lang/spec/blob/b10eaed/spec/lifetimes.md#18-passing-a-host-to-a-t-parameter-downgrades-it-to-a-guest)) simply grew a fourth entry — a verb may now *borrow* an argument as well as take a guest, relay, or consume — and the entry it grew is, satisfyingly, the one that leaves the caller in the strongest position: still the host, with the callee unable to keep anything. diff --git a/stories/memory.md b/stories/memory.md index 4b90e61..cb84f4d 100644 --- a/stories/memory.md +++ b/stories/memory.md @@ -182,7 +182,7 @@ We considered giving `&T` parameters a special exemption: a bare symbol may not So the passing mode had to split. A parameter that merely reads or mutates the caller's object for the duration of the call is a genuinely different contract from one that keeps a guest past the call, and the old `&T` was carrying both. Separating them gives the three modes in [`memory.md` §2.9](https://github.com/zane-lang/spec/blob/b10eaed/spec/memory.md#29-function-parameters-swallow-guest-and-borrow): `T` **swallows**, taking hosting access and downgrading the caller; `&T` takes a **guest**, which the callee may store or return and which only a guest source can supply; `'T` **borrows**, accepting any place at all — bare symbols included — and granting read and `mut` access that expires with the call. The borrow was not a new concept: value types had always been passed exactly this way, and the reference world had simply never been given the same option. -That third mode turned out to pay for itself immediately in a place we had not been aiming at. A reference-type receiver had been an implicit guest, which under the new source rule would have made `node!setScale(...)` illegal on a bare local — an absurdity. But a receiver almost never needs to be kept; it needs to be read and written for the duration of the call. So a bare `this T` on a reference type is now an implicit `'T` borrow ([`functions.md` §2.4](https://github.com/zane-lang/spec/blob/b10eaed/spec/functions.md#24-mutating-methods-use-mut)), and `this &T` is what a method writes in the rarer case where it stores or returns the receiver. The pleasing part is that this makes the two type worlds agree: a `mut` receiver is a mutable borrow of the caller's slot whether the type is a value or a reference, and the special-casing that used to sit in that sentence is gone. +That third mode turned out to pay for itself immediately in a place we had not been aiming at. A reference-type subject had been an implicit guest, which under the new source rule would have made `node!setScale(...)` illegal on a bare local — an absurdity. But a subject almost never needs to be kept; it needs to be read and written for the duration of the call. So a bare `this T` on a reference type is now an implicit `'T` borrow ([`functions.md` §2.4](https://github.com/zane-lang/spec/blob/b10eaed/spec/functions.md#24-mutating-methods-use-mut)), and `this &T` is what a method writes in the rarer case where it stores or returns the subject. The pleasing part is that this makes the two type worlds agree: a `mut` subject is a mutable borrow of the caller's slot whether the type is a value or a reference, and the special-casing that used to sit in that sentence is gone. Naming the mode took longer than designing it. The tempting move was to hand the constrained meaning to the bare `&` and mark the escaping one, on the general principle that the marked form should be the restricted form. That principle does not apply here, and noticing why was the turn: **both** forms are marked. The unmarked form is `T`, the swallow. Between `&` and a new sigil there is no asymmetry of markedness to appeal to, so the argument has to be made on continuity instead — and there `&` has a large incumbent claim. It means *guest* in a field type, in a storage declaration, in a return type, in the glossary, and across every chapter above this one. Redefining it in the parameter position alone would make the same character mean two things depending on where it sits, which is precisely the kind of context-dependence [the two-vocabulary chapter](#two-vocabularies-host-and-guest-above-anchor-and-tether) had just finished removing from `tether`. So `&` keeps meaning guest everywhere, and the new concept takes the new mark. From dd1cbcb0820439bd0409efa7e6e279cdb4c55e00 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 17:18:47 +0000 Subject: [PATCH 30/31] docs: tell the receiver-to-subject story MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Appends "What does a receiver receive?" to stories/functions.md: the word arrived free with the object model and went unexamined until a rule about `this` forced us to say which of its three senses we meant, and then until someone asked what a receiver receives. Records the dead metaphor, the grammar register that supplied the replacement, the candidates weighed, and the cost — merged chapters keep the old word, so the two doc trees disagree by design. Pointer added from functions.md §2.1; README row updated. --- README.md | 2 +- spec/functions.md | 2 ++ stories/functions.md | 18 ++++++++++++++++++ 3 files changed, 21 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index ac81053..9f587e7 100644 --- a/README.md +++ b/README.md @@ -72,7 +72,7 @@ The spec states *what* the language does; the **why** lives in a parallel set of | [`stories/concurrency.md`](stories/concurrency.md) | [`spec/concurrency.md`](spec/concurrency.md) — the parallelism/concurrency split and the refusal of `async` coloring, why `spawn` marks only a call, water-tower lifetimes, signature-based safety without locks, and value-typed mutation closing the aliased-write gap | | [`stories/error-handling.md`](stories/error-handling.md) | [`spec/error-handling.md`](spec/error-handling.md) — the two-doors model and why failure is control flow rather than a `Result` value, `resolve` as expression-substitution rather than assignment, typed abort paths and the deliberately-absent propagate operator, keeping abortability orthogonal to effects, and explicit path values through `Unit` | | [`stories/control-flow.md`](stories/control-flow.md) | [`spec/control-flow.md`](spec/control-flow.md) — `guard` as an active exit that opens no scope of its own, doing without `while` behind a written loop bound, one-based counting after the loop that forced the question, and why control-flow contracts use fundamental semantic types | -| [`stories/functions.md`](stories/functions.md) | [`spec/functions.md`](spec/functions.md) — pulling methods out of the type body and the verb model that revealed, mutation made visible with `:`/`!`, overloading on parameter shape alone, why callables are call-only while self-typed lambdas are values, and why every return carries an explicit value | +| [`stories/functions.md`](stories/functions.md) | [`spec/functions.md`](spec/functions.md) — pulling methods out of the type body and the verb model that revealed, mutation made visible with `:`/`!`, overloading on parameter shape alone, why callables are call-only while self-typed lambdas are values, why every return carries an explicit value, and dropping the inherited word "receiver" for `subject` | | [`stories/operators.md`](stories/operators.md) | [`spec/operators.md`](spec/operators.md) — the fixed vocabulary worth overloading, `~` as the universal flip, laws enforced through derived operators, grammar-only grouping, and home-package coherence | | [`stories/packages.md`](stories/packages.md) | [`spec/packages.md`](spec/packages.md) — the directory as namespace and compilation unit, declarations as move checks, explicit qualified access through `$`, and keeping mutable state inside values so the effect model can see it | diff --git a/spec/functions.md b/spec/functions.md index bbc6f67..84fc502 100644 --- a/spec/functions.md +++ b/spec/functions.md @@ -34,6 +34,8 @@ Int scaledId(this Node, factor Int) { } ``` +> **Story:** [`stories/functions.md`](../stories/functions.md#what-does-a-receiver-receive) — "What does a receiver receive?". + ### 2.2 `this` grants private-field access Naming the first parameter `this` is the only thing that makes a declaration a method. That token grants access to `_`-prefixed fields on the subject type regardless of which package declares the method; home-package status does not matter. The same parameter type written with another name is a function and does not grant private-field access. diff --git a/stories/functions.md b/stories/functions.md index 96300a8..e611de5 100644 --- a/stories/functions.md +++ b/stories/functions.md @@ -53,3 +53,21 @@ Making `Unit` a real value raised a choice between convenient procedure syntax a The problem was not whether synthesizing `Unit()` was safe; it was where the knowledge had to live. `Unit` is declared by the bundled `core` implementation like the other fundamental types. Giving it fallthrough would require a special case in return-path analysis and AST lowering solely for that nominal declaration. `Bool` does not set a precedent: an `if` node already checks a condition, so choosing `Bool` as the expected type only fills an existing slot. A return node with no expression has no such slot to check; the compiler would have to create the expression. We therefore took the strict rule we had initially rejected. Every returning path carries an explicit value, so a `Unit` verb ends in `return Unit()` and an expression body writes `Unit noOperation() => Unit()`. The cost is repetition where the signature already proves there is only one possible value. What it buys is more fundamental: `Unit` stays ordinary all the way through the compiler, and every return AST has the same shape regardless of the type travelling through it. + +## What does a receiver receive? + +The rules in this document had said "receiver" from the first draft, and we never looked at the word once. It arrived free with the object model, the way it arrives free in Go, Swift, Ruby, and Java's documentation, and a word that every neighbouring language already uses does not attract the scrutiny a coined one does. It survived the whole of the passing-mode work by being invisible. + +What eventually broke it was a rule that had to talk about `this` specifically. Taking the bare symbol away as a guest source forced us to say what a bare `this T` means on a reference type, and the sentence we reached for — *the receiver is never a swallow position* — turned out not to parse. A reader asked which receiver was meant, and the honest answer was that the word had been covering three things at once: the object a method is called on, the `this` parameter whose surface form decides how that object arrives, and the expression standing left of `:` or `!` at the call site. The rule was about the second; the reason behind it was about the third; "stores or returns the receiver" was about the first. All three were true, and the sentence let a reader pick the wrong one. So we split them — subject, subject parameter, subject expression ([`glossary.md` §3.38](https://github.com/zane-lang/spec/blob/97e5bf2/spec/glossary.md#338-subject--subject-parameter--subject-expression)) — and thought that was the end of it. + +It was not, because the next question was the one that actually landed: *what does a receiver receive?* In Smalltalk the answer is a message. A call **was** a message, sent to an object, and the object that took delivery was its receiver — the metaphor was exact, and it was load-bearing, because the message could be forwarded, reified, or not understood. Every language that copied the object-dot-method shape kept the noun and dropped the machinery underneath it. Zane has no messages. A method here is a package-scope verb whose first parameter is `this`, and `player!setScale(...)` desugars to a plain call with the object in first position ([`functions.md` §2.1](https://github.com/zane-lang/spec/blob/97e5bf2/spec/functions.md#21-methods-are-verbs-whose-first-parameter-is-this)). Nothing is sent, so nothing is received. + +That is a worse failure than it first sounds, and our own naming guide is what makes it legible. The test we apply to a candidate term is whether a fresh reader's existing sense of the word points toward the concept or away from it — whether it feeds or fights. `receiver` did neither. It is a dead metaphor: a label with no analogy left underneath, teaching nothing on contact and quietly inviting the reader to look for a delivery that does not exist. We had rejected `origin form` for being flat and descriptive; `receiver` is flatter, and it describes something untrue. + +The replacement was sitting in the register we had already committed to. This document calls callables **verbs** because a verb is the word that acts — and grammar, having lent us that, has the rest of the sentence to lend as well. A call reads *subject–verb–object*: in `player!setScale(scale)` the subject is `player`, the verb is `setScale`, the object is what it acts on. The subject is what the verb acts *from*, which is precisely what `this` is, and a reader who has met the word in a sentence already knows that much before reading a definition. It is short, ordinary, unremarkable in dense prose, and it makes the whole parameter list nameable in one register instead of one word borrowed from grammar and one from a message-passing model we do not implement. + +We weighed keeping `receiver` anyway, on the strength of it being what every other language says — a real argument, since a term's job is to be understood, and familiar-but-hollow may beat apt-but-strange. We decided the familiarity was worth less than it looks: a reader who knows `receiver` from Go knows it as a *position*, not as a metaphor, so nothing is actually lost when the position keeps its meaning under a better name. `target` was the other candidate and was rejected quickly — "call target" already means the function being called, so it would have collided with the one thing in a call it must never be confused with. + +`subject` is not free of prior claims. It is the observer pattern's `Subject`, and in security it is the principal a policy is written about. That is a weaker collision than the one that sank `matrix` — neither of those meanings is likely to be in a reader's mind while reading a method signature — but it is not nothing, and it is the honest cost of the choice. + +The larger cost is one this file is currently demonstrating. The spec is rewritten to the present, so every rule now says *subject*; the stories are not, so every chapter above this one still says *receiver*, and always will. A reader working through the design history meets both words for one concept and must carry the mapping themselves. We accepted that deliberately rather than quietly reaching back to rewrite the earlier chapters: an accurate record of the reasoning we actually had, in the words we actually had it in, is worth more than a uniform vocabulary. This chapter is the mapping. From 042a7bd00ef0da929ade6033cd37e4b0cc66f887 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Jul 2026 17:27:22 +0000 Subject: [PATCH 31/31] docs(meta): make the stories guide the source of truth for append-only MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The rule lived in two places at once, and CLAUDE.md — which is agent notes, not contributor documentation — had grown a full normative restatement of it. Now the story guide §5 owns the rule and says how it is enforced (by hand, by author and reviewer; there is no CI), and CLAUDE.md keeps only the command to run and the two ways sessions have actually got it wrong: editing a merged chapter, and refusing to edit an unmerged one. --- CLAUDE.md | 40 +++++++++++++++------------- contributing/writing-stories-docs.md | 4 ++- 2 files changed, 25 insertions(+), 19 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 3c953c5..0204a53 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -110,29 +110,33 @@ quality bar — dense, opinionated, long-form prose. Writing a story is two halves: write the narrative, then integrate it into the spec. Don't skip the second half. -### Append-only is literal, and it is checked by diffing -A **merged** story chapter is never edited — not to correct a claim the design -has since retired, and **not** to append a forward pointer to it. A new chapter -goes after every merged one, never between chapters already on `main`. Say what -stopped being true from the *new* chapter, naming the older chapter's claim. -Both of these were caught in review on this repo rather than by the author, so -verify before committing: +### Append-only: run the check, and read the rule where it lives +**Story guide §5 owns this rule** — what may be edited, what the PR-versus-commit +distinction means, and the rare consolidation exception. Read it there; it is +the source of truth for contributors and agents alike, and this section adds +only what a session keeps getting wrong. + +Nothing enforces it automatically — no CI, no hook. Run the check yourself +before every commit that touches `stories/`: ```sh git diff origin/main -- stories/.md | grep -E "^-[^-]" ``` -Additions only is the passing result. Any output means a merged chapter was -edited, or a chapter was inserted ahead of one. Story guide §5 has the -reasoning. - -**The frozen unit is the PR, not the commit.** Chapters your own branch adds are -drafts until it merges: rewrite them, reorder them, insert a new chapter among -them freely — a decision reached late in review often belongs *before* the -chapters already drafted. The grep above is exactly the right check because it -diffs against `main`, so it stays quiet through all of that and fires only when -something merged moves. Don't over-apply the rule to your own unmerged work; a -previous session did, and had to be corrected by the maintainer. +Additions only is the passing result. + +Two failure modes, both from real sessions on this repo: + +- **Too loose.** Editing a merged chapter to fix a retired claim, or bolting a + forward pointer onto one. Say what stopped being true from the *new* chapter + instead, naming the older chapter's claim. Caught in review, not by the author. +- **Too strict.** Refusing to touch chapters *your own branch* added, because + they were already written. They are drafts until the PR merges — rewrite, + reorder, and insert among them freely; a decision reached late in review often + belongs before them. The grep is quiet through all of that by design. + +If the grep is clean, you have not violated the rule, whatever your instinct +says. ### Interview the maintainer — you cannot reconstruct the real reasoning The actual thread — which roads were tried and rejected, in what order the diff --git a/contributing/writing-stories-docs.md b/contributing/writing-stories-docs.md index b8b48ed..20cf437 100644 --- a/contributing/writing-stories-docs.md +++ b/contributing/writing-stories-docs.md @@ -120,7 +120,7 @@ The href ends in the chapter's heading **anchor** so the link scrolls straight t This is the discipline that makes the folder a *history* rather than a stale snapshot. -**Append, don't overwrite.** When the design changes, the old reasoning did not become false — it became *the previous chapter*. So when the spec moves, add to the story: open a **new chapter at the end of the file** that names the cause and what it forced — *"The shift to X meant the old Y no longer held, so we…"* — and pin its spec references to the new commit (§4.2). The discarded path stays on the page as the record of why the design used to be one way and is now another; that causal trail is often the most illuminating thing in the file, and rewriting it away destroys it. +**Append, don't overwrite.** When the design changes, the old reasoning did not become false — it became *the previous chapter*. So when the spec moves, add to the story: open a **new chapter at the end of the file** — after everything already published, see the publication note below — that names the cause and what it forced — *"The shift to X meant the old Y no longer held, so we…"* — and pin its spec references to the new commit (§4.2). The discarded path stays on the page as the record of why the design used to be one way and is now another; that causal trail is often the most illuminating thing in the file, and rewriting it away destroys it. "Append" is meant literally, and it has two teeth: @@ -137,6 +137,8 @@ git diff origin/main -- stories/.md | grep -E "^-[^-]" Any output is a violation: a removed or rewritten line means a published chapter was edited, and a `-` next to a chapter heading means a chapter was inserted ahead of one that had already merged. The clean result is additions only — which is also why the check is the right one to run: it compares against what is published, so it stays silent while you rearrange your own branch's new chapters and speaks up the moment you disturb a merged one. +Nothing runs this for you. There is no CI job and no hook; the rule is enforced by the author running the diff before committing and by the reviewer running it again on the branch. That is deliberate — the "consolidate dead threads" exception below is a judgement call no check could make, so a green check would have to be overridable anyway — but it does mean a violation reaches `main` if both people skip it. Treat the command as part of the commit, not as an optional audit. + **Consolidate dead threads, sparingly.** Appending forever would bury the present under history. So a chapter *may* be rewritten or folded down — but only when its narrative has become pure dead weight: it no longer illuminates the present design *and* is not interesting as history. That is a high bar. The default is to append; consolidation is the rare exception, not routine cleanup, and when in doubt you keep the history. The contrast to hold in mind: the **spec** is rewritten to the present on every change — it states only what is true now. The **story** accumulates — it states how what is true now came to be. They have opposite update rules on purpose.