perf(runtime): Uint8Array.subarray copies every suffix; 1,251x Node at 10k views

[proggeramlug]

What happened

Creating n suffix views with input.subarray(i) becomes nearly quadratic in Perry. The measured ratio rises from 11.70x at n=100 to 23.81x at 1,000 and 1,251.19x at 10,000; Perry times out at the next size. The source copies the suffix before recording alias metadata, so logically shared views still pay payload-copy cost.

Measured against Node v26.5.1 using Perry perry 0.5.1531 at 9495bfc95e2afcfb5a7cb535e440e61ec0722cb1. This is evidence from that pinned revision, not a claim that current main was remeasured. First reproduce on current main; if it is already fixed, identify the fixing commit and attach the comparison.

Measurements

Times are median milliseconds per workload invocation. Ratios are Perry/Node. A correctness or timeout classification takes precedence over performance; successful smaller-size timings on those rows are diagnostic evidence.

binary-uint8array-subarray — TIMEOUT

n	Node ms / status	Perry ms / status	ratio	Node checksum	Perry checksum
100	0.011710	0.137018	11.70×	18622	18622
1000	0.054758	1.303522	23.81×	630318	630318
10000	0.428716	536.405209	1251.19×	51281607	51281607
100000	3.598368	TIMEOUT	—	12816444	—
1000000	34.806917	SKIPPED	—	127857054	—

Log(time)/log(n) least-squares slopes: Perry 1.796, Node 0.876, delta 0.920.

Workload: n suffix views over n source bytes; result length/first byte consumption detects incorrect offsets without scanning each view. This Uint8Array path uses BufferHeader storage and js_buffer_slice; its copied storage plus alias metadata differs from a zero-copy view. Read-only checksums do not prove write aliasing.

Slopes cover different completed sizes: Node [100, 1000, 10000, 100000, 1000000], Perry [100, 1000, 10000].

perry n=100000: TIMEOUT, exit -9

Process exceeded 60 s

What is expected / acceptance criteria

• Reproduce current main, then remeasure the unchanged workload at 100, 1k, 10k and larger sizes with matching Node checksums; remove the growing suffix-payload-copy cost and document before/after slopes.
• Verify writes through the source and through a subarray are visible in both directions, including overlapping and nested subarrays. The timing benchmark is read-only and does not prove these semantics.
• Preserve negative/clamped bounds, empty views, byteOffset/byteLength and backing-buffer identity where exposed; retain Buffer and Uint8Array parity coverage.
• Exercise view lifetime across moving GC, including a live view whose original receiver is no longer directly referenced, and reclamation of dead views without retaining the backing forever.

Implementation to inspect

Hypothesis: This Uint8Array uses the BufferHeader path. dispatch_buffer_method calls js_buffer_slice, which allocates slice_len bytes, copies the entire requested range, and registers an alias. Across suffixes the copied byte count is n(n+1)/2, before registry and GC costs. remove_entries_for_dead_buffer can additionally scan view registries when views die. A backing-store/offset representation should be investigated, while preserving all alias and lifetime semantics; the existing alias-propagation machinery cannot simply be removed without a replacement.

• crates/perry-runtime/src/buffer/access.rs: js_buffer_slice
• crates/perry-runtime/src/buffer/view.rs: remove_entries_for_dead_buffer
• crates/perry-runtime/src/object/buffer_dispatch.rs: dispatch_buffer_method

Source reading narrows the investigation; it does not establish exclusive runtime/compiler attribution. No compiler or runtime changes were made to obtain these measurements.

Agent scope and coordination

Own the BufferHeader subarray/view path in buffer/access.rs, buffer/view.rs and object/buffer_dispatch.rs. This is distinct from the Unicode string slicing issue. Related buffer-classification and GC changes may touch the same structures, so coordinate before changing their layout.

Implementation work can proceed in separate branches. Serialize benchmark runs on any shared host; parallel timing runs invalidate small performance comparisons. Preserve language semantics and moving-GC safety.

• Related history/context: Improve node:buffer slice/subarray shared backing-store parity #1205
• Related history/context: audit: lookup_typed_array_kind miss-consumers vs buffer-kind Uint8Array (split-brain classification, #9342 siblings) #9347

Reproduce and remeasure

Everything needed for the workload is embedded below; no private repository, fixture, npm package, or shared prelude is required. Save a complete benchmark block under its indicated filename in /tmp/perry-builtin-repro/. Use Node 26.5.1 to match this baseline; it runs these TypeScript files directly.

From the Perry checkout/branch being evaluated:

mkdir -p /tmp/perry-builtin-repro
cargo build --release --locked -p perry -p perry-runtime-static -p perry-stdlib-static
export PERRY_RUNTIME_DIR="$PWD/target/release"
export TZ=UTC LC_ALL=en_US.UTF-8
git rev-parse HEAD
node --version
target/release/perry compile /tmp/perry-builtin-repro/binary-uint8array-subarray.ts --no-auto-optimize -o /tmp/perry-builtin-repro/app

To reproduce the historical baseline, use the pinned commit above in a separate checkout and build the compiler and both libraries there. Repeat compilation for each additional benchmark below. Run this small driver from the same checkout, changing name and sizes for that benchmark:

import json, math, subprocess
name = 'binary-uint8array-subarray'
sizes = [100, 1000, 10000, 100000, 1000000]
result_on_stderr = True
stopped = set()
points = {"node": [], "perry": []}
for n in sizes:
    pair = {}
    for engine, cmd in [("node", ["node", f"/tmp/perry-builtin-repro/{name}.ts"]),
                        ("perry", ["/tmp/perry-builtin-repro/app"])]:
        if engine in stopped: continue
        try:
            p = subprocess.run(cmd + [str(n)], capture_output=True, text=True, timeout=60)
        except subprocess.TimeoutExpired:
            print(engine, n, "TIMEOUT"); stopped.add(engine); continue
        if p.returncode:
            print(engine, n, "ERROR", p.returncode, p.stderr, p.stdout); continue
        r = json.loads(p.stderr if result_on_stderr else p.stdout)
        pair[engine] = r
        points[engine].append((n, r["ms_per_run"]))
        print(engine, r)
    if len(pair) == 2:
        print("ratio", n, pair["perry"]["ms_per_run"] / pair["node"]["ms_per_run"],
              "checksum_match", pair["perry"]["checksum"] == pair["node"]["checksum"])
def slope(rows):
    if len(rows) < 2: return None
    x = [math.log(n) for n, t in rows]; y = [math.log(t) for n, t in rows]
    mx = sum(x)/len(x); my = sum(y)/len(y)
    return sum((a-mx)*(b-my) for a,b in zip(x,y))/sum((a-mx)**2 for a in x)
print("slopes", {engine: slope(rows) for engine, rows in points.items()})

Record before/after results from the same unchanged source, engine versions and host. The measured driver uses seeded setup outside timers, at least 200 ms AND five warmup runs, then seven samples with at least 20 ms measured work each. Fresh input is prepared before each timer for mutating workloads. The median per-run time is reported, with checksum consistency checked on every invocation. Timeouts cover setup, warmup and sampling, not just one builtin call.

Environment and limits

• CPU: Apple M1 Max; 10 logical cores; arm64.
• OS: macOS-26.5-arm64-arm-64bit-Mach-O; target: native host.
• Node: v26.5.1; Perry: perry 0.5.1531; build: release from source.
• Compile flag: --no-auto-optimize; compiler and both matching runtime archives were rebuilt together.
• The pinned source revision and compiler/runtime/Node artifact hashes were unchanged throughout the sweep.
• Load average at measurement start: [58.3896484375, 53.017578125, 57.90087890625]; end: [25.240234375, 30.416015625, 24.8681640625].
• Host contention limits precise constant-factor claims; repeat on a quiet host before asserting an improvement.
• Timings include timer overhead and checksum calculation. String hashes bound lookup count, not Unicode lookup cost; indexed consumption may also force Node string materialization.

Minimal correctness reductions

This issue is a performance workload; the complete checksum-gated reproducer follows.

Complete standalone benchmark sources

binary-uint8array-subarray.ts — sizes [100, 1000, 10000, 100000, 1000000]

Size meanings and fresh-input policy are in the leading metadata. result_on_stderr for this file: True.

// @runtime {"name": "binary-uint8array-subarray", "category": "binary-node", "verification": "checksum", "sources": [{"file": "crates/perry-runtime/src/buffer/access.rs", "function": "js_buffer_slice"}, {"file": "crates/perry-runtime/src/buffer/view.rs", "function": "remove_entries_for_dead_buffer"}, {"file": "crates/perry-runtime/src/object/buffer_dispatch.rs", "function": "dispatch_buffer_method"}], "hypothesis": "Buffer-backed Uint8Array.subarray copies each complete suffix before registering alias metadata, causing quadratic copied bytes across n suffixes; per-view GC cleanup can also scan the full view registry.", "notes": "n suffix views over n source bytes; result length/first byte consumption detects incorrect offsets without scanning each view. This Uint8Array path uses BufferHeader storage and js_buffer_slice; its copied storage plus alias metadata differs from a zero-copy view. Read-only checksums do not prove write aliasing.", "asynchronous": false, "output_stderr": true, "fresh_input": false}
// Standalone file. Shared helpers/driver are inlined by common.py.

let seed = 0x12345678;
function rnd(): number {
  seed ^= seed << 13; seed ^= seed >>> 17; seed ^= seed << 5;
  return (seed >>> 0) / 4294967296;
}
function numbers(n: number): number[] {
  const a: number[] = [];
  for (let i = 0; i < n; i++) a.push(Math.floor(rnd() * 1000000));
  return a;
}
function hashArray(a: number[]): number {
  let h = a.length;
  for (let i = 0; i < a.length; i++) h = (h * 31 + a[i]) % 1000000007;
  return h;
}
// Bounded checksum work avoids making string slicing/indexing part of every
// string benchmark's asymptotic cost. The workload itself consumes its result.
function hashString(s: string): number {
  let h = s.length;
  const step = Math.max(1, Math.floor(s.length / 32));
  for (let i = 0; i < s.length; i += step) h = (h * 31 + s.charCodeAt(i)) % 1000000007;
  return h;
}

function hashBytes(a: Uint8Array): number {
  let h = a.length;
  const step = Math.max(1, Math.floor(a.length / 32));
  for (let i = 0; i < a.length; i += step) h = (h * 31 + a[i]) % 1000000007;
  return h;
}

function setup(n: number): Uint8Array {
  const a = new Uint8Array(n);
  for (let i = 0; i < n; i++) a[i] = Math.floor(rnd() * 256);
  return a;
}
function run(input: Uint8Array): number {
  let h = 0;
  for (let i = 0; i < input.length; i++) {
    const view = input.subarray(i);
    h = (h + view.length + view[0]) % 1000000007;
  }
  return h;
}

// Size is the final argument: both native Perry and Node expose it reliably.
const n = Number(process.argv[process.argv.length - 1]);
if (!(n > 0)) throw new Error("Expected a positive size argument");
function benchmarkMain(): void {
  seed = 0x12345678;
  const preparedInput = setup(n);
  let checksum = 0;
  let seen = false;
  let warmMs = 0;
  let warmRuns = 0;
  while (warmMs < 200 || warmRuns < 5) {
    seed = 0x12345678;
    const input = preparedInput;
    const start = performance.now();
    const value = run(input);
    const elapsed = performance.now() - start;
    if (!(elapsed >= 0)) throw new Error("Invalid monotonic timer");
    warmMs += elapsed;
    warmRuns++;
    if (seen && value !== checksum) throw new Error("CORRECTNESS: unstable checksum during warmup");
    checksum = value;
    seen = true;
  }
  const samples: number[] = [];
  let runs = 0;
  for (let sample = 0; sample < 7; sample++) {
    let elapsed = 0;
    let count = 0;
    // Mutable workloads prepare fresh input BEFORE each timer; immutable
    // workloads reuse setup. Neither preparation nor validation is measured.
    while (elapsed < 20) {
      seed = 0x12345678;
      const input = preparedInput;
      const start = performance.now();
      const value = run(input);
      const duration = performance.now() - start;
      if (!(duration >= 0)) throw new Error("Invalid monotonic timer");
      elapsed += duration;
      count++;
      if (value !== checksum) throw new Error("CORRECTNESS: unstable checksum during sampling");
    }
    samples.push(elapsed / count);
    runs += count;
  }
  // Do not depend on Array.sort to compute the median of a sort benchmark.
  for (let i = 1; i < samples.length; i++) {
    const v = samples[i];
    let j = i - 1;
    while (j >= 0 && samples[j] > v) { samples[j + 1] = samples[j]; j--; }
    samples[j + 1] = v;
  }
  console.error(JSON.stringify({name: "binary-uint8array-subarray", category: "binary-node", n,
    ms_per_run: samples[3], runs, checksum}));
}
benchmarkMain();

[proggeramlug]

Fixed by #10071, landed on main via merge train #10095 at 1ae0f8449.

The close keyword did not fire because the train merges as its own branch, so closing manually.