cruft-verify

What's up with Cruft?

Reproduce cruft's performance and conformance numbers yourself. Companion to cruftless-dev/cruft and the claims on cruft.sh. Bench suites run through the orchestrator (run-cruftbench.mjs), which normalizes every suite into one results.jsonl + summary.md shape (schema: bench/schemas/normalized-row.schema.json); conformance normalizes via conformance/to-normalized.mjs. Run a suite directly and you get that runner's native output instead.

Setup

• cruft on PATH (or CRUFT_BIN). Install: npm i -g cruftjs.
• Cross-runtime suites also want node, bun, deno (NODE_BIN/BUN_BIN/DENO_BIN); missing ones are skipped.
• Results go to ./results/ (gitignored; override TEST_ARTIFACTS_DIR).

Benchmarks

node bench/scripts/run-cruftbench.mjs --help   # orchestrator (12 suites)
node bench/scripts/run-cruftbench.mjs run --suite all --smoke --engines node,bun,deno,cruft

bench/worker-loads/run-prod-worker-loads.sh    # worker isolation memory/latency
node bench/run-matrix.mjs --runs 5             # cross-runtime matrix
node bench/scripts/run-neutral.mjs             # third-party CLBG/GCBench corpus

Machine-dependent; each summary.md records the host + engine versions. cruft wins on worker memory and cold-start, loses raw single-threaded throughput to mature JITs. Losses are reported, not hidden.

Conformance (test262)

git clone https://github.com/tc39/test262
H="$PWD/test262/harness"

# one test:
T262_HARNESS_DIR="$H" cruft test262 run "$PWD/test262/test/<path>.js" --runner conformance/runner-full.mjs

# whole corpus -> per-test.jsonl -> normalized rows:
find test262/test -name '*.js' ! -name '*_FIXTURE*' > paths.txt
cruft test262 sweep --paths paths.txt --runner conformance/runner-full.mjs --harness "$H" --out per-test.jsonl
node conformance/to-normalized.mjs per-test.jsonl results/test262 cruft

Unflagged tests run in both strict and non-strict mode (pass = both pass), per INTERPRETING.md; strict runs in a fresh child. T262_MODE = dual (default) / sloppy / strict.

conformance/test262-fyi-adapter/ — web-reality lane vs V8/JSC/SM (legacy .caller/.arguments on). See its README.

Not completely 100%; there are a few wall-clock-bound tests that only pass at 1min or so. There is also an opt-in legacy-caller. There are a several fails: fail_count and some reasons why.

Output

{ "schema_version": 1, "suite": "worker-loads", "benchmark": "webhook_transform@64",
  "engine": "cruft", "status": "PASS", "metric": "peak_rss", "value": 226.4, "unit": "MiB" }

status ∈ PASS | FAIL | DIVERGE | UNSUPPORTED | TIMEOUT | INVALID. Every suite shares this same shape, so results.jsonl concatenates across suites.

Third-party & license

Harnesses and original fixtures: Apache-2.0 OR MIT. Some fixtures reimplement published algorithms (CLBG, GCBench) — provenance in bench/THIRD-PARTY.md, upstream license texts in licenses/.