v1.14.0
v1.14 Tranches 1–2 + 4 — the measurement foundation, then the intent channel it licensed
(roadmap docs/research/2026-07-roadmap-proposal.md). Tranches 1–2 were repo-internal (eval/ and
CI only; the one core exception is the repair() purity fix under Fixed): fix the ruler before
measuring capability. Gate G1's PASS then cleared Tranche 4, which DOES extend the published
surface — the intent channel below.
Added — Tranche 4: the intent channel (2026-07-12; core + CLI, zero new runtime deps)
src/intent.ts— the judge-v2 scoring core, lifted into the core package. A brief's
checkable expectations as data (Intent), lowered to the shallow predicate kinds
(room-count/room-exists/room-area/total-area/adjacent/reachable, plus a
new gatingroom-windows), checked againstdescribe()facts.
validateIntent(source, intent)→{ ok, satisfied, total, violations, subscores, assertions, diagnostics }with typed, catalogued violations and Nickel-style spanless blame
messages (intent /roomsInclude/1: no room matching concept "bathroom" …);
intentFromJson(zero-dep pathed shape walker);feedbackForResult(deterministic
per-violation correction prompts — advisory data, ADR 0005, never auto-applied).src/intent-concepts.ts— the concept vocabulary, now production name resolution. Byte-
mirrors the eval's table; a known concept resolves exactly as the eval judges (label →
room_type→uses, token-bounded), an unknown one falls back to a literal
id → label → uses → room_type match.- Eight catalogued codes:
E_INTENT_ROOM_MISSING/_ROOM_COUNT/_ROOM_AREA/
_TOTAL_AREA/_NO_WINDOWgate;E_INTENT_NOT_ADJACENT/_NO_DOOR/_UNREACHABLEare
advisory (gate: false— scored, never failingok;reachableblames by cause: no entrance
→NO_DOOR, cut-off rooms →UNREACHABLE). Promoting adjacency/reachability to gating stays
parked on T3's still-open loop-vs-resampling question. schemas/intent.schema.json(npm run gen:intent-schema, drift-tested, served by the docs
site). Its field docs make Gate G1's two lessons normative: the area band conventions
("about/~/bare N m²" → ±10%; "at least N" →minonly; qualitative words → no assertion) and
the count discipline ("assert a room count only when the brief enumerates it").- CLI:
arch validate --intent <intent.json>(the gate — exit 2 on a gating violation;
composes with--graph/--strict;--feedbackappends the correction prompts) and
arch score <file> --brief <intent.json>(the continuous meter —satisfied/total+
subscores, exit 0 on any successful measurement). describe()windows gainfacing: "N"|"S"|"E"|"W"(append-only; the outward normal of the
window's host wall), and intentwindowsassertions take an optionalfacing.- The eval now consumes the same implementation (
eval/assertions.ts/synonyms.tsare thin
re-export shims; run.ts'sExpectis the productionIntent) — one judge, zero eval↔prod
skew.JUDGE_VERSIONstays "2", proven by a pinned fixture (eval/judge-fixture.json+
test/eval-fixture.test.ts) that every corpus per-assertion judgment is byte-identical across
the lift; the fixture is regenerated only to record an approved bump, never to green a red suite.
Added — release engineering: tokenless OIDC publishing (npm + MCP registry)
.github/workflows/release.yml— av*tag push (or manual dispatch) publishes the core,
then the MCP shim, to npm via OIDC trusted publishing with provenance (no npm token exists
anywhere; each package carries a one-time Trusted Publisher registration on npmjs.com pointing
at this workflow), then syncs the MCP registry withmcp-publisher login github-oidc—
also tokenless. Every step is idempotent (versions already on a registry are skipped, and the
registry sync is guarded by the registry's own state), so partial failures re-run safely.
Replaces the local granular-token publish flow that npm is deprecating through 2026–2027.- MCP shim 0.2.0 (
@chanmeng666/archlang-mcp, registry entry updated): thevalidatetool
takes an optionalintent(gating assertions fail it; advisory ones score), a newscore
tool is the continuous satisfaction meter, andintent.schema.jsonships as the
intent-schemaresource — the sameintentFromJson/validateIntent/feedbackForResultpath
the CLI uses. - Provenance gotcha, recorded: npm E422-rejects a publish whose
package.json
repository.urlcasing differs from the OIDC-attested repo (ChanMeng666, not
chanmeng666) — fixed in both package.json files + server.json.
Added — Gate G1 verdict + the L2 experiment harness (2026-07-12; still repo-internal)
- Gate G1: PASS (
eval/g1/— generator harness, generated intents, double-blind scores,
report). NL→intent-JSON per-assertion faithfulness on all 26 briefs: 154/157 (98.1%) vs
93.4% per-assertion accuracy of direct.archgeneration (one-tailed z = 2.08, p = .019;
valid-only sensitivity variant below resolution — recorded). The intent channel (roadmap T4:
src/intent.ts,arch validate --intent,intent.schema.json) is cleared for a future
release. The generation prompt is oracle-isolated andtest/g1.test.tsenforces it. - T3 harness: the L2 tier (
eval/l2.ts+eval/l2-run.ts+.github/workflows/eval-l2.yml,
npm run eval:l2). Diagnostic feedback loop (≤2 rounds, fed only compile/lint diagnostics +
fix --dry-runpreviews + trimmeddescribe()— oracle-isolated) against an equal-token-budget
i.i.d. resampling control (Olausson accounting, round-up favours the control), per-metric
best-of, mean±σ across trials,pass@n/pass^n, retrying author + per-brief error isolation.
Offline-tested (14 tests). The live experiment has not been run (cost declined) — the
loop-vs-resampling question remains open and no loop-gain claim is made.
Added — judge v2: brief-grounded intent scoring
- Intent-assertion scoring core (
eval/assertions.ts,JUDGE_VERSION = "2").scoreSourceno
longer greps the goldens for label substrings and golden-derived area bands; it lowers each brief
to a small intent-assertion data structure — the shallow five kindsroom-count/
room-exists/room-area/total-area/adjacent/reachable— and checks the model's
plan against those. The five-kind boundary is deliberately the one a futuresrc/intent.tscan
lift wholesale (Tranche 4 hook).Scoregains append-onlysubscores/assertions/
judgeVersionfields. - Oracle-isolated synonym table (
eval/synonyms.ts,SYNONYMS_VERSION = 1). Room-label matching
runs through a versioned, never-shown-to-the-model concept table with token-bounded,
one-room-one-concept greedy assignment — so "wc"/"toilet"/"bath" resolve to one concept without
leaking the answer into the prompt. - Brief-grounded area checks. Area is verified only where the brief states a number, in a
±10–15% band around the brief's number; all 20 golden-derived bands were deleted. Qualitative
size words ("compact", "generous") carry no cap yet (a documented tier-b hook, added the day a
real "oversized compact" instance appears). - Frozen corpus-review rubric (
eval/rubric.md). Blind-drafted by an isolated agent, then frozen
with the approver's decisions: room-count policy B (a ±1 surplus passes the gate only when the
extra room is pure circulation, operationalized asplanCirc >= expectedCirc + 1); one-room-one-concept
greedy assignment; qualitative size words carry no cap. Adjacency and reachability score as
subscores only, never a gate (Tranche 4 hook). - Corpus 22 → 26. Three prompts amended so every room count is brief-derivable
(two-bath-flat,against-wall-bath,accessible-bath), plus a new per-room-area slice
(sized-kitchen-flat,sized-bedrooms,sized-wet-room,sized-office-mix) so the area dimension
is no longer total-only (H5) — every band carries the brief-source quote it came from. - L1 deterministic-tool gate (
eval/faults/,eval/l1.ts,test/fault-injection.test.ts, in CI
vianpm test). Six single-defect fixtures (off-wall door/window/opening, furniture-through-wall,
blocked-doorway, and a combined case) prove thel1Pipeline— a bounded machine-applicable-fix
fixpoint (mirroringarch fix) followed byrepair(), in the ADR 0011 → ADR 0006 order — heals
every defect class deterministically and idempotently, and is a byte no-op on a clean golden. --l1live overlay (eval/run.ts, live runs only). Reports the deterministic dividend
ΔL0→L1 (whatfix+repairrecover for free, zero extra API calls) with a per-row heal column;
the committed baseline delta stays L0-only so cross-run comparisons don't silently fold the tool
tier into a model score.- eval-live workflow inputs (
.github/workflows/eval-live.yml): a--l1toggle (default on) and
the corpus-coveringmaxdefault of 26.
Changed — live-harness integrity
- Token budget & determinism. Anthropic
max_tokens2048 → 16384 (reasoning models spend
thinking tokens out of the completion cap — the 2048 ceiling truncated output into false
invalidity) withtemperature: 0and ephemeral prompt caching; the OpenAI path pins
seed = 20260711and recordssystem_fingerprint(temperature deliberately not sent). --budget <n>tok|<n>usdcircuit breaker — a pre-call estimate halts the run before it
overspends; skipped briefs are excluded from the denominators (over-estimating direction, verified
price map).- Cross-judge guard.
Baselinenow carries ajudgefield andrenderDeltaflags any delta
taken across a judge-version change as non-comparable — a judge change is never a capability
result. - Calibrated judge-v2 baseline (
eval/live-baseline.json; 26 briefs,gpt-5.5-2026-04-23,
seed-pinned, GitHub Actions): L0 valid 25/26 (96%) · intent 13/26 (50%) · sound 4/26 (15%);
the--l1overlay lifts it to intent 18/26 (69%, ΔL0→L1 +5) · sound 6/26 (+2) — 7 briefs
healed by 47 repair moves, 0fixedits. The old 9% one-shot intent was ~55–65% measurement
artifact (deep-dive H2); the calibrated 50% sits inside the roadmap's predicted 45–60% band.
Residual true failures are dominated by physical violations (which L1 clears), with a few
room-count and placeholder-label misses and one compile failure (the model inventing alabel
statement). Judge-v1 numbers are kept only as history.
Fixed
repair()is pure again across repeated calls. It mutated the shared parse-stage
memo's AST in place (moving furnitureatnodes), so a secondrepair()of the
byte-identical source saw already-moved pieces and reported zero changes — same input,
history-dependent output, violating ADR 0006's determinism promise.repairnow works
on a private deep clone of the parsed plan;compile()output was never affected
(regression-tested intest/repair.test.ts). Found by the new fault-injection L1 gate.