agent-reference can hand an agent one slice instead of the whole document (arpd-6iis)
Nothing changes unless you ask for it. satsuma agent-reference with no flags
still prints the whole document, byte for byte — 28,109 bytes, asserted against
AI-AGENT-REFERENCE.md by a test, so anything already piping the command keeps
working. What is new is that the reference is no longer all-or-nothing.
The document is now cut into eight canonical sections, each with a measured token
cost, and three additive flags select from them:
--list— every section id, its cost, and the profiles that need it.--section <id>— one named section.--profile writeor--profile read— the slice a task profile needs.
Costs are real measurements, not bytes/4 estimates: o200k_base via
js-tiktoken, counted by one function the CLI's prebuild and the measurement
script both call, so the printed figure cannot drift from the text it describes.
| Section | Tokens | Profiles |
|---|---|---|
grammar |
948 | write |
conventions |
1450 | write, read |
mistakes |
448 | write |
examples |
441 | write |
cli-index |
1136 | read |
cli-composition |
1672 | read |
workflow-generate |
460 | write |
workflow-read |
264 | read |
The whole document is 6813 tokens. --profile write is 3743 and
--profile read is 4520, so an agent that only authors Satsuma pays 55% of
the full cost and one that only traces an existing workspace pays 66%.
Worth recording because it contradicts the plan: the feature was scoped expecting
read to be the cheaper profile. Measurement said otherwise — reading a workspace
needs the CLI's command surface, which is bulkier than the grammar. The estimates
that shaped the design were wrong in direction, and the numbers above are what
replaced them.
Unknown ids fail loudly rather than falling back to the whole document: an
unrecognised --section or --profile exits 1 and lists the valid values, and
combining any two of the three flags exits 2.
AI-AGENT-REFERENCE.md is now generated. It stays at the same repo path with
the same content, but the canonical text lives in reference/*.md and the
committed file is composed from it by npm run regenerate:agent-reference. Editing
it directly will be overwritten. A drift guard makes that safe rather than
surprising: scripts/agent-reference-compose.test.mjs asserts that composing every
section in canonical order reproduces the file byte for byte, and it runs in both
the pre-commit hook and CI, so a hand-edit or a missed regeneration fails before it
can ship. A second guard, cli-index-flags.test.ts, checks every flag the
reference documents against the real command registration — renaming a CLI flag now
breaks the docs test that describes it.
A new satsuma-language skill ships the same reference as a lazy-loading
envelope for skill-aware agents: 164 resident tokens of frontmatter, with the
full document loaded only once the skill triggers. That is the whole reference
deferred, where the CLI's flags are the opposite trade — pay for exactly the slice
you asked for, resident cost zero.
One comparison point measured along the way, for anyone weighing an MCP server
against the CLI: the 23 commands' tool schemas come to 2253 tokens, resident on
every request whether a tool is called or not. No MCP server was built; the
schemas were generated by introspecting the real command registrations so the
figure cannot drift from --help.
The visualisation can paint coverage, and trace one field's whole chain (sl-3de8)
Two additions to the visualisation, plus the VS Code wiring that reaches them.
A coverage overlay on the overview. A new Coverage button in the toolbar —
overview mode only, off by default — repaints each schema card's header with what is
mapped rather than what merely exists. The header count changes from 9 fields to
the leaf ratio 7/9, gains a percentage badge, and gains a proportional fill bar
behind the header text. Hovering reads
7/9 leaf fields mapped (78%) — 2 records partly mapped, so the records a
percentage cannot describe are still visible.
The overlay is deliberately paint-only: toggling it re-renders but never re-runs
layout, and the overview's layout object is asserted to be the same object across a
toggle, so a card cannot move because you asked what was covered. Where coverage
could not be computed the header reads Coverage unavailable rather than 0/N —
"not measured" is not "nothing is mapped". Two card kinds are out of scope and show
no overlay: metric cards and fragment cards.
Field rows show their coverage as a shape, not just a fill. A field's port dot
is now solid for a covered field, a half-moon inside an accent ring for a partly
covered record, a hollow ring for an uncovered one, and dashed where coverage was
not computed. That distinction is the point: a record whose subtree was only partly
mapped previously painted the same solid dot as a fully mapped one, so partial
coverage was invisible unless you hovered the exact row (sl-f0x6). It renders in
both the overview's expanded card and the mapping detail view's source and target
columns, and it is not gated on the coverage overlay — wherever coverage is
known, the dots show it.
A chain view for one field's lineage. The per-field lineage icon on a field row
now opens a left-to-right rail: upstream hops furthest-first, the focused field in
the middle, then downstream hops, joined by connector arrows so the rail reads as one
flow rather than a row of separate cards. Each hop names the mapping responsible for
it as a button that opens that mapping's detail, badges its classification (NL, and
NL-derived in its own palette), and lets you click any field to re-focus the chain
there. Wide fans collapse by namespace once a column carries more than three hops
and expand on click. A hop at the traversal limit is badged ⋮ depth limit, so a
truncated chain says so instead of looking complete. Editing the file while the
chain is open re-traces it rather than dropping you back to the overview.
examples/multi-hop-lineage/ is in the corpus as a demo fixture for it: a
deliberately artificial seven-schema, six-hop chain with varying record nesting,
which gives three upstream and three downstream hops from the middle field. It is
labelled in examples/README.md as exactly that — a fixture, not a plausible
pipeline — so nobody mistakes it for a pattern worth copying.
In VS Code, Satsuma: Show Field Lineage now opens that rail inside the main
visualisation panel, answered by a new satsuma/fieldChain request against the
language server. The chain is scoped to the entry file's import-reachable closure
and parses files that are not open in any editor from disk, so it can cross into an
imported file you have never opened. This replaces the bespoke field-lineage
webview, which is deleted along with its CLI shell-out — the old horizontal
source.field → [transform] → target.field presentation is gone. A new command,
Satsuma: Show Coverage Overlay, opens the panel with the overlay already on,
taking the extension from eight commands to nine.
Opening the chain view is a host's job — the component renders a model it is handed
rather than tracing lineage itself — and all three hosts now do it, so the same
gesture works in the extension, in the Playwright harness, and in the public
playground, which is built from the harness bundle.
The whole of this section is covered by browser tests rather than by unit tests
alone: the coverage overlay's toggle and badge values, the port-dot treatments in
both themes, and the chain view's hop ordering, classification badges, depth-limit
affordance, namespace-fan collapse and connector arrows are each asserted against a
rendered DOM. That matters here more than usual, because none of it is observable
from a function's return value — a class that never lands on a row, or a connector
that paints outside the gap it is meant to fill, passes every unit test there is.
For the component's consumers, everything here is additive: FieldChainModel and
FieldChainHop are new types delivered by a new request rather than folded into
VizModel, so a host holding a cached payload renders the overview exactly as
before and simply never enters chain mode. One caveat for a host that switches
exhaustively over the automation view modes — that union has gained chain.
field-lineage --json reports how far each hop is (sl-4czz)
Two additive fields, so nothing that reads the existing keys changes: depth on
every hop in upstream[] and downstream[], counting hops from the focused field
(1 for a direct neighbour), and maxDepth at the top level, echoing the
requested --depth (default 10). Together they let a consumer tell a chain that
ended because it ran out of lineage from one that ended because it hit the limit.
The human-readable output is unchanged.
BREAKING: graph --json spells every entity canonically (lgc-wtz1)
Any consumer keying off graph --json ids for a non-namespaced entity needs
updating: "raw_data" is now "::raw_data". The payload previously spelled the
same entity two ways in one document — nodes[].id and schema_edges[] endpoints
used the bare index-key form, while edges[] endpoints were already canonical — so
joining a node to the field edges that referenced it required knowing which form
each key used, and a caller who assumed one form silently matched nothing.
Every id in the payload is now the canonical form: ::name for a file-scope entity,
ns::name for a namespaced one. The keys that moved are nodes[].id,
nodes[].sources, nodes[].targets, and schema_edges[].from/.to, plus
edges[].mapping. edges[].from/.to were already canonical and are unchanged.
Namespaced workspaces see no difference at all — warehouse::staged is identical
in both spellings, which is exactly why the inconsistency survived this long. It is
non-namespaced workspaces, the common case, where every id gains a :: prefix.
The :: prefix marks the global namespace explicitly. It buys a human nothing — it
is not even valid Satsuma syntax, so it cannot be pasted back into a file — which is
why human-readable graph output is unchanged, --compact included, and gained
tests to keep it that way.
Two limits worth knowing. ::name is now also accepted as command input, across
the thirteen commands that resolve an entity by name (arrows, coverage,
field-lineage, fields, lineage, mapping, match-fields, meta, metric,
nl, nl-refs, schema, where-used), so an id read out of graph --json can be
fed straight back in — previously the canonical form was write-only and every such
lookup failed. It is an explicit "no namespace" marker, not a wildcard. And the
cross-command join is not complete: schema-level lineage --json still emits bare
names, so a file-scope entity cannot yet be matched between graph --json and
lineage --json. coverage --json, where-used --json and field-lineage --json
were already canonical.
BREAKING: satsuma lint publishes its own exit-code table (sl-1u6r)
Any CI job keying off lint's exit codes needs to be re-read. lint previously
returned 2 both for error-severity findings and for failing to run at all — an
unreadable workspace, a bad path, an unusable satsuma.config.yaml — so a script
could not tell a failing lint gate from a broken checkout. It now has its own
table, following the fmt --check precedent:
| Code | Meaning |
|---|---|
0 |
No findings, or warnings only without strict mode |
1 |
Warnings present and strict mode active |
2 |
Error-severity findings present |
3 |
Lint could not run — unusable config, unknown rule id, unreadable workspace |
What changed in practice:
2no longer means "could not run".satsuma lint /nonexistent/pathexited
2and now exits3. A job that treated2as "the workspace has lint errors"
was previously also catching setup failures under that label; it now sees them
separately.2continues to mean error-severity findings, which is what it has
meant in practice all along.1is new, and opt-in. Warnings remain advisory by default and still exit
0. They exit1only under--strictorlint.strict: true, so no existing
job changes behaviour by upgrading.
--strict and --no-strict both exist and win over lint.strict in either
direction, so a workspace can commit a gate and one job can still opt out. A
suppressed rule produces no findings and therefore can never trigger a strict
failure.
A workspace can commit its lint settings to satsuma.config.yaml (sl-npi6)
New, optional, and inert if absent — a workspace with no config runs every rule
exactly as before. The file sits at the workspace root and currently carries one
section:
lint:
suppress: [unenumerated-record-target] # rule ids excluded from every run
strict: true # warnings exit 1 instead of 0
typeAliases:
- [STRING, TEXT, VARCHAR] # these three spellings mean the same typesuppress is the persistent form of --ignore, for a rule a workspace has decided
it does not want rather than one being silenced for a single run. strict promotes
warnings to a failing exit code, so a team can commit a gate instead of remembering
a flag; --strict and --no-strict both exist and win over it in either direction,
so one job can still opt out of a gate the workspace sets. typeAliases declares
type spellings as equivalent for type-mismatch-direct-arrow, which is what makes
that rule usable in a workspace whose layers spell one type three ways.
Alias groups stay separate rather than being flattened into one set — flattening
would make STRING equivalent to INT as soon as a workspace declared both a
string group and an integer group. An unrecognised key earns a warning rather than
silence, so a typo in a config is visible; an unusable config fails the run outright
with lint's new exit code 3 rather than being ignored.
It is not a dotfile on purpose: .satsuma is a first-class Satsuma source
extension, so a config named .satsumacfg would sit one character away from a
source glob and would get no editor YAML association or schema support.
The shape and its precedence rule live in @satsuma/core, reached through the
@satsuma/core/config subpath so the yaml dependency stays out of the browser
bundle — the LSP can mirror lint diagnostics without a second interpretation of the
same file.
Two structural lint rules (sl-j30s, sl-hysg)
satsuma lint reports two new warnings. Both are warning-severity and not
fixable, so no existing exit code changes unless you also opt into strict mode.
type-mismatch-direct-arrow— a baresrc -> tgtasserts the value passes
through unchanged, so connecting aSTRINGto aDATEis almost always a
mis-picked field, a schema edit that outran its mappings, or a missing transform.
Types are compared on their base token, case-insensitively:Stringmatches
STRINGandVARCHAR(255)matchesVARCHAR. Nothing else is presumed
equivalent — declare your own equivalences inlint.typeAliases, which this rule
is the first consumer of. An arrow carrying any transform body is never
type-checked,map { … }value maps included: a transform may legitimately change
type, and judging that is interpretation the CLI leaves to agents.lineage-cycle— the schema-level mapping graph (the edgessatsuma lineage
andgraph --compactdraw) contains a cycle. Self-mappings are excluded, since
that is how an increment is expressed. One finding is reported per
strongly-connected component rather than per elementary cycle, with a canonical
representative path, the mapping behind each hop, and the component members the
path does not visit — nothing is capped or truncated.
Detection for both lives in @satsuma/core, so the LSP can mirror them as editor
diagnostics without a second implementation.
lint.typeAliases and lint.strict are now enforced, so the warnings the CLI
printed to say they were inert are gone. lint --rules also derives its id column
width from the registry rather than a hardcoded number, so these two ids — the
longest yet — no longer push their descriptions out of alignment (sl-n4rb).
A namespaced mapping can target a global schema again (lgc-3f13)
Four commands were wrong about the same file. A mapping inside a namespace whose
target is a schema declared at file scope had that target silently rewritten as
namespace-local:
schema s1 { field_0 STRING }
namespace ns_a {
schema s0 { field_0 STRING }
mapping m0 { source { s0 } target { s1 } field_0 -> field_0 }
}
s1 is declared globally, and the mapping says so. But extraction pre-qualified bare
target refs with the enclosing namespace, inventing ns_a::s1 — a schema that exists
nowhere — and it did so before any workspace index was available, destroying the
information the resolver needs to try the current namespace and then fall back to
global. So validate reported a false undefined-ref warning against a
perfectly good file; graph --json emitted a schema_edges endpoint with no
corresponding node, and a field edge into ns_a::s1.field_0, a field on a
non-existent schema; and lineage --from ns_a::s0 reported data flowing into a
schema nowhere declared. The source side of the same mapping resolved correctly,
which is what hid it.
All four now name s1. A downstream trace that previously terminated in a phantom
schema continues into the real one, so lineage results grow where this shape occurs.
For @satsuma/core consumers: ExtractedMapping.targets now holds refs exactly as
authored rather than pre-qualified. Resolve them against the index — current
namespace first, then global — as the source side already required.
VS Code: warnings land on their real line, and the summary says what it found (sl-6osm)
Two live defects, both found by typing the CLI's JSON envelopes rather than trusting
them.
Every warning and question diagnostic was reported on line 0. Not off by one —
all of them, at the top of the file, however far down the note actually was. The
extension read a row field that the CLI had renamed to line, so the ?? 0
fallback fired every time. Satsuma: Show Warnings now puts each diagnostic on the
line it belongs to.
The summary panel was dropping most of what it had. It looked for a note field
on mappings, fragments, transforms and metrics that the CLI only ever emits for
schemas, so those sections rendered as bare lists of names; and it read files where
the CLI emits fileCount, so the Files: line never printed at all. Mappings now
show their sources, targets and arrow count; metrics their display name and grain;
fragments their field count; and the file count appears. Two headings that keyed off
arrays the CLI never populates are gone.
The visualisation stops drawing lineage that is not there (lgc-4bxl, lgc-fu7o)
Two edge-drawing defects on the overview, both found by the structural invariants
added this cycle rather than by a screenshot.
A computed arrow invented a source. An arrow with no source — -> stamp { "Set at load time." } — fell back to looking the target's own name up in the source
schema, so it drew a confident line from a same-named source field that the Satsuma
explicitly denies is involved. Where no such name existed it drew nothing and looked
identical to an unmapped field. Computed arrows now emit no field-to-field edge at
all, and the target's filled coverage port is what marks them as deliberate.
A multi-source arrow drew only one line. For s0.field_0, s1.field_0 -> field_0
the layout collapsed every arrow to its first source, so one of the two sources went
undrawn — and hovering the second source field highlighted the single line that
existed, pointing at the wrong schema card. satsuma graph --json had been emitting
both edges correctly all along. The layout now emits one edge per source and
resolves each independently, and highlighting matches the edge actually resolved.
The visualisation highlights the fields a hovered arrow really names (sl-rj78, sl-d7fz)
Both filed from screenshots, and both about hovering an arrow row in the mapping
detail view and watching the wrong thing light up.
An arrow nested inside a flatten or each block highlighted nothing. The hover
path resolved the raw authored, container-relative path (.adults) instead of the
absolute path every other resolution surface uses
(transects.sightings.adults), so it matched no field row.
A computed arrow highlighted its target but none of the source fields its prose
names. The hover only ever read the arrow's structural source list, which is empty
by design for a computed arrow — even though the @refs in its transform text were
already visibly coloured a few pixels away. Those refs are now resolved against every
source schema on screen and highlighted like any other source.
One example's coverage figure moves
examples/xml-to-parquet/pipeline.stm had a transform note referencing a field in
backticks where an @ref was meant. A backtick reference is prose; a resolved @ref
confers coverage (ADR-036), so making it one changes what the example reports:
source coverage for commerce_order goes from 22/24 91% (21 declared, 1 nl) to
23/24 95% (21 declared, 2 nl), and Order.Discounts.DiscountCode leaves the
"covered by no mapping" list. validate and graph output for the file are
unchanged. Mentioned because anyone diffing example figures across releases will see
it; no tooling behaviour changed.
@satsuma/core source contracts: FieldDecl variants, and field endpoints (cbdr-hqhh, sl-jyee)
Two TypeScript source-contract changes for consumers of @satsuma/core. No Satsuma
syntax, CLI JSON field, LSP payload, or VizModel field changes, and runtime output is
byte-identical. They are grouped here because both are compile-time only, and both
turn a class of wrong answer into a type error.
FieldDecl now rejects impossible field shapes. It is a union of scalar, record,
scalar-list and record-list variants: a scalar cannot carry children or
fragment-spread state, and a record list must carry its record body, even when that
body is empty. Code constructing scalar declarations should pass dynamic type strings
through createScalarTypeExpression. LSP- or visualisation-style rendered strings
such as list_of STRING can use fieldDeclFromRenderedType, which normalises the
spelling into core's existing { type: "STRING", isList: true } shape. Use
classifyFieldDecl with assertNever when handling every variant exhaustively.
See ADR-045.
qualifyField is gone, replaced by resolveFieldEndpoint. This one will break a
build rather than warn. It is not a rename: qualifyField ended in an unconditional
`${schemas[0]}.${field}` guess, silently picking the first schema when a bare
token was ambiguous. resolveFieldEndpoint returns a FieldEndpointResolution that
reports the ambiguity and makes the caller decide, which is what stopped several
lineage defects being expressible. createCanonicalFieldEndpoint, fieldEndpointOf,
fieldEndpointPath, fieldEndpointSchema and the CanonicalFieldEndpoint type are
newly exported alongside it. The one remaining undecided reading — a bare token that
also names a declared schema — now lives at a single named, tested site instead of
being spread across callers; r0-7w76 tracks the decision.
Field-lineage traversal also moved into core (buildFieldEdges, traceFieldLineage
and their types), so the CLI's field-lineage and graph emitters share one
implementation. Purely additive, and output is unchanged by construction — the
existing tests pass untouched.
The eleven packages are one npm workspace behind Turborepo (mbt-5l7g, ADR-049)
Nothing changes for anyone installing Satsuma; this is how the repository builds
itself. The released satsuma-cli and satsuma-lsp tarballs and the .vsix were
verified equivalent to v0.12.0's through the migration — the .vsix's file set is
byte-for-byte identical to one packaged before it, and the server extracted from it
completes a real initialize round-trip.
The one thing that will catch a contributor out:
npm --prefix tooling/<package> test no longer builds that package's
dependencies. Every prebuild/pretest hook that shelled out to
npm --prefix ../sibling run build is gone — that hand-maintained duplication is
what Turborepo replaced. A bare per-package npm test on a freshly installed or
partly built tree now fails against missing or stale output. Use
turbo run test --filter=<package>, which builds what it needs first and takes a
package's declared name (@satsuma/core, satsuma-cli) rather than its
directory; or run npm run build:all once. Three hooks survive, none of them
building a sibling.
Eleven lockfiles across ten packages became one root package-lock.json, and the
cross-package build order is now derived from the dependency graph in the
manifests rather than written down as a sequence anywhere — changing it means
editing a package's dependencies, nothing else.
With the task graph in place, CI could cache against it. End-to-end wall clock for
the whole pipeline, all 28 checks green:
| Run | Total |
|---|---|
Before (ec45cfba) |
4m35s |
| After, cold cache | 3m36s |
| After, warm cache | 1m56s |
That is 58% off the baseline warm. Quote the cold figure for a first run after a
dependency change — it is the honest number when nothing can be replayed. A
comment-only edit to one @satsuma/viz source file now replays the CLI's 1061
tests from cache in 571ms against 59.8s of real work. The ~119MB wasi-sdk
toolchain that four separate WASM-building jobs each re-downloaded is cached too
(sl-wz2h).
Agents working in the sandbox must export TURBO_CONFIG_DIR_PATH and
VERCEL_CONFIG_DIR_PATH before any turbo run and before git commit, since
the pre-commit hook invokes turbo. See AGENTS.md.
Test counts come from the test runners (sl-unr3)
Every per-package test count, the parser corpus count, and the CLI command count
now live in one generated file, test-stats.json, which the docs and the public
site read from instead of hardcoding. They had already drifted and begun
contradicting each other — cli.njk advertised 22 CLI commands while learn.njk
advertised 23, and AGENTS.md credited @satsuma/core with 679 tests when it had 689.
The numbers come from each tool's own summary output rather than from anyone's
memory: the test runner's tests N line, tree-sitter's Total parses: N, and the
built CLI's own --help. The pre-commit hook regenerates the file from logs the
checks already produced (no second test run) and stages it; CI fails if it drifts.
There is deliberately no timestamp field, so a diff only ever means a count really
changed. Do not hand-edit it, and do not hardcode a count in prose that could read
{{ stats.* }} instead.
TypeScript 6, and why not 7 (di-xup1, sl-bblz, ADR-051)
The repository is on TypeScript 6.0.3 across all nine packages that declare it.
Nothing user-visible changed; the work was in two tsconfigs. TypeScript 6 turns the
deprecated moduleResolution: "node" and baseUrl into errors rather than
warnings, so satsuma-viz-backend and satsuma-viz-harness moved to nodenext
and dropped their baseUrl/paths blocks — module resolution now goes through each
dependency's exports map like every other consumer in the workspace.
nodenext rather than the node16 a reader might expect, recorded as ADR-051
so it does not get "corrected" back: both packages are still "type": "commonjs"
while value-importing ESM-only siblings, and TypeScript's frozen node16 mode
forbids a CJS file from requiring an ESM-only package (TS1479/TS1541). nodenext
tracks Node's require(esm) support, stable since Node 22.12, which is what CI
runs.
TypeScript 7.0 went GA on 2026-07-08 with a Go-native compiler and much faster
builds, and the repository cannot adopt it yet: it ships no stable programmatic
API, which typescript-eslint's type-aware rules require. The upstream support
request was closed as not planned, and typescript-eslint's peer range still caps
below 6.1, so installing TypeScript 7 fails resolution and forcing past it crashes
ESLint. The stable API is targeted for 7.1. Dependabot is told to ignore the major
until then; sl-bblz tracks it.
The public site says what it can support (saf-dmvx)
Eleven fixes from an audit of the marketing site. Two pages were restructured
rather than corrected: vscode.njk traded a feature inventory of LSP internals for
workflow-framed sections, and now names the three webviews that actually exist
instead of miscounting the coverage gutter as one of them; cli.njk traded an
exhaustive command grid for the command families plus a link to the reference, and
its hero now describes token efficiency qualitatively rather than with an invented
benchmark.
On correctness: seven examples.njk snippets had been fabricated or had drifted
from the fixtures they claimed to show, and were re-derived from the real
examples/*.stm files; a Kimball star-schema (SCD Type 2) card was added from a
real example; a dead link, a stale example count, and a page that denied the
existence of a command it documented elsewhere were all fixed.
Three unsubstantiated numbers are still on the site, deliberately. "40-60%
smaller", "3-8x less token usage than mapping spreadsheets", and "LLMs generate
valid Satsuma >90%" are all still rendered, unchanged. The last is a future target
lifted from a success-metrics list and repeated as though it were a measured
result. Removing or replacing them needs a measurement, which is a proposed feature
and not yet built — so they remain, and remain unsupported. Feature 45's token
figures above do not substantiate them: they measure the agent reference's own
resident cost, not Satsuma against a spreadsheet.