ELIDE: default-value column elision for sparse API payloads

[ysnghr]

Problem

Real API payloads (Stripe / Shopify / GitHub / logs) are dominated by columns where 80%+ of rows share the same value — status: "active", currency: "usd", is_deleted: false, null in a hundred optional fields. SOON currently spends full tokens on every cell of every one of those columns.

Proposal

Add an ELIDE column mechanism to table headers. If a column's dominant value covers ≥ threshold of rows, hoist it into the header and emit per-row overrides:

SHAPE orders = {id,status,currency}
orders[100]<orders | status=active,currency=usd>:
(1,,)
(2,,cancelled)      # override on second column
(3,,)
...

Semantics:

• Header | f=v,f=v binds default values, matched by field name.
• An empty positional cell means "use the default".
• Any non-empty cell overrides the default for that row.
• null remains explicit (null), distinct from empty.

Column selection is deterministic and cost-driven: compute per-column
saving = repetitions * len(value) - override_syntax_overhead with the token-aware cost function (#1); emit iff net-positive.

Acceptance criteria

• SPEC §4/§6 amended for the | defaults … header and the empty-cell semantics.
• Conformance fixtures: one column elided, multi-column elided, all-same-value column, override with null.
• New L10 dataset in benchmarks/datasets.py: a sparse API-response shape (many optional nulls, dominant-value fields) — SOON+ELIDE ≥ +15pt over the current v0.1 encoding.
• Ablation column in the accuracy harness ([Retrieval-accuracy benchmark harness (adoption blocker) #8]) — ELIDE could hurt retrieval accuracy on rare rows; measure before shipping as default.

Risk / open questions

• Retrieval accuracy: models may miss elided defaults. Ship gated behind an encoder option initially; enable by default only after the accuracy harness passes.
• Interaction with ? optional fields: an optional field can also carry a default — need to define precedence (default applies only when the field is present; _ still means absent).