Skip to content

DS4 Context Engine 0.4.8

Choose a tag to compare

@Alucard24 Alucard24 released this 28 Sep 04:20
· 31 commits to main since this release

Release 0.4.8 — Retract the quoting of every unsupported exact value

Coordinated packages: ds4-context-core, ds4-context-reference-adapter, and ds4-context-engine 0.4.8.
Implementation commit: f0a457a.

Summary

Behaviour patch that ends the recurrent compaction.custom_fallback with
unsupported-exact-value. Validation is unchanged: a backticked value the
evidence does not hold literally, or as its canonical JSON-escaped rendering, is
still invalid. What changes is the repair: it retracts the two backticks of
every unsupported span in place, keeps the text, and the repaired summary is
re-validated. No bullet is deleted for an exact-value failure, so the
eight-bullet and 25% removal bounds no longer apply to that path, and the repair
no longer consults the span classification, which stays as class-only
diagnostics on the fallback warning. Fallback remains for structural, transport
and budget failures. SQLite schema 16, canonical JSONL history and every
provider-facing default are unchanged.

Changes

  • packages/core/src/compaction/summary-contract.ts: the repair API is now
    analyzeUnsupportedExactValueDowngrade and downgradeUnsupportedExactValues,
    with ExactValueDowngradeResult / ExactValueDowngradeAttempt replacing the
    ExactValuePrune* types and statuses.
    • downgradeUnsupportedExactSpans retracts the backticks of every span
      returned by the existing unsupportedExactMatches and leaves the text.
    • affectedBulletCount reports the bullets that contained a rejected span.
    • RENDERING_EQUIVALENT_RELATIONS and renderingEquivalentSpans are removed:
      the repair no longer needs the classifier.
    • The classifier keeps its bounded analysis and the class-only report, now
      documented as observation-only again.
  • src/pi-adapter/summary-generator.ts: the downgrade outcome is recorded as
    unsupported-exact-spans-unquoted; unsupported-exact-bullets-pruned no
    longer exists, and a summary still invalid after the retraction is reported as
    post-downgrade-invalid.
  • tests/unit/summary-contract.test.ts and
    tests/unit/summary-generator.test.ts cover the downgrade of supported,
    composed, one-character-deviant, over-length and absent spans, the
    disappearing bounds, and the class-only report on a structural failure.
    tests/unit/compaction-optimizations.test.ts and
    tests/integration/compaction.test.ts cover the completed compaction with
    validationStatus: "warning" and the privacy-safe counters.
  • docs/COMPACTION.md, docs/ADR/069-retract-quoting-of-every-unsupported-exact-value.md
    and the ADR index record the decision; README.md links the release.

Why

The affected-bullet series from production was 30/20 → 23/19 → 19/15 → 17/11 →
16/12, every run ending in compaction.custom_fallback with
unsupported-exact-value; repair=too-many-bullets. The class-only report that
closed the attribution carried 25 spans across 15 bullets: spansClassifiedCheap
was 0, so no span was a rendering variant of contiguous evidence; 11 spans were
compositions whose parts sit at unrelated positions, 5 spent their share of the
analysis budget, 4 exceeded the length limit, 3 were adjacent compositions and 2
had no near-miss at all.

On that input the 0.4.7 repair could retract the quoting of at most the three
adjacent spans. The remaining 22 spans kept the bullet-removal path, and
15 − 3 = 12 bullets still exceeded the eight-bullet bound, so 0.4.7 would have
fallen back by construction on the same input. The classes are not a finite list
of odd renderings to eliminate one release at a time: they are the summarizer's
normal composing and paraphrasing behaviour, and an association the evidence
does not contain (composed-two-present-parts) must never be accepted.

The repair now applies the only deterministic answer that keeps both the fact and
the verification: the fact stays as prose, the exactness claim goes, and every
value that remains backticked is still verified against the evidence.
Instructing the model differently (0.4.1, 0.4.5) was already shown not to
converge.

Privacy and safety

  • Logs, warnings and diagnostics keep carrying counters only: the number of
    retracted spans, never span text, fragments or evidence hashes.
  • The class-only report is observation-only and runs only after a validation
    failure; it never changes validation or the repair.
  • Validation strictness is preserved for every surviving backticked value. The
    deliberate trade is that a value the evidence does not carry — including an
    invented one — is no longer deleted: it stays as prose and in the recorded
    counter, instead of costing the fact or the whole DS4 compaction. The fallback
    this replaces (Pi's default compaction) does not check exact values at all.
  • No provider call, network access or configuration change is involved in the
    repair.

Compatibility and persistence

  • No migrations, no schema change (still 16), no new configuration key, and no
    change to any provider-facing default: chars-v1 remains the default
    estimator, modelAwareness.autoTune and DS4 native continuation stay
    disabled, storage.scope keeps its project default.
  • The core repair API is renamed as listed above; the engine/core compatibility
    guard turns a mixed artifact pair into one actionable line instead of a
    missing function.
  • Fallback still exists for missing, duplicate or empty sections, unsupported
    file paths, unknown sections or headings, malformed content, transport and
    budget failures, and post-downgrade-invalid.

Validation

On Node 26.5.1 the release passed npm run check (100 Vitest files, 650 tests,
TypeScript builds and root typecheck), npm run pack:check in a clean consumer
for all three packages at 0.4.8 (251 core files, 7 reference-adapter files, 112
extension files), deterministic npm run quality:compare (candidate 0.9875
against the frozen baseline 0.808156), and
npm run schema:context-persistence (passed: true). No provider calls are
involved in this release procedure. jev_verify is not available in this
environment (project verification disabled), so the checks above were run
directly.