DS4 Context Engine 0.5.3

DS4 Context Engine 0.5.3

Changes

• Integrate PR #2 (squash commit d2519f3) and follow-up b8614d0.
• Add context.supplementalPlacement with unchanged default latest-user, opt-in stable-prefix for all supplements after leading system/developer messages, and opt-in hybrid for pin/memory/project at that prefix with retrieval immediately before the newest native user request.
• Map native pin indices, classifications, privacy reasons and supplemental provenance through the same actual planned positions across both insertion points. Ignore source-less supplements without shifting indices.
• Isolate synthetic supplements as independent atomic groups so they cannot absorb a following native assistant/tool prefix, misattribute its provenance or incorrectly consume its budget. Native tool call/result binding and validation remain enforced.
• Preserve native no-user behavior and fail-open fallback, without retaining synthetic supplements in fallback.
• Add regressions for leading roles, two-point mapping, selected/excluded provenance, changing retrieval, no-user inputs, budget fallback, native tool-batch retention and incomplete/orphan selected exchanges. Config tests round-trip all three modes, reject invalid values and cover the default-shape loader requirement. The golden default is latest-user.
• Document the trade-offs and limitations in the planner contract and ADR 075.

Configuration

Save context.supplementalPlacement through Pi's configuration interface or in ds4-context.json:

{
  "context": {
    "supplementalPlacement": "hybrid"
  }
}

stable-prefix remains available for installations with unchanged supplements, including the author's local-engine comparison. Existing configurations without this field retain latest-user. Fully restart Pi after upgrading to load the matching engine/core artifacts and saved settings.

Scope and limitations

Stable placement removes one source of prefix divergence; it does not guarantee cache hits or a byte-for-byte request prefix. Memory/project selection, changing blocks, sliding tails, compaction, request options and provider serialization may invalidate the following suffix. The PR author's Strata measurements were not reproduced during this review. No provider calls or source-history rewrites are introduced. Privacy, budgets, schema and default BPE/auto-tuning behavior remain unchanged.

The separate real long-conversation DS4/Pi BPE/auto-tuning objective is not established by these synthetic checks. No new provider budget was requested or used.

Validation

The pre-version-change source at b8614d0 passed the acquired local checks and the observed GitHub CI matrix (Node 22.19.0 and 24.x). Synthetic regressions reproduced supplement absorption before the group-isolation fix and passed afterward.

npm ci completed on supported Node 22.23.1 with the synchronized 0.5.3 manifests, constants and lockfile. The dev-inclusive installation audit reported six advisories (three moderate and three high); no dependency changes were made.

On final coordinated 0.5.3 source after the required verification-policy reload:

• Acquired node-typecheck, node-test and git-diff-check passed after installation.
• npm run quality:compare completed successfully.
• npm run schema:context-persistence passed: 1,266 bytes / 317 estimated tokens.
• npm run pack:check verified a clean consumer: core 275 files, reference adapter 7 files, Pi extension 139 files.
• All three npm pack --dry-run --json inventories contained no private-state paths.
• npm audit --omit=dev reported zero production vulnerabilities.
• Release-source CI passed on Node 22.19.0 and 24.x, including 112 files / 894 tests and package-consumer checks.

Local installation and package/registry checks used Node 22.23.1. Windows and live-engine cache savings were not verified locally. Publication-record edits are documentation-only; source results are reused because code, tests, dependencies and configuration remain unchanged.

Publication and registry verification

Published manually to npm on 2026-10-06 by alucard_24, in dependency order:

1. ds4-context-core@0.5.3
2. ds4-context-reference-adapter@0.5.3
3. ds4-context-engine@0.5.3

All three publication commands succeeded. Two initial fresh-cache registry checks returned ETARGET for core, and an intermediate dist-tag read still reported latest: 0.5.2 for the reference adapter. After waiting, a new fresh-cache npm run registry:check -- 0.5.3 verified all three exact public artifacts on Node 22.23.1. Subsequent npm dist-tag ls reads confirmed latest: 0.5.3 for every package. These observations do not guarantee immediate consistency at every client or cache.

The annotated v0.5.3 tag points to release-preparation commit 212dc43 and was pushed only after registry verification. GitHub Release v0.5.3 is public and stable. CI remains validation-only, with no publishing credentials or publishing steps.

DS4 0.5.2

DS4 Context Engine 0.5.2

Changes

• Fix rebase tool validation to use the exact compaction-aware Pi context serialized into the checkpoint. Reuse that captured context for validation, privacy classification and handoff construction.
• Preserve incomplete tool exchanges already excluded by Pi compaction in the unchanged canonical archive without transferring them into the handoff or blocking rebase.
• Keep active incomplete calls/results blocked before journal or target creation, including results whose matching call exists only in archived history.
• Add content-free toolExchangeDiagnostics with activeIssueCount and historicalIssueCount. No tool IDs, arguments, results or message bodies are returned by these diagnostics.
• Add six synthetic regressions covering archived calls/results, active calls/results, compaction-crossing result binding and repeated IDs. Archived cases verify successful activation, unchanged source bytes and absence of archived tool payloads from the checkpoint.
• Clarify ADR 074 and the history/rebase contract. Correct the guide's stale enabled-default description; runtime defaults are unchanged.

Scope and limitations

The shared atomic validator and archive/header/hash/ancestry/lineage/memory checks remain unchanged. There is no source rewrite, tool-result fabrication, validation bypass, schema migration, budget increase or extra provider request. Rebase remains unavailable if the transferred active context really contains incomplete tool exchanges. This patch does not repair source history damaged by another extension or change FTS rebuild costs.

Windows-host execution and live BPE/auto-tuning verification are not established by these synthetic checks.

Validation

Before the fix, the new regression file reproduced four failures, including archived exchanges incorrectly blocking rebase and an active orphan result incorrectly accepted when its call existed only in archived history. After the fix, acquired node-typecheck, node-test and git-diff-check passed on the source change.

On final coordinated 0.5.2 source after the required policy reload:

• npm ci completed on supported Node 22.23.1.
• jev_verify all passed node-typecheck, node-test and git-diff-check after installation.
• npm run pack:check passed in a clean consumer: core 275 files, reference adapter 7 files, Pi extension 137 files.
• npm run quality:compare completed successfully.
• npm run schema:context-persistence passed: 1,266 bytes / 317 estimated tokens.
• npm audit --omit=dev reported zero production vulnerabilities. The dev-inclusive installation audit reported five vulnerabilities: three moderate and two high.

Package/consumer checks used Node 22.23.1. Windows and the additional Node CI matrix were not executed locally. Publication-record edits are documentation-only; source results are reused only while code, tests, dependencies and configuration remain unchanged.

Publication and registry verification

Published manually to npm on 2026-10-05 by alucard_24, in dependency order:

1. ds4-context-core@0.5.2
2. ds4-context-reference-adapter@0.5.2
3. ds4-context-engine@0.5.2

All publication commands succeeded. Initial fresh-cache registry checks returned ETARGET for core while metadata was not yet consistent across the three packages. A subsequent fresh-cache npm run registry:check -- 0.5.2 verified all three exact artifacts on Node 22.23.1. npm dist-tag ls confirmed latest: 0.5.2 for every package.

The annotated v0.5.2 tag was created at release-preparation commit e1b3a43 and pushed after registry verification. GitHub Release v0.5.2 was created from that verified tag. CI remains validation-only with no publishing credentials.

DS4 0.5.1

DS4 0.5.1

Changes

• Repair rebuildable summary projections after an external session purge removes original source entries or child nodes. Discard only unavailable graphs and their transitive parents; retain valid nodes and other sessions.
• Skip canonical graph records whose original provenance is unavailable instead of degrading startup because cascading source-link deletion left incompatible metadata.
• Preserve the surviving canonical summary in the next compaction as a new warning-level unverified branch node. Do not inherit missing raw-source or child links; record previous-summary-provenance-unavailable alongside the existing unverified-summary marker.
• Preserve IDs declared in canonical compaction metadata even after their derived rows are discarded. Publish in-memory graph changes only after successful database persistence.
• Reject unindexed current-segment sources before generation. Recovery itself makes no additional model requests; new recovery diagnostics contain counters only.
• Add six synthetic regressions covering startup, a previously cleared derived graph, next-compaction recovery, restart, retained canonical text, cross-session isolation, transitive child loss, strict malformed-metadata rejection, and pre-generation failure for unindexed current sources.

Scope and limitations

• Canonical JSONL is never changed by this repair. Deleted raw history is not recovered or presented as verified evidence; the retained summary is explicitly unverified.
• The separate synchronous FTS deletion cost when rebuilding a large externally truncated index is not fixed here. Non-destructive session rebase remains preferable to purge.
• Schema, immutable-node, source-link, topology and generated-summary validation remain strict. Compaction budgets, transport retries, semantic retrieval, memory/pin contracts, native continuation and feature defaults are unchanged.
• Provider calls were not made for this patch; compaction generation in the regression tests uses local mocks. Windows behavior and the reporting host's current JSONL/database have not been verified directly.
• The independent real-context BPE/auto-tuning objective remains unverified and is not completed by these synthetic regressions.
• Contract: summary graph recovery.

Validation

On final coordinated 0.5.1 source after the acquired-policy reload:

• npm ci completed successfully on supported Node 22.23.1.
• jev_verify all passed node-typecheck, node-test and git-diff-check after installation.
• npm run quality:compare completed successfully.
• npm run schema:context-persistence passed: 1,266 bytes / 317 estimated tokens; the existing memory-tool schema remains unchanged.
• npm run pack:check passed on Node 22.23.1: core 275 files, reference adapter 7 files, Pi extension 136 files, exercised in a clean consumer.
• npm audit --omit=dev reported zero production vulnerabilities. The dev-inclusive installation audit reported five vulnerabilities: three moderate and two high.

The Windows and additional Node CI matrix were not executed locally. Subsequent publication-record edits are documentation-only; reuse the source/behavior results only while code, tests, dependencies and configuration remain unchanged.

Publication and registry verification

Published manually to npm on 2026-10-05 by alucard_24, in dependency order:

1. ds4-context-core@0.5.1
2. ds4-context-reference-adapter@0.5.1
3. ds4-context-engine@0.5.1

All publication commands succeeded. The initial fresh-cache registry check returned ETARGET for the newly published core; a delayed fresh-cache npm run registry:check -- 0.5.1 then verified all three exact artifacts on Node 22.23.1. Separate npm dist-tag ls checks confirmed latest: 0.5.1 for every package.

The annotated v0.5.1 tag was created at release-preparation commit de4a74a and pushed to origin after registry verification. GitHub Release v0.5.1 was created from that verified tag. CI remains validation-only with no publishing credentials.

0.5.0 — History Recall and recoverable Session Rebase

0.5.0 — History Recall and recoverable Session Rebase

Coordinated stable release of ds4-context-core, ds4-context-reference-adapter and ds4-context-engine. Both adapters depend exactly on ds4-context-core@0.5.0; Pi compatibility remains 0.84.3 and Node remains >=22.19.0.

Changes

• Explicit context_history_recall, context_history_read and context_history_status tools reuse DS4's canonical session index and exact/FTS retrieval. Branch/type/date/role filters apply before SQL LIMIT; summaries and trusted project scope require explicit permission.
• Bounded original-source reads verify canonical byte locations and SHA-256. Opaque references expire and are reauthorized. Historical content remains quoted evidence, with privacy checks at tool and provider egress; recall output and queries are not recursively re-indexed.
• Manual /context rebase --dry-run, /context rebase and /context rebase --recover <operationId> use deterministic bounded checkpoints, verified lineage, canonical target staging, an inter-process lock and recoverable journal. Recovery validates the staged target prefix before activation; the original session is never deleted or truncated.
• Curated memory/pins carry forward with original provenance, existing mutation/confirmation requirements and continuation lifecycle overlays. SQLite migrations 17–18 add rebuildable source-locator, checkpoint, lineage and operation projections.
• History Recall and Session Rebase are enabled by default, as explicitly requested by the user. Configure or disable them through Pi's /context config set historyTools.enabled false and /context config set sessionRebase.enabled false; saved settings apply at the next session start. Existing explicit overrides take precedence. Rebase never runs automatically or as a model-callable tool.

Scope and limitations

• No session-generation provider calls are needed for recall or rebase. Source data and crash tests use isolated synthetic sessions; existing user sessions/settings are not changed by release preparation.
• The default recall scope remains current-branch. Use explicit current-lineage scope to search verified ancestors after rebase; project-scope permission remains off by default. Rebase reduces the active session, not total archive/storage size.
• A handoff that exceeds the checkpoint budget fails closed rather than inventing a summary. Windows crash/filesystem behavior has not been executed on this Linux host.
• Semantic retrieval, chars-v1, BPE, auto-tuning, compaction validation and native continuation defaults are unchanged. The separate real-context BPE/auto-tuning objective remains unverified; synthetic tests do not complete it.
• Details: feature contract, implementation record, ADR 074.

Validation

On final 0.5.0 source after the acquired-policy reload:

• npm ci completed successfully on supported Node 22.23.1.
• jev_verify all passed node-typecheck, the complete node-test scope and git-diff-check after installation. This includes canonical recall/scope/privacy/budget tests, compatibility goldens, subprocess crash recovery at every rebase boundary, target-corruption rejection, inter-process lock recovery and the installed Pi 0.84.3 offline spike.
• npm run quality:compare completed successfully.
• npm run schema:context-persistence passed: 1,266 bytes / 317 estimated tokens; the existing memory-tool schema remains unchanged.
• npm run pack:check passed on Node 22.23.1: core 275 files, reference adapter 7 files, Pi extension 135 files, all exercised in a clean consumer.

The Windows and additional Node CI matrix were not executed locally. Subsequent publication-record edits are documentation-only; reuse the source/behavior results only while code, tests, dependencies and configuration remain unchanged.

Publication and registry verification

Published manually to npm on 2026-10-05 by alucard_24, in dependency order:

1. ds4-context-core@0.5.0
2. ds4-context-reference-adapter@0.5.0
3. ds4-context-engine@0.5.0

All publication commands succeeded. Initial fresh-cache registry checks returned ETARGET for the newly published core; a delayed fresh-cache npm run registry:check -- 0.5.0 then verified all three exact artifacts on Node 22.23.1. Separate npm dist-tag ls checks confirmed latest: 0.5.0 for every package.

The annotated v0.5.0 tag was created at release-preparation commit 7ee50a9 and pushed to origin after registry verification. GitHub Release v0.5.0 was created from that verified tag. CI remains validation-only with no publishing credentials.

DS4 Context Engine 0.4.13

Release 0.4.13

Coordinated patch release for ds4-context-core, ds4-context-reference-adapter, and ds4-context-engine.

Changes

• Historical retrieval filters Italian/English grammar, acknowledgements, workflow and generic finish requests from inferred keywords/simple symbols. Stopword lookup folds case, accents and compatibility characters without rewriting literal evidence. Explicit backticks, paths, qualified identifiers, flags and quoted phrases remain searchable, as do UI terms such as schermate, premere and input.
• Lexical-only retrieval rows require a verified identifier, phrase or FTS term match. FTS order, role, recency and unused budget can rank evidence but cannot independently qualify it. Semantic candidates retain their separate eligibility; the distinct-topic bonus stays bounded.
• Managed planning records complete native excluded-group/message totals and at most 32 content-free group details, oversized-first: planner-generated IDs/positions, estimator-unit cost, message count, kind, oversized/predecessor flags and allowlisted exclusion/rescue reasons.
• /context explain and the default /context excluded show compact native-group diagnostics separately from the historical inventory. /context excluded all preserves explicit item-level provenance and truthful persisted-rollup labelling. Legacy, observer and fallback manifests default to at most 40 item details when group diagnostics are unavailable.
• context.excluded_oversized_turn adds at most eight oversized-group details and a truncation flag. Native group totals/details survive persisted excluded-item rollup independently.
• ADR 073 records the decision and its limitations.

Scope and limitations

Recent-tail/input budgets, contiguous-tail selection, immediate-predecessor rescue, tool-call/result atomicity, privacy enforcement, provider transport and provider-facing defaults are unchanged. chars-v1 remains the default; BPE and auto-tuning remain opt-in. Genuinely oversized older turns can still be excluded even below the global message target and remain warnings, now attributable to a specific group. Retrieving a relevant excerpt does not restore the entire excluded turn.

Canonical Pi history is untouched; manifests remain rebuildable, non-authoritative projections. New diagnostic metadata contains no message bodies, canonical entry IDs, paths, tool arguments/results, provider bodies or credentials. Existing manifests without the optional diagnostic block remain usable.

Deterministic lexical retrieval is not a guarantee of semantic correctness for every conversation or language. This patch has synthetic local multi-turn coverage through real DS4/Pi hooks and SQLite retrieval, including 40 older generic reminders, topic recovery and topicless requests, plus atomic tool exchanges, rescue outcomes and a 68,000-item historical inventory/persisted rollup. It does not establish behavior on the reporting host, live provider calibration, headroom transitions or auto-tuning safety across provider turns. No additional provider call is made for this release.

Validation

Executed locally on the coordinated 0.4.13 code state with Node.js v26.5.1 and npm 11.17.0:

• npm ci completed, including both workspace builds.
• npm run check passed: workspace builds, root typecheck, 106 test files / 814 tests, including the frozen 0.2.0 compatibility golden and the new retrieval/group-diagnostic regressions.
• The acquired jev_verify all checks (node-typecheck, node-test, git-diff-check) each exited 0.
• npm run quality:compare passed; candidate quality score 0.9875.
• npm run schema:context-persistence passed: 1266 bytes / 317 estimated tokens, within both bounds.
• npm run pack:check passed in a clean consumer: core 251 files, reference adapter 7 files, Pi extension 124 files.
• npm audit --omit=dev --audit-level=moderate reported 0 production vulnerabilities. The dev-inclusive install audit reported 5 findings (3 moderate, 2 high); this patch does not claim to resolve them.

The code checks remain applicable after the release-record-only edits: source, tests, dependencies, configuration and environment are unchanged. These results are local/offline and are not reporting-host or live-provider validation.

Publication and registry verification

Published manually in this order, all with public access and the stable latest tag:

1. ds4-context-core@0.4.13
2. ds4-context-reference-adapter@0.4.13
3. ds4-context-engine@0.4.13

npm run registry:check -- 0.4.13 passed against the exact published versions in a fresh temporary consumer. The check validates exact adapter/core dependencies, public core exports, compiled reference-adapter conformance, the packaged quality corpus, the installed ds4-context-storage CLI shim, and isolated offline Pi RPC startup/shutdown without extension errors. Separate registry dist-tag reads confirmed latest: 0.4.13 for all three packages.

The first registry-consumer attempt returned ETARGET for ds4-context-core@0.4.13, while dist-tag reads still showed 0.4.12. After a short wait and a fresh npm cache, exact-version verification and all three dist-tag reads succeeded. No package was republished, rewritten or unpublished to resolve this visibility delay.

Annotated tag v0.4.13 was created and pushed on release-preparation commit 053e628; implementation commit d761167 and release preparation are on origin/main. CI remains validation-only and receives no publishing credentials.

v0.4.12 — Deterministic compaction-summary structure

Release 0.4.12 — Deterministic compaction-summary structure

Coordinated packages: ds4-context-core, ds4-context-reference-adapter, and ds4-context-engine 0.4.12.

Changes

• DS4 now normalizes the twelve required summary sections before validation, on every segment, aggregate and direct update: recognizable headings are canonicalized, duplicate bodies are concatenated in encounter order, and the contract order is restored. Missing or empty semantic sections receive an explicit reporting-omission marker (- Not reported in the generated summary; absence of facts is not established.) instead of failing the whole graph, so a repairable Markdown omission no longer discards an otherwise usable compaction.
• Unknown headings and unsectioned text are retained as quoted notes in Current State; they are never promoted into decisions, constraints or completion claims. A whole-output Markdown fence is unwrapped. Files Read and Files Modified are always regenerated from Pi's sanitized file-operation inventory, including when their headings are missing or duplicated, and Markdown-unsafe known-file evidence still fails closed without leaking the path.
• Exact-value quote retraction now runs whenever unsupported spans are present, independently of other validation errors, and only when the summary fell back to fail-closed. Both repairs are reported as warning codes with counter-only messages: summary-structure-normalized and summary-sections-not-reported. No generated heading, path or body text appears in the diagnostics, and no additional provider repair request is issued.
• Adds the new core export normalizeSummaryStructure, verified by the engine↔core compatibility guard, plus synthetic regressions for all twelve sections (missing, empty, duplicated, reordered), heading-level/case/emphasis normalization, every JavaScript line terminator and heading whitespace form, unknown headings and free-form output, whole-output fences, canonical file inventories, unsafe evidence, empty responses, tool-call responses, idempotence, all 4,096 omission subsets, and segment, aggregate and direct-update coordinator paths. Each combined-failure regression reproduces the reported missing-section, unsupported-exact-value error on the prior implementation.

Implementation commit: bcc0a02. Decision: ADR 072 (supersedes ADR 069's structural-fallback policy).

Scope and limitations

The problem was reproduced locally from code and tests; the reported live failure is not confirmed to have recurred after this patch, and no live session, database or session JSONL was inspected. Normalization preserves what the model reported: it cannot recover facts that were never reported, and the omission marker states that absence is not established. Unrecognized section titles remain notes rather than being guessed into a semantic category.

The strict validator and its error severities, the eight-bullet bound, the 25% removal limit, output-limit handling, transport retry policy, graph atomicity, JSONL authority and persistence schemas are unchanged. Empty responses, tool-call responses, unsafe file evidence, incomplete transport, output limits, aborts and storage failures still fall back to Pi. No provider repair request is added. The summary contract version, default tokenizer, opt-in BPE/auto-tuning, privacy enforcement and provider-facing defaults are unchanged. Core and the reference adapter receive only the coordinated version bump.

No live-provider, real-compaction or Windows-host confirmation is claimed, and the separate real-context BPE/auto-tuning objective remains open.

Validation

On the final 0.4.12 source, version and dependency state, using Node 26.5.1 / npm 11.17.0:

• npm ci completed successfully.
• npm run check completed successfully: workspace builds, strict TypeScript typecheck and the full Vitest suite, 104 files / 789 tests, including compatibility, privacy and integration tests. During development the new regressions were confirmed to fail on the unpatched implementation before passing with it. git diff --check was clean. The repository verification policy gate was unavailable in this session because the coordinated version bump invalidated its acquired fingerprint; the same commands were executed directly instead, and the gate can only be re-armed by the user's own reload.
• npm run quality:compare: candidate quality 0.9875 versus baseline 0.808156 on the packaged synthetic corpus, the same deterministic deltas as 0.4.11.
• npm run schema:context-persistence: passed: true, 1,266 bytes / 317 estimated tokens, within the 320-token relative gate.
• npm run pack:check: verified core 251 files, reference adapter 7 files and engine 122 files in a clean consumer, including packaged exports, reference conformance and isolated offline Pi RPC.
• npm audit --omit=dev: no production findings. npm ci reports 5 development-inclusive audit findings (3 moderate, 2 high); the lockfile diff changes only coordinated versions and exact internal dependencies, not third-party package versions. These warnings are not hidden or fixed by this patch.

No live-provider, live-session or Windows-host confirmation is claimed.

Publication and registry verification

• Published manually to npm in dependency order: ds4-context-core@0.4.12 → ds4-context-reference-adapter@0.4.12 → ds4-context-engine@0.4.12, all public with latest. Registry dist-tags.latest reached 0.4.12 for all three packages after propagation.
• npm run registry:check -- 0.4.12 first reported ETARGET for the core dependency and then E404 for the engine until propagation completed, as in previous releases; with a fresh temporary npm cache it then passed: “Verified all DS4 registry packages at exact version 0.4.12.” This covers exact internal dependencies, core/public exports, reference conformance, the packaged quality corpus, the storage CLI shim and isolated offline Pi RPC.
• Annotated tag v0.4.12 points to release-preparation commit ebc9fd0957e1bbcfae9a9302b16bd4821dd1d202 and was pushed after successful registry verification.
• GitHub Release: https://github.com/Alucard24/ds4-context-engine/releases/tag/v0.4.12.
• CI remains validation-only; no publishing credentials, provider calls or session data were added.

0.4.11 — Topic-aware retrieval ranking and per-occurrence tool binding

Release 0.4.11 — Topic-aware retrieval ranking and per-occurrence tool binding

Coordinated packages: ds4-context-core, ds4-context-reference-adapter, and ds4-context-engine 0.4.11.

Changes

• Stops treating generic workflow vocabulary as a historical query. devi, procedi, allora, aggiungi, continua, considerare, ovvero, quelli, fix, please and similar words no longer become keywords, symbols or query terms. Evidence that is explicitly present in the request — backticked spans, qualified identifiers, file paths, flags, error codes and quoted phrases — stays searchable even when its text is otherwise a stopword, so a legitimate topic named Continue or devi is still found. A bare acknowledgement now produces no-query instead of retrieving earlier approvals.
• Adds a bounded distinct-topic bonus of 40 per additional matched query term, capped at 80. FTS order, source role, recency and token length combined can no longer outweigh a candidate that matches one more topic term, while literal identifier and literal phrase tiers stay strictly ahead of numeric score.
• Counts matches with word boundaries and Latin accent folding consistent with the canonical unicode61 remove_diacritics 2 index. A keyword prefix inside an unrelated larger word no longer earns an extra match, and accented evidence matches its unaccented query term. Quoted evidence text is unchanged.
• Binds each tool result to the latest preceding occurrence of its tool-call ID. When a completion reuses an ID, independent turns are no longer merged into one oversized atomic group, an incomplete call can no longer be validated by a later reuse, and orphan results never bind to a future call. containsToolExchange is derived from completed call indices rather than rescanning every relation per group.
• Adds synthetic unit and integration regressions for acknowledgement queries, Italian topic extraction, stopword-named identifiers, workflow-only versus two-topic ranking, duplicate rows with a longer lower-ranked candidate, prefix and accent match accounting, reused IDs across independent turns, incomplete and orphan exchanges, a real exchange spanning a user boundary, a 200-exchange reused-ID sequence, a long native context with a 64k tail and an oversized immediate predecessor with input headroom. Each fails on the prior implementation and passes with the patch.

Implementation commit: c0298ab. Decision: ADR 071.

Scope and limitations

Both defects were reproduced locally from the code and its tests. Neither is confirmed to have occurred in a reported live session, and no live session, database or session JSONL was inspected for this patch.

No retrieval budget, result limit, token ceiling, recent-tail policy, recurrence ceiling, rescueImmediatePredecessor behavior, deduplication, branch isolation, evidence rendering, privacy handling or provider-facing default changed. context.recentTailTokens keeps its model-dependent automatic ceiling; the 64k ceiling applies only from a 1,050,000-token context window, not at 262,144. A genuine older oversized turn can still be excluded even when the whole native input alone would fit the global budget.

The ranking bonus is lexical, not semantic: it changes which historical evidence reaches the provider for a given request and cannot guarantee relevance. No live provider calls, real compactions or BPE/auto-tuning verification were performed, and the separate real-context objective remains open. Full real-context BPE/auto-tuning verification remains a separate objective.

Compaction validation, exact-value downgrading, compaction transport retries, graph atomicity, JSONL authority, persistence schemas and opt-in BPE/auto-tuning defaults are unchanged. Core and the reference adapter receive only the coordinated version bump.

Validation

On the final 0.4.11 source, version and dependency state, using Node 26.5.1 / npm 11.17.0:

• npm ci completed successfully.
• The approved jev_verify all gate passed npm run typecheck, npm test and git diff --check: workspace builds, strict TypeScript and the full Vitest suite, 103 files / 717 tests, including compatibility, privacy and integration tests. During development the new regressions were confirmed to fail on the unpatched implementation before passing with it. These are the build/typecheck/test scopes of npm run check, executed through the acquired verification policy rather than claiming a separate execution of that command.
• npm run quality:compare: candidate quality 0.9875 versus baseline 0.808156 on the packaged synthetic corpus.
• npm run schema:context-persistence: passed: true, 1,266 bytes / 317 estimated tokens, within the 320-token relative gate.
• npm run pack:check: verified core 251 files, reference adapter 7 files and engine 120 files in a clean consumer, including packaged exports, reference conformance and isolated offline Pi RPC.
• npm audit --omit=dev --json: no production findings. npm ci reports development-inclusive audit findings; the lockfile diff changes only coordinated versions and exact internal dependencies, not third-party package versions. These warnings are not hidden or fixed by this patch.

No live-provider, live-session or Windows-host confirmation is claimed.

Publication and registry verification

• Published manually to npm in dependency order: ds4-context-core@0.4.11 → ds4-context-reference-adapter@0.4.11 → ds4-context-engine@0.4.11, all public with latest.
• Annotated tag v0.4.11 and the GitHub Release were created after successful registry verification.
• CI remains validation-only; no publishing credentials, provider calls or session data were added.

DS4 Context Engine 0.4.10

Release 0.4.10 — Bounded incomplete-stream retry and safe compaction failure metadata

Coordinated packages: ds4-context-core, ds4-context-reference-adapter, and ds4-context-engine 0.4.10.

Changes

• Recognizes Pi's exact Stream ended without finish_reason error in both rejected requests and error responses as transport / stream-incomplete. Uses the existing attempt cap, exponential backoff, fresh routing sessions, cooperative cancellation, usage accounting and pre-dispatch operation-input budget; defaults and limits are unchanged.
• Adds adapter-local typed provider-failure diagnostics: stage, allowlisted category/reason, attempt count/limit and optional validated HTTP status. No raw body, headers, arbitrary provider code, finish-reason string or exception cause is exposed. HTTP client/server errors, content filters, unknown finish reasons and unclassified failures are not indiscriminately retried; HTTP error bodies cannot turn into transport retries merely by containing transport keywords.
• /context compaction distinguishes planned from successfully completed segments. The existing segmentCount field remains the planned count; the additive completedSegmentCount can show partial completion before fallback, without claiming graph persistence. Each new custom attempt resets counters and failure metadata.
• Pi fallback preserves the original custom-attempt trigger, including proactive requests, along with safe provider-failure metadata. Metadata-only fallback warnings and command output expose the bounded diagnostic.
• Adds synthetic unit/integration coverage for failure classification, replay bounds, cancellation, usage, fresh routing sessions, operation-budget enforcement, metadata privacy, partial counters/reset, trigger preservation and TUI output after native fallback. The missing-terminator regressions fail on the prior implementation and pass with the patch.

Implementation commit: 7cc8a23. Decision: ADR 070.

Scope and limitations

The reported host's 0.4.9 preparation took 2.48 seconds and generation took 141.15 seconds, with an unclassified provider error before aggregation. That report does not establish a missing terminator as its cause. This release corrects a reproducible classification/retry gap and makes remaining failures attributable; it does not promise successful compaction or a specific real-provider latency. No live provider calls, real compactions, session/database inspection or Windows configuration changes were made for this patch. Full real-context BPE/auto-tuning verification remains a separate objective.

Compaction validation, exact-value downgrading, graph atomicity, concurrency limits, input/output budgets, JSONL authority, persistence schemas and opt-in BPE/auto-tuning defaults remain unchanged. Core and the reference adapter receive only the coordinated version bump.

Validation

On the final 0.4.10 source, version and dependency state, using Node 26.5.1 / npm 11.17.0:

• npm ci completed successfully.
• The approved jev_verify all gate passed npm run typecheck, npm test and git diff --check: workspace builds, strict TypeScript and the full Vitest suite, including compatibility/privacy/integration tests. These are the build/typecheck/test scopes of npm run check, executed through the acquired verification policy rather than claiming a separate execution of that command.
• npm run quality:compare: candidate quality 0.9875 versus baseline 0.808156 on the packaged synthetic corpus.
• npm run schema:context-persistence: passed: true, 1,266 bytes / 317 estimated tokens, within the 320-token relative gate.
• npm run pack:check: verified core 251 files, reference adapter 7 files and engine 118 files in a clean consumer, including packaged exports, reference conformance and isolated offline Pi RPC.
• npm audit --omit=dev --json: no production findings. npm ci reports five development-inclusive audit findings (three moderate, two high); the lockfile diff changes only coordinated versions/exact internal dependencies, not third-party package versions. These warnings are not hidden or fixed by this patch.

No live-provider or Windows-host confirmation is claimed.

Publication and registry verification

• Published manually to npm in dependency order: ds4-context-core@0.4.10 → ds4-context-reference-adapter@0.4.10 → ds4-context-engine@0.4.10, all public with latest. All three dist-tags.latest values were verified as 0.4.10.
• The first exact-version consumer check returned ETARGET for core; subsequent engine metadata lookups returned E404 until the registry exposed 0.4.10. After propagation, npm run registry:check -- 0.4.10 passed with a fresh temporary npm cache: “Verified all DS4 registry packages at exact version 0.4.10.” This includes exact internal dependencies, core/public exports, reference conformance, the packaged quality corpus, the storage CLI shim and isolated offline Pi RPC.
• npm 11.17.0 emitted a misleading bin-normalization warning during engine publication. Exact registry metadata retains ds4-context-storage at scripts/ds4-context-storage.mjs, and the installed CLI usage probe passed; no CLI was removed from the release.
• Annotated tag v0.4.10 points to release-preparation commit 1397362f767728c829165e27a62a8acc1985984b and was pushed after successful registry verification.
• GitHub Release: https://github.com/Alucard24/ds4-context-engine/releases/tag/v0.4.10.
• CI remains validation-only; no publishing credentials, provider calls or session data were added.

v0.4.9 — Compaction preparation performance and attribution

Release 0.4.9 — Compaction preparation performance and attribution

Coordinated packages: ds4-context-core, ds4-context-reference-adapter, and ds4-context-engine 0.4.9.

Changes

• The Pi compaction adapter indexes each canonical message fingerprint once and consumes matching entry IDs in source order, instead of searching and re-serializing earlier candidates for every message. Exact canonical matching and fail-closed behavior on missing messages remain unchanged.
• Compaction diagnostics report non-overlapping local preparation, generation, aggregation and persistence wall times. Preparation is split into index sync, source mapping, prompt planning and segment planning. Index sync additionally reports its nested canonical JSONL and semantic sync durations (not to be added twice). The latest diagnostics appear in /context compaction and metadata-only compaction.timings logs, including when in-process summary generation fails.
• A summary halted by the provider's output limit reports whether the failed stage was segment, aggregate or update. This is a diagnostic change, not an automatic increase of output budgets or a guarantee that Pi's fallback succeeds.
• Adds synthetic adapter, runtime and coordinator tests and tests/benchmarks/compaction-preparation.bench.ts; the benchmark uses a local fake provider and logs only counts and timings when explicitly requested.

Scope and limitations

The real report that motivated this work showed roughly 1,131 seconds of local preparation within a 1,268-second operation, followed by an output-limit failure. The prior report did not divide preparation among indexing, source mapping and segmentation. A synthetic 1,283-message case measured about 208 ms preparation (about 183 ms segment planning and 23 ms source mapping) with a no-op index sync. This one-sample result neither reproduces nor explains the real delay. The 0.4.9 diagnostics are needed on the affected host before attributing that delay or claiming it resolved. No live provider run was made for these changes. BPE/auto-tuning defaults and compaction validation policy remain unchanged.

Implementation commit: 53de88c.

Validation

• npm ci rebuilt the three synchronized workspaces. It warned of four moderate vulnerabilities in installed development dependencies; npm audit --omit=dev --audit-level=moderate found zero production dependency vulnerabilities.
• npm run check: build, typecheck and all 100 test files / 655 tests passed on 0.4.9.
• npm run quality:compare: candidate quality score 0.9875; no regression against the comparison baseline.
• npm run schema:context-persistence: passed: true.
• npm run pack:check: clean consumer verified core (251 files), reference adapter (7 files) and engine (115 files).
• DS4_BENCH_DIAGNOSTICS=1 npx vitest bench tests/benchmarks/compaction-preparation.bench.ts: synthetic local fake provider; one measured iteration, not a statistically robust latency claim.
• npm run registry:check -- 0.4.9, with a fresh npm cache: verified exact published packages, dependencies, CLI shim, compiled exports/conformance and offline packaged Pi startup after npm registry propagation.

Publication

Published manually from eb28572 in dependency order: ds4-context-core@0.4.9, ds4-context-reference-adapter@0.4.9, then ds4-context-engine@0.4.9, all public with latest. An immediate registry check failed with ETARGET for core, and the next fresh-cache check failed with ETARGET for the engine; after propagation, a third fresh-cache check succeeded. At verification time, npm dist-tag ls showed latest: 0.4.9 for all three packages.

npm publish printed a manifest auto-correction warning about the storage CLI entry in the engine; the exact published-artifact check subsequently confirmed that the installed manifest still declares the CLI and that the installed shim runs. This warning did not require republishing or bypassing the registry check.

Annotated tag v0.4.9 targets eb28572; GitHub Release: https://github.com/Alucard24/ds4-context-engine/releases/tag/v0.4.9. No DS4 compaction provider calls or real-session data were used to validate this release's performance.

DS4 Context Engine 0.4.8

Release 0.4.8 — Retract the quoting of every unsupported exact value

Coordinated packages: ds4-context-core, ds4-context-reference-adapter, and ds4-context-engine 0.4.8.
Implementation commit: f0a457a.

Summary

Behaviour patch that ends the recurrent compaction.custom_fallback with
unsupported-exact-value. Validation is unchanged: a backticked value the
evidence does not hold literally, or as its canonical JSON-escaped rendering, is
still invalid. What changes is the repair: it retracts the two backticks of
every unsupported span in place, keeps the text, and the repaired summary is
re-validated. No bullet is deleted for an exact-value failure, so the
eight-bullet and 25% removal bounds no longer apply to that path, and the repair
no longer consults the span classification, which stays as class-only
diagnostics on the fallback warning. Fallback remains for structural, transport
and budget failures. SQLite schema 16, canonical JSONL history and every
provider-facing default are unchanged.

Changes

• packages/core/src/compaction/summary-contract.ts: the repair API is now
  analyzeUnsupportedExactValueDowngrade and downgradeUnsupportedExactValues,
  with ExactValueDowngradeResult / ExactValueDowngradeAttempt replacing the
  ExactValuePrune* types and statuses.
  • downgradeUnsupportedExactSpans retracts the backticks of every span
    returned by the existing unsupportedExactMatches and leaves the text.
  • affectedBulletCount reports the bullets that contained a rejected span.
  • RENDERING_EQUIVALENT_RELATIONS and renderingEquivalentSpans are removed:
    the repair no longer needs the classifier.
  • The classifier keeps its bounded analysis and the class-only report, now
    documented as observation-only again.
• src/pi-adapter/summary-generator.ts: the downgrade outcome is recorded as
  unsupported-exact-spans-unquoted; unsupported-exact-bullets-pruned no
  longer exists, and a summary still invalid after the retraction is reported as
  post-downgrade-invalid.
• tests/unit/summary-contract.test.ts and
  tests/unit/summary-generator.test.ts cover the downgrade of supported,
  composed, one-character-deviant, over-length and absent spans, the
  disappearing bounds, and the class-only report on a structural failure.
  tests/unit/compaction-optimizations.test.ts and
  tests/integration/compaction.test.ts cover the completed compaction with
  validationStatus: "warning" and the privacy-safe counters.
• docs/COMPACTION.md, docs/ADR/069-retract-quoting-of-every-unsupported-exact-value.md
  and the ADR index record the decision; README.md links the release.

Why

The affected-bullet series from production was 30/20 → 23/19 → 19/15 → 17/11 →
16/12, every run ending in compaction.custom_fallback with
unsupported-exact-value; repair=too-many-bullets. The class-only report that
closed the attribution carried 25 spans across 15 bullets: spansClassifiedCheap
was 0, so no span was a rendering variant of contiguous evidence; 11 spans were
compositions whose parts sit at unrelated positions, 5 spent their share of the
analysis budget, 4 exceeded the length limit, 3 were adjacent compositions and 2
had no near-miss at all.

On that input the 0.4.7 repair could retract the quoting of at most the three
adjacent spans. The remaining 22 spans kept the bullet-removal path, and
15 − 3 = 12 bullets still exceeded the eight-bullet bound, so 0.4.7 would have
fallen back by construction on the same input. The classes are not a finite list
of odd renderings to eliminate one release at a time: they are the summarizer's
normal composing and paraphrasing behaviour, and an association the evidence
does not contain (composed-two-present-parts) must never be accepted.

The repair now applies the only deterministic answer that keeps both the fact and
the verification: the fact stays as prose, the exactness claim goes, and every
value that remains backticked is still verified against the evidence.
Instructing the model differently (0.4.1, 0.4.5) was already shown not to
converge.

Privacy and safety

• Logs, warnings and diagnostics keep carrying counters only: the number of
  retracted spans, never span text, fragments or evidence hashes.
• The class-only report is observation-only and runs only after a validation
  failure; it never changes validation or the repair.
• Validation strictness is preserved for every surviving backticked value. The
  deliberate trade is that a value the evidence does not carry — including an
  invented one — is no longer deleted: it stays as prose and in the recorded
  counter, instead of costing the fact or the whole DS4 compaction. The fallback
  this replaces (Pi's default compaction) does not check exact values at all.
• No provider call, network access or configuration change is involved in the
  repair.

Compatibility and persistence

• No migrations, no schema change (still 16), no new configuration key, and no
  change to any provider-facing default: chars-v1 remains the default
  estimator, modelAwareness.autoTune and DS4 native continuation stay
  disabled, storage.scope keeps its project default.
• The core repair API is renamed as listed above; the engine/core compatibility
  guard turns a mixed artifact pair into one actionable line instead of a
  missing function.
• Fallback still exists for missing, duplicate or empty sections, unsupported
  file paths, unknown sections or headings, malformed content, transport and
  budget failures, and post-downgrade-invalid.

Validation

On Node 26.5.1 the release passed npm run check (100 Vitest files, 650 tests,
TypeScript builds and root typecheck), npm run pack:check in a clean consumer
for all three packages at 0.4.8 (251 core files, 7 reference-adapter files, 112
extension files), deterministic npm run quality:compare (candidate 0.9875
against the frozen baseline 0.808156), and
npm run schema:context-persistence (passed: true). No provider calls are
involved in this release procedure. jev_verify is not available in this
environment (project verification disabled), so the checks above were run
directly.