Skip to content

feat(runtime-host): own session catalog and configuration - #1651

Merged
Astro-Han merged 7 commits into
apache:mainfrom
M4n5ter:feat/runtime-host-session-catalog-config
Jul 31, 2026
Merged

feat(runtime-host): own session catalog and configuration#1651
Astro-Han merged 7 commits into
apache:mainfrom
M4n5ter:feat/runtime-host-session-catalog-config

Conversation

@M4n5ter

@M4n5ter M4n5ter commented Jul 30, 2026

Copy link
Copy Markdown
Member
English

Summary

Make Runtime Host the canonical authority for Session catalog reads, stable creation, metadata changes, configuration replacement, and message-based read markers, while keeping execution-sensitive configuration transitions under Runtime authority.

Multiple Clients now observe one revisioned Session projection and serialize competing mutations through the Host instead of owning independent Session state.

Catalog and creation

  • Add closed typed operations for Session catalog query, creation, metadata update, configuration update, and read-marker changes.
  • Provide bounded revision-pinned keyset pages, exact Session lookup, typed filters, and explicit truncation for legacy labels that cannot fit the wire contract.
  • Bind continuation cursors to their canonical filter so omitted or changed filters cannot mix result sets.
  • Store catalog activity, message preview, epoch, and generation in SQLite; publish each transcript projection atomically and repair interrupted projections after restart.
  • Use matching ordered indexes for unfiltered, archived, flagged, label, and linked-child catalog pages.
  • Create Sessions with a stable Client-supplied identity and durable request fingerprint.
  • Preserve create identity across response loss, Host restart, partial transcript publication, and Session removal.
  • Recover an exact marker-only transcript without overwriting non-empty or unrelated state.

Metadata and configuration

  • Use metadata revisions as compare-and-set tokens for concurrent Client mutations.
  • Update name, labels, and flagged state through a closed metadata patch.
  • Replace model, thinking, permission, collaboration, and orchestration configuration as one complete value.
  • Commit configuration and its sandbox execution boundary in one SQLite transaction.
  • Keep no-op configuration replacements revision-stable and avoid unnecessary backend invalidation.
  • Route configuration transitions through SessionManager so active and pending Runs, Plan state, Deep Research labels, shell lineage, and backend snapshots share one authority boundary.
  • Fence existing and future execution claims and linked-child creation across configuration validation, resource disposal, and the storage commit.
  • Set read markers through visible user or assistant message identity instead of timestamps or tool-tail records.
  • Retry ordinary read-marker revision races within a fixed bound and return a typed conflict if they do not converge.

Coordination and lifecycle

  • Serialize operations for one Session while allowing unrelated Sessions to proceed independently.
  • Reject archived Sessions, pending Interactions, and affected linked Sessions with active Runs before configuration side effects.
  • Refresh only the affected Session lineage while preserving unrelated active work.
  • Publish the committed metadata revision through Session continuity.
  • Return typed revision conflicts and fail closed with Host drain when a mutation outcome or post-commit invalidation is uncertain.

Boundaries

This slice does not add archive, unarchive, remove, working-directory moves, branching, or revision operations. It does not wire Desktop or TUI production adapters and does not change production Host activation.

Configuration changes that require shell-lineage cleanup or Plan-state validation fail closed until the serving Host composition provides those authorities. The pre-release Runtime Host wire protocol remains v0.

Validation

  • Biome lint passed across 2,362 files.
  • Root build and typecheck passed.
  • Storage: 819 passed, 1 skipped.
  • Runtime: 2,827 passed, 9 skipped.
  • Runtime Host: 371 passed.
  • Real two-Client UDS coverage for stable creation, concurrent configuration CAS, revision-pinned pagination, continuity publication, read markers, and legacy label projection.
  • git diff --check passed.

Part of #1167 and #853.

简体中文

概要

让 Runtime Host 成为 Session catalog 读取、稳定创建、metadata 变更、configuration 完整替换和基于 message 的 read marker 的权威所有者;涉及执行资源的 configuration transition 仍由 Runtime authority 负责。

多个 Client 现在会观察同一份带 revision 的 Session projection,并通过 Host 串行化竞争 mutation,不再各自持有独立的 Session 状态。

Catalog 与创建

  • 为 Session catalog query、创建、metadata update、configuration update 和 read-marker mutation 增加封闭的 typed operation。
  • 提供有界且绑定 revision 的 keyset catalog 分页、精确 Session 查询、typed filter,以及无法满足 wire contract 的旧 labels 的显式截断状态。
  • 将 continuation cursor 绑定到规范化 filter,避免省略或更换 filter 时混入其它结果集。
  • 在 SQLite 中保存 catalog activity、message preview、epoch 和 generation;原子发布每次 transcript projection,并在重启后修复中断的 projection。
  • 为无过滤、archived、flagged、label 和 linked-child catalog page 使用匹配排序的索引。
  • 使用稳定的 Client-supplied identity 和持久化 request fingerprint 创建 Session。
  • 在响应丢失、Host 重启、transcript 部分发布和 Session 删除后继续保留 create identity。
  • 可以恢复精确的 marker-only transcript,同时不会覆盖非空或无关状态。

Metadata 与 Configuration

  • 使用 metadata revision 作为并发 Client mutation 的 compare-and-set token。
  • 通过封闭 metadata patch 更新名称、labels 和 flagged 状态。
  • 将 model、thinking、permission、collaboration 和 orchestration configuration 作为一个完整值替换。
  • 在同一个 SQLite transaction 中提交 configuration 及其 sandbox execution boundary。
  • no-op configuration replacement 不增加 revision,也不触发不必要的 backend invalidation。
  • 通过 SessionManager 执行 configuration transition,使 active/pending Run、Plan 状态、Deep Research label、shell lineage 和 backend snapshot 共用同一 authority boundary。
  • 在 configuration 校验、resource disposal 和 storage commit 期间 fence 已存在和后续的 execution claim,以及 linked-child 创建。
  • 使用可见的 user 或 assistant message identity 设置 read marker,不依赖 timestamp 或 tool-tail record。
  • 在固定次数内重试普通 read-marker revision race;无法收敛时返回 typed conflict。

协调与生命周期

  • 同一个 Session 的 operation 串行执行,不相关 Session 仍可并行。
  • 在产生 configuration side effect 前拒绝 archived Session、pending Interaction,以及存在 active Run 的受影响 linked Session。
  • 只刷新受影响的 Session lineage,不影响无关的 active work。
  • 通过 Session continuity 发布已提交的 metadata revision。
  • 返回 typed revision conflict;mutation outcome 或提交后的 invalidation 不确定时 fail closed 并触发 Host drain。

边界

本 slice 不增加 archive、unarchive、remove、working-directory move、branch 或 revision operation;也不接入 Desktop/TUI production adapter,不改变生产 Host activation。

需要清理 shell lineage 或校验 Plan 状态的 configuration change 会 fail closed,直到 serving Host composition 提供对应 authority。预发布 Runtime Host wire protocol 继续保持 v0。

验证

  • 2,362 个文件通过 Biome lint。
  • 根级 build 和 typecheck 通过。
  • Storage:819 项通过,1 项跳过。
  • Runtime:2,827 项通过,9 项跳过。
  • Runtime Host:371 项通过。
  • 真实双 Client UDS 覆盖稳定创建、并发 configuration CAS、绑定 revision 的分页、continuity publication、read marker 和旧 label projection。
  • git diff --check 通过。

属于 #1167#853 的一部分。

@Astro-Han

Copy link
Copy Markdown
Contributor

I went through the storage and Runtime Host paths. The ownership split is coherent: Host exposes closed operations, while execution-sensitive configuration stays in Runtime.

I found one P1 and three P2 findings.

  • [P1] Make the transcript durable before publishing the SQLite commit

    packages/storage/src/session-store.ts publishes durable SQLite state after non-durable transcript writes. The stable-create marker uses writeFile, and transcript append does not enable the existing durable appendJsonl path. Because SQLite uses synchronous = FULL, a power loss can preserve the metadata commit while losing the marker or JSONL tail.

    Recovery can then see an existing Session with a missing transcript, or a catalog projection for messages that no longer exist. The transcript file and directory entry need to be synced before the SQLite publish commit.

  • [P2] Keep execution-semantic labels outside catalog metadata replacement

    Catalog metadata can replace labels such as mode:deep_research and mode:expert-team:*. These labels currently affect root admission, while their owning capabilities are explicitly deferred.

    A metadata update can therefore make a Session unexecutable, or remove a mode label without running that mode's transition and cleanup. Could we reserve these labels and let catalog mutations replace user-owned labels only?

  • [P2] Reject active execution claims instead of waiting for them

    runSessionQuiescentMutation waits for existing execution claims before transitionSessionConfiguration checks hasActiveRuns. A long Turn can leave the request pending until the Turn finishes, so the later session_busy check is effectively unreachable for that claim.

    This also differs from the PR contract, which says configuration changes reject active Runs before side effects. I think this needs an admission fence that blocks new claims while immediately rejecting if an existing claim is active.

  • [P2] Add direct coverage for commit-unknown and Host-drain classification

    The SQLite tests cover CAS, projection repair, pagination, and stable-create identity well. I could not find direct coordinator tests for the commit_outcome_unknown and Host-drain branches.

    A small failure-injection suite around HostSessionCatalogCoordinator should distinguish pre-commit failure, post-commit uncertainty, typed conflicts, and the cases that must or must not request drain.

I do not think the PR needs to be split. The slice is cohesive, but I would fix these durability and authority gaps before it lands.

@M4n5ter

M4n5ter commented Jul 31, 2026

Copy link
Copy Markdown
Member Author
English

@Astro-Han Thanks for the detailed review. I addressed all four findings in 9c491e11d:

  • synced the transcript file and directory chain before publishing SQLite state, including stable-create retries;
  • preserved execution-semantic labels during catalog metadata replacement;
  • made configuration mutation reject existing execution claims immediately while retaining the admission fence for idle commits;
  • added direct failure-injection coverage for pre-commit failure, commit uncertainty, typed conflicts, and Host-drain classification.

Validated with the full Storage (820 passed), Runtime (2,827 passed), and Runtime Host (376 passed) suites, plus the root build and typecheck.

简体中文

感谢详细审查。9c491e11d 已处理全部四项问题:

  • SQLite 发布前同步 transcript 文件及目录链,并覆盖 stable-create 重试路径;
  • catalog metadata 替换时保留执行语义 labels;
  • 配置变更遇到已有 execution claim 时立即拒绝,空闲提交期间继续持有 admission fence;
  • 补充 pre-commit failure、commit uncertainty、typed conflict 与 Host drain 分类的直接故障注入测试。

完整 Storage(820 项)、Runtime(2,827 项)、Runtime Host(376 项)测试,以及根目录 build 和 typecheck 均已通过。

@M4n5ter
M4n5ter force-pushed the feat/runtime-host-session-catalog-config branch from 9c491e1 to cc6b9ee Compare July 31, 2026 04:05

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. The original four findings are fixed, and the PR still reads as one cohesive change.

A few follow-ups remain:

  • [P2] Recover an existing empty or truncated stable-create transcript. If the process exits after the create claim becomes durable but while session.jsonl is being written, ensureMarkerOnlyTranscript() finds the file but rejects it instead of restoring the marker. An exact retry can then fail permanently for that Session ID.

  • [P2] Repair a pending catalog projection before serving more reads. If the JSONL append succeeds and commitCatalogProjectionWrite() fails, the same Host can continue returning stale catalog data because projection recovery only runs during store initialization.

  • [P2] Reject reserved execution labels in session.create. Labels such as mode:deep_research and mode:expert-team:* are accepted at creation, but root admission rejects those modes and metadata replacement preserves the labels. This can create a Session that cannot run and cannot be repaired through the catalog API.

  • [P2] Apply the existing Plan-mode restriction to session.configuration.update. Creation rejects collaborationMode: "plan", but configuration updates still accept it, leaving the Session unusable until another update changes it back.

  • [P2] Canonicalize cwd with realpath() before fingerprinting and persistence. resolve() plus stat() leaves the stored execution identity tied to a symlink path whose target can later change without a Session revision.

  • [P3] Add focused coverage for the 48 KiB pagination boundary, forbidden configuration states, read-marker retry exhaustion, and response loss followed by Host restart. Splitting the long UDS scenario would also make failures easier to isolate.

None of these requires another architectural layer or a PR split. They fit the seams already introduced here.

@M4n5ter
M4n5ter force-pushed the feat/runtime-host-session-catalog-config branch from 7fa3ac8 to 4d58677 Compare July 31, 2026 06:37
@Astro-Han

Copy link
Copy Markdown
Contributor

The latest fixes close the earlier P2 findings. I have two non-blocking follow-ups:

  • [P2] Keep exact stable-create retries independent of later filesystem changes. prepareCreate() currently calls realpath() before probeStableSessionCreate() and includes the resolved target in the request fingerprint. If the first response is lost and the symlink is then removed or retargeted, an identical retry returns invalid_request or operation_conflict instead of the existing Session. The request fingerprint could use the normalized request path, while realpath() is applied only when an absent claim requires a new Session to be persisted.

  • [P3] Add direct coverage for read-marker revision retries reaching their limit and returning the typed conflict.

These do not block this PR under the current scope and severity threshold.

@Astro-Han
Astro-Han merged commit 059c15f into apache:main Jul 31, 2026
3 checks passed
@M4n5ter
M4n5ter deleted the feat/runtime-host-session-catalog-config branch July 31, 2026 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants