Skip to content

perf(sqlitestore): rows 表升级为完整持久层,消除每次写的全量 payload 重放 - #2182

Merged
DeliciousBuding merged 1 commit into
masterfrom
chore/cleanup-round-37
Sep 1, 2026
Merged

perf(sqlitestore): rows 表升级为完整持久层,消除每次写的全量 payload 重放#2182
DeliciousBuding merged 1 commit into
masterfrom
chore/cleanup-round-37

Conversation

@DeliciousBuding

Copy link
Copy Markdown
Collaborator

背景

#2154 后端性能最后一项:edge SQLiteStore.syncPersist 每次写(39 处写路径同步触发 + 启动校验 + Close/Flush)都把整个内存 Store 全量 JSON 编码并 UPSERT 到 agenthub_store_snapshots 大字段——O(store 规模) 写放大 + WAL 膨胀。而 load() 本就优先读 agenthub_store_rowsplanSQLiteLoadSource),payload 仅在 rows 为空时回退;既有两套 durability 测试(sqlite_durable_hardening / sqlite_row_store)也是删掉 snapshots 表后靠 rows 恢复。即 payload 每写一次的全量重放是纯死重。

但不能直接跳过 payloadfileSnapshot.Settings/SettingsMtime/Checkpoints 三个字段此前只存在于 payload、不在 rows delta。本 PR 先补齐再做跳过。

变更

  1. rows 升级为完整持久层:新增 settingscheckpoint 两个 row kind。
    • applySQLiteRow 支持两类解码(settings 单行、checkpoint 按 run id)。
    • deltaSQLiteRows 追加 checkpoint 增量(无序 map 用 sortedKeys 确定性化)+ settings 单行(编码变化才写,写入后永不删除,兼作迁移标记)。
  2. rowsSeeded 门控跳过 payload
    • load() 命中 rows 时置位。
    • syncPersist 在「快照有内容」的提交后置位(空库不翻转,避免空 store 反复写也无陈旧回退可丢);置位后整段 payload 编码 + UPSERT 被跳过
  3. 存量升级路径:rows 已存在但早于新 kind 的库,load() 从遗留 payload 一次性收养 Settings/SettingsMtime/Checkpoints;首次持久化即写成 rows,settings 行成为「已迁移」标记,之后不再收养。

验证

  • go test ./internal/store/ 全绿:既有 sqlite_durable_hardening_test / sqlite_crash_recovery_test / sqlite_row_store_test(含删 snapshots 靠 rows 恢复用例)零改动通过TestSQLiteSettingsSurviveReopen 在跳过 payload 后仍绿(settings 已走 rows)。
  • 新增 3 个回归(sqlite_store_payload_skip_test.go):
    • TestSQLitePersistSkipsLegacyPayloadOnceRowsSeeded:rows 播种后的写不再刷新 payload(断言 payload 无新线程、且旧状态仍在),重开仍从 rows 完整恢复。
    • TestSQLiteCheckpointSurvivesReopen:checkpoint 走 rows,删 payload 后重开仍恢复。
    • TestSQLiteLegacyRowsAdoptsSettingsAndCheckpointsFromPayload:存量库一次性收养 + 写成 rows(settings 标记行=1),删 payload 二次重开仍恢复。
  • go vet ./...make test(edge+hub)全绿;5 项 verifier + git diff --check 全绿。

未验证 / 备注

  • 纯 edge 进程内存储改动,无 hub/API/契约变化;合并后需重建并重启 dev-edge 生效。
  • 空 store 永不翻转(其持久化本就廉价,且不留陈旧回退);仅 settings 无实体内容的极端库会保留 payload 写(罕见且载荷极小,已在代码注释说明)。
  • 至此 [整理] 静态审计 backlog 跟踪(探索 lane 剩余项) #2154 探索 lane 后端性能项全部消化完毕。

@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: e7265f52-dbb8-48a8-8099-c0efb5af626a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@DeliciousBuding
DeliciousBuding merged commit 3021c6a into master Sep 1, 2026
39 checks passed
@DeliciousBuding
DeliciousBuding deleted the chore/cleanup-round-37 branch September 1, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant