perf(sqlitestore): rows 表升级为完整持久层,消除每次写的全量 payload 重放 - #2182
Merged
Conversation
Co-authored-by: Cursor <cursor@vectorcontrol.tech>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Team Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
#2154 后端性能最后一项:edge
SQLiteStore.syncPersist每次写(39 处写路径同步触发 + 启动校验 + Close/Flush)都把整个内存 Store 全量 JSON 编码并 UPSERT 到agenthub_store_snapshots大字段——O(store 规模) 写放大 + WAL 膨胀。而load()本就优先读agenthub_store_rows(planSQLiteLoadSource),payload 仅在 rows 为空时回退;既有两套 durability 测试(sqlite_durable_hardening/sqlite_row_store)也是删掉 snapshots 表后靠 rows 恢复。即 payload 每写一次的全量重放是纯死重。但不能直接跳过 payload:
fileSnapshot.Settings/SettingsMtime/Checkpoints三个字段此前只存在于 payload、不在 rows delta。本 PR 先补齐再做跳过。变更
settings、checkpoint两个 row kind。applySQLiteRow支持两类解码(settings 单行、checkpoint 按 run id)。deltaSQLiteRows追加 checkpoint 增量(无序 map 用sortedKeys确定性化)+ settings 单行(编码变化才写,写入后永不删除,兼作迁移标记)。rowsSeeded门控跳过 payload:load()命中 rows 时置位。syncPersist在「快照有内容」的提交后置位(空库不翻转,避免空 store 反复写也无陈旧回退可丢);置位后整段 payload 编码 + UPSERT 被跳过。load()从遗留 payload 一次性收养 Settings/SettingsMtime/Checkpoints;首次持久化即写成 rows,settings 行成为「已迁移」标记,之后不再收养。验证
go test ./internal/store/全绿:既有sqlite_durable_hardening_test/sqlite_crash_recovery_test/sqlite_row_store_test(含删 snapshots 靠 rows 恢复用例)零改动通过;TestSQLiteSettingsSurviveReopen在跳过 payload 后仍绿(settings 已走 rows)。sqlite_store_payload_skip_test.go):TestSQLitePersistSkipsLegacyPayloadOnceRowsSeeded:rows 播种后的写不再刷新 payload(断言 payload 无新线程、且旧状态仍在),重开仍从 rows 完整恢复。TestSQLiteCheckpointSurvivesReopen:checkpoint 走 rows,删 payload 后重开仍恢复。TestSQLiteLegacyRowsAdoptsSettingsAndCheckpointsFromPayload:存量库一次性收养 + 写成 rows(settings 标记行=1),删 payload 二次重开仍恢复。go vet ./...、make test(edge+hub)全绿;5 项 verifier +git diff --check全绿。未验证 / 备注