Skip to content

docs: 添加 CLAUDE.md 协作规范 + CI 集成测试#34

Merged
NeverENG merged 7 commits into
mainfrom
feature/snapshot
May 20, 2026
Merged

docs: 添加 CLAUDE.md 协作规范 + CI 集成测试#34
NeverENG merged 7 commits into
mainfrom
feature/snapshot

Conversation

@NeverENG
Copy link
Copy Markdown
Owner

@NeverENG NeverENG commented May 20, 2026

Summary

  • 新增 CLAUDE.md:约定方案先行、外科手术式修改、原子性 commit、自动 PR/merge 规则
  • CI 添加 integration job:编译 Server → 启动 → benchmark 验证

规则要点

  1. 方案先行:先提多方案让用户选
  2. 精准修改:不连带改动无关代码
  3. 原子 commit:改完立刻提交
  4. 自动 PR/merge:commit 后自动 PR,CI 通过自动 rebase merge

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added session lifecycle notifications (session established/terminated messages).
    • Introduced block-indexed storage format for improved query performance.
  • Bug Fixes

    • Improved error handling and validation across components.
  • Improvements

    • Restructured logging system with standardized message prefixes.
    • Updated server name to "BanDB."
    • Enhanced CI pipeline to test feature branches.
    • Optimized persistence strategy for reduced write overhead.
  • Documentation

    • Added comprehensive storage optimization design document.

Review Change Stack

NeverENG and others added 7 commits May 12, 2026 21:46
约定三条核心规则:方案先行、外科手术式修改、原子性 commit。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
新增 integration job:编译并启动 server,等待就绪后运行 benchmark,
全部通过即为 CI green。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
commit 后自动创建 PR 并开启 auto-merge,CI 通过后自动 rebase merge 到 main。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@NeverENG NeverENG merged commit 4aada05 into main May 20, 2026
0 of 5 checks passed
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 20, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 21fa120c-b8e7-447e-b45c-1c4770736336

📥 Commits

Reviewing files that changed from the base of the PR and between 5c6d0c8 and 775d39e.

⛔ Files ignored due to path filters (9)
  • Raft/raft_data/raft_log.dat is excluded by !**/*.dat
  • Raft/raft_test_data_install_snap/raft_log.dat is excluded by !**/*.dat
  • Raft/raft_test_data_log/raft_log.dat is excluded by !**/*.dat
  • Raft/raft_test_data_snap_create/raft_log.dat is excluded by !**/*.dat
  • Raft/raft_test_data_snap_persist/raft_log.dat is excluded by !**/*.dat
  • Raft/raft_test_data_term/raft_log.dat is excluded by !**/*.dat
  • Server/raft_data/raft_log.dat is excluded by !**/*.dat
  • Server/raft_data/raft_state.dat is excluded by !**/*.dat
  • Server/raft_data/snapshots/155040_2.snap is excluded by !**/*.snap
📒 Files selected for processing (20)
  • .claude/settings.local.json
  • .github/workflows/ci.yml
  • CLAUDE.md
  • Raft/raft.go
  • Raft/raft_test.go
  • Raft/raft_wal.go
  • Raft/rpc.go
  • Server/server.go
  • Server/server_pprof.go
  • config/config.json
  • log/sstable_1777264896266900900.sst
  • network/banNet/DataPack.go
  • network/banNet/connection.go
  • network/banNet/msgHandle.go
  • network/banNet/server.go
  • service/fsm.go
  • service/router.go
  • storage/zstorage/SSTable.go
  • storage/zstorage/memtable.go
  • 迭代方案/storage-bottleneck-optimization.md

📝 Walkthrough

Walkthrough

This PR consolidates incremental Raft persistence refactoring, storage engine block-indexed SSTable format migration with MemTable WAL removal, connection lifecycle callback integration, and work-stealing message dispatch, alongside infrastructure updates and comprehensive optimization planning documentation.

Changes

Raft Persistence & Logging Refactor

Layer / File(s) Summary
Raft incremental persistence and log append refactor
Raft/raft.go, Raft/rpc.go, Raft/raft_wal.go
Introduces persistStateLocked() to persist only Term and votedFor; updates RequestVote, AppendEntries, and InstallSnapshot RPC paths to use incremental persistence instead of full persistLocked(); modifies AppendEntry to call wal.AppendLog() for individual entries; updates WAL SaveState to open/seek instead of truncate; adds RebuildLogFile() method.
Election fast path and snapshot handling with slog
Raft/raft.go
Adds single-node election fast path (directly become leader when peer count is zero); migrates election startup, leader transition, snapshot auto-triggering, and snapshot replay to structured slog logging; maintains non-blocking ApplyCh behavior with logged skip on channel full.
Raft test infrastructure with per-test data directories
Raft/raft_test.go
Updates all persistence and snapshot tests to use NewRaftWithDataDir with per-test dedicated directories, replacing prior shared "raft_data" cleanup; each test creates, removes before run, defers cleanup, and reloads via NewRaftWithDataDir to verify persisted state.

Storage Engine: Block-Indexed SSTable & MemTable WAL Removal

Layer / File(s) Summary
SSTable block-indexed format and index cache
storage/zstorage/SSTable.go
Introduces BlockIndexEntry type and SSTableBlockSize constant; adds indexCache and idxMu to SSTable struct; updates LoadSSTableMetaList to use slog, sort files, and asynchronously preheat block indices; replaces WriteToSSTable with block-based writer that builds per-block indices and writes footer.
SSTable read with block index and merge operations
storage/zstorage/SSTable.go
Updates ReadAllFromSSTable to detect footer and stop at data end offset; implements new ReadFromSSTable using cached block index search with fallback to full scan; updates MergeSSTable to write merged output in block-indexed format and cache the merged index.
MemTable removes WAL persistence and updates logging
storage/zstorage/memtable.go
Removes WAL write from Put and WAL tombstone write from Delete; updates WAL recovery, Flush, FlushToSSTable, and Compaction to use slog; removes debug prints from Get; maintains lookup order and persistence semantics.

Network & Service Integration: Connection Lifecycle & Message Dispatch

Layer / File(s) Summary
Connection buffering and lifecycle logging
network/banNet/connection.go, network/banNet/DataPack.go
Changes msgChan from unbuffered to buffered (capacity 10); removes msgChan write case from StartWriter fallback; updates logging strings to "commenced/closed/established/terminated" format with connection IDs; adds [ERROR] and [WARN] prefixes to DataPack logs.
Message handle work-stealing dispatch and logging
network/banNet/msgHandle.go
Implements SendMsgToTaskQueue with worker ID derived from connection ID, priority enqueue to dedicated queue, bounded work-stealing, and blocking fallback; updates duplicate route and unregistered MsgID logging to [WARN]/[ERROR] format; updates worker startup and shutdown messaging.
Server lifecycle callbacks and connection management
network/banNet/server.go, service/router.go, Server/server.go, Server/server_pprof.go
Adds OnConnStart and OnConnStop methods to Router that send session notifications via SendBuffMsg; updates Server start/stop logging messages; changes callback nil checks to [WARN] format; wires router callbacks to server via SetConnStartFunc/SetConnStopFunc.
Router message buffering and response paths
service/router.go
Updates handlePut, handleGet, handleDelete to use SendBuffMsg instead of SendMsg for all responses; removes success-path log from handlePut.
FSM logging and Get error semantics
service/fsm.go
Migrates KVServer.Run and KVServer.Apply to slog; changes Get to return "key not found" only when value is nil and err is nil, otherwise returning value/error without conversion; removes prior stdout snapshot messages.

Infrastructure, Configuration & Documentation

Layer / File(s) Summary
CI workflow with integration test job
.github/workflows/ci.yml
Extends on.push.branches to include feature/**; adds integration job that builds server/benchmark, starts server with GOMEMLIMIT=512MiB, polls port 8080 readiness, runs benchmark, and verifies final connectivity.
Configuration and Claude local settings
config/config.json, .claude/settings.local.json
Updates server name to "BanDB"; adds Claude settings with permissions.allow for Go commands and Python -c pattern.
Comprehensive storage optimization design document
迭代方案/storage-bottleneck-optimization.md
Outlines storage bottlenecks and multi-phase optimization roadmap: P0 priorities (WAL batching, state persist merging, SSTable block indexing), P1-P3 enhancements (MANIFEST, connection pooling, snapshot caching), and architectural improvements (Leader lease reads, Multi-Raft partitioning, pipelined replication); includes Phase 1-7 timeline, file change summary, and risk assessment.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

  • NeverENG/BanDB#24: Refactors MemTable Put/Delete/Get behavior with active/dirty double-buffer, directly related to this PR's removal of WAL persistence and error semantics changes in the same methods.
  • NeverENG/BanDB#33: Modifies Raft snapshot flow and InstallSnapshot handling, overlapping with this PR's snapshot replay and incremental persistence changes.

Poem

🐰 Hops through storage blocks with glee,
Raft logs persist incrementally,
Work-stealing queues and snapshots rebuild,
LSM teeth sharp, architecture skilled.
From chaos comes the roadmap clear—
Optimizations year by year!

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/snapshot

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant