perf(storage): SSTable 迭代器改缓冲读，归并快 17 倍

[NeverENG]

迭代器逐字段直接从 *os.File 读取（键长、键、值长、值），每条记录约四次 read syscall。它同时服务 compaction 的每个归并源与范围扫描 —— 是全量读的热点，10 万条记录即约 40 万次系统调用。

加一层 64KiB 缓冲，让内核往返次数与记录数解耦。

实测（本机 macOS，10 万条 × 200B value）

	改前	改后	倍数
顺序迭代	261 ms（38 万条/秒）	11 ms（923 万条/秒）	24×
8 路归并（compaction）	1.391 s（7 万条/秒）	80 ms（124 万条/秒）	17×
投递 Fetch 201 条（游标居中）	3.5 ms	1.57 ms	2.2×
投递 Fetch 201 条（游标靠后）	0.9 ms	0.12 ms	7.6×

归并那一项用 git stash 只回退 iterator.go 做前后对照，排除其它改动干扰。

两个必须一起改的细节

不能再用 Seek(0, SeekCurrent) 判断是否越过数据区终点 —— 那返回的是底层文件偏移，已被预读推到缓冲末尾，会让迭代提前结束（表现为条目莫名减少）。改为自行累计已消费字节数。

带 seek 的构造在定位后必须 Reset 缓冲 —— 否则缓冲里残留的是 seek 之前预读的字节，会从错误位置开始解析。

过程记录

首次替换时搜索串写错（原文是 it.f，我写成 it.r），替换静默未生效，测出 261ms→270ms「无提升」。没有就此下结论说「syscall 不是瓶颈」，而是先去核对代码是否真的改了 —— 一看 Next() 原样未动。修正后才是真实结果。

验证

新增 iterator_bench_test.go 固化这两项测量，便于后续改动对照。go build ./...（含 -tags pprof）、go vet、gofmt、go test -race ./... 连跑 2 次全绿 —— 这条路径被 compaction、范围扫描、新旧判定（recency）、故障注入等既有用例大量覆盖。

🤖 Generated with Claude Code

Summary by CodeRabbit

• Performance

  • Improved SSTable iteration efficiency through buffered reading and explicit position tracking.
  • Improved reliability when seeking within indexed data blocks.

• Tests

  • Added throughput coverage for sequential SSTable scans.
  • Added performance testing for merging multiple SSTable files.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 46da94b0-5ae3-405e-bcdd-d6b8dd684bb1

📥 Commits

Reviewing files that changed from the base of the PR and between 048ee8e and 6c4e362.

📒 Files selected for processing (2)

• storage/iterator.go
• storage/iterator_bench_test.go

---

📝 Walkthrough

Walkthrough

The SSTable iterator now uses a 64 KiB buffered reader and explicit byte-position tracking. New throughput tests measure sequential scans and multi-file compaction merges.

Changes

SSTable iteration performance

Layer / File(s)	Summary
Buffered iterator parsing storage/iterator.go	The iterator uses buffered reads, resets the reader after indexed seeks, tracks consumed bytes, and parses complete record fields with io.ReadFull.
Iterator and merge throughput tests storage/iterator_bench_test.go	Tests measure three full scans of 100,000 entries and a merge of eight SSTables containing 12,500 entries each. Tests validate counts and errors. התח

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SSTableIterator
  participant BufferedReader
  participant SSTableFile
  SSTableIterator->>SSTableFile: Seek to block offset
  SSTableIterator->>BufferedReader: Reset at seek offset
  SSTableIterator->>BufferedReader: Read record fields
  BufferedReader->>SSTableFile: Read buffered bytes
  SSTableIterator->>SSTableIterator: Update consumed position

Loading

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/block-based-iterator

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}