Skip to content

fix(core): bound WAL-backed rollout pagination memory - #230

Merged
beinan merged 1 commit into
lance-format:mainfrom
beinan:fix/bound-lsm-pagination
Jul 31, 2026
Merged

fix(core): bound WAL-backed rollout pagination memory#230
beinan merged 1 commit into
lance-format:mainfrom
beinan:fix/bound-lsm-pagination

Conversation

@beinan

@beinan beinan commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • page source=all and source=wal through an id-only LSM scan so wide claim-check columns never enter the full LSM sort
  • locate selected ids in each physical base/WAL dataset using only id and _rowid, then fetch exactly the page rows with take_rows
  • preserve global filter, offset, ordering, has_more, schema-evolution alignment, and inline-blob projection behavior
  • add regression coverage for mixed base/WAL pagination, deep-page completion, filters, wide text values, and WAL-only reads

Testing

  • ./.codex/skills/ci-pr-helper/scripts/run_ci_checks.sh
  • 216 core/workspace Rust tests passed, 3 ignored; integration, master, and server tests passed
  • 213 Python tests passed, 6 skipped, 3 xfailed
  • cargo fmt, cargo clippy, ruff format/check, and pyright passed
  • all GitHub Actions checks passed

@beinan
beinan force-pushed the fix/bound-lsm-pagination branch from bbcd514 to e25a238 Compare July 31, 2026 20:09
@beinan
beinan merged commit 1e65d1d into lance-format:main Jul 31, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant