Skip to content

feat(c): expose batch incremental reads - #748

Closed
JingsongLi wants to merge 1 commit into
apache:mainfrom
JingsongLi:codex/expose-c-incremental-read
Closed

feat(c): expose batch incremental reads#748
JingsongLi wants to merge 1 commit into
apache:mainfrom
JingsongLi:codex/expose-c-incremental-read

Conversation

@JingsongLi

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: N/A

Expose the existing fixed-range incremental scan and Arrow read capabilities through the C ABI. C consumers can now plan (start_exclusive, end_inclusive] snapshot ranges instead of being limited to full batch scans.

Brief change log

  • Add C mode constants for Delta, Changelog, Auto, and Diff incremental scans.
  • Add opaque incremental scan and plan handles with result and free functions.
  • Add incremental planning, work-unit counting, and ranged Arrow reading APIs.
  • Preserve ReadBuilder projection and predicates across incremental planning and reading.
  • Add ABI signature guards plus behavioral and null-boundary tests.

Tests

  • cargo test -p paimon-c
  • cargo clippy -p paimon-c --all-targets -- -D warnings
  • mkdocs build --strict
  • Generated the C header with cbindgen and verified all incremental declarations.

API and Format

This is an additive C ABI change. It does not change the Paimon storage format. The new API performs fixed-range batch incremental reads and does not wait for future snapshots.

Documentation

Updated the C integration guide with mode semantics, a complete lifecycle example, ownership rules, and the fixed-range limitation. Cross-linked the batch incremental reading guide to the new C API.

@JingsongLi JingsongLi closed this Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant