Skip to content

[python] Keep native planning for Blob descriptor reads - #9144

Merged
JingsongLi merged 2 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/native-plan-blob-descriptor
Aug 10, 2026
Merged

[python] Keep native planning for Blob descriptor reads#9144
JingsongLi merged 2 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/native-plan-blob-descriptor

Conversation

@XiaoHongbo-Hope

@XiaoHongbo-Hope XiaoHongbo-Hope commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Purpose

Keep native scan planning enabled for dynamic reader options which do not affect split planning.

query.read_blobs() copies the table with blob-as-descriptor=true. The native capability gate treated this reader-only option as unsupported and silently fell back to Python planning.

Changes

  • allow blob-as-descriptor, read.batch-size, and read.parallelism without disabling native planning; these options remain effective in the PyPaimon reader and are not needed by the Rust planner
  • continue forwarding only the scan options explicitly supported by the Rust planner
  • keep unknown dynamic options on the conservative Python fallback path because Rust 0.4 does not strictly reject every unknown option
  • preserve conservative handling for option removal
  • cover Blob descriptor planning and decoding, reader-only options, and unknown-option fallback

Tests

  • native plan unit and integration tests: 52 passed, 12 subtests passed
  • multimodal read_blobs / stream_blobs tests: 6 passed
  • flake8 on changed files
  • compileall
  • git diff --check
  • OSS-backed Data Evolution BLOB table: the high-level query.read_blobs() path invoked native planning once, planned one split at the pinned snapshot, and returned the expected row and Blob payload

@XiaoHongbo-Hope XiaoHongbo-Hope changed the title [python] Forward dynamic options to native planner [python] Keep native planning for Blob descriptor reads Aug 10, 2026
@JingsongLi

Copy link
Copy Markdown
Contributor

+1

@JingsongLi
JingsongLi merged commit 7cd451f into apache:master Aug 10, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants