Skip to content

[python] Add DataFusion extra for merge conditions - #9463

Merged
JingsongLi merged 2 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/datafusion-extra
Aug 29, 2026
Merged

[python] Add DataFusion extra for merge conditions#9463
JingsongLi merged 2 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/datafusion-extra

Conversation

@XiaoHongbo-Hope

@XiaoHongbo-Hope XiaoHongbo-Hope commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Purpose

merge_into condition expressions use the Python DataFusion package but currently direct users to pypaimon[sql], which also installs pypaimon-rust. This makes the condition dependency unnecessarily coupled to the full SQL stack.

Changes

  • Add pypaimon[datafusion] with only datafusion>=54,<55 on Python 3.10+.
  • Keep pypaimon[sql] unchanged for the full Rust and DataFusion SQL stack.
  • Point merge condition errors and documentation to the new extra.
  • Fail with an explicit Python upgrade message on Python 3.6-3.9.
  • Quote the installation command for shells such as zsh.

Validation

  • Merge condition tests: 13 passed, 36 skipped, 109 deselected.
  • Missing-dependency and unsupported-Python regression tests passed.
  • Generated package metadata contains the new datafusion extra while sql retains both dependencies.
  • flake8 and git diff --check passed.

@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as ready for review August 29, 2026 08:11
@JingsongLi

Copy link
Copy Markdown
Contributor

+1

@JingsongLi
JingsongLi merged commit 6e7ba7c into apache:master Aug 29, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants