Skip to content

feat: add ASOF join physical operator#23828

Draft
Xuanwo wants to merge 2 commits into
apache:mainfrom
Xuanwo:xuanwo/asof-physical
Draft

feat: add ASOF join physical operator#23828
Xuanwo wants to merge 2 commits into
apache:mainfrom
Xuanwo:xuanwo/asof-physical

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 23, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

Rationale for this change

This is the first layer of the ASOF JOIN stack. It establishes the physical
execution contract independently so later logical-plan, SQL, DataFrame, and
serialization changes can be reviewed as smaller follow-up PRs.

What changes are included in this PR?

  • Add AsOfJoinExec for left-preserving, Snowflake-style ASOF semantics.
  • Require compatible equality partitioning and ordering on both inputs.
  • Preserve merge state across input and output batch boundaries.
  • Define output properties, statistics, and ASOF-specific metrics.
  • Add physical operator tests covering match directions, equality groups,
    batch boundaries, unmatched rows, and invalid contracts.

Are these changes tested?

Yes:

  • cargo fmt --all
  • cargo clippy --all-targets --all-features -- -D warnings
  • cargo test -p datafusion-physical-plan joins::asof_join --all-features
  • Extended workspace tests from the contributor guide

Are there any user-facing changes?

This adds a new physical operator API. SQL and DataFrame APIs are intentionally
left to dependent PRs in the ASOF JOIN stack.

@github-actions github-actions Bot added the physical-plan Changes to the physical-plan crate label Jul 23, 2026
@codecov-commenter

codecov-commenter commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.94042% with 202 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.75%. Comparing base (47bd094) to head (68ac8ec).

Files with missing lines Patch % Lines
datafusion/physical-plan/src/joins/asof_join.rs 79.94% 147 Missing and 55 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #23828      +/-   ##
==========================================
- Coverage   80.75%   80.75%   -0.01%     
==========================================
  Files        1089     1090       +1     
  Lines      368948   369955    +1007     
  Branches   368948   369955    +1007     
==========================================
+ Hits       297952   298749     +797     
- Misses      53237    53387     +150     
- Partials    17759    17819      +60     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants