feat: add ASOF joins to DataFrame API by Xuanwo · Pull Request #23831 · apache/datafusion

Xuanwo · 2026-07-23T05:59:35Z

Which issue does this PR close?

Part of ASOF join support / Specialize Range Joins #318.
Umbrella PR: feat: support ASOF joins #23738.
Depends on feat: add ASOF join logical semantics #23829 (and transitively feat: add ASOF join physical operator #23828).

Rationale for this change

This is the DataFrame API layer of the ASOF JOIN stack. It exposes the logical
contract through the fluent API without coupling that API review to SQL parsing
or protobuf changes.

GitHub shows the cumulative stack because fork branches cannot be selected as
upstream PR bases. The isolated diff for this layer is:
Xuanwo/datafusion@xuanwo/asof-logical...xuanwo/asof-dataframe

What changes are included in this PR?

Add DataFrame::join_asof for expression-based equality keys.
Add DataFrame::join_asof_using for same-named equality keys.
Export AsOfMatch and Operator from the DataFusion prelude.
Add a DataFrame-only execution test that does not depend on SQL syntax.

Are these changes tested?

Yes:

cargo fmt --all
cargo clippy --all-targets --all-features -- -D warnings
cargo test -p datafusion --test core_integration dataframe::join_asof --all-features
Extended workspace tests from the contributor guide

Are there any user-facing changes?

Users can build ASOF joins through DataFrame::join_asof and
DataFrame::join_asof_using.

This PR should be reviewed and merged after #23829. After its dependencies
squash-merge, this branch will be refreshed from main without rewriting its
published history.

github-actions · 2026-07-23T06:22:57Z

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details

     Cloning apache/main
    Building datafusion v54.1.0 (current)
       Built [  85.071s] (current)
     Parsing datafusion v54.1.0 (current)
      Parsed [   0.029s] (current)
    Building datafusion v54.1.0 (baseline)
       Built [  83.772s] (baseline)
     Parsing datafusion v54.1.0 (baseline)
      Parsed [   0.028s] (baseline)
    Checking datafusion v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.686s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [ 172.381s] datafusion
    Building datafusion-expr v54.1.0 (current)
       Built [  21.246s] (current)
     Parsing datafusion-expr v54.1.0 (current)
      Parsed [   0.063s] (current)
    Building datafusion-expr v54.1.0 (baseline)
       Built [  20.994s] (baseline)
     Parsing datafusion-expr v54.1.0 (baseline)
      Parsed [   0.064s] (baseline)
    Checking datafusion-expr v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   1.434s] 223 checks: 221 pass, 1 fail, 1 warn, 30 skip

--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/enum_variant_added.ron

Failed in:
  variant LogicalPlan:AsOfJoin in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:245
  variant LogicalPlan:AsOfJoin in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:245

--- warning partial_ord_enum_variants_reordered: enum variants reordered in #[derive(PartialOrd)] enum ---

Description:
A public enum that derives PartialOrd had its variants reordered. #[derive(PartialOrd)] uses the enum variant order to set the enum's ordering behavior, so this change may break downstream code that relies on the previous order.
        ref: https://doc.rust-lang.org/std/cmp/trait.PartialOrd.html#derivable
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/partial_ord_enum_variants_reordered.ron

Failed in:
  LogicalPlan::Repartition moved from position 7 to 8, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:249
  LogicalPlan::Union moved from position 8 to 9, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:253
  LogicalPlan::TableScan moved from position 9 to 10, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:256
  LogicalPlan::EmptyRelation moved from position 10 to 11, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:260
  LogicalPlan::Subquery moved from position 11 to 12, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:263
  LogicalPlan::SubqueryAlias moved from position 12 to 13, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:265
  LogicalPlan::Limit moved from position 13 to 14, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:267
  LogicalPlan::Statement moved from position 14 to 15, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:269
  LogicalPlan::Values moved from position 15 to 16, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:274
  LogicalPlan::Explain moved from position 16 to 17, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:277
  LogicalPlan::Analyze moved from position 17 to 18, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:281
  LogicalPlan::Extension moved from position 18 to 19, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:284
  LogicalPlan::Distinct moved from position 19 to 20, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:287
  LogicalPlan::Dml moved from position 20 to 21, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:289
  LogicalPlan::Ddl moved from position 21 to 22, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:291
  LogicalPlan::Copy moved from position 22 to 23, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:293
  LogicalPlan::DescribeTable moved from position 23 to 24, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:296
  LogicalPlan::Unnest moved from position 24 to 25, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:299
  LogicalPlan::RecursiveQuery moved from position 25 to 26, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:301
  LogicalPlan::Repartition moved from position 7 to 8, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:249
  LogicalPlan::Union moved from position 8 to 9, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:253
  LogicalPlan::TableScan moved from position 9 to 10, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:256
  LogicalPlan::EmptyRelation moved from position 10 to 11, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:260
  LogicalPlan::Subquery moved from position 11 to 12, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:263
  LogicalPlan::SubqueryAlias moved from position 12 to 13, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:265
  LogicalPlan::Limit moved from position 13 to 14, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:267
  LogicalPlan::Statement moved from position 14 to 15, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:269
  LogicalPlan::Values moved from position 15 to 16, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:274
  LogicalPlan::Explain moved from position 16 to 17, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:277
  LogicalPlan::Analyze moved from position 17 to 18, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:281
  LogicalPlan::Extension moved from position 18 to 19, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:284
  LogicalPlan::Distinct moved from position 19 to 20, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:287
  LogicalPlan::Dml moved from position 20 to 21, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:289
  LogicalPlan::Ddl moved from position 21 to 22, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:291
  LogicalPlan::Copy moved from position 22 to 23, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:293
  LogicalPlan::DescribeTable moved from position 23 to 24, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:296
  LogicalPlan::Unnest moved from position 24 to 25, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:299
  LogicalPlan::RecursiveQuery moved from position 25 to 26, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:301

     Summary semver requires new major version: 1 major and 0 minor checks failed
     Warning produced 1 major and 0 minor level warnings
    Finished [  44.938s] datafusion-expr
    Building datafusion-optimizer v54.1.0 (current)
       Built [  21.393s] (current)
     Parsing datafusion-optimizer v54.1.0 (current)
      Parsed [   0.026s] (current)
    Building datafusion-optimizer v54.1.0 (baseline)
       Built [  21.433s] (baseline)
     Parsing datafusion-optimizer v54.1.0 (baseline)
      Parsed [   0.026s] (baseline)
    Checking datafusion-optimizer v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.177s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [  44.026s] datafusion-optimizer
    Building datafusion-physical-plan v54.1.0 (current)
       Built [  29.095s] (current)
     Parsing datafusion-physical-plan v54.1.0 (current)
      Parsed [   0.121s] (current)
    Building datafusion-physical-plan v54.1.0 (baseline)
       Built [  29.097s] (baseline)
     Parsing datafusion-physical-plan v54.1.0 (baseline)
      Parsed [   0.118s] (baseline)
    Checking datafusion-physical-plan v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.790s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [  61.029s] datafusion-physical-plan
    Building datafusion-proto v54.1.0 (current)
       Built [  47.744s] (current)
     Parsing datafusion-proto v54.1.0 (current)
      Parsed [   0.016s] (current)
    Building datafusion-proto v54.1.0 (baseline)
       Built [  47.508s] (baseline)
     Parsing datafusion-proto v54.1.0 (baseline)
      Parsed [   0.017s] (baseline)
    Checking datafusion-proto v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.272s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [  96.999s] datafusion-proto
    Building datafusion-sql v54.1.0 (current)
       Built [  33.250s] (current)
     Parsing datafusion-sql v54.1.0 (current)
      Parsed [   0.027s] (current)
    Building datafusion-sql v54.1.0 (baseline)
       Built [  33.230s] (baseline)
     Parsing datafusion-sql v54.1.0 (baseline)
      Parsed [   0.027s] (baseline)
    Checking datafusion-sql v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.254s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [  68.385s] datafusion-sql
    Building datafusion-substrait v54.1.0 (current)
       Built [ 268.354s] (current)
     Parsing datafusion-substrait v54.1.0 (current)
      Parsed [   0.021s] (current)
    Building datafusion-substrait v54.1.0 (baseline)
       Built [ 270.965s] (baseline)
     Parsing datafusion-substrait v54.1.0 (baseline)
      Parsed [   0.015s] (baseline)
    Checking datafusion-substrait v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.229s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [ 541.789s] datafusion-substrait

codecov-commenter · 2026-07-23T06:24:24Z

Codecov Report

❌ Patch coverage is 75.85499% with 353 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.73%. Comparing base (47bd094) to head (2e3c976).

Files with missing lines	Patch %	Lines
datafusion/physical-plan/src/joins/asof_join.rs	85.70%	89 Missing and 55 partials ⚠️
datafusion/expr/src/logical_plan/plan.rs	43.71%	83 Missing and 11 partials ⚠️
datafusion/expr/src/logical_plan/builder.rs	60.97%	25 Missing and 7 partials ⚠️
datafusion/optimizer/src/push_down_filter.rs	0.00%	24 Missing ⚠️
datafusion/core/src/dataframe/mod.rs	43.33%	15 Missing and 2 partials ⚠️
datafusion/core/src/physical_planner.rs	70.90%	14 Missing and 2 partials ⚠️
datafusion/expr/src/logical_plan/display.rs	0.00%	10 Missing ⚠️
...tafusion/optimizer/src/optimize_projections/mod.rs	14.28%	6 Missing ⚠️
datafusion/expr/src/logical_plan/tree_node.rs	93.02%	0 Missing and 3 partials ⚠️
datafusion/proto/src/logical_plan/mod.rs	0.00%	3 Missing ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #23831      +/-   ##
==========================================
- Coverage   80.75%   80.73%   -0.03%     
==========================================
  Files        1089     1090       +1     
  Lines      368948   370407    +1459     
  Branches   368948   370407    +1459     
==========================================
+ Hits       297952   299053    +1101     
- Misses      53237    53510     +273     
- Partials    17759    17844      +85

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Xuanwo added 3 commits July 23, 2026 13:05

feat: add ASOF join physical operator

e546de5

feat: add ASOF join logical semantics

bd58e1d

feat: add ASOF joins to DataFrame API

ca0e943

github-actions Bot added sql SQL Planner logical-expr Logical plan and expressions optimizer Optimizer rules core Core DataFusion crate substrait Changes to the substrait crate proto Related to proto crate physical-plan Changes to the physical-plan crate labels Jul 23, 2026

github-actions Bot added the auto detected api change Auto detected API change label Jul 23, 2026

Xuanwo mentioned this pull request Jul 23, 2026

feat: support ASOF joins #23738

Open

Xuanwo added 3 commits July 23, 2026 14:30

Merge remote-tracking branch 'origin/main' into xuanwo/asof-physical

68ac8ec

Merge branch 'xuanwo/asof-physical' into xuanwo/asof-logical

c8c4813

Merge branch 'xuanwo/asof-logical' into xuanwo/asof-dataframe

2e3c976

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add ASOF joins to DataFrame API#23831

feat: add ASOF joins to DataFrame API#23831
Xuanwo wants to merge 6 commits into
apache:mainfrom
Xuanwo:xuanwo/asof-dataframe

Xuanwo commented Jul 23, 2026

Uh oh!

github-actions Bot commented Jul 23, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Jul 23, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Xuanwo commented Jul 23, 2026

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actions Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Jul 23, 2026 •

edited

Loading

codecov-commenter commented Jul 23, 2026 •

edited

Loading