Skip to content

refactor(proto): migrate scalar subquery serde - #23915

Open
Phoenix500526 wants to merge 1 commit into
apache:mainfrom
Phoenix500526:issue/23515
Open

refactor(proto): migrate scalar subquery serde#23915
Phoenix500526 wants to merge 1 commit into
apache:mainfrom
Phoenix500526:issue/23515

Conversation

@Phoenix500526

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

ScalarSubqueryExpr nodes must share the results container populated by their enclosing ScalarSubqueryExec. A normal child decode loses that scope when serialization moves into the plan implementation.

Add a scoped child decode operation so the new plan hooks preserve the shared container while removing the central serialization path.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

ScalarSubqueryExpr nodes must share the results container populated by
their enclosing ScalarSubqueryExec. A normal child decode loses that
scope when serialization moves into the plan implementation.

Add a scoped child decode operation so the new plan hooks preserve the
shared container while removing the central serialization path.

Closes apache#23515

Signed-off-by: Jiawei Zhao <Phoenix500526@163.com>
@github-actions github-actions Bot added proto Related to proto crate physical-plan Changes to the physical-plan crate labels Jul 27, 2026
@github-actions

Copy link
Copy Markdown

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details
     Cloning apache/main
    Building datafusion-physical-plan v54.1.0 (current)
       Built [  37.668s] (current)
     Parsing datafusion-physical-plan v54.1.0 (current)
      Parsed [   0.140s] (current)
    Building datafusion-physical-plan v54.1.0 (baseline)
       Built [  38.173s] (baseline)
     Parsing datafusion-physical-plan v54.1.0 (baseline)
      Parsed [   0.141s] (baseline)
    Checking datafusion-physical-plan v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.650s] 223 checks: 222 pass, 1 fail, 0 warn, 30 skip

--- failure trait_method_added: pub trait method added ---

Description:
A non-sealed public trait added a new method without a default implementation, which breaks downstream implementations of the trait
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#trait-new-item-no-default
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/trait_method_added.ron

Failed in:
  trait method datafusion_physical_plan::proto::ExecutionPlanDecode::decode_plan_with_scalar_subquery_results in file /home/runner/work/datafusion/datafusion/datafusion/physical-plan/src/proto.rs:108

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [  78.025s] datafusion-physical-plan
    Building datafusion-proto v54.1.0 (current)
       Built [  59.774s] (current)
     Parsing datafusion-proto v54.1.0 (current)
      Parsed [   0.019s] (current)
    Building datafusion-proto v54.1.0 (baseline)
       Built [  59.824s] (baseline)
     Parsing datafusion-proto v54.1.0 (baseline)
      Parsed [   0.020s] (baseline)
    Checking datafusion-proto v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.229s] 223 checks: 222 pass, 1 fail, 0 warn, 30 skip

--- failure trait_method_marked_deprecated: trait method #[deprecated] added ---

Description:
A trait method is now #[deprecated]. Downstream crates will get a compiler warning when using this method.
        ref: https://doc.rust-lang.org/reference/attributes/diagnostics.html#the-deprecated-attribute
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/trait_method_marked_deprecated.ron

Failed in:
  method try_into_scalar_subquery_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:664
  method try_from_scalar_subquery_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:664

     Summary semver requires new minor version: 0 major and 1 minor checks failed
    Finished [ 121.351s] datafusion-proto

@github-actions github-actions Bot added the auto detected api change Auto detected API change label Jul 27, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 67.56757% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.65%. Comparing base (e8a65f2) to head (6f12a16).

Files with missing lines Patch % Lines
datafusion/proto/src/physical_plan/mod.rs 38.46% 16 Missing ⚠️
datafusion/physical-plan/src/scalar_subquery.rs 80.00% 2 Missing and 6 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #23915      +/-   ##
==========================================
- Coverage   80.66%   80.65%   -0.02%     
==========================================
  Files        1095     1095              
  Lines      372294   372319      +25     
  Branches   372294   372319      +25     
==========================================
- Hits       300324   300296      -28     
- Misses      54055    54101      +46     
- Partials    17915    17922       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto detected api change Auto detected API change physical-plan Changes to the physical-plan crate proto Related to proto crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proto: migrate ScalarSubqueryExec (adds a subquery-results scope to the decode ctx)

2 participants