Skip to content

Add optimizer-focused benchmarks for logical plan optimization#20834

Draft
adriangb wants to merge 1 commit intoapache:mainfrom
pydantic:add-optimizer-benchmarks
Draft

Add optimizer-focused benchmarks for logical plan optimization#20834
adriangb wants to merge 1 commit intoapache:mainfrom
pydantic:add-optimizer-benchmarks

Conversation

@adriangb
Copy link
Contributor

@adriangb adriangb commented Mar 9, 2026

Summary

  • Add 17 new benchmarks to sql_planner.rs that stress the logical optimizer across different dimensions:
    • Plan node scaling: join chains with 4, 8, and 16 tables
    • Expression scaling: 50-200 filter predicates, 100 aggregate expressions, 50 CASE expressions
    • Mixed deep+wide: joins combined with aggregates, filters, sorts, limits
    • Subqueries: correlated EXISTS, IN, scalar subqueries, and multiple combined subqueries
    • Unions: 4 and 8 branch UNION ALL queries
    • Other: DISTINCT over many columns, nested CTEs (4 levels)
  • Uncomment existing TPC-H and TPC-DS full-suite logical plan benchmarks

These complement the existing simple-plan benchmarks (2-6 node plans) by covering the optimizer's behavior as plan complexity grows — useful for tracking optimizer performance improvements (ref #15045).

Test plan

  • cargo bench --bench sql_planner runs successfully with all new benchmarks
  • Verified no overlap with existing benchmarks in sql_planner_extended.rs or optimizer/benches/

🤖 Generated with Claude Code

Add 17 new benchmarks to sql_planner.rs that stress the logical optimizer
across different dimensions: deep join chains (4/8/16 tables), wide
expressions (50-200 filter predicates, 100 aggregates, 50 CASE exprs),
mixed deep+wide plans, correlated subqueries, unions, distinct, and
nested CTEs. Also uncomment TPC-H and TPC-DS full-suite benchmarks.

These benchmarks help identify optimizer bottlenecks as plan complexity
grows, complementing the existing simple-plan benchmarks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions bot added the core Core DataFusion crate label Mar 9, 2026
@adriangb adriangb marked this pull request as draft March 9, 2026 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant