Skip to content

Propagate query plan optimizations to correlated subquery input#85455

Merged
novikd merged 28 commits intomasterfrom
correlated-subqueries-input-buffer
Oct 25, 2025
Merged

Propagate query plan optimizations to correlated subquery input#85455
novikd merged 28 commits intomasterfrom
correlated-subqueries-input-buffer

Conversation

@novikd
Copy link
Copy Markdown
Member

@novikd novikd commented Aug 12, 2025 •
edited
Loading

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Make query plan optimizations visible to the correlated subquery input subplan by postponing it's materialization. Part of #79890.

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Aug 12, 2025 •
edited
Loading

Workflow [PR], commit [7728c54]

Summary: ⏳

job_name test_name status info comment
Stress test (arm_asan) failure
Server died FAIL cidb
Hung check failed, possible deadlock found (see hung_check.log) FAIL cidb
Killed by signal (in clickhouse-server.log) FAIL cidb
Fatal message in clickhouse-server.log (see fatal_messages.txt) FAIL cidb
Killed by signal (output files) FAIL cidb

@clickhouse-gh clickhouse-gh bot added the pr-improvement Pull request with some product improvements label Aug 12, 2025
@novikd novikd mentioned this pull request Aug 4, 2025
Open
14 tasks
@KochetovNicolai KochetovNicolai self-assigned this Aug 14, 2025
KochetovNicolai
KochetovNicolai reviewed Aug 18, 2025
View reviewed changes
KochetovNicolai
KochetovNicolai reviewed Aug 18, 2025
View reviewed changes
KochetovNicolai
KochetovNicolai reviewed Aug 18, 2025
View reviewed changes
KochetovNicolai
KochetovNicolai reviewed Aug 18, 2025
View reviewed changes
KochetovNicolai
KochetovNicolai reviewed Aug 18, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@KochetovNicolai KochetovNicolai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall good.

I would prefer creating intermediate steps like dependent join + reference first, and after convert to buffering steps.

Technically, in this case (later) we can swap join sides togther with dependent join refernece steps.

@qoega
Copy link
Copy Markdown
Member

qoega commented Sep 23, 2025

Will it only work for correlated subqueries for now? Will it be possible to extend it to more CTEs?

@novikd
Copy link
Copy Markdown
Member Author

novikd commented Sep 30, 2025

Will it only work for correlated subqueries for now? Will it be possible to extend it to more CTEs?

This PR aims to support only the case of correlated subqueries, but it'll be easy to extend it to CTEs in the future (I have such plans in mind)

@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 24, 2025
@novikd
Copy link
Copy Markdown
Member Author

novikd commented Oct 25, 2025

PR / Stateless tests (amd_ubsan, parallel) (pull_request)

#88893

PR / Performance Comparison (amd_release, master_head, 2/3) (pull_request)

Unrelated

PR / AST fuzzer (amd_ubsan) (pull_request)

Unrelated

PR / Stress test (arm_asan) (pull_request)

#84669

CH Inc sync

Unrelated

@novikd novikd added this pull request to the merge queue Oct 25, 2025
Merged via the queue into master with commit cb4e143 Oct 25, 2025
118 of 123 checks passed
@novikd novikd deleted the correlated-subqueries-input-buffer branch October 25, 2025 17:09
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 mentioned this pull request Oct 25, 2025
Merged
@robot-clickhouse robot-clickhouse added the pr-synced-to-cloud The PR is synced to the cloud repo label Oct 25, 2025
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 added pr-backports-created-cloud deprecated label, NOOP pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR labels Oct 25, 2025
robot-clickhouse-ci-1 added a commit that referenced this pull request Oct 25, 2025
@robot-clickhouse-ci-1
Merge pull request #88989 from ClickHouse/cherrypick/25.10/85455
7926b06 
Cherry pick #85455 to 25.10: Propagate query plan optimizations to correlated subquery input
robot-clickhouse added a commit that referenced this pull request Oct 25, 2025
@robot-clickhouse
Backport #85455 to 25.10: Propagate query plan optimizations to corre…
905af02 
…lated subquery input
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 mentioned this pull request Oct 25, 2025
Merged
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label Oct 25, 2025
clickhouse-gh bot added a commit that referenced this pull request Oct 25, 2025
@clickhouse-gh
Merge pull request #88990 from ClickHouse/backport/25.10/85455
3ee76d8 
Backport #85455 to 25.10: Propagate query plan optimizations to correlated subquery input
@ianton-ru
Copy link
Copy Markdown
Contributor

ianton-ru commented Dec 8, 2025

@novikd Hi,
I guess, fix for 03644_object_storage_correlated_subqueries.sql must be backported in 25.8.

@vdimir vdimir added pr-must-backport Pull request should be backported intentionally. Use this label with great care! and removed pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-backports-created-cloud deprecated label, NOOP pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR v25.10-must-backport labels Dec 10, 2025
@robot-ch-test-poll robot-ch-test-poll added the pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR label Dec 10, 2025
This was referenced Dec 10, 2025
Closed
Closed
Closed
@vdimir
Copy link
Copy Markdown
Member

vdimir commented Dec 11, 2025

Backports from this PR contains lot's of other changes let's create a new PR to backport only this change #91957

@robot-clickhouse-ci-1 robot-clickhouse-ci-1 added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label Dec 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@KochetovNicolai KochetovNicolai KochetovNicolai approved these changes

Assignees

@KochetovNicolai KochetovNicolai

Labels

pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-improvement Pull request with some product improvements pr-must-backport Pull request should be backported intentionally. Use this label with great care! pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@novikd @qoega @ianton-ru @vdimir @KochetovNicolai @nikitamikhaylov @robot-clickhouse @robot-ch-test-poll @rschu1ze @robot-clickhouse-ci-1