Support the plan-based parallel replicas in automatic parallel replicas - #115788
Conversation
Automatic parallel replicas builds a plan with parallel replicas, finds the node that would sit at the top of the plan executed on the replicas and instruments the corresponding node of the single-node plan to collect runtime dataflow statistics. `findTopNodeOfReplicasPlan` located that node by matching the branch that reads from the other replicas against `ReadFromParallelRemoteReplicasStep`. With `parallel_replicas_plan_based` that branch is a `ReadFromParallelReplicas` step, which ships a serialized plan fragment instead of a query. The rest of the plan is shaped exactly the same - a `Union` of the local branch and the branch reading from the other replicas - but the top node was never found, so no statistics were collected and automatic parallel replicas never enabled parallel replicas for any query. Recognize both steps. Two shapes are still not handled and keep the previous behaviour of skipping the optimization: a join whose plan-based local branch carries an extra pass-through `Expression` (the node hashes of the two plans then differ), and a plain read whose plan fragment is the reading step itself. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Workflow [PR], commit [bb89c23] Summary: ✅
AI ReviewSummaryThis PR teaches Final Verdict
LLVM Coverage Report
Changed lines: Changed C/C++ lines covered: 9/9 (100.00%) · Uncovered code |
clickhouse-gh
Bot
added
the
pr-bugfix
|
Both points are right — Updated the changelog entry to the suggested wording. For the category I went with |
Build profile diff (arm_release)Comparing ✅ No significant changes. Binary sizes
Only the stripped binary is compared: the official master build keeps debug symbols while PR builds strip them, so the other binaries differ by construction. Compile time of recompiled translation units7 translation units recompiled, 15 s compile time in total, 7 of them have a recent master baseline. |
The flaky check hit `Logical error: 'Bucket 7 is pushed twice'` in `GroupingAggregatedTransform` twice out of 41 runs of the new test. The same stack trace is already reported for the distributed plan on a release branch, see #115663, so it is a race in the memory efficient merging of partial aggregation results, not in what this test checks - the decision of automatic parallel replicas. Merge the partial results of the replicas without `GroupingAggregatedTransform` so the test stops tripping over it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
clickhouse-gh
Bot
added
pr-experimental
|
💋 |
robot-clickhouse-ci-1
added
the
pr-synced-to-cloud
3 participants
Automatic parallel replicas builds a plan with parallel replicas, finds the node that would sit at the top of the plan executed on the replicas, and instruments the corresponding node of the single-node plan to collect runtime dataflow statistics.
findTopNodeOfReplicasPlanlocated that node by matching the branch that reads from the other replicas againstReadFromParallelRemoteReplicasStep.With
parallel_replicas_plan_basedthat branch is aReadFromParallelReplicasstep, which ships a serialized plan fragment instead of a query. The rest of the plan is shaped exactly the same - aUnionof the local branch and the branch reading from the other replicas - but the top node was never found, so no statistics were collected and automatic parallel replicas never enabled parallel replicas for any query.This recognizes both steps. Verified against a local server with
parallel_replicas_plan_based=1: 10 of the 12*_autopr_*stateless tests that fail with the plan-based implementation as the default now pass. The remaining two are left for a follow-up and keep the previous behaviour of skipping the optimization:04341_autopr_mode1_apply_join- with join runtime filters the plan-based local branch carries an extra pass-throughExpression, so the node hashes of the two plans differ and no match is found.03634_autopr_input_bytes_estimation_compact- for a plain read the plan fragment is the reading step itself, which cannot estimate the number of bytes sent to the initiator.Related: #112351
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
Automatic parallel replicas can now collect runtime statistics and enable parallel replicas for supported queries when
parallel_replicas_plan_basedis enabled.Workflow [PR]
Sync PR [sync-upstream/pr/115788]
Version info
26.9.1.47(included in26.9and later)