sql: fix distsql planning issue with optimizer (streaming aggregation) #31825
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
RaduBerinde
force-pushed
the
distsql-agg-ordering-bug
branch
from
3b023a3
to
d089bf9
Compare
RaduBerinde
force-pushed
the
distsql-agg-ordering-bug
branch
2 times, most recently
from
c028b93
to
8cf5e62
Compare
andy-kimball
approved these changes
Oct 25, 2018
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained (and 1 stale)
rytaft
approved these changes
Oct 25, 2018
There was a problem hiding this comment.
Reviewed 13 of 13 files at r1.
Reviewable status:
pkg/sql/opt/exec/execbuilder/relational_builder.go, line 98 at r1 (raw file):
} // reqOrdering converts an OrderingChoice to a ColumnOrdering (according to the
reqOrdering -> sqlOrderingFromChoice
RaduBerinde
force-pushed
the
distsql-agg-ordering-bug
branch
from
8cf5e62
to
54420ef
Compare
RaduBerinde
commented
Oct 25, 2018
There was a problem hiding this comment.
TFTRs!
Reviewable status:
pkg/sql/opt/exec/execbuilder/relational_builder.go, line 98 at r1 (raw file):
Previously, rytaft wrote…
reqOrdering -> sqlOrderingFromChoice
Done.
|
bors r+ |
Build failed |
The distsql planner looks at physical properties of various plan nodes to figure out what orderings it needs to preserve when merging data from multiple nodes. These are passed as required orderings in the exec.Factory, but not for all operators - only those for which it would actually matter. There is a code path related to setting up streaming aggregations which was unaccounted for which looks at the ordering of the input node. To fix this, we now pass required orderings to filters and projections. I looked at all the operators and the rest either: - never produce an ordering as far as the optimizer is concerned (VirtualScan, HashJoin, ScalarGroupBy, SetOp, ProjectSet); or - they pass through the input ordering unchanged (Distinct); or - they cannot be distributed (Limit, Ordinality). Release note (bug fix): Fixed an issue which causes invalid results or an "incorrectly ordered stream" error with streaming aggregations (in some cases).
RaduBerinde
force-pushed
the
distsql-agg-ordering-bug
branch
from
54420ef
to
f04fc30
Compare
|
bors r+ |
craig bot
pushed a commit
that referenced
this pull request
Oct 25, 2018
31825: sql: fix distsql planning issue with optimizer (streaming aggregation) r=RaduBerinde a=RaduBerinde The distsql planner looks at physical properties of various plan nodes to figure out what orderings it needs to preserve when merging data from multiple nodes. These are passed as required orderings in the exec.Factory, but not for all operators - only those which would actually matter. There is a code path related to setting up streaming aggregations which was unaccounted for which looks at the ordering of the input node. To fix this, we now pass required orderings to filters and projections. I looked at all the operators and the rest either: - never produce an ordering as far as the optimizer is concerned (VirtualScan, HashJoin, ScalarGroupBy, SetOp, ProjectSet); or - they pass through the input ordering unchanged (Distinct); or - they cannot be distributed (Limit, Ordinality). Release note (bug fix): Fixed an issue which causes invalid results or an "incorrectly ordered stream" error with streaming aggregations (in some cases). Co-authored-by: Radu Berinde <radu@cockroachlabs.com>
Build succeeded |
This was referenced Oct 30, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The distsql planner looks at physical properties of various plan nodes
to figure out what orderings it needs to preserve when merging data
from multiple nodes. These are passed as required orderings in the
exec.Factory, but not for all operators - only those which would
actually matter.
There is a code path related to setting up streaming aggregations
which was unaccounted for which looks at the ordering of the input
node.
To fix this, we now pass required orderings to filters and
projections. I looked at all the operators and the rest either:
(VirtualScan, HashJoin, ScalarGroupBy, SetOp, ProjectSet); or
Release note (bug fix): Fixed an issue which causes invalid results or
an "incorrectly ordered stream" error with streaming aggregations (in
some cases).