New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
distsql: avoid unnecessary flushes in outbox #22471
Conversation
pkg/sql/distsqlrun/outbox.go, line 246 at r1 (raw file):
See the doc, I think Stop needs to be called. Comments from Reviewable |
Review status: 0 of 1 files reviewed at latest revision, 1 unresolved discussion, some commit checks pending. pkg/sql/distsqlrun/outbox.go, line 201 at r1 (raw file):
pkg/sql/distsqlrun/outbox.go, line 246 at r1 (raw file): Previously, RaduBerinde wrote…
See my comment above. Comments from Reviewable |
pkg/sql/distsqlrun/outbox.go, line 201 at r1 (raw file):
not a big deal, but this will probably make us go to sleep and require some other goroutine to wake us up immediately. Maybe just start with a large timeout Comments from Reviewable |
Very nice that this achieves the performance benefit you were previously seeing with a longer flush interval. Review status: 0 of 1 files reviewed at latest revision, 3 unresolved discussions, some commit checks pending. Comments from Reviewable |
Previously, the outbox would have a flush ticker that would fire whether or not the outbox was empty. This change makes it so that we only ever fire if we have rows to flush. Addresses cockroachdb#22385 Release note (performance improvement): performance improvement for distributed sql queries.
f167f20
to
0d5bf14
Compare
Review status: 0 of 1 files reviewed at latest revision, 3 unresolved discussions. pkg/sql/distsqlrun/outbox.go, line 201 at r1 (raw file): Previously, petermattis (Peter Mattis) wrote…
Done. pkg/sql/distsqlrun/outbox.go, line 201 at r1 (raw file): Previously, RaduBerinde wrote…
Solved by the use of Comments from Reviewable |
the ratio of lines changed to performance improvement on this one is off the charts - awesome! Reviewed 1 of 1 files at r2. Comments from Reviewable |
Review status: all files reviewed at latest revision, 2 unresolved discussions, some commit checks pending. Comments from Reviewable |
You mentioned earlier that you saw a 25% improvement from a larger Review status: all files reviewed at latest revision, 1 unresolved discussion, all commit checks successful. Comments from Reviewable |
Maybe, I'll definitely do some more experimenting. |
Previously, the outbox would have a flush ticker that would fire whether
or not the outbox was empty. This change makes it so that we only ever
fire if we have rows to flush.
Addresses #22385
Release note (performance improvement): performance improvement for
distributed sql queries.
SELECT tpch.lineitem ORDER BY l_extendedprice LIMIT 1000
:16%
improvement on a local 3-node cluster,22%
improvement on a 6-noderoachprod
cluster.This should be an across-the-board improvement noticed more with longer-running queries.