Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

colexec: tune reserved memory for partitions #45593

Open
asubiotto opened this issue Mar 2, 2020 · 1 comment
Open

colexec: tune reserved memory for partitions #45593

asubiotto opened this issue Mar 2, 2020 · 1 comment
Labels
C-performance Perf of queries or internals. Solution not expected to change functional behavior. T-sql-queries SQL Queries Team

Comments

@asubiotto
Copy link
Contributor

asubiotto commented Mar 2, 2020

Sorts:

  • We currently reserve half of the work memory in sorts. With a partition cache size of 64KiB and a default work mem of 64MiB, this allows us to have 1024 partitions open at any one time. This seems like too many partitions open at one time and reduces the partition size to 32MiB. We might be able to find a better balance.
    Joins:
  • We limit ourselves to 16 partitions in this case. A lower number of partitions increases the chances that the partitions are large and the chances that we'll have to repartition. At least it should be fair and the number of partitions should be equal to sorts.

To find a good number of partitions to set the defaults to, it would be good to run end-to-end performance benchmarking on tpch.

Jira issue: CRDB-5143

@asubiotto asubiotto created this issue from a note in BACKLOG, NO NEW ISSUES: SQL Execution (20.1 Commitments) Mar 2, 2020
@asubiotto asubiotto moved this from 20.1 Commitments to 20.1.8 in BACKLOG, NO NEW ISSUES: SQL Execution Mar 9, 2020
@asubiotto asubiotto self-assigned this Mar 9, 2020
@asubiotto asubiotto moved this from 20.1.8 to 20.1 Commitments in BACKLOG, NO NEW ISSUES: SQL Execution Mar 13, 2020
@asubiotto asubiotto moved this from 20.1 Commitments to 20.1 Stabilization Period in BACKLOG, NO NEW ISSUES: SQL Execution Mar 27, 2020
@asubiotto asubiotto moved this from 20.1 Stabilization Period to [VECTORIZED BACKLOG] Enhancements/Features in BACKLOG, NO NEW ISSUES: SQL Execution Apr 14, 2020
@jlinder jlinder added the T-sql-queries SQL Queries Team label Jun 16, 2021
@github-actions
Copy link

We have marked this issue as stale because it has been inactive for
18 months. If this issue is still relevant, removing the stale label
or adding a comment will keep it active. Otherwise, we'll close it in
10 days to keep the issue queue tidy. Thank you for your contribution
to CockroachDB!

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 26, 2023
@yuzefovich yuzefovich reopened this Sep 27, 2023
@yuzefovich yuzefovich added the C-performance Perf of queries or internals. Solution not expected to change functional behavior. label Sep 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-performance Perf of queries or internals. Solution not expected to change functional behavior. T-sql-queries SQL Queries Team
Projects
BACKLOG, NO NEW ISSUES: SQL Execution
[VECTORIZED BACKLOG] Enhancements/Fea...
Status: Backlog
Development

No branches or pull requests

3 participants