release-21.2: cli: support SQL disk spilling in tenants #71218
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 1/1 commits from #71040 on behalf of @jaylim-crl.
/cc @cockroachdb/release
Previously, temp storage for SQL tenants was configured to point to memory
with a limit of 100MB. As a result, all ephemeral data when processing large
queries end up going to memory, and there was no way to configure this to
point to disk. This commit changes that behavior, and the default temp storage
for SQL tenants is now the same as SQL for dedicated, i.e. disk, with a limit
of 32GB. The operator can configure this through the
--max-disk-temp-storage
flag.
Release note (cli change):
cockroach mt start-sql
will now support thefollowing flags to configure ephemeral storage for SQL when processing large
queries:
--store
,--temp-dir
, and--max-disk-temp-storage
.Release note (sql change): SQL tenants will now spill to disk by default
when processing large queries, instead of memory.
Release justification: The upcoming Serverless MVP release plans to allow
spilling to ephemeral disk for SQL operations (See CC-4983), but the
existing
start-sql
for tenants doesn't support that. This commit changesthe default spilling behavior in multi-tenant scenarios, and allows an operator
to configure ephemeral SQL storage through the CLI, and should have no
impact on dedicated customers.