Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kv: guardrails for range size #96549

Closed
shralex opened this issue Feb 4, 2023 · 0 comments · Fixed by #96725
Closed

kv: guardrails for range size #96549

shralex opened this issue Feb 4, 2023 · 0 comments · Fixed by #96725
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
KV

Comments

@shralex
Copy link
Contributor

shralex commented Feb 4, 2023 •
edited by cockroach-jira-scripts

We've seen that changing max_range_size to a low value can break multiple sub-systems. We should implement guardrails for setting min/max range size (for user and/or system ranges).

Jira issue: CRDB-24182
@shralex shralex added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-kv Anything in KV that doesn't belong in a more specific category. T-kv KV Team and removed A-kv Anything in KV that doesn't belong in a more specific category. labels Feb 4, 2023
@blathers-crl blathers-crl bot added this to Incoming in KV Feb 4, 2023
@shralex shralex mentioned this issue Feb 7, 2023
Merged
shralex added a commit to shralex/cockroach that referenced this issue Feb 7, 2023
@shralex
kv: increasing the minimal max_range_size to 64MiB
9b206f7 
We've seen that small range sizes can be detrimental to various
components. This PR makes it so users can't lower max_range_size
below 64MiB (half of the default min_range_size), instead of 64KiB previously.

Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182
Fixes: cockroachdb#96549
shralex added a commit to shralex/cockroach that referenced this issue Feb 10, 2023
@shralex
kv: increasing the minimal max_range_size to 64MiB
938d746 
We've seen that small range sizes can be detrimental to various
components. This PR makes it so users can't lower max_range_size
below 64MiB (half of the default min_range_size), instead of 64KiB previously.

Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182
Fixes: cockroachdb#96549
shralex added a commit to shralex/cockroach that referenced this issue Feb 11, 2023
@shralex
kv: increase the minimal max_range_size to 64MiB
4cb71df 
We've seen that small range sizes can be detrimental to various
components. This PR makes it so users can't lower max_range_size
below 64MiB (half of the default min_range_size), instead of 64KiB previously.

Release Note: Small ranges have been known to cause problems in various
CRDB subsystems. This PR prevents setting max_range_size below
COCKROACH_MIN_RANGE_MAX_BYTES, an environment variable which defaults
to 64MiB (half of the default minimum range size).

Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182
Fixes: cockroachdb#96549
shralex added a commit to shralex/cockroach that referenced this issue Feb 21, 2023
@shralex
kv: increase the minimal max_range_size to 64MiB
e9f46f4 
We've seen that small range sizes can be detrimental to various
components. This PR makes it so users can't lower max_range_size
below 64MiB (half of the default min_range_size), instead of 64KiB previously.

Release Note: Small ranges have been known to cause problems in various
CRDB subsystems. This PR prevents setting max_range_size below
COCKROACH_MIN_RANGE_MAX_BYTES, an environment variable which defaults
to 64MiB (half of the default minimum range size).

Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182
Fixes: cockroachdb#96549
shralex added a commit to shralex/cockroach that referenced this issue Feb 23, 2023
@shralex
kv: increase the minimal max_range_size to 64MiB
6abd7dd 
We've seen that small range sizes can be detrimental to various
components. This PR makes it so users can't lower max_range_size
below 64MiB (half of the default min_range_size), instead of 64KiB previously.

Release Note: Small ranges have been known to cause problems in various
CRDB subsystems. This PR prevents setting max_range_size below
COCKROACH_MIN_RANGE_MAX_BYTES, an environment variable which defaults
to 64MiB (half of the default minimum range size).

Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182
Fixes: cockroachdb#96549
craig bot pushed a commit that referenced this issue Mar 25, 2023
@shralex @msbutler @renatolabs
Merge #96725 #98759 #99371
86ad731 
96725: sql: increase minimal max_range_size to 64MiB r=shralex a=shralex

We've seen that small range sizes can be detrimental to various components. This PR makes it so users can't lower max_range_size below 64MiB (half of the default min_range_size), instead of 64KiB previously.

Release note: Small ranges have been known to cause problems in various CRDB subsystems. This PR prevents setting max_range_size below COCKROACH_MIN_RANGE_MAX_BYTES, an environment variable which defaults to 64MiB (half of the default minimum range size). 

Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182
Fixes: #96549

98759: backupccl: create stripped crdb_internal.fingerprint overload r=stevendanna a=msbutler

This patch adds a variant of crdb_internal.fingerprint(), which creates a "stripped" fingerprint of the target span. Namely, `crdb_internal.fingerping(span,true)` will return a fingerprint that is agnostic to the mvcc timestamps and the index prefix of the key, and considers only the latest mvcc history of the key span.

For example, suppose the user fingerprinted a table at some system time, then backed up and restored it to that same system time. The restored table should have the same fingerprint!

This fingerprint variant is signicantly more scalable than SHOW EXPERIMENTAL FINGERPRINT, as it uses export requests compared to a simple scan.

Fixes #98570

Release note: None

99371: roachtest: lower default timeout r=srosenberg a=renatolabs

The previous default of 10 hours if unnecessarily long. As a consequence, if a test has a bug that causes it to hang, it will take 10h's worth of cluster usage, even if it's a test that generally succeeds in half an hour.

This should hopefully help with the timeouts we have been seeing in roachtest nightly runs.

Epic: none

Release note: None

Co-authored-by: shralex <shralex@gmail.com>
Co-authored-by: Michael Butler <butler@cockroachlabs.com>
Co-authored-by: Renato Costa <renato@cockroachlabs.com>
@craig craig bot closed this as completed in a37e053 Mar 25, 2023
KV automation moved this from Incoming to Closed Mar 25, 2023
@blathers-crl blathers-crl bot mentioned this issue Mar 25, 2023
Closed
blathers-crl bot pushed a commit that referenced this issue Mar 25, 2023
@shralex
kv: increase the minimal max_range_size to 64MiB
ebe4a43 
We've seen that small range sizes can be detrimental to various
components. This PR makes it so users can't lower max_range_size
below 64MiB (half of the default min_range_size), instead of 64KiB previously.

Release Note: Small ranges have been known to cause problems in various
CRDB subsystems. This PR prevents setting max_range_size below
COCKROACH_MIN_RANGE_MAX_BYTES, an environment variable which defaults
to 64MiB (half of the default minimum range size).

Epic: https://cockroachlabs.atlassian.net/browse/CRDB-24182
Fixes: #96549
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
KV
Closed
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sql: increase minimal max_range_size to 64MiB shralex/cockroach
1 participant
@shralex