Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Repair-based node operations write SSTables with useless filters as a result of not feeding correct key estimation #15748

Closed
raphaelsc opened this issue Oct 18, 2023 · 3 comments
Closed

Repair-based node operations write SSTables with useless filters as a result of not feeding correct key estimation #15748

raphaelsc opened this issue Oct 18, 2023 · 3 comments
Assignees
@asias
Labels
area/repair-based operations repare based node operations area/sstable
Milestone
5.4

Comments

@raphaelsc
Copy link
Member

The repair writer, which creates new SSTables, incorrectly picked the default value for partition estimation, rather than using the one received. That causes those new SSTables to have useless filters that will potentially add to read amplification until they're rewritten in the off-strategy step. So this can manifest as temporary latency spikes, which can of course, degrade service temporarily.
@raphaelsc
Copy link
Member Author

Closed by 4592bbe

@raphaelsc
Copy link
Member Author

@asias please proceed with backport of 4592bbe to branches 5.2 and 5.1. It's an important fix.

@raphaelsc raphaelsc changed the title Repair-based node operations write SSTables with useless filters due to a bug Repair-based node operations write SSTables with useless filters as a result of not feeding correct key estimation Oct 18, 2023
@asias asias mentioned this issue Oct 18, 2023
Closed
@mykaul mykaul added backport/5.2 Issues that should be backported to 5.2 branch once they'll be fixed Requires-Backport-to-5.1 Backport candidate labels Oct 18, 2023
denesb pushed a commit that referenced this issue Oct 18, 2023
@asias @denesb
repair: Use the updated estimated_partitions to create writer
ac45d8d 
The estimated_partitions is estimated after the repair_meta is created.

Currently, the default estimated_partitions was used to create the
write which is not correct.

To fix, use the updated estimated_partitions.

Reported by Petr Gusev

Closes #14179
Fixes #15748

(cherry picked from commit 4592bbe)
denesb pushed a commit that referenced this issue Oct 18, 2023
@asias @denesb
repair: Use the updated estimated_partitions to create writer
23f9fdf 
The estimated_partitions is estimated after the repair_meta is created.

Currently, the default estimated_partitions was used to create the
write which is not correct.

To fix, use the updated estimated_partitions.

Reported by Petr Gusev

Closes #14179
Fixes #15748

(cherry picked from commit 4592bbe)
@denesb
Copy link
Contributor

denesb commented Oct 18, 2023

Backport queued to 5.2 and 5.1.

@denesb denesb removed Backport candidate backport/5.2 Issues that should be backported to 5.2 branch once they'll be fixed Requires-Backport-to-5.1 labels Oct 18, 2023
@mykaul mykaul added this to the 5.4 milestone Oct 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@asias asias

Labels
area/repair-based operations repare based node operations area/sstable
Projects
None yet
Milestone
5.4
Development

No branches or pull requests
4 participants
@asias @denesb @raphaelsc @mykaul