Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

A periodic RocksDB metadata consistency check #1248

Closed
mbautin opened this issue Apr 24, 2019 · 0 comments
Closed

A periodic RocksDB metadata consistency check #1248

mbautin opened this issue Apr 24, 2019 · 0 comments
Assignees
@zhaoalex
Labels
area/docdb YugabyteDB core features
Projects
YBase features Data integrity

Comments

@mbautin
Copy link
Collaborator

mbautin commented Apr 24, 2019

E.g. check that all files mentioned in the manifest are indeed present.
This is useful to guard against e.g. external file deletions.
@mbautin mbautin changed the title A periodic RocksDB consistency check A periodic RocksDB metadata consistency check Apr 24, 2019
@bmatican bmatican mentioned this issue Apr 25, 2019
Open
@bmatican bmatican added the area/docdb YugabyteDB core features label Feb 4, 2020
@bmatican bmatican added this to To Do in YBase features via automation Feb 4, 2020
@bmatican bmatican added this to To do in Data integrity via automation Feb 4, 2020
@bmatican bmatican moved this from Backlog to Short term in Data integrity Nov 10, 2020
@zhaoalex zhaoalex self-assigned this Mar 16, 2021
@zhaoalex zhaoalex mentioned this issue Apr 1, 2021
Open
zhaoalex added a commit that referenced this issue Apr 2, 2021
@zhaoalex
[#1248] ysql: Create background task for verifying tablet data integrity
aa58dab 
Summary:
We add an online recurring background task to verify the integrity of each tablet's
rocksdb data. RocksDB will catch corruption errors relating to missing sst files and
incorrect file sizes, and we can expand checks to cover block checksums in the future.
If any corruption is detected, we bump a new metric `tablet_data_corruptions`
and log a warning.

Test Plan:
New test suite `TabletDataIntegrityTest` testing no corruption, missing files,
and incorrect file sizes.

Reviewers: mbautin, bogdan, rahuldesirazu

Reviewed By: rahuldesirazu

Subscribers: zyu, ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D11031
@zhaoalex zhaoalex closed this as completed Apr 2, 2021
YBase features automation moved this from Backlog to Done Apr 2, 2021
Data integrity automation moved this from Short term to Done Apr 2, 2021
@karishmalalani karishmalalani mentioned this issue May 18, 2021
Open
This was referenced May 21, 2021
Open
Open
YintongMa pushed a commit to YintongMa/yugabyte-db that referenced this issue May 26, 2021
@zhaoalex @YintongMa
[yugabyte#1248] ysql: Create background task for verifying tablet dat…
1a23fed 
…a integrity

Summary:
We add an online recurring background task to verify the integrity of each tablet's
rocksdb data. RocksDB will catch corruption errors relating to missing sst files and
incorrect file sizes, and we can expand checks to cover block checksums in the future.
If any corruption is detected, we bump a new metric `tablet_data_corruptions`
and log a warning.

Test Plan:
New test suite `TabletDataIntegrityTest` testing no corruption, missing files,
and incorrect file sizes.

Reviewers: mbautin, bogdan, rahuldesirazu

Reviewed By: rahuldesirazu

Subscribers: zyu, ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D11031
@bmatican bmatican mentioned this issue Aug 4, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zhaoalex zhaoalex

Labels
area/docdb YugabyteDB core features
Projects
Data integrity
  
Done
YBase features
  
Done
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bmatican @mbautin @zhaoalex