Smooth the deletion of WAL files #5116
Conversation
| @@ -367,14 +367,16 @@ TEST_F(DBSSTTest, RateLimitedDelete) { | |||
| auto sfm = static_cast<SstFileManagerImpl*>(options.sst_file_manager.get()); | |||
| sfm->delete_scheduler()->SetMaxTrashDBRatio(1.1); | |||
|
|
|||
| WriteOptions wo; | |||
| wo.disableWAL = true; | |||
There was a problem hiding this comment.
If I understand correctly, you are disabling WAL here so that the test can pass, as the test is written to the previous specification of WAL files not going though the DeleteScheduler.
Instead of disabling WAL, why not fix the asserts in the test, to account for the WAL files as well? It might offer more robust test coverage.
There was a problem hiding this comment.
Its a little complicated as this test makes assumptions about the order of file deletion, which is broken if WAL is enabled. I added a new test specifically to test for WAL deletion by DeleteScheduler to ensure coverage
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:
anand1976
force-pushed
the
wal_sst_file_mgr
branch
from
e7d6038
to
f8bc607
Compare
There was a problem hiding this comment.
@anand1976 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
There was a problem hiding this comment.
@anand1976 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
There was a problem hiding this comment.
@anand1976 is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
|
@anand1976 merged this pull request in dae3b55. |
Summary: WAL files are currently not subject to deletion rate limiting by DeleteScheduler. If the size of the WAL files is significant, this can cause a high delete rate on SSDs that may affect other operations. To fix it, force WAL file deletions to go through the SstFileManager. Original PR for this is facebook#2768 Pull Request resolved: facebook#5116 Differential Revision: D14669437 Pulled By: anand1976 fbshipit-source-id: c5f62d0640cebaa1574de841a1d01e4ce2faadf0
WAL files are currently not subject to deletion rate limiting by DeleteScheduler. If the size of the WAL files is significant, this can cause a high delete rate on SSDs that may affect other operations. To fix it, force WAL file deletions to go through the SstFileManager. Original PR for this is #2768
Test:
New unit test in db_sst_test