Use unique DB directory when TEST_TMPDIR is set #14249
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Some of the stress tests script run tests multiple times with TEST_TMPDIR set. When TEST_TMPDIR is set, the db directory is a fixed string. This caused the same DB directory was reused across db_crashtest.py script run. Each db_crashtest.py run would randomize some of the parameters. This caused different parameters to be used with same DB directory, violating some of the assumption such as use_put_entity_one_in parameter. This change added a suffix to DB directory, so that each db_crashtest.py script run would generate a unique DB directory, which prevents the issue from happening again. Test Plan: Stress test Reviewers: Subscribers: Tasks: Tags:
|
@xingbowang has imported this pull request. If you are a Meta employee, you can view this in D91069655. |
hx235
approved these changes
Jan 21, 2026
Contributor
hx235
left a comment
hx235
left a comment
There was a problem hiding this comment.
Good finding—this improvement makes the stress test more resilient to directory-deletion errors.
|
@xingbowang merged this pull request in eb5e1a2. |
pdillinger
added a commit
to pdillinger/rocksdb
that referenced
this pull request
Jan 21, 2026
Summary: Clearing DB dir for crash test is currently a hodgepodge of 1. Caller of db_crashtest.py maybe tries to clear the dir 2. db_crashtest.py tries to clear the dir in get_dbname() (but ignoring failure) 3. db_crashtest.py passes --destroy_db_initially to some db_stress calls as needed 4. db_crashtest.py tries to clear the dir between some db_stress calls 5. db_crashtest.py tries to clear the dir after everything is done and successful (no artifacts to investigate or save) (but ignoring failure) 6. Try to add more uniqueness to the directory from facebook#14249 This change reverts or replaces 2, 4, 5, and 6 by doubling-down on (expanding) 3 and a small variant of it: * crash_test.mk passes --destroy_db_initially=1 so that the first run of db_stress clears the db dir. * After each db_stress invocation, db_crashtest.py resets destroy_db_initially=0 so that the next invocation reuses the same DB, except in cases where there is an incompatibility that requires a fresh DB (from cases 3 and 4 above). * On success, uses new `db_stress --destroy_db_and_exit` option to clean up the DB dir without needing a custom cleanup_cmd (now ignored) Note that although case 1 is likely obsolete, it is out of control of an open source PR. Test Plan: some manual runs
meta-codesync bot
pushed a commit
that referenced
this pull request
Jan 22, 2026
Summary: Clearing DB dir for crash test is currently a hodgepodge of 1. Caller of db_crashtest.py maybe tries to clear the dir 2. db_crashtest.py tries to clear the dir in get_dbname() (but ignoring failure) 3. db_crashtest.py passes --destroy_db_initially to some db_stress calls as needed 4. db_crashtest.py tries to clear the dir between some db_stress calls 5. db_crashtest.py tries to clear the dir after everything is done and successful (no artifacts to investigate or save) (but ignoring failure) 6. Try to add more uniqueness to the directory from #14249 This change reverts or replaces 2, 4, 5, and 6 by doubling-down on (expanding) 3 and a small variant of it: * crash_test.mk passes --destroy_db_initially=1 so that the first run of db_stress clears the db dir. * After each db_stress invocation, db_crashtest.py resets destroy_db_initially=0 so that the next invocation reuses the same DB, except in cases where there is an incompatibility that requires a fresh DB (from cases 3 and 4 above). * On success, uses new `db_stress --destroy_db_and_exit` option to clean up the DB dir without needing a custom cleanup_cmd (now ignored) Note that although case 1 is likely obsolete, it is out of control of an open source PR. Pull Request resolved: #14254 Test Plan: some manual runs Reviewed By: xingbowang Differential Revision: D91164731 Pulled By: pdillinger fbshipit-source-id: 0a66c8c0e130c9eeacc55af411a18a09bc9debdf
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Some of the stress tests script run tests multiple times with TEST_TMPDIR set. When TEST_TMPDIR is set, the db directory is a fixed string. This caused the same DB directory was reused across db_crashtest.py script run. Each db_crashtest.py run would randomize some of the parameters. This caused different parameters to be used with same DB directory, violating some of the assumption such as use_put_entity_one_in parameter. This change added a suffix to DB directory, so that each db_crashtest.py script run would generate a unique DB directory, which prevents the issue from happening again.
Test Plan:
Stress test local run
Reviewers:
Subscribers:
Tasks:
Tags: