Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix startup on existing data folder issues #1245

Merged
merged 6 commits into from Jul 27, 2023

Conversation

afranken
Copy link
Member

@afranken afranken commented Jul 25, 2023

Description

  • Always delete directories if retainFilesOnExit==false
  • Do not exclude buckets called "test"
  • Lock bucket when loading its objects
  • Log # of objects loaded for each bucket.

Related Issue

Fixes #1224

Tasks

  • I have signed the CLA.
  • I have written tests and verified that they fail without my change.

Not sure where this bit of code came from, but if a bucket was named
"test", S3Mock would have ignored its existence when starting...

Fixes #1224
Not sure this can ever happen, but if S3Mock is loading a bucket, we
should not allow concurrent modifications on it.

Fixes #1224
This makes it easier to see if there were errors, and if the correct
# of objects were loaded when starting S3Mock on an existing data
folder.

Fixes #1224
@afranken afranken self-assigned this Jul 26, 2023
@afranken afranken added the bug label Jul 26, 2023
The data folder should only be used by S3Mock, so this should not
happen. Still, guard against the possibility that somehow, folders
without a "bucketMetadata.json" (=folders that are not a bucket) are
present in the data folder and skip them while loading buckets on
startup.

Fixes #1224
@afranken afranken merged commit f40c20c into main Jul 27, 2023
4 checks passed
@afranken afranken deleted the 1224-startup-on-existing-data-directory-fixes branch July 27, 2023 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Random unrecoverable "bucketMetadata" is null error
1 participant