Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BlobDB: is_fifo=true also evict non-TTL blob files #4049

Closed
wants to merge 3 commits into from

Conversation

yiwu-arbug
Copy link
Contributor

Summary:
Previously with is_fifo=true we only evict TTL file. Changing it to also evict non-TTL files from oldest to newest, after exhausted TTL files.

Test Plan:
updated blob_db_test

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yiwu-arbug has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Copy link
Contributor

@sagar0 sagar0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

@facebook-github-bot
Copy link
Contributor

@yiwu-arbug has updated the pull request. View: changes, changes since last import

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yiwu-arbug has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@@ -54,14 +54,7 @@ WalFilter::WalProcessingOption BlobReconcileWalFilter::LogRecordFound(

bool blobf_compare_ttl::operator()(const std::shared_ptr<BlobFile>& lhs,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This name needs to be updated now that we are no longer using ttl to compare.

Copy link
Contributor

@sagar0 sagar0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise, lgtm.

@facebook-github-bot
Copy link
Contributor

@yiwu-arbug has updated the pull request. View: changes, changes since last import

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yiwu-arbug has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@@ -373,7 +376,7 @@ class BlobDBImpl : public BlobDB {

// all the blob files which are currently being appended to based
// on variety of incoming TTL's
std::set<std::shared_ptr<BlobFile>, blobf_compare_ttl> open_ttl_files_;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost make a mistake.. The TTL version of comparator is also needed here. I'm keeping it now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, ok. Wow, at least updating the name to be consistent with the logic caught a sleeper-bug.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup.. not sure why tests not catching it.

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yiwu-arbug is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@yiwu-arbug yiwu-arbug deleted the blob_fifo branch July 18, 2018 00:16
rcane pushed a commit to rcane/rocksdb that referenced this pull request Sep 13, 2018
Summary:
Previously with is_fifo=true we only evict TTL file. Changing it to also evict non-TTL files from oldest to newest, after exhausted TTL files.
Closes facebook#4049

Differential Revision: D8604597

Pulled By: yiwu-arbug

fbshipit-source-id: bc4209ee27c1528ce4b72833e6f1e1bff80082c1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants