Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Secondary cache error injection #9002

Closed
wants to merge 5 commits into from

Conversation

anand1976
Copy link
Contributor

Implement secondary cache error injection in db_stress.

@@ -0,0 +1,110 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe just use "secondary" instead of "sec" will be easier to remember and search

const Cache::CacheItemHelper* helper) {
ErrorContext* ctx = GetErrorContext();
if (ctx->rand.OneIn(prob_)) {
return Status::IOError();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we return Corrupted? Or IOError is better. I'm not sure what we expect from the implementation layer of secondary cache. We may also need to consider to unify the error code returned in secondary cache since it uses the storage and IOError might make sense in some cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All insert errors are currently treated the same by RocksDB, so it doesn't matter much what the error is. We can always add more here if the implementation changes.

SecondaryCache* cache_;
};

class FaultInjectionSecondaryCache : public SecondaryCache {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe some comments about the general behavior of the fault injection and how it will be used for testing.

}

void FaultInjectionSecondaryCache::ResultHandle::WaitAll(
FaultInjectionSecondaryCache* cache,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not inject error in WaitAll?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do inject error in WaitAll. Its done in FaultInjectionSecondaryCache::ResultHandle::UpdateHandleValue.

@facebook-github-bot
Copy link
Contributor

@anand1976 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@@ -841,6 +841,9 @@ DEFINE_int32(open_metadata_write_fault_one_in, 0,
#ifndef ROCKSDB_LITE
DEFINE_string(secondary_cache_uri, "",
"Full URI for creating a customized secondary cache object");
DEFINE_int32(secondary_cache_fault_one_in, 0,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't use DEFINE_bool ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its the probability, not a flag.

Copy link
Contributor

@zhichao-cao zhichao-cao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, some testing failure need to be addressed.

@facebook-github-bot
Copy link
Contributor

@anand1976 has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@anand1976 has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@anand1976 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@anand1976 has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot
Copy link
Contributor

@anand1976 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@anand1976 merged this pull request in 78556c1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants