Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/curvefs/client/change disk cache conf on fly #2728

Conversation

Cyber-SiKu
Copy link
Contributor

@Cyber-SiKu Cyber-SiKu commented Sep 1, 2023

What problem does this PR solve?

  1. Enables disk cache-related policies to be dynamically adjusted
  2. The strategy for updating and cleaning the cache is as follows
//       ok           nearfull               full
// |------------|-------------------|----------------------|
// 0     trimRatio*safeRatio    safeRatio               fullRatio
//
// 1. 0<=ok<trimRatio*safeRatio;
// 2. trimRatio*safeRatio<=nearfull<safeRatio
// 3. safeRatio<=full<=fullRatio
// If the status is ok or ok->nearfull does not clean up
// If the status is full or
// full->nearfull clean up

Issue Number: #xxx

Problem Summary:

What is changed and how it works?

What's Changed:

How it Works:

Side effects(Breaking backward compatibility? Performance regression?):

Check List

  • Relevant documentation/comments is changed or added
  • I acknowledge that all my contributions will be made under the project's license

@Cyber-SiKu
Copy link
Contributor Author

cicheck

@@ -245,9 +245,10 @@ diskCache.trimCheckIntervalSec=5
# the interval of check to trim load file to s3
diskCache.asyncLoadPeriodMs=5
# start trim file when disk cache use ratio is Greater than fullRatio,
# util less than safeRatio
# util less than safeRatio * trimeRatio / 100
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trimeRatio -> trimRatio
Is the time point of start trim reaching fullRatio or trimRatio? Fix code comments if reach trimRatio

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trimeRatio -> trimRatio Is the time point of start trim reaching fullRatio or trimRatio? Fix code comments if reach trimRatio

fullRatio start to trim

@wu-hanqing wu-hanqing self-requested a review September 1, 2023 03:21
curvefs/conf/client.conf Outdated Show resolved Hide resolved
<< ", usedBytes is: " << usedBytes
<< ", use ratio is: " << ratio;
<< ", usedBytes is: " << usedBytes << ", use ratio is: " << ratio
<< ", baseRatio is: " << baseRatio;
return false;
}

// TODO(wuhongsong):
// See Also: https://github.com/opencurve/curve/issues/1534
bool DiskCacheManager::IsExceedFileNums() {
uint64_t fileNums = cachedObjName_->Size();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation of Size() may have problem, access size_ should protected by lock.

curvefs/src/client/s3/disk_cache_manager.h Outdated Show resolved Hide resolved
curvefs/src/client/s3/disk_cache_manager.cpp Show resolved Hide resolved
curvefs/src/client/s3/disk_cache_manager.cpp Show resolved Hide resolved
@@ -53,6 +56,19 @@ DEFINE_validator(avgReadFileBytes, &pass_uint64);
DEFINE_uint64(avgReadFileIops, 0, "the read throttle iops of disk cache");
DEFINE_validator(avgReadFileIops, &pass_uint64);

DEFINE_uint32(diskNearFullRatio, 70, "the nearfull ratio of disk cache");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: nearful

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: nearful

fix

curvefs/src/client/s3/disk_cache_manager.cpp Outdated Show resolved Hide resolved
curvefs/src/client/s3/disk_cache_manager.cpp Show resolved Hide resolved
curvefs/src/client/s3/disk_cache_manager.cpp Outdated Show resolved Hide resolved
curvefs/src/client/s3/disk_cache_manager.cpp Outdated Show resolved Hide resolved
@Cyber-SiKu Cyber-SiKu force-pushed the feat/curvefs/client/change_diskCacheConf_onFly branch from 177837e to 14731c9 Compare September 4, 2023 06:42
@Cyber-SiKu Cyber-SiKu force-pushed the feat/curvefs/client/change_diskCacheConf_onFly branch 2 times, most recently from 62b1d54 to d5a9de4 Compare September 4, 2023 11:22
Signed-off-by: Cyber-SiKu <Cyber-SiKu@outlook.com>
@Cyber-SiKu Cyber-SiKu force-pushed the feat/curvefs/client/change_diskCacheConf_onFly branch from d5a9de4 to d5af8dc Compare September 4, 2023 11:42
@Cyber-SiKu
Copy link
Contributor Author

cicheck

1 similar comment
@Cyber-SiKu
Copy link
Contributor Author

cicheck

@Cyber-SiKu Cyber-SiKu merged commit b1eb512 into opencurve:master Sep 4, 2023
3 checks passed
@Cyber-SiKu Cyber-SiKu deleted the feat/curvefs/client/change_diskCacheConf_onFly branch September 4, 2023 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants