Scale the dbuf cache with arc_c instead of arc_c_max #6561
Conversation
|
I had a quick look at how I'm afraid I don't have the bandwidth for that at this time ...especially as it's now beers o'clock on a Friday! 🍻 |
No need to rush, but merging this without a test case feels wrong: this issue has been flying under the radar for quite some time (~1 year) because we were missing a test case: i think we need to add it. |
|
@chrisrd The numbers in this part of the commit comment don't make sense:
With 1GB of RAM, the default Since it seems the newly-introduced dbuf cache is the cause of the problem in #6505, I have to wonder if a better solution would be to register a shrinker for the dbuf cache. Or does the whole "filesystem shrinker" schem rely in some way on there being only a single shrinker? |
|
@loli10K whilst I think it would be beneficial to start adding @tuxoko I'm not sure exactly what value might be added to @dweezil you're right of course! My commit message was confusing ARC and metadata min/max sizes. And I mis-quoted "GB" instead of "MB" as well. Sigh. I'll update the pull request with that part of the commit message changed to: We could potentially do something about explicitly shrinking the dbuf cache by adding something into What I think would be a nice addition would be a way of monitoring the effectiveness of the dbuf cache, e.g. hit/miss ratio, so you can see if changing the dbuf cache size makes a significant difference to a specific workload. But I can't really see where to instrument the hit/miss ratio. If anyone has any idea, feel free to point it out for me! Or even better, submit a patch. |
Yeah, that should do it.
That should be fine, the shrinker shouldn't normally free all the stuff at once, but we should return a more accurate value so it will shrink with a more accurate proportion. Though, we might need to see if dbuf_cache will free fast enough. |
|
@chrisrd could you please rebase this against master. I do think we need something like this, and probably some additional adjustments too. I'd like to do some local testing with this patch. |
|
@behlendorf Rebased |
Codecov Report
@@ Coverage Diff @@
## master #6561 +/- ##
===========================================
- Coverage 73.57% 60.02% -13.55%
===========================================
Files 294 294
Lines 93863 93867 +4
Branches 0 28198 +28198
===========================================
- Hits 69059 56344 -12715
+ Misses 24804 24569 -235
- Partials 0 12954 +12954
Continue to review full report at Codecov.
|
|
Repushed to see if we can clear the error on buildbot/Ubuntu 14.04 i686 |
Commit d3c2ae1 introduced a dbuf cache with a default size of the minimum of 100M or 1/32 maximum ARC size. (These figures may be adjusted using dbuf_cache_max_bytes and dbuf_cache_max_shift.) The dbuf cache is counted as metadata for the purposes of ARC size calculations. On a 1GB box the ARC maximum size defaults to c_max 493M which gives a dbuf cache default minimum size of 15.4M, and the ARC metadata defaults to minimum 16M. I.e. the dbuf cache is an significant proportion of the minimum metadata size. With other overheads involved this actually means the ARC metadata doesn't get down to the minimum. This patch dynamically scales the dbuf cache to the target ARC size instead of statically scaling it to the maximum ARC size. (The scale is still set by dbuf_cache_max_shift and the maximum size is still fixed by dbuf_cache_max_bytes.) Using the target ARC size rather than the current ARC size is done to help the ARC reach the target rather than simply focusing on the current size. Signed-off-by: Chris Dunlop <chris@onthe.net.au>
There was a problem hiding this comment.
This makes good sense and works as expected in my local testing. I'll get it merged.
@tuxoko's recommendation to include the dbuf_cache_size in the arc_evictable_memory() is a good idea but let's do it in a followup PR.
Commit d3c2ae1 introduced a dbuf cache with a default size of the minimum of 100M or 1/32 maximum ARC size. (These figures may be adjusted using dbuf_cache_max_bytes and dbuf_cache_max_shift.) The dbuf cache is counted as metadata for the purposes of ARC size calculations. On a 1GB box the ARC maximum size defaults to c_max 493M which gives a dbuf cache default minimum size of 15.4M, and the ARC metadata defaults to minimum 16M. I.e. the dbuf cache is an significant proportion of the minimum metadata size. With other overheads involved this actually means the ARC metadata doesn't get down to the minimum. This patch dynamically scales the dbuf cache to the target ARC size instead of statically scaling it to the maximum ARC size. (The scale is still set by dbuf_cache_max_shift and the maximum size is still fixed by dbuf_cache_max_bytes.) Using the target ARC size rather than the current ARC size is done to help the ARC reach the target rather than simply focusing on the current size. Reviewed-by: Chunwei Chen <tuxoko@gmail.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: George Melikov <mail@gmelikov.ru> Signed-off-by: Chris Dunlop <chris@onthe.net.au> Issue openzfs#6506 Closes openzfs#6561
Commit d3c2ae1 introduced a dbuf cache with a default size of the minimum of 100M or 1/32 maximum ARC size. (These figures may be adjusted using dbuf_cache_max_bytes and dbuf_cache_max_shift.) The dbuf cache is counted as metadata for the purposes of ARC size calculations. On a 1GB box the ARC maximum size defaults to c_max 493M which gives a dbuf cache default minimum size of 15.4M, and the ARC metadata defaults to minimum 16M. I.e. the dbuf cache is an significant proportion of the minimum metadata size. With other overheads involved this actually means the ARC metadata doesn't get down to the minimum. This patch dynamically scales the dbuf cache to the target ARC size instead of statically scaling it to the maximum ARC size. (The scale is still set by dbuf_cache_max_shift and the maximum size is still fixed by dbuf_cache_max_bytes.) Using the target ARC size rather than the current ARC size is done to help the ARC reach the target rather than simply focusing on the current size. Reviewed-by: Chunwei Chen <tuxoko@gmail.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: George Melikov <mail@gmelikov.ru> Signed-off-by: Chris Dunlop <chris@onthe.net.au> Issue #6506 Closes #6561
Description
Scale the size of the dbuf cache according to the ARC target size rather than use a static size based on the maximum ARC size.
Motivation and Context
Commit d3c2ae1 introduced a dbuf cache with a default size of the minimum of 100M or 1/32 maximum ARC size. (These figures may be adjusted using dbuf_cache_max_bytes and dbuf_cache_max_shift.)
On a 1GB box, the ARC defaults to maximum 370GB and minimum 16G, and the dbuf cache size comes out at 12G. I.e. the dbuf cache is an significant proportion of the minimum ARC size. With other overheads involved this actually means the ARC can't get down to the minimum.
This patch dynamically scales the dbuf cache to the target ARC size instead of statically scaling it to the maximum ARC size. (The scale is still set by dbuf_cache_max_shift and the maximum size is still fixed by dbuf_cache_max_bytes.) Using the target ARC size rather than the current ARC size is done to help the ARC reach the target rather than simply focusing on the current size.
Relates to #6506
How Has This Been Tested?
Tests per #6506:
Various
arcstats:Types of changes
Checklist:
Signed-off-by.