keep more metaslabs loaded #9197
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
pcd1193182
added
Status: Code Review Needed
|
What would you think about taking this even further and including DLPX-65016? I think that would make sense assuming this lands after #9181. |
ahrens
reviewed
Aug 21, 2019
Codecov Report
@@ Coverage Diff @@
## master #9197 +/- ##
=========================================
- Coverage 79.22% 77.82% -1.4%
=========================================
Files 400 389 -11
Lines 122001 121530 -471
=========================================
- Hits 96656 94585 -2071
- Misses 25345 26945 +1600
Continue to review full report at Codecov.
|
|
@ahrens I'll fold in both of those changes as well. |
…etaslabs loaded Reviewed at: http://reviews.delphix.com/r/50808/
…val is cancelled with loaded metaslabs, assertion fails Reviewed at: http://reviews.delphix.com/r/50310/
behlendorf
reviewed
Aug 22, 2019
behlendorf
approved these changes
Aug 27, 2019
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
ahrens
approved these changes
Aug 28, 2019
| @@ -15,7 +15,7 @@ | |||
| # | |||
|
|
|||
| # | |||
| # Copyright (c) 2018 by Delphix. All rights reserved. | |||
| # Copyright (c) 2018, 2019 by Delphix. All rights reserved. | |||
There was a problem hiding this comment.
[nit] looks like you can revert this
I talked with Paul about this and he felt it would be fine to merge them in either order. So I suggest we go ahead with this one once the last nits have been resolved. |
behlendorf
added
Status: Accepted
allanjude
pushed a commit
to KlaraSystems/zfs
that referenced
this pull request
Apr 28, 2020
With the other metaslab changes loaded onto a system, we can significantly reduce the memory usage of each loaded metaslab and unload them on demand if there is memory pressure. However, none of those changes actually result in us keeping more metaslabs loaded. If we don't keep more metaslabs loaded, we will still have to wait for demand-loading to finish when no loaded metaslab can satisfy our allocation, which can cause ZIL performance issues. In addition, performance is traditionally measured by IOs per unit time, while unloading is currently done on a txg-count basis. Txgs can take a widely varying range of times, from tenths of a second to several seconds. This can result in confusing, hard to predict behavior. This change simply adds a time-based component to metaslab unloading. A metaslab will remain loaded for one minute and 8 txgs (by default) after it was last used, unless it is evicted due to memory pressure. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Matt Ahrens <mahrens@delphix.com> Signed-off-by: Paul Dagnelie <pcd@delphix.com> External-issue: DLPX-65016 External-issue: DLPX-65047 Closes openzfs#9197
12 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
With the other metaslab changes loaded onto a system, we can significantly reduce the memory usage of each loaded metaslab and unload them on demand if there is memory pressure. However, none of those changes actually result in us keeping more metaslabs loaded. If we don't keep more metaslabs loaded, we will still have to wait for demand-loading to finish when no loaded metaslab can satisfy our allocation, which can cause ZIL performance issues. In addition, performance is traditionally measured by IOs per unit time, while unloading is currently done on a txg-count basis. Txgs can take a widely varying range of times, from tenths of a second to several seconds. This can result in confusing, hard to predict behavior.
Description
This change simply adds a time-based component to metaslab unloading. A metaslab will remain loaded for one minute and 8 txgs (by default) after it was last used, unless it is evicted due to memory pressure.
How Has This Been Tested?
zfs-test and zloop, and performance testing to verify no regressions.
Types of changes
Checklist:
Signed-off-by.