Limit in-memory use for inodes #58
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This matches what ext4 does and saves some space. Signed-off-by: Alexander Larsson <alexl@redhat.com>
This avoids the module being unloaded while the fs is mounted. Signed-off-by: Alexander Larsson <alexl@redhat.com>
Signed-off-by: Alexander Larsson <alexl@redhat.com>
Signed-off-by: Alexander Larsson <alexl@redhat.com>
alexlarsson
force-pushed
the
dentry-size
branch
from
d2d2bf0
to
e9afa89
Compare
|
Wth is up with armv7: Surely the printf type |
This changes the way a directory inode stores the dentry list. Instead of a list of all dentries followed by all the filenames we store a list of chunks, where each chunk is at most 4k of such dentries and names. Additionally we have a header in the beginning with the size and dentry count of all such chunks. On the reader side we don't read (and alloc) all the dentries with the inode, instead we read the blocks as needed, meaning we use a lot less memory per in-memory inode. We preload the chunk headers for typical size dirs (4 chunks max) as a cheap way to avoid having to re-read it. However for large dirs we read the table as needed. Since we don't read all dentries when creating the inode we now also use the st_nlink from the file rather than computing it from the dentries. Signed-off-by: Alexander Larsson <alexl@redhat.com>
This means we're not using potentially unbounded kernel memory for the inode for the xattrs. I think in practice we're not going to see such large xattrs anyway, they are mainly used to store things like ACLs, file caps or selinux contexts. Signed-off-by: Alexander Larsson <alexl@redhat.com>
alexlarsson
force-pushed
the
dentry-size
branch
from
e9afa89
to
0fbff50
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This limits the xattr data size to 4k, and splits the dentry data into chunks of (max) 4k which we read in chunks as needed. This drops the in-memory requirement for inodes a lot.
@rhvgoyal does this take care of your issues wrt memory use?
Note: For memory use I will also look at re-using xattr chunks in memory between inodes as they are already shared on disk. Should be rather easy.