[RFC] Ideas to reduce memory usage

[rfjakob]

I am looking at the memory usage of a gocryptfs mount right now, containing 531300 inodes. After an ls -R, we are at 470MB RSS, with all of the inodes cached by the kernel. EncFS uses 92MB RSS for the same workload.

Looking at the memory profile, most of the memory is used in

• new(Inode): 48MB
• Inode.children: 65MB (46.5+18.8)
• Inode.parents: 115MB (17.5+97.52)

One could argue that the problem solves itself once the kernel starts evicting inodes from the cache, but still, there might be room for improvement.

For future reference, I'm pasting the most interesting part of the profile below and attaching the full memory profile and gocryptfs binary.

(pprof) top
311.35MB of 313.85MB total (99.20%)
Dropped 84 nodes (cum <= 1.57MB)
Showing top 10 nodes out of 37 (cum >= 22.02MB)
      flat  flat%   sum%        cum   cum%
   99.52MB 31.71% 31.71%   143.33MB 45.67%  runtime.mapassign
   64.04MB 20.40% 52.11%    64.04MB 20.40%  runtime.makemap
   48.01MB 15.30% 67.41%   112.04MB 35.70%  github.com/hanwen/go-fuse/fuse/nodefs.newInode
   43.31MB 13.80% 81.21%    43.31MB 13.80%  runtime.hashGrow
   21.50MB  6.85% 88.06%   247.37MB 78.82%  github.com/hanwen/go-fuse/fuse/pathfs.(*pathInode).createChild
      21MB  6.69% 94.75%       21MB  6.69%  runtime.rawstringtmp
       6MB  1.91% 96.66%       35MB 11.15%  github.com/hanwen/go-fuse/fuse/pathfs.(*pathInode).setClientInode
    4.44MB  1.41% 98.08%     4.44MB  1.41%  github.com/hanwen/go-fuse/fuse/nodefs.(*portableHandleMap).Register
    3.01MB  0.96% 99.04%     3.01MB  0.96%  github.com/rfjakob/gocryptfs/internal/contentenc.newBPool.func1
    0.52MB  0.17% 99.20%    22.02MB  7.02%  github.com/rfjakob/gocryptfs/internal/fusefrontend_reverse.(*ReverseFS).OpenDir

(pprof) list newInode
Total: 313.85MB
ROUTINE ======================== github.com/hanwen/go-fuse/fuse/nodefs.newInode in /home/jakob/go/src/github.com/hanwen/go-fuse/fuse/nodefs/inode.go
   48.01MB   112.04MB (flat, cum) 35.70% of Total
         .          .     48:	// NodeFileSystem.
         .          .     49:	mountPoint *fileSystemMount
         .          .     50:}
         .          .     51:
         .          .     52:func newInode(isDir bool, fsNode Node) *Inode {
   48.01MB    48.01MB     53:	me := new(Inode)
         .    17.50MB     54:	me.parents = make(map[parentData]struct{}, 1)
         .          .     55:	if isDir {
         .    46.54MB     56:		me.children = make(map[string]*Inode, initDirSize)
         .          .     57:	}
         .          .     58:	me.fsInode = fsNode
         .          .     59:	me.fsInode.SetInode(me)
         .          .     60:	return me
         .          .     61:}

(pprof) list addChild
Total: 313.85MB
ROUTINE ======================== github.com/hanwen/go-fuse/fuse/nodefs.(*Inode).addChild in /home/jakob/go/src/github.com/hanwen/go-fuse/fuse/nodefs/inode.go
         0   114.33MB (flat, cum) 36.43% of Total
         .          .    197:		ch := n.children[name]
         .          .    198:		if ch != nil {
         .          .    199:			log.Panicf("Already have an Inode with same name: %v: %v", name, ch)
         .          .    200:		}
         .          .    201:	}
         .    16.81MB    202:	n.children[name] = child
         .    97.52MB    203:	child.parents[parentData{n, name}] = struct{}{}
         .          .    204:	if w, ok := child.Node().(TreeWatcher); ok && child.mountPoint == nil {
         .          .    205:		w.OnAdd(n, name)
         .          .    206:	}
         .          .    207:}
         .          .    208:

2.memprofile.tar.gz

[rfjakob]

Good enough. Not a single user complaint about memory usage.

[Slartibartfast27]

I have major problems with RSS usage using the ubuntu 20.04 LTS version of armhf:
gocryptfs 1.7.1; go-fuse 0.0~git20190214.58dcd77; 2019-12-26 go1.13.5 linux/arm
see #569