Ram loss in vfs when accessing rump fs

[Cochimetl]

When reading many large files from a rump ext2fs fs, vfs demands more RAM.
It seems the used RAM increases in large portions (~2M stepts) in repos/dde_rump/src/lib/vfs/rump/vfs_rump.cc line 134 in a rump_sys_pread call.

This was observed under 20.08.

[Cochimetl]

Quadrupled amount of files read, needed RAM never went above 16M despite having needed 12M beforehand. So it's probably ok?

[chelmuth]

@ssumpf @cnuke do you have any thoughts about this?

[ssumpf]

@Cochimetl: The back end allocator in repos/dde_rump/src/include/util/allocator_fap.h will allocate 2MB chunks on demand (which are super pages on x86_64) until VM_SIZE or RAM quota is reached in case it runs out of memory. VM_SIZE is 256MB on 32 bit and 4GB on 64 bit systems.

[nfeske]

I wonder, what is the state of this issue? Is it resolved by @ssumpf's comment, or is there the need for further steps?

I think that the stated upper limit of 256 MiB is rather high. E.g., the limit is certainly not anticipated by Sculpt OS, which assigns merely 64 MiB to the file-system server. On the other hand, I haven't run into the reported issue myself. It would be really nice to have an artificial work load to reproduce the greedy behavior.

[cnuke]

@nfeske in Sculpt the amount of memory rump is supposed to consume is limited by setting the ram attribute to 48 MiB and it should stay below this limit. The VM_SIZE is the theoretical maximum rump can consume if not limited. So if rump_set_memlimit, which is used to enfource this limit, works as intended we should be fine.

[nfeske]

@cnuke Thanks for dispelling my worries. ;-)

This leaves only the question of how to go about this issue?

@Cochimetl I wonder, are you using the ram attribute mentioned by @cnuke already?

[Cochimetl]

Ram attribute was used and set to 7M. This fits the observed behaviour (three increases of 2M).
Thanks for the help and the explanations.