Calculation of freePhysicalMemorySize does not currently take disk cache into account

[robsyme]

Linux can sometimes utilise unused RAM as disk cache. This RAM is freed immediately if any other application requests it. At the moment, Nextflow calculates how much RAM is available by calling

ManagementFactory.getOperatingSystemMXBean().freePhysicalMemorySize

Unfortunately, this freePhysicalMemorySize calculation by the OperatingSystemMXBean includes RAM being used for disk cache, even though the OS is very happy to give this RAM up at a moment's notice. The result is that a system that has recently moved a bunch of files into cache will not accept new jobs, thinking that there is no ram available.

It would be great if the freeMemory calculation also included RAM being used for disk space, as it really is available to nextflow for jobs.

A quick-and-dirty workaround is to manually empty the cache. As root:

# sync && echo 3 > /proc/sys/vm/drop_caches