LSF clusters have varying resource units -- toil needs to detect this and adapt accordingly

[mr-c]

Use of the correct units is needed at
https://github.com/BD2KGenomics/toil/blob/105aa24/src/toil/batchSystems/lsf.py#L38

mem = '' if mem is None else '-R "select[type==X86_64 && mem > ' + str(int(mem/ 1000000)) + '] rusage[mem=' + str(int(mem/ 1000000)) + ']" -M' + str(int(mem/ 1000000)) + '000'

For example -- the trailing + '000' needs to be left off when running on EBI's LSF clusters

[mr-c]

One way to discover the units is with the following command

grep LSF_UNIT_FOR_LIMITS ${LSF_ENVDIR:-/etc}/lsf.conf | awk -F= '{print $2}'

Could some other LSF users verify this and report back what they see?

For example:

[mcrusoe@ebi-cli-001 /]$ grep LSF_UNIT_FOR_LIMITS ${LSF_ENVDIR:-/etc}/lsf.conf | awk -F= '{print $2}'
MB

[mr-c]

One way to discover the units is with the following command

grep LSF_UNIT_FOR_LIMITS ${LSF_ENVDIR:-/etc}/lsf.conf | awk -F= '{print $2}'

Could some other LSF users verify this and report back what they see?

For example:

[mcrusoe@ebi-cli-001 /]$ grep LSF_UNIT_FOR_LIMITS ${LSF_ENVDIR:-/etc}/lsf.conf | awk -F= '{print $2}'
MB

[mr-c]

Here's the comprehensive solution from ipython-cluster-helper for determining varying the LSF unit size

https://github.com/roryk/ipython-cluster-helper/blob/2042f4c/cluster_helper/lsf.py#L94

[mr-c]

Here's the comprehensive solution from ipython-cluster-helper for determining varying the LSF unit size

https://github.com/roryk/ipython-cluster-helper/blob/2042f4c/cluster_helper/lsf.py#L94

[mr-c]

Other helpful LSF code at https://github.com/roryk/ipython-cluster-helper/blob/8a6a8a0/cluster_helper/cluster.py#L142

[mr-c]

Other helpful LSF code at https://github.com/roryk/ipython-cluster-helper/blob/8a6a8a0/cluster_helper/cluster.py#L142

[lordzappo]

grep LSF_UNIT_FOR_LIMITS ${LSF_ENVDIR:-/etc}/lsf.conf | awk -F= '{print $2}'
GB

[lordzappo]

grep LSF_UNIT_FOR_LIMITS ${LSF_ENVDIR:-/etc}/lsf.conf | awk -F= '{print $2}'
GB