Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
LSF: Add unit scaling other than MB #274
Dask Jobqueue assumes that the units are MB, which is true under
This PR adds the ability for users to both specify or allow auto
Auto-detection routine is based on the LSF 9.1.3 docs for where to
The `lsf.conf` file can specify units other than MB for memory, and any other large units of scaling, through the `LSF_UNIT_FOR_LIMITS` variable. Dask Jobqueue assumes that the units are MB, which is true under the default setup of LSF, but not always. * If the admin has changed it to something else in the file. * If the file does not exist, LSF assumes KB. This PR adds the ability for users to both specify or allow auto detection of the unit system through an `lsf_units` kwarg or `lsf-units` YAML arg. Auto-detection routine is based on the LSF 9.1.3 docs for where to look, defaults, and variable names.
For reference, here are the official LSF docs on the mater
guillaumeeb left a comment
Okay thanks for the work here !
This looks really good, the only thing that is missing is actual tests. Could you provide a test with some given unit, and a test for
I made some changes to include the LSF 10.1+ way of doing things, you can apparently specify everything from KB up to ZB, and even then in single letter short-hand. I think there may be an issue with the
For reference, here is the 10.1 docs:
I made the following changes as well as requested: