Add an argument to `--flex-alloc-nodes` option that will allow you to select only the nodes that are up

[vkarak]

Currently, the all argument selects all the nodes in the system partition that fulfil any additional constraints placed from the command line. We also have the STATE argument that allows you to pick up nodes in certain state. I will suggest to add an up argument that will automatically select only the nodes that are actually up.

[akesandgren]

--distribute=idle should also only pick up on nodes that are actually IDLE and only IDLE, i.e. nodes that are in IDLE+DRAINED or IDLE+whatever should not be taken into consideration.
I.e. the check should be for exactly what is specified, not just checking if it matches anywhere in the string for state.

[vkarak]

I.e. the check should be for exactly what is specified, not just checking if it matches anywhere in the string for state.

We don't do a string matching there, but rather treat the states given as a set, so IDLE+DRAINED means (for us) that the node is in two states {IDLE, DRAINED}. Therefore if you ask "give me the nodes that they are IDLE", it means that it will get that one, too. The right solution would be to extend the syntax and basically implement what you suggest: "give me the nodes that are exclusively in state X." I think the way Slurm reports the node state is a bit problematic; I guess it simply sets the DRAINED bit (leaving the IDLE bit unchanged) and the node state simply checks that bitmask and reports whatever it finds. The fact that DRAINED is stronger from IDLE is application logic. So the best would be to add some syntactic sugar to request nodes in a specific state only, e.g., --flex-alloc-nodes=%IDLE%.

[vkarak]

I hit that too with the --distribute option and it is indeed a bit annoying that we don't have a way to select the nodes that are exclusively in IDLE state.