Slurm will not mark non responsive node as down, if node is within ResumeTimeout

[rexcsn]

Please see wiki page for details regarding this issue

Root cause of #2117

We will track any feedback/comments in this Github issue, thank you!

[demartinofra]

Here is the fix which should be included in Slurm 20.11 to address this issue: SchedMD/slurm@9140c2a

[keien]

will the next parallelcluster release have a version of slurm with the above fix?

[demartinofra]

Unfortunately Slurm 20.11 which contains the fix hasn't been released yet so we can't guarantee the very next release of ParallelCluster will have it included.

[demartinofra]

Slurm 20.11 got released but the following regression is preventing ParallelCluster from upgrading: https://bugs.schedmd.com/show_bug.cgi?id=10298

[tbugfinder]

looks like fixed:

Quote:

This is fixed in 20.11.1:

SchedMD/slurm@466c287

[demartinofra]

Upgrade to Slurm 20.11 has been merged in develop:

• Upgrade to Slurm 20.11.4 aws-parallelcluster-cookbook#871
• Use cloud_reg_addrs option and do not manually reset nodeaddr on power_down aws-parallelcluster-node#278
• Upgrade to Slurm 20.11.4 #2459

[rexcsn]

pcluster v2.11.0 is released, which contains slurm v20.11.7 that will fix the issue.
Resolving the issue now, please let us know if you have any question, thanks!