Report all nodes in lease request

[JamesMurkin]

We now send all nodes in the cluster in the lease call

We used to only send nodes that could currently be scheduled on however that caused issues:

• Gang jobs were not getting pre-empted if they were on a node that was excluded for reasons such as being cordoned
• The armada-scheduler expects all jobs being run to be returned each lease call. The jobs are on the node object, so if we excluded nodes that were cordoned, the armada-scheduler assumed the job has been lost

We now set Unschedulable to mark if the executor thinks jobs can be scheduled on this node or not

┆Issue is synchronized with this Jira Task by Unito

[codecov]

Codecov Report

Patch coverage: 22.22% and no project coverage change.

Comparison is base (cbcbd5d) 58.49% compared to head (f17fc7f) 58.49%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2401   +/-   ##
=======================================
  Coverage   58.49%   58.49%           
=======================================
  Files         234      234           
  Lines       29296    29296           
=======================================
  Hits        17136    17136           
  Misses      10844    10844           
  Partials     1316     1316           

Flag	Coverage Δ
armada-server	58.49% <22.22%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ternal/executor/utilisation/cluster_utilisation.go	41.07% <18.75%> (ø)
internal/executor/node/node_group.go	87.87% <50.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.