[Pulsar Scheduler] Don't Schedule Jobs When Executors Are Lagging Behind

[d80tb7]

┆Issue is synchronized with this Jira Task by Unito

[JamesMurkin]

@d80tb7

Is this good to review? I made a minor change to the unit test as it was failing - but looks like it was just a copy paste error

[codecov]

Codecov Report

Patch coverage: 71.42% and project coverage change: +0.01 🎉

Comparison is base (b405422) 58.97% compared to head (ccbeb2c) 58.98%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2308      +/-   ##
==========================================
+ Coverage   58.97%   58.98%   +0.01%     
==========================================
  Files         225      225              
  Lines       28470    28498      +28     
==========================================
+ Hits        16790    16810      +20     
- Misses      10388    10394       +6     
- Partials     1292     1294       +2     

Flag	Coverage Δ
armada-server	58.98% <71.42%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
internal/scheduler/scheduling_algo.go	67.22% <71.42%> (+0.43%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[d80tb7]

@JamesMurkin yup I think this one is god to review

[JamesMurkin]

Possibly we should re-adjust numbers in config.

We allow scheduling 5000 per round, but only allow 2500 lag - which can easily happen in a single round

Possibly maximumJobsToSchedule should be 1000 instead