Autoscale should only scale down if no tasks were accepted during keepalive period

[naktinis]

Checklist

• I have checked the issues list
  for similar or identical enhancement to an existing feature.
• I have checked the pull requests list
  for existing proposed enhancements.
• I have checked the commit log
  to find out if the same enhancement was already implemented in the
  main branch.
• I have included all related issues and possible duplicate issues in this issue
  (If there are none, check this box anyway).

Related Issues and Possible Duplicates

Related Issues

• None

Possible Duplicates

• None

Brief Summary

Currently it scales down if there was no scale up in the keepalive period. However, I propose it should not scale down if any tasks has been received in the keepalive period.

I'll illustrate this with an example. Let's say we've just finished processing 100 tasks (so there's a relatively high load for an extended period of time). Let's say the last scaling up happened an hour ago.

Currently there's a maybe_scale periodic check set up to run once every keepalive seconds so it happens sometimes that it runs just after a task finished running. So let's say maybe_scale runs after our 100th task just finished. It sees that last scale up happened an hour ago and there are no tasks queued up right now (which it may just be a couple of seconds without tasks) and it scales down immediately.

Now a second later another burst of tasks comes in and we have to scale up again. So basically we just had a few seconds without any tasks in a relatively high load period and it already scaled down, so now we have to initialize the worker again.

It should not have scaled down just because of a second or two of inactivity. Rather, it should wait for at least keepalive seconds of inactivity before scaling down.

Design

Architectural Considerations

None

Proposed Behavior

Autoscaler.scale_down should also check if monotonic() - self._last_task_received > self.keepalive. There's no self._last_task_received at the moment, but we can register another on_task_message which memorizes the latest task message time with self._last_task_received = monotonic().

If this change makes sense, I can produce a PR.

Proposed UI/UX

Diagrams

N/A

Alternatives

None