Introduce Scheduler#load and Async::Idler for load shedding and saturation.

[ioquatix]

Introduce Scheduler#load which is a 1-second load average. This load average can be used to detect overload conditions in the event loop. In addition, introduce Async::Idler which will schedule tasks up to a given maximum_load.

Typical use cases include connection load shedding:

scheduler = Fiber.scheduler
while true
  server.wait_readable 
  if scheduler.load < 0.8
    handle_connection(server.accept)
  end
end

Request load shedding:

while request = client.read_request
  if scheduler.load > 0.95
    return too_many_requests #429
  end
  # ...handle request...

And load saturation:

idler = Async::Idler.new(0.95)
jobs.each do |job|
  idler.async do # schedule as many jobs concurrently up to 95% loading
    job.process
  end
end

While I'm confident the interface is good enough, there is a small chance that I've not anticipated some critical use case. We may introduce changes to this interface. Async::Idler is unlikely to change.

Fixes #277.

Types of Changes

• New feature.

Contribution

• I added tests for my changes.
• I tested my changes locally.
• I agree to the Developer's Certificate of Origin 1.1.