Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Policy to rate-limit action executions #3720
According to https://docs.stackstorm.com/reference/policies.html, we have
And while concurrency is useful, it's not enough in some cases.
The proposal is to implement rate-limiting policy so we could limit the action executions based on time
Proposed policy example
--- name: rate-limit-http-downloads description: > Run max 3 'core.http' actions/second and cancel everything that exceeds the limit enabled: true resource_ref: core.http # alternative: # policy_type: action.limit policy_type: action.rate_limit parameters: # maximum number of executions per time interval window threshold: 3 # 100ms, 1s, 315s, 15m, 1h interval: 1s # cancel actions exceeding the 3 actions/s limit action: cancel
We'll need additionally to filter/limit the executions by input arguments, same as https://docs.stackstorm.com/reference/policies.html#action-concurrency-attr
I believe this is typical and pragmatic instrument that might be very popular in a wild.
+1 I've fielded several questions on Slack regard this feature request. Most of the use cases revolved around monitoring and auto-remediation. Example, a user wants to try and remedy a problem of a disk filling up, however the alert is flapping and injecting events into StackStorm at a high rate. In this case the programmer only wanted to react to the event if it was unique within the last X seconds. I'm probably over-simplifying the explanation here, but the general idea is the same.
I agree that something like that is needed, but that's actually quite a complex problem and a solution is non-trivial.
Imo, the best solution for that probably is event aggregation and triggering rules based on the aggregated events.
@lakshmi-kannan and I discussed something like that recently and we don't really see a simple solution. Right now events are stateless and introducing support for aggregate events would make them stateful which introduces many unique challenges (now we need to persist a count / time based window of events in memory and in some persistent storage, etc.).
Having said that, that's quite a common problem in the monitoring world (only alert on state changes and aggregate alerts) so we should draw some inspiration from there.
In fact, @lakshmi-kannan and I worked on Rackspace Monitoring product in the past which aimed to solve those problems. There we used a project called Esper which handled windowing and event-correlation for us.
We should also look into open-source projects which we can leverage for windowing and event-correlation because implementing that ourselves correctly is non-trivial in a distributed system.
@safarsi Are we talking about implementation as StackStorm policy within st2 core? https://docs.stackstorm.com/reference/policies.html
If yes, we'd be happy to accept a pull request.