Replies: 1 comment
|
Thanks for starting this discussion, @rdsubhas! I think from a broad view this idea is interesting and worth considering. There are some interesting details we'd need to think about, though:
I also note that the use-case you identified for this was rate limits but the solution you've proposed is instead concurrency limits. Limiting concurrency does often limit request rate to some extent by reducing throughput, but if rate limiting is the primary motivation then perhaps we should aim for a feature that directly limits request rate, rather than trying to fake it with a concurrency limit. I think (the questions I raised above notwithstanding) the general idea you proposed could still apply to rate limits, though: instead of specifying a number of concurrent requests per group, the configuration could specify the rate limit directly. For example: request_limits {
members = [
aws_instance.example, # a resource in the same module as the request limit
module.example.aws_instance.other, # a resource in a child module
# ...and perhaps also some way to say "all aws_instance everywhere
# beneath this module", and other similar broad rules.
]
# Perform only up to 300 requests every five minutes
period = "5m"
request_limit = 300
# (some systems unfortunately express their rate limits in
# terms of fixed time periods like calendar days instead of
# in rolling totals, so perhaps we'd need some other options
# here to describe that situation too.
# Could also support concurrency limits to use instead of
# or alongside request rate limits.
concurrency_limit = 2
}Hypothetically then, with a configuration like the above OpenTofu would check as it begins working with each resource instance which request limit groups it belongs to and make sure all of those groups are ready to absorb the request. If any one of the limiting groups the resource instance belongs to is currently constrained then perhaps OpenTofu would emit a short message mentioning that the request is being delayed and then just wait until the request group becomes ready again. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Problem
Different providers have different rate limits and throttling. This becomes an excruciating pain when you have one "low-limit" provider sandwiched between many high-limit ones.
The global
-concurrencyflag is a rather blunt instrument to deal with this.Outsourcing this to providers brings it's own set of challenges: Handling shared and in-flight rate limits, etc. But mainly, it makes it impossible for tofu users to model cross-provider rate limits, because in the end, one provider cannot influence another providers.
Proposal:
concurrency_keyattribute inlifecycleblockWhen a
concurrency_keyis supplied, tofu ensures that resource operations within the sameconcurrency_keyare sequential.Behaviorally Compatible
This merely enforces the equivalent of the current
-concurrency=1behavior, within operations of the sameconcurrency_key, without any behavioral changes to the dependency graph or ordering or any other functionality.No changes are required to the provider ecosystem. It works transparently with all providers, and does not change anything at the provider contract or SDK level.
The attribute itself is optional. Not specifying it changes nothing.
Further Improvements
Initially, tofu would apply effectively
concurrency=1for resource operations with sameconcurrency_key. But in future, users could perform fine-grained control by extending the-concurrencyCLI flag, e.g.-concurrency=10 -concurrency:<concurrency-key>=5Conclusion
This feature would offer both tofu users and provider implementors relief:
-concurrencyglobal flagComment or upvote if you like this proposal ❤️
All reactions