Add wait/delay between invocations for heavily rate limited downstream dependencies

[ryancormack]

When downstream dependencies are heavily rate limited, there's no easy way to natively 'throttle' the power tuner to accommodate.

One existing potential solution would be to have a Lambda function that essentially does a Thread.Sleep(time) to add a delay (and of course not run the tuner in parallel).

Given that involves paying for Lambda and a State Transition, could it be possible to add an additional parameter of sleepBetweenRunsMs or something similar to indicate a sleep period between power tuner runs?

There are some downsides to this approach though. It would involve a state transition for every run, even if the wait/delay time was set to 0. A possible approach to this is to have a Choice state at the start that either does the current branch work flow, or does a 'new' branch/workflow that is all essentially a clone of the current flow but now has a Wait task as part of the map/loop. This could look something like:

This approach would only add a single additional state transition, which whilst not ideal as every user would incur that cost, it may be an acceptable addition for the benefit of an easy to configure/use Wait/Delay between each invocation.

Maybe this has been discussed before and was rejected because of the ability to add a function that does a naïve sleep?

[alexcasalboni]

Hi @ryancormack 👋 thanks for sharing 🙏

I like the idea of a configurable way to sleep X milliseconds between each invocation.

I wouldn't implement it as an actual Wait state in the Step Functions definition. Imho, there are easier ways to achieve the same without making the overall state machine more complex (and expensive) for everyone.

In the past, I've done this by implementing the Thread.sleep(...) logic directly in the function I was power-tuning. The function would sleep only if a special input parameter was given in the payload (something obvious like event.sleepMsForPowertuning.

Alternatively, we could implement an optional sleepBetweenRunsMs input parameter for the state machine, like you suggested. This way, you don't have to implement custom logic in your Lambda functions, across multiple runtimes, etc.

From the point of view of the "usability", as a user you shouldn't care much if it's implemented as a Wait state or as a Thread.sleep(...) in the Executor step. You need to pay for both (there is probably a way to compare how many ms of sleep correspond to a Step Function wait state in terms of $$$). Personally, I'd prefer the second option because it keeps the state machine simple (visually).

Also, each Executor invokes your Function num times, so we actually need to sleep multiple times within the Executor invocation. A Wait state wouldn't be enough.

---

If we move forward with the second option, the implementation would be fairly simple. We could add a couple of sleeps here and here:

...
if (sleepBetweenRunsMs) {
    await sleep(sleepBetweenRunsMs);
}
...

where sleep looks like this:

function sleep(ms) {
  return new Promise((resolve) => {
    setTimeout(resolve, ms);
  });
}

Do you feel like opening a pull-request (PR) for this? Otherwise, I could look into it next week.

[ryancormack]

Hi Alex,

Thanks for the fast reply.

Your suggestion is much nicer, I think I just started down a rabbit hole and totally looked over the fact there is code invoking the function rather than a Step Function integration invoking it.

I'm happy to go look and pick this proposed solution up 👍🏻