[LIB-820] add time.sleep in some places to avoid throttling when migrating the data;

[opalczynski]

No description provided.

[23doors]

6? Why when we have only rate limit per second?

[opalczynski]

Hmm, because the batch make 10 creates/updates 60/10 = 6 - or am I bad at math? :)

[23doors]

But we only have limits per second. So every second it's a fresh limit. So time.sleep(1) should always "refresh" it. And this way it will be only possible to run into rate limit when we use more than 60 calls before sleeping

[opalczynski]

Oh, get it :)

[23doors]

Maybe we could add some support for Retry-After in library instead? When we send a request we could check if code == 429, get header value for Retry-After, parse it as float and sleep for the time it specifies and then retry. This way even if we change to some burst throttles (per minute window), it'll still work. And less obnoxious errors for users ;)

[opalczynski]

@23doors it's nice! I will create a task for that and handle it in next or next next sprint.

[opalczynski]

@23doors I will merge that for now and in few weeks when add retry to python LIB - will change the logic here. Is this ok?

[23doors]

Sure! fine with me :)