Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Throttling Support? #1425

Open
EliFinkelshteyn opened this Issue Apr 5, 2013 · 1 comment

Comments

Projects
None yet
2 participants

Since we run a lot of EMR jobs from different places, we wind up hitting throttling rate limit errors pretty frequently. To add support for those, I've added a little wrapper around boto that just sleeps instead of dying when throttling errors happen. It's pretty simple, and of the form:

while True:
    sleep_time = 10
    try:
        return self.conn.describe_jobflow(job_id).state
    except boto.exception.EmrResponseError:
        self.logger.info("Getting throttled. Sleeping for 10 secs.")
        time.sleep(10)
        sleep_time = sleep_time*2

I was thinking it would really be nice if there was a standard "throttle_safe" option for all boto operations. If I wrote this into the code and made a pull request, is that something you guys would be interested in merging?

Sounds like a good idea.

Would be nice to have a standard value and back-off multiplier which would work most of the time without even triggering Throttling which amazon could validate or at least publish some ballpark values.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment