Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'handle_rate_limit' incorrect behaviour #300

Open
pep4eto1211 opened this issue Jul 27, 2022 · 1 comment
Open

'handle_rate_limit' incorrect behaviour #300

pep4eto1211 opened this issue Jul 27, 2022 · 1 comment

Comments

@pep4eto1211
Copy link

Description:

When creating an instance of Client (found in twitter_ads.client) you can pass a dict with additional options, using the options parameter. One of this options is handle_rate_limit. When passed, the library is supposed to look for the rate limit headers returned by the API and sleep the execution when such a limit is encountered. However, it would appear that the wait time is insufficient, as the next request after the wait period fails with a rate limit error.

How to replicate:

client = Client(**config, options={'handle_rate_limit': True})
...
for chunk_ids in split_list(ids, 20):
    analytics_data: List[dict] = Campaign.all_stats(
        account,
        chunk_ids,
        [METRIC_GROUP.ENGAGEMENT, METRIC_GROUP.BILLING],
        granularity=GRANULARITY.TOTAL,
        start_time=export_range.start_date,
        end_time=export_range.end_date
    )

The above code is ran for an export range of one day. It is repeated for multiple accounts. This is the output when a rate limit is reached:

time=2022-07-26 17:22:50 name=twitter_ads.http level=WARNING message=Request reached Rate Limit: resume in 592 seconds
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "*********/twitter-ads-env/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "*********/twitter-ads-env/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "*********/twitter-ads-env/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "*********/twitter-ads-env/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "*********/utils/backfill.py", line 110, in run_backfill
    for entity_data in results:
  File "*********/utils/backfill.py", line 74, in _download_data
    analytics_data: List[dict] = Campaign.all_stats(
  File "*********/twitter-ads-env/lib/python3.8/site-packages/twitter_ads/analytics.py", line 78, in all_stats
    response = Request(account.client, 'get', resource, params=params).perform()
  File "*********/twitter-ads-env/lib/python3.8/site-packages/twitter_ads/http.py", line 70, in perform
    raise Error.from_response(response)
twitter_ads.error.RateLimit: <RateLimit object at 0x7f2b79a75520 code=429 details=None>

Expected behaviour:

The library should wait the appropriate amount of time before attempting to call the API again to completely avoid RateLimit errors.

Observed behavior:

At the current state, the handle_rate_limit option serves no purpose. Despite waiting a certain amount of time, the rate limit error is encountered anyway.

@tushdante
Copy link
Collaborator

@pep4eto1211 i believe this is this related to #298. can you confirm?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants