Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing retry_after parameter in rate limits #2544

Closed
Luc1412 opened this issue Jan 29, 2020 · 3 comments
Closed

Missing retry_after parameter in rate limits #2544

Luc1412 opened this issue Jan 29, 2020 · 3 comments
Labels

Comments

@Luc1412
Copy link

@Luc1412 Luc1412 commented Jan 29, 2020

Summary

Context:
I got a bot that broadcast messages (Game updates) to multiple thousand servers (who subscribed to it). I try to do this task as fast as possible. Normally I post it with multiple threads and discord.py auto handles the rate limits for me.

It seems that since 01/26 it sometimes happens that the retry_after parameter isn't included?
This only happens after approx. 4min of posting at max speed.

Reproduction Steps

I'm not sure. I think sending messages at max speed.

Expected Results

The API auto handles the missing retry_after parameter and might do a default timeout.

Actual Results

An error gets thrown:

    message = await channel.fetch_message(int(message_data['message_id']))
  File "/opt/EasyFortniteStats/bot_venv/lib/python3.8/site-packages/discord/abc.py", line 921, in fetch_message
    data = await self._state.http.get_message(channel.id, id)
  File "/opt/EasyFortniteStats/bot_venv/lib/python3.8/site-packages/discord/http.py", line 189, in request
    retry_after = data['retry_after'] / 1000.0
KeyError: 'retry_after'
NoneType: None

Checklist

  • I have searched the open issues for duplicates.
  • I have shown the entire traceback, if possible.
  • I have removed my token from display, if visible.

System Information

  • Python v3.8.0-final
  • discord.py v1.3.1-final
  • aiohttp v3.5.4
  • websockets v6.0
  • system info: Linux 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11)
@Luc1412 Luc1412 changed the title Missing retry_after parameter in rate limit messages Missing retry_after parameter in rate limits Jan 29, 2020
@Rapptz

This comment has been minimized.

Copy link
Owner

@Rapptz Rapptz commented Jan 30, 2020

I can't reproduce this.

Also I'm unsure why you're using multiple threads since this stuff isn't thread safe -- which might actually invalidate this bug report.

If you have DEBUG logging enabled to see the raw API response it'd be more helpful but right now I'm not sure this is even possible since that's a wide scope of breakage at the API layer.

@Rapptz Rapptz added the no repro label Jan 30, 2020
@Luc1412

This comment has been minimized.

Copy link
Author

@Luc1412 Luc1412 commented Jan 30, 2020

Also I'm unsure why you're using multiple threads since this stuff isn't thread safe -- which might actually invalidate this bug report.

My current code looks simplified like this:

recipients = [gid1, gid2, gid3, gid4]  # List of guild ids
recipients_chunks = someCustomMethodToChunk(recipients, 2)  # Create sublists with the defined number of entries eg. [[gid1, gid2], [gid3, gid4]]

broadcast_tasks = []
for recipients_chunks in recipients_chunks:
    broadcast_tasks.append(self.bot.loop.create_task(self.bc_chunk(premium_guild_chunk)))
for broadcast_task in broadcast_tasks:
    result = broadcast_task.result()
    # doing stuff with the result
print('finished')

async def _broadcast_chunk(self, chunk_recipients):
    for recipients in chunk_recipients:
        # doing some stuff and getting channel id from db
        await channel.send('My Message')

Is this wrong?

If you have DEBUG logging enabled to see the raw API response it'd be more helpful but right now I'm not sure this is even possible since that's a wide scope of breakage at the API layer.

For today's broadcast, I enable DEBUG to get more details.

Rapptz added a commit that referenced this issue Feb 2, 2020
At some point Cloudflare started returning some JSON. I don't know how
this JSON looks like.

See #2544
@Rapptz

This comment has been minimized.

Copy link
Owner

@Rapptz Rapptz commented Feb 5, 2020

It turns out that this is received when you do something that causes around 5000 429s in a short time frame from our testing in the d.py server. Cloudflare bans you for an hour with a message like this:

{
  "code": 0,
  "message": "You are being blocked from accessing our API temporarily due to exceeding our rate limits frequently. Please read our docs at https://discordapp.com/developers/docs/topics/rate-limits to prevent this moving forward."
}

A commit, 25a04ed, addressed this earlier. However, I'm unsure how you managed to get that many 429s since the library handles it. In order to reproduce it we had to not use the library but rather a new HTTP session with no 429 handling whatsoever.

@Rapptz Rapptz closed this Feb 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.