Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Reddit cog occassionally fails to relay new /r/python messages #390

Open
SebastiaanZ opened this issue Aug 5, 2019 · 0 comments

Comments

@SebastiaanZ
Copy link
Member

commented Aug 5, 2019

Occasionally, the Reddit cog stops relaying new messages to the #reddit channel. From what I've seen, there are two ways in which the cog breaks and both should be easy to fix.

1. The background task fails on unexpected http responses

The background task to fetch new posts, the self.new_posts_task created in the on_ready event listener, assumes that the response we get from the Reddit API is valid json in this line. If, for some temporary reason, we don't get a json response, but a text/html response (404, probably), the task fails with the following exception:

In [160]: bot.get_cog("Reddit").new_posts_task
Out[160]: 
  File "/bot/bot/cogs/eval.py", line 167, in _eval
    res = await func()
  File "<string>", line 8, in func
  File "/bot/bot/cogs/eval.py", line 167, in _eval
    res = await func()
  File "<string>", line 8, in func
  File "/bot/bot/cogs/reddit.py", line 142, in poll_new_posts
    posts = await self.fetch_posts(f"{subreddit}/new")
  File "/bot/bot/cogs/reddit.py", line 55, in fetch_posts
    content = await response.json()
  File "/bot/.venv/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 938, in json
    headers=self.headers)
aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8'

This line is in the fetch_posts utility function used by both the new_posts_task as well as the top_weekly_posts_task. This means that can break because of this error.

The likely solution is to either handle the exception or check the response status code before trying to parse it as json. We probably want to either have some kind of retry logic in the handling of the non-200 response, since otherwise the weekly top posts task skips a week.

2. The channel can't be initialized during on_ready

Similar to the issue we had with the watch channels, it can occur that the bot's internal channel cache isn't fully loaded when the on_ready event fires. This means that the channel will not be retrieved and the tasks will never be started. This only happens intermittently, but it does occasionally happen.

In such cases, the self.bot.get_channel returns None, because the channel has not been loaded into the internal bot cache yet at that point.

    async def on_ready(self):
        self.reddit_channel = self.bot.get_channel(Channels.reddit)

While this could be fixed by adding a slight delay or a bit of retry logic, another option is to wait for the migration to a later version of discord.py and use await self.bot.fetch_channel here instead. That will fetch the channel from the API, bypassing any issues we may have with the internal bot channel cache.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.