Python News implemention#899
Merged
Merged
Conversation
Added general content of cog: class and setup.
Function sync maillists listing with API, that hold IDs of message that have news. PEPs handling is over RSS, so this will added manually in this function.
`News.get_webhook` fetch discord.Webhook by ID provided in config. `self.webhook` use webhook that it got from this function.
…nnel and webhook. This use local dev environment IDs.
Removed Webhook and Channel from their listings, created new class `PythonNews` that hold them + mail lists.
Replaced in-file mail lists with constants.py's, replaced webhook ID getting.
- Created task `post_pep_news` that pull existing news message IDs from API, do checks and send new PEP when it's not already sent. - Removed `get_webhook` - Removed `self.webhook`
…ook_names` Function fetch display names of these mail lists, that bot will post. These names will be used on Webhook author names. `News.webhook_names` storage these name and display name pairs.
…date check - Created helper function `News.get_thread_and_first_mail` - Created helper function `News.send_webhook` - Created helper function `News.check_new_exist` - Task `post_maillist_news`, that send latest maillist threads to news, when they don't exist. - Implemented helper functions to PEP news - Added date check
- Created new helper function `News.get_webhook_and_channel` to will be run in Cog loading and will fetch #python-news channel and webhook. - Fixed `News.send_webhook` when you pass `None` as author, this will not add author. - Replaced individual channel and webhook fetches with `News.webhook` and `News.channel`. - Replaced positional arguments with kwargs in `send_webhook` uses. - Moved maillists syncing from `News.__init__` to `News.post_maillist_news`. - Simplified `News.post_pep_news` already exist checks.
kwzrd
added
a: dependencies
jb3
requested changes
Apr 26, 2020
Member
jb3
left a comment
jb3
left a comment
There was a problem hiding this comment.
Thank you for the contribution ks, I've left a couple of proposals on improvements to the feature.
- Added footer to webhook sent message - Made `send_webhook` return `discord.Message` instead ID of message - Added waiting for Webhook on `send_webhook` - Added message publishing in new loops
Added `features="lxml"` to `BeautifulSoup` class creating to avoid warning.
jb3
reviewed
Apr 27, 2020
jb3
reviewed
Apr 27, 2020
Co-Authored-By: Joseph <joseph@josephbanks.me>
jb3
requested changes
Apr 30, 2020
Member
jb3
left a comment
jb3
left a comment
There was a problem hiding this comment.
I've got some concerns on how well this will scale. If we have 100 posts in the channel (not an unachievable number) and there are 10 new ones without any caches in the bot (not unlikely since we restart a lot) we will be making 1,000 API calls. A better way to do this is not use Discord for this and instead store the ID of the thread, using the thread_id property of the mailman API (see below) and compare against that.
- Removed (now) unnecessary helper function `News.check_new_exist`. - Use thread IDs instead message IDs on maillists checking to avoid Discord API calls. - Use PEP number instead message IDs on PEP news checking to avoid Discord API calls.
jb3
reviewed
May 1, 2020
In `News` cog PEP news posting, define `utf-8` as encoding on response parsing to avoid the error. Co-authored-by: Joseph Banks <joseph@josephbanks.me>
jb3
reviewed
May 3, 2020
jb3
requested changes
May 3, 2020
Member
jb3
left a comment
jb3
left a comment
There was a problem hiding this comment.
Getting very close with this one now, looking almost ready!
Co-authored-by: Joseph Banks <joseph@josephbanks.me>
jb3
approved these changes
May 6, 2020
Member
jb3
left a comment
jb3
left a comment
There was a problem hiding this comment.
This looks good to me! Seems we've handled most of the oddities provided by the PEP URL and the mailing list API!
SebastiaanZ
suggested changes
May 6, 2020
SebastiaanZ
suggested changes
May 6, 2020
Co-authored-by: Sebastiaan Zeeff <sebastiaan.zeeff@gmail.com>
jb3
requested changes
May 7, 2020
Member
jb3
left a comment
jb3
left a comment
There was a problem hiding this comment.
Not everything has been updated to use the new filename
Co-authored-by: Joseph Banks <joseph@josephbanks.me>
jb3
approved these changes
May 7, 2020
SebastiaanZ
approved these changes
May 13, 2020
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #890
Site issue (stalled until it merged): python-discord/site#352
Summary of what is done
feedparser(used in PEP news fetching) andbeautifulsoup4(used in maillist news fetching)chardetto warning to avoid log spam.bot.cogs.newsextension loading added to__main__.pyNewscogsync_maillists,get_webhook_names,check_new_exist,send_webhook,get_thread_and_first_mailandget_webhook_and_channel.