Accomodate special case : Title attribute missing from rss #17
Conversation
|
Hi, thanks for your contribution! The reason why there's no checks in that title, description and link fields are required fields by RSS specification |
|
Oh yes definitely! It would help parse non-standard rss files like this one. While it seems to be standard in the most part, the first item was making my program crash when it had to parse it because of the lack of title given. |
Nice! Would you want to create a separate PR or force-push this one to update the logic and add this feature? |
|
I think I will update this PR when I'll have time. 👍 |
|
@BBArikL Have you gotten around to this? Otherwise I could implement this the coming week. |
|
Hello! I have been quite busy with life since last time. I should be way more free after this thursday and I'll try update the branch somewhere in the next weekend. Thank you for reminding me of it, I'll add a reminder so I do not forget :) |
…e additional entries.
|
@dhvcc @Thewildweb Here is the commit that I promised! I added some whitespaces to help the code review and added a functionality to have additional entries to the RSS scraped. Let me know what you think! |
|
To add a field, call parse() with entries equal to a list of fields the scraper should look for. Then you can retrieve the information (let's say for the first item) by callying: And now the variable |
|
Hi @BBArikL @Thewildweb |
rss_parser/_parser.py
Outdated
| "publish_date": getattr(item.pubDate, "text", ""), | ||
| "category": getattr(item.category, "text", ""), | ||
| "description": description_soup.text, | ||
| "title": getattr(getattr(item, "title", ""), "text", ""), |
There was a problem hiding this comment.
May be we need to to something about those double getattrs. Perhaps move them into a separate function?
There was a problem hiding this comment.
Right, I'll do that!
rss_parser/_parser.py
Outdated
| default: str, | ||
| item_dict: Optional[str] = None, | ||
| default_dict: Optional[str] = None, | ||
| item: object, |
There was a problem hiding this comment.
I think black uses 4 spaces instead of 8, let's try to not cause any extra diffs
There was a problem hiding this comment.
Oh I did not, see that I put more spaces, I will fix this in the next commit.
|
@BBArikL please fix linting issues |
|
That last commit should fix the linting issues 😅 . First time working with strict code checkers. |
When parsing a rss file, it might be possible for the item to not have any title property. I added a check for the title to be sure it does not crash any user's application. When there is no title found in the rss, the "title" property is changed to a empty string.