Skip to content

Fix StaleElementReferenceException in Crawler#2591

Merged
bogdankostic merged 1 commit intomasterfrom
fix_crawler_stale_element
May 24, 2022
Merged

Fix StaleElementReferenceException in Crawler#2591
bogdankostic merged 1 commit intomasterfrom
fix_crawler_stale_element

Conversation

@bogdankostic
Copy link
Contributor

When trying to crawl for example sub-pages of the following page (https://taz.de/!s=&OnlineRessort=5097/?search_page=0), we get the following error:

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document

This PR makes sure that we only try to crawl elements where the href attribute is available.

@bogdankostic bogdankostic marked this pull request as ready for review May 23, 2022 17:39
@bogdankostic bogdankostic requested a review from julian-risch May 23, 2022 17:39
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍 Just not forget to label even these small PRs with a tag. I added topic:preprocessing for this one.

@bogdankostic bogdankostic merged commit 1ab2b97 into master May 24, 2022
@bogdankostic bogdankostic deleted the fix_crawler_stale_element branch May 24, 2022 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants