Skip to content

Added support for scraping multiple urls

Choose a tag to compare

View all tags
@ryan-blunden ryan-blunden released this 20 May 05:04
0.2
7b60c3a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

0.2 Release

This release adds support for scraping multiple urls as well as minor code improvements and new code checks.

Updates

  • Updated the fetch_page_content function to require a list of URLs instead of a single URL.
  • Added type hints and type checking withmypy
  • Added ruff for linting and format checking.
  • Improved error handling in the fetch_page_content function.
  • Replaced explicit timeout waiting with Playwright's wait_for_selector.
  • Updated usage instructions in the README.md
  • Changed default timeout to 10 seconds

Fixes

  • Fixed an issue in the README.md file where the PyPI package name was incorrect.
Assets 2
Loading