Skip to content

Royale High API v2.0.1-beta

Choose a tag to compare

View all tags
@ProjectAnndromeda ProjectAnndromeda released this 24 Jul 14:51
· 5 commits to main since this release
2.0.1-beta
30d191d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Royale High API v2.0.1-beta

Major Version

Changelog

[2.0.1-beta] - 2024-07-24

Overview

Made returning data over 3 times faster. Migrated from Flask to Quart and replaced Selenium with Playwright.

Breaking Changes

  • Framework Change:

    • Migrated from Flask to Quart. Quart is an asynchronous web framework based on Flask, which requires updates to routing, request handling, and middleware usage.

  • Browser Automation Library:

    • Replaced Selenium with Playwright for browser automation and web scraping. This change impacts how browser interactions are scripted and managed, including async operations and timeout handling.

Features

  • Asynchronous Support:

    • Updated codebase to leverage asynchronous operations using Quart’s async/await syntax and Playwright’s async API, improving concurrency and responsiveness.

  • Improved Web Scraping:

    • Enhanced web scraping with Playwright, which supports better handling of modern web applications and dynamic content.

  • Dynamic Request Handling:

    • Implemented dynamic request management in scraping. Requests are now staggered with a delay, allowing up to a maximum of three concurrent requests, improving efficiency and reducing server load.

Fixes and Improvements

  • Timeout and Retry Logic:

    • Improved timeout and retry logic for fetching web pages. Increased timeouts and implemented more robust retry mechanisms to handle network or page loading issues.

  • Error Handling:

    • Enhanced error handling for scenarios where pages fail to load or where no items are found. Added checks for specific no-results messages to stop further scraping when necessary.

  • Logging and Debugging:

    • Improved logging to provide detailed information on page requests, errors, and the number of items found. This helps in debugging and monitoring the scraping process.

Dependencies

  • Updated Dependencies:
    • Switched from Selenium WebDriver to Playwright for browser automation.
    • Updated web framework dependency from Flask to Quart.

Migration Notes

  • Code Updates Required:
    • Update route handling and request processing to be compatible with Quart’s asynchronous model.
    • Refactor browser automation scripts to use Playwright’s API instead of Selenium’s WebDriver API.
    • Ensure all async functions are properly awaited and handle exceptions appropriately in the new async environment.
Assets 2
Loading