Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: adaptive playwright crawler #2316

Merged
merged 29 commits into from Feb 21, 2024
Merged

feat: adaptive playwright crawler #2316

merged 29 commits into from Feb 21, 2024

Commits on Feb 21, 2024

  1. WIP adaptive playwright crawler

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    5b5dff8 View commit details
    Browse the repository at this point in the history

  2. WIP Implement basic adaptive crawling flow

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    41dee0f View commit details
    Browse the repository at this point in the history

  3. Implement RequestHandlerResult methods

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    3797c28 View commit details
    Browse the repository at this point in the history

  4. Initial commitResult implementation

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    a22fd3c View commit details
    Browse the repository at this point in the history

  5. Remove imports from index.js

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    fc99b95 View commit details
    Browse the repository at this point in the history

  6. Export AdaptivePlaywrightCrawler

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    6a69bf0 View commit details
    Browse the repository at this point in the history

  7. Working prototype

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    3ad175e View commit details
    Browse the repository at this point in the history

  8. Better RequestHandlerResult implementation

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    e7ec8fe View commit details
    Browse the repository at this point in the history

  9. Less scientific notation

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    b90f188 View commit details
    Browse the repository at this point in the history

  10. Scope rendering type predictions by label

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    7adb0bd View commit details
    Browse the repository at this point in the history

  11. Export extractUrlsFromPage for use in playwright crawling

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    dcdea04 View commit details
    Browse the repository at this point in the history

  12. Improve RequestHandlerResult typings and doc and move it to the core … 

    …package
    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    a4bc015 View commit details
    Browse the repository at this point in the history

  13. Add missing dependencies to package.json

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    c6d79b3 View commit details
    Browse the repository at this point in the history

  14. Move RenderingTypePredictor to playwright utils

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    365bb6f View commit details
    Browse the repository at this point in the history

  15. Implement default result checker and comparator

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    1d87b5a View commit details
    Browse the repository at this point in the history

  16. Add querySelector context helper

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    fadeb56 View commit details
    Browse the repository at this point in the history

  17. Bug

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    b85107c View commit details
    Browse the repository at this point in the history

  18. Lint

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    6020f9c View commit details
    Browse the repository at this point in the history

  19. Rendering type detection test

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    435353b View commit details
    Browse the repository at this point in the history

  20. Test link enqueueing

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    4ea950e View commit details
    Browse the repository at this point in the history

  21. Extended tests

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    59de28d View commit details
    Browse the repository at this point in the history

  22. Remove portadom

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    f00083f View commit details
    Browse the repository at this point in the history

  23. Mark experimental things

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    e59f097 View commit details
    Browse the repository at this point in the history

  24. Remove unnecessary Promise.resolve

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    909e760 View commit details
    Browse the repository at this point in the history

  25. Formatting

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    c599c38 View commit details
    Browse the repository at this point in the history

  26. Add example usage

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    acb701b View commit details
    Browse the repository at this point in the history

  27. Track additional statistics

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    3636934 View commit details
    Browse the repository at this point in the history

  28. Do not accept handlePageFunction

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    0e5eacd View commit details
    Browse the repository at this point in the history

  29. yarn

    @janbuchar
    janbuchar committed Feb 21, 2024
    Copy the full SHA
    de31f1b View commit details
    Browse the repository at this point in the history