Skip to content

docs: add example resume a crawl #1143

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Apr 10, 2025
Merged

docs: add example resume a crawl #1143

merged 8 commits into from
Apr 10, 2025

Conversation

@Mantisus
Copy link
Collaborator

@Mantisus Mantisus commented Apr 8, 2025

Description

  • add example resume a crawl

Issues

@Mantisus Mantisus requested a review from Copilot April 8, 2025 01:17
Copilot AI reviewed Apr 8, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 2 out of 3 changed files in this pull request and generated no comments.

Files not reviewed (1)
  • docs/examples/resume_crawl.mdx: Language not supported

@Mantisus Mantisus requested review from janbuchar and vdusek April 8, 2025 01:18
vdusek
vdusek requested changes Apr 8, 2025
View reviewed changes
Copy link
Collaborator

@vdusek vdusek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we can demonstrate this better:

  • Do not use enqueue_links, instead start with a predefined set of URLs.
  • Remove unnecessary calls and comments, like the export_data_json, to reduce the noise.
  • During the first crawl, stop the run manually using CTRL+C.
  • Execute a second run, let it finish, and compare the statistics (logs) of both runs.

Mantisus and others added 2 commits April 8, 2025 16:01
janbuchar
janbuchar reviewed Apr 8, 2025
View reviewed changes
@Mantisus Mantisus self-assigned this Apr 8, 2025
@Mantisus Mantisus requested a review from vdusek April 9, 2025 01:38
janbuchar
janbuchar reviewed Apr 9, 2025
View reviewed changes
janbuchar
janbuchar reviewed Apr 9, 2025
View reviewed changes
Mantisus and others added 2 commits April 9, 2025 13:51
vdusek
vdusek approved these changes Apr 10, 2025
View reviewed changes
Copy link
Collaborator

@vdusek vdusek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vdusek vdusek merged commit 40edf70 into apify:master Apr 10, 2025
23 checks passed
@vdusek vdusek added the t-tooling Issues with this label are in the ownership of the tooling team. label Apr 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@janbuchar janbuchar janbuchar approved these changes

@vdusek vdusek vdusek approved these changes

Assignees

@Mantisus Mantisus

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Explain how to resume a crawl

3 participants

@Mantisus @janbuchar @vdusek