-
-
Notifications
You must be signed in to change notification settings - Fork 60
Add --failOnFailedSeed checkbox to URL list workflows #1236
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- Fixes title case
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice!
|
I think the intention of the If we also add a failed threshold setting, that could then be use for more fine-grained control, eg. with this setting, and 100 URLs, 99/100 pages crawled would be considered a failure. But, if we a a fail threshold, then that could be set to 5, and 95/100 would be a failure by 99/100 would be success. |
|
Nitpick and happy to move this topic to Discord, but could we move towards sentence case for long checkbox labels? Personally I find the long checkbox labels in title case hard to read, which I do think is validated rather than only personal preference. I also remember seeing that it makes localization/translation more difficult down the road, though I can't find any resources to back that up. |
|
With webrecorder/browsertrix-crawler#402, the --failOnFailedSeed should also mark crawl as failed (needs a bit more testing). |
Co-authored-by: Ilya Kreymer <ikreymer@users.noreply.github.com>
Co-authored-by: Ilya Kreymer <ikreymer@users.noreply.github.com>
|
Tested with latest crawler, works as expected. |
Fixes #1180
Screenshot of option as presented in UI:
Checkbox does not appear for seeded crawls.
Crawls that fail this way currently show as "Partial Complete", and the fatal message is clearly visible in the Error Logs.