Skip to content

v2.2.0: Fix for Large Authenticated Archives

Choose a tag to compare

View all tags
@agude agude released this 03 Sep 03:22
· 92 commits to main since this release
v2.2.0
60aae9f

Release Notes:

This release addresses a critical bug that caused the authenticated archiving mode (SPN2) to fail for large websites. The previous logic could lead to an infinite loop when archiving many pages, which has now been resolved with a more robust processing strategy.

What's New

  • Reliable Large Archives: Fixed a race condition where the status of an archive job could expire before the script checked it. The script now interleaves submitting new URLs and polling for the status of pending jobs, ensuring that large archives complete successfully in authenticated mode. (Closes #28)
  • Automatic Submission Retries: The script is now more resilient to temporary network or API errors. If submitting a URL fails, it will be automatically re-queued and retried up to three times.
  • Gentler API Polling: The status polling logic now includes a small delay between each check, making the script a better citizen of the Internet Archive's API infrastructure.
Assets 2
Loading