Skip to content

v0.7.5 — fix infinite hang archiving CSS-heavy sites

Choose a tag to compare

@Xafloc Xafloc released this 04 Jun 17:10

Fixed

  • Archiving CSS-heavy sites hung forever. inline_css_resources() used a re-scan loop with a literal str.replace to inline CSS url(...) and @import references. The regex captured URLs without their surrounding quotes, so for the common url("...") form the replace never matched — the reference stayed in the CSS, the loop's termination flag was reset on every pass, and it spun on the CPU indefinitely. (Any url() that failed to fetch caused the same non-termination.)

    In practice this hung archiving of large sites such as foxnews.com: the archive never completed, no file was written, and nothing was logged after "Found +link". Sites with little or no CSS (e.g. example.com) were unaffected, which is why the bug was intermittent.

    The function now uses re.sub with callbacks — each reference is rewritten exactly once (quotes included) with no re-scan loop — plus a recursion-depth guard for circular @imports. Archiving such sites now terminates at the archive deadline (~30s) and writes a partial archive instead of hanging.

Builds on v0.7.4 (archiving runs off the event loop) and v0.7.3 (archive_ssl_verify toggle).