Skip to content

CrawlerX 1.3.0

Choose a tag to compare

@github-actions github-actions released this 04 Oct 17:59
· 295 commits to develop since this release
c601eaf

Added

  • Add the lockfile-derived browser sidecar and isolated fetch lab (PR #50).
  • Add a fetch-strategy contract with the new error values not_found, gone, rate_limited, timeout, challenge, network, and auth_required, plus retryability metadata (PR #51).
  • Add benchmark and live-canary tooling (PR #52).
  • Reuse Chromium across browser fetch requests (PR #55).
  • Add shared sessions, FlareSolverr cookie replay, and LoginCookieProvider (PR #56).

Changed

  • Require jooservices/client ^4.4; redirect cookies are enabled by default and stay separate from the CrawlerX session store and explicit cookie handoff (PR #59).
  • Enable fetch-lab TC-M08, make Compose host ports configurable or disableable, and make CaptainHook work in Git worktrees (PR #58).

Fixed

  • Apply configured ClientBuilder options in ClientFactory (PR #49).
  • Refresh Jable performer fixtures/readiness (PR #53) and the 141Jav live detail sample (PR #54).
  • Refresh Aisex fixtures and update the pagination expectation to page 1214 (PR #57).

Security

  • Block browser requests to private, loopback, link-local, CGNAT, unique-local IPv6, unspecified, multicast, and metadata addresses, including redirects. Support an optional hostname allowlist; private-network access is limited to the fetch lab (PR #60). Rejected requests return non-retryable ssrf_blocked.

Upgrading from 1.2

Applications that switch on CrawlErrorCode must handle the new values: not_found, gone, rate_limited, timeout, challenge, network, auth_required, and ssrf_blocked. jvmeta currently maps unknown codes to parse_drift until its mapping adopts these cases. rate_limited, timeout, challenge, and network are retryable by default; the other new values are terminal by default.