v2.0.0

Public proxy types are removed and renamed (see Removed and Changed), which is a breaking
change to the library API under SemVer.

Added

• Generalized retry for every backend (static, JS, headless), applied whether or not a proxy
  pool is configured. Transient failures — connection errors, timeouts, 429, and 5xx — are
  retried with exponential backoff and jitter.
• RetryOptions on CrawlerOptions.Retry (MaxRetries, BaseDelay, MaxDelay, JitterFactor,
  DelayOnRateLimit, AttemptTimeout). Headless backends default to MaxRetries = 1.
• Per-attempt timeout (RetryOptions.AttemptTimeout) that cancels and retries a stalled request.
• CLI flags --retries, --retryDelay, --maxRetryDelay, and --attemptTimeout.
• New SimpleCrawler.Core.Retry namespace: RetryExecutor, RetryClassifier, RetryReason,
  RetryAttempt<T>, RetryHandler.
• AbstractHeadlessCrawler<TPage, TResult> in SimpleCrawler.Core: a shared base for real-browser
  backends that owns the proxy-keyed page pool, the acquire/retry loop, and single-evaluation
  extraction. Backends supply four vendor primitives (NewPageAsync, NavigateAsync,
  ClosePageCore, EvaluateExtractorAsync) plus an optional AfterSuccessfulLoad hook.

Changed

• Proxy rotation is now the "alternative route" case of the shared retry loop: a multi-proxy pool
  rotates instantly (no delay); a single-proxy or absent pool backs off; 429 backs off even when
  another proxy is free (DelayOnRateLimit, default on).
• Breaking: IProxyPool.ReportFailure now takes RetryReason instead of ProxyFailureKind.
• Retry is installed at the HTTP layer for HttpClient-based backends, so robots.txt/sitemap
  probes and JS sub-resource fetches inherit it. The crawler HttpClient request timeout is left
  uncapped; RetryOptions.AttemptTimeout bounds each attempt instead.
• PlaywrightCrawler<TResult> and PuppeteerCrawler<TResult> now derive from
  AbstractHeadlessCrawler<IPage, TResult> instead of AbstractRobotsCrawler<IPage, IPage, TResult>,
  removing ~90% of the duplication between them. This is source-compatible: the new base still
  derives from AbstractRobotsCrawler<IPage, IPage, TResult>, no public member was removed, and
  subclasses still need only override GetResult. Pre-compiled third-party subclasses should be
  recompiled against the new base.

Removed

• Breaking: removed ProxyFailureKind, ProxyAttempt<T>, ProxyFailureClassifier,
  ProxyRetryExecutor, and ProxyRoutingHandler — superseded by the Retry namespace.
• Breaking: removed ProxyPoolOptions.MaxRetries; the retry budget moved to
  RetryOptions.MaxRetries.

Compatibility

• The smpcrawl CLI stays backward-compatible: --proxyRetries is retained as a hidden alias for
  --retries.