v0.3.0

What's Changed

• fix: detect current managed and interactive challenge pages by @luinor223 in #1

New Contributors

• @luinor223 made their first contribution in #1

Full Changelog: v0.2.0...v0.3.0

chromesolver v0.1.0

First release of github.com/luinor223/cloudfetch/chromesolver: a cloudfetch.Solver (requires cloudfetch v0.3.0) that clears Cloudflare Managed and Interactive Challenges in headless Chrome, clicking the Turnstile checkbox when one appears, and returns the cookies and identity headers Chrome sent.

go get github.com/luinor223/cloudfetch/chromesolver@v0.1.0

v0.2.0

This release moves cloudfetch to an API modelled on net/http. It has breaking changes; see the upgrade notes below.

go get github.com/luinor223/cloudfetch@v0.2.0

Breaking changes

• Do takes an *http.Request. Its context and headers apply to the whole call, including challenge solving, and req is not modified. DoContext is removed: use http.NewRequestWithContext instead.
• Post takes a content type, like net/http: Post(url, contentType, body).
• WithBrowser takes a Browser (Chrome, Firefox, Safari, Edge) instead of a utls.ClientHelloID. Each preset pins a TLS fingerprint and headers for the same version: Chrome 133, Firefox 120, Safari 16.0 and Edge 85.
• MaxChallengeRetries is replaced by WithMaxRetries(n). The default is 3; 0 disables solving.
• Unsolvable challenges return sentinel errors you can check with errors.Is: ErrManagedChallenge, ErrCaptcha and ErrChallengeFailed. Running out of retries now returns ErrChallengeFailed instead of the challenge page.
• Header precedence: headers set on the request now override WithHeader, which overrides the browser defaults.

Upgrading

// v0.1.0
resp, err := client.DoContext(ctx, "POST", url, body)
client := cloudfetch.New(cloudfetch.WithBrowser(utls.HelloFirefox_120))

// v0.2.0
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, url, body)
resp, err := client.Do(req)
client := cloudfetch.New(cloudfetch.WithBrowser(cloudfetch.Firefox))

New

• Client.CloseIdleConnections(), which http.Client.CloseIdleConnections() also reaches.
• Runnable programs in examples/.

Fixes

• The IUAM solver handles real Cloudflare pages. It is tested against challenge pages captured by cloudscraper, including pages with decoy inputs, setInterval and answers read from hidden elements.
• Concurrent first requests to one host no longer leak a connection, and HTTP/2 connections that get replaced now close once they are idle.
• Bare IPv6 hosts such as https://[::1]/ get the default port.
• Non-Cloudflare 403, 429 and 503 responses are no longer read into memory just to check for a challenge.

Full changelog: v0.1.0...v0.2.0

v0.1.0

First release of cloudfetch, a Go HTTP client that gets past Cloudflare bot detection by impersonating a real browser.

go get github.com/luinor223/cloudfetch@v0.1.0

Features

• Browser TLS fingerprints via uTLS: Chrome 133 (default), Firefox 120, Safari 16 and Edge 85, selected with WithBrowser
• Matching headers: User-Agent, Sec-CH-UA, Sec-Fetch-* and friends for the same browser and version as the TLS fingerprint
• IUAM challenge solver: solves the classic "Checking your browser" JS challenge with goja and retries automatically
• HTTP/2 with ALPN negotiation and connection reuse, falling back to HTTP/1.1
• Transparent decompression: brotli, gzip, deflate and zstd via resp.Text() / resp.Bytes()
• HTTP CONNECT proxies with optional basic auth (WithProxy("http://user:pass@host:port"))
• Cancellation: DoContext applies a context to every request and to challenge waits
• Cookie jar shared across requests

Limitations

• Header order and HTTP/2 settings come from Go's net/http, so sites that fingerprint those can still tell it apart from a browser.
• Newer challenge-platform JS challenges are detected but usually not solved.
• Turnstile and CAPTCHAs need a real browser.

See the README for usage.