Skip to content

v0.2.0

Choose a tag to compare

@7and1 7and1 released this 27 Oct 08:05
· 1 commit to main since this release

fetch can use a free key. Static binaries for Linux (amd64, arm64), macOS (amd64, arm64) and Windows (amd64); checksums in SHA256SUMS. Or: go install github.com/verifiedproxyfeed/proxy-scraper@v0.2.0

  • fetch -key psk_…, or PROXY_SCRAPER_KEY in the environment: reads https://api.proxyscraper.com/k/{key}/v1/… instead of /v1/…, with the same paths, parameters and rows, at the key's limits (1,000 requests a minute, 100,000 a day) instead of 10 requests in 10 seconds. A free key: https://proxyscraper.com/account.
  • -key wins over the variable, and -key '' runs without one. Prefer the variable: a flag shows up in ps and in shell history.
  • A key is checked before any request (psk_ and 32 lower-case letters or digits), goes only to an https data host (or one on this machine), and is never printed: errors show /k/psk_masked/….
  • A keyed HTTP 404 means the key is unknown or revoked. A keyed HTTP 429 names the cap it hit and its Retry-After; fetch reports it and does not wait.
  • check and scrape never send the key. Without a key, every request and message is the same as in v0.1.0.

The binaries are not signed; on macOS a browser download may need xattr -d com.apple.quarantine proxy-scraper_*.