Skip to content

Releases

v0.2.1

Choose a tag to compare

@7and1 7and1 released this 28 Oct 01:09

Without a key, fetch's HTTP 429 now says what the free limit is and where a key comes from. Static binaries for Linux (amd64, arm64), macOS (amd64, arm64) and Windows (amd64); checksums in SHA256SUMS. Or: go install github.com/verifiedproxyfeed/proxy-scraper@v0.2.1

  • A keyless HTTP 429 reads HTTP 429 (10 requests in 10 seconds per address; more with a free key: https://proxyscraper.com/account). A key raises the limits to 1,000 requests a minute and 100,000 a day: fetch -key psk_… or PROXY_SCRAPER_KEY.
  • Every other request and message is v0.2.0's: keyed errors, check and scrape are unchanged.

The binaries are not signed; on macOS a browser download may need xattr -d com.apple.quarantine proxy-scraper_*.

v0.2.0

Choose a tag to compare

@7and1 7and1 released this 27 Oct 08:05

fetch can use a free key. Static binaries for Linux (amd64, arm64), macOS (amd64, arm64) and Windows (amd64); checksums in SHA256SUMS. Or: go install github.com/verifiedproxyfeed/proxy-scraper@v0.2.0

  • fetch -key psk_…, or PROXY_SCRAPER_KEY in the environment: reads https://api.proxyscraper.com/k/{key}/v1/… instead of /v1/…, with the same paths, parameters and rows, at the key's limits (1,000 requests a minute, 100,000 a day) instead of 10 requests in 10 seconds. A free key: https://proxyscraper.com/account.
  • -key wins over the variable, and -key '' runs without one. Prefer the variable: a flag shows up in ps and in shell history.
  • A key is checked before any request (psk_ and 32 lower-case letters or digits), goes only to an https data host (or one on this machine), and is never printed: errors show /k/psk_masked/….
  • A keyed HTTP 404 means the key is unknown or revoked. A keyed HTTP 429 names the cap it hit and its Retry-After; fetch reports it and does not wait.
  • check and scrape never send the key. Without a key, every request and message is the same as in v0.1.0.

The binaries are not signed; on macOS a browser download may need xattr -d com.apple.quarantine proxy-scraper_*.

v0.1.0

Choose a tag to compare

@7and1 7and1 released this 20 Oct 04:48

First release. Static binaries for Linux (amd64, arm64), macOS (amd64, arm64) and Windows (amd64); checksums in SHA256SUMS. Or: go install github.com/verifiedproxyfeed/proxy-scraper@v0.1.0

  • fetch: the verified feed from https://api.proxyscraper.com, fastest first.
  • scrape: the public lists marked "publish": true in sources.json, de-duplicated, unchecked.
  • check: each proxy fetches an HTTPS judge with full certificate verification. A judge's HTTP 429 over that verified tunnel counts as working, and stderr says how many.

The binaries are not signed; on macOS a browser download may need xattr -d com.apple.quarantine proxy-scraper_*.