Skip to content

feat: fetch multiple proxies at once via /random?count=N (#194)#232

Closed
Germey wants to merge 1 commit into
security/harden-apifrom
feature/random-count
Closed

feat: fetch multiple proxies at once via /random?count=N (#194)#232
Germey wants to merge 1 commit into
security/harden-apifrom
feature/random-count

Conversation

@Germey

@Germey Germey commented Jul 4, 2026

Copy link
Copy Markdown
Member

背景

Issue 收敛计划 Phase 4a(增强:一次获取多个代理),对应 #194

⚠️ 本 PR 基于 security/harden-api(Phase 3)分支,建议 #231 合并后再合并本 PR。GitHub 会在 #231 合入 master 后自动把本 PR 的 base 重定向到 master。

需求(#194

多线程爬虫希望一次请求拿到多个代理,减少对 /random 的调用次数。

改动

  • storages/redis.py:新增 RedisClient.randoms(count, ...),优先返回满分代理,不足时按分数从高到低回退,最终用 random.sample 去重随机抽取,返回 List[Proxy]
  • processors/server.py/random 支持 count 参数,一次返回多个随机代理(每行一个)。count 不传或 ≤1 时行为完全不变(返回单个代理);可与 key 组合,子池不足时沿用 PROXY_RAND_KEY_DEGRADED 的降级逻辑。
  • README.md:补充 count 参数说明。

用法

GET /random            # 单个代理(不变)
GET /random?count=5    # 5 个随机代理,每行一个
GET /random?count=5&key=proxies:weibo

验证

  • server.py / redis.py 编译通过。
  • count > 可用数量 时返回全部可用(min(count, len) 封顶);count<=1 走原单代理分支。

关联 issue

- redis: add RedisClient.randoms(count, ...) which prefers max-score proxies and falls back to rank, returning a de-duplicated random sample

- server: /random accepts a count parameter and returns N random proxies (one per line); count<=1 keeps the original single-proxy behavior; works with key too

- README: document the new count parameter

Closes #194
@Germey
Germey deleted the branch security/harden-api July 4, 2026 08:44
@Germey Germey closed this Jul 4, 2026
@Germey
Germey deleted the feature/random-count branch July 4, 2026 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant