fix: resolve core runtime bugs (tester event loop, /random 500, scheduler crash)#229
Merged
Conversation
- utils/proxy: fix ValueError on multi-colon proxy strings in the str branch of convert_proxy_or_proxies, which made /random return 500 (#218) - scheduler: guard None process handles in finally so ENABLE_*=false no longer raises AttributeError (#62) - tester: replace deprecated get_event_loop()/asyncio.wait() with a fresh loop + asyncio.gather to fix 'Passing coroutines is forbidden' on Python 3.10+ (#191, #201) - tester: reuse one aiohttp session with a bounded connector instead of one per proxy, reducing memory growth and fd exhaustion (#119, #155) - tester: catch ClientResponseError/ContentTypeError + return_exceptions so bad httpbin responses no longer raise 'Task exception was never retrieved' (#208)
This was referenced Jul 4, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
这是 issue 收敛计划的 Phase 1(核心运行期 Bug 修复)。只修 Bug,不动依赖/功能,保证行为稳定。
改动内容
1.
/random返回 500 —ValueError: too many values to unpackproxypool/utils/proxy.py中convert_proxy_or_proxies的 str 分支 用了host, port = data.split(':'),当 Redis 里存在形如ip:port:port/ 多冒号的脏数据时会抛异常,直接导致/random500。list 分支早已用*_兜底,这里对齐为host, port, *_ = data.split(':')。2.
ENABLE_TESTER/GETTER/SERVER=false启动崩溃proxypool/scheduler.py的finally块无条件调用tester_process.is_alive(),当对应进程被禁用时该变量为None→AttributeError。改为判空后再打印状态。3. tester 在 Python 3.10+ 报错 / 内存增长 / fd 耗尽
proxypool/processors/tester.py:asyncio.new_event_loop()+asyncio.gather()取代已弃用的asyncio.get_event_loop()与对协程调用的asyncio.wait(),修复TypeError: Passing coroutines is forbidden, use tasks explicitly.aiohttp.ClientSession(原来每个代理都新建一个 session),并给TCPConnector设置limit=TEST_BATCH上限,缓解长跑内存增长与 Windowstoo many file descriptors in select()。EXCEPTIONS增加ClientResponseError、ContentTypeError,gather(..., return_exceptions=True),避免 httpbin 返回非 JSON/超长 header 时出现Task exception was never retrieved。验证
py_compile通过,无 lint 错误。convert_proxy_or_proxies('1.2.3.4:80:90')由「抛异常」变为返回1.2.3.4:80;混合列表、正常输入均正常。影响面
仅内部实现调整,API 行为、评分逻辑、配置项均不变。