fix: stabilize API & CLI integration tests in CI#2105
Merged
qin-ctx merged 1 commit intoMay 18, 2026
Conversation
1. Skip find/add_skill tests on 401/500 when embedding unavailable
- PR CI (fork repos) cannot access VLM/Embedding API keys, causing
embedding service to return 401/500 with dummy keys
- test_response_types.py: skip test_find_response_types on 401
- test_skill_api.py: skip add_skill tests on 500, find tests on 401
- build_test_helpers.py: skip assert_resource_findable on 401
2. Run CLI tests serially to avoid CONFLICT and timeout
- CLI tests share session-scoped fixtures (test_dir_uri, test_pack_uri)
- Concurrent workers cause CONFLICT Resource is busy and timeout errors
- Remove -n 4 from CLI compatibility and integration test steps
3. Reduce parallelism in effect tests to avoid server overload
- Lightweight tests: -n 4 -> -n 2
- Heavy tests: -n 2 -> serial (resource-intensive operations)
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
PR Code Suggestions ✨No code suggestions found for the PR. |
qin-ctx
approved these changes
May 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
摘要
修复 CI 中 API 和 CLI 集成测试的稳定性问题，涵盖 PR 模式（fork 仓库无 API KEY）和主仓库 effect 测试（并发过载）两类场景。
修改内容
1. Embedding 服务不可用时跳过相关测试
涉及文件：
tests/api_test/common/test_response_types.pytests/api_test/resources/slow/test_skill_api.pytests/api_test/scenarios/resources_retrieval_slow/build_test_helpers.py问题：PR CI 中 fork 仓库无法访问
VLM_API_KEY和EMBEDDING_API_KEY密钥，CI 使用 dummy API key 配置 embedding 服务。依赖 embedding 的接口（find、add_skill）会返回 401 或 500，导致测试误报失败。根因链：
find调用 embedding 服务 → 上游返回 401 → 服务端映射为UNAUTHENTICATED→ HTTP 401add_skill调用 embedding 做向量化 → 服务端内部错误 → HTTP 500修复：仅在 401/500 时
pytest.skip()，其他错误仍正常断言失败，保证测试有效性。2. CLI 测试改为串行执行
涉及文件：
.github/workflows/api_test.yml问题：CLI 集成测试使用
-n 4（4 个并行 worker）运行，但共享同一服务器和 session 级别 fixture（test_dir_uri、test_pack_uri）。并发 worker 导致：add-resource→CONFLICT Resource is busyov write --wait/ov wait等待被其他 worker 锁定的资源 → 超时 120s影响的失败用例：
test_pack_urifixture → CONFLICT（级联影响TestContentRead、TestContentAbstract、TestContentOverview、TestContentDownload）TestContentWrite.test_write_append→ 超时TestSystemWait.test_wait_with_timeout→ 超时TestContentReindex.test_reindex→ CONFLICT修复：移除 CLI 兼容性测试和集成测试的
-n 4参数，改为串行执行。3. Effect 测试降低并行度
涉及文件：
.github/workflows/api_test_effect.yml问题：effect 测试虽有 API KEY，但资源操作密集的测试并发运行时服务端过载，导致 500 错误和索引构建超时。
修复：
-n 4→-n 2-n 2→ 串行测试验证
test_find_response_types在 embedding 不可用（401）时 SKIP，其他错误仍 FAILtest_add_skill_basic在 embedding 不可用（500）时 SKIP，其他错误仍 FAILassert_resource_findable在 find 返回 401 时 SKIP