fix(knowledge-graph): 修复 KG 构建管线四项级联缺陷(事务/端点流失/日志/退避) - #536
Merged
ThreeFish-AI merged 2 commits intoMay 13, 2026
Conversation
…etry_after 失尊; 针对一次完整 KG 构建日志暴露的多处缺陷做系统性修复(详见 ISSUE-085): 1. 社区摘要事务冲突:B3 阶段从 shared_session 剥离,使用独立 AsyncSessionLocal 会话承载 corpus 配置查询与 summarizer 调用;同时移除 summarizer 内部三处 db.commit(),事务边界回归调用方(解决 "A transaction is already begun on this Session" 异常)。 2. 关系端点 unresolved 45% 流失:EntityResolver.ResolutionResult 新增 id_merge_map(old_entity_id → surviving_entity_id),Exact / Token / ANN 三阶段同步维护;新增 _flatten_chain 工具函数展平传递链;service.py _resolve_ref 重写为优先 ID 直查(含 ANN→DB UUID 跨表场景),删除旧的 "32 位 hex hash unresolved" 启发式分支。 3. 日志 run_id 双身份:repository.update_build_run 新增 human_run_id 参数; build_run_updated / build_run_created / state_guard 三处日志统一输出 run_uuid(DB PK)+ run_id(人类可读 build-xxx-ts)双字段;service.py 所有调用点传入 human_run_id=run_id。 4. LLM retry_after 失尊:新增 _extract_retry_after_seconds 支持 JSON body 与 HTTP header 双源解析;_compute_retry_backoff 仅对 502/503/429 等瞬时 故障启用 retry_after,叠加 floor(≥ 默认指数)与 cap(≤ 120s)+ jitter。 测试覆盖:新增 test_extractors_retry_backoff.py 18 例、 TestEntityResolverIdMergeMap 6 例、TestFlattenChain 5 例;适配既有 test_entity_resolver_token_overlap.py 与 test_resolve_ref.py 结构断言; 范围内 158 例全过,KG 单元测试目录 788/788 通过。 🤖 Generated with [Claude Code](https://github.com/claude), [CodeX](https://openai.com), [Gemini](https://github.com/apps/gemini-code-assist) Co-Authored-By: Aurelius Huang<threefish.ai@gmail.com>
…语义反转 / DB UUID 关系静默丢失; 1. 消除 _compute_retry_backoff 双重 jitter:提取无 jitter 的 base_backoff 作为 retry_after floor 计算基线,避免 [0,2) 的超宽 jitter 范围;收紧测试断言 至单层 jitter 区间。 2. 修正 _ann_stage primary_keys 语义:从 remaining_indices(存留实体)填充 survivor_keys,从已合并 secondary 经 prior_id_merge_map 回溯填充 secondary_survivor_keys;排除 normalize_label 碰撞导致的自合并 (如 "OpenAI" vs "OpenAI Inc.");同步维护 merge_map(label→label) 以支持下游标签级 fallback。 3. 修复 ANN→DB UUID 关系持久化静默丢失:_resolve_ref 对 id_merge_map 返回的 DB UUID 做可达性检查,通过 db_uuid_to_label 反向映射转写为标签,确保 _create_relation_with_session 和 sync_relation 均能正确处理;无 label 映射时标记 unresolved 保持计数器可见性;sync_relation edge_dicts 构建 补充 db_uuid_to_label fallback。 KG 单元测试 792/793 通过(1 个预存失败与本次无关)。 🤖 Generated with [Claude Code](https://github.com/claude), [CodeX](https://openai.com), [Gemini](https://github.com/apps/gemini-code-assist) Co-Authored-By: Aurelius Huang<threefish.ai@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
43bacd7e,20 chunks,6.5 分钟)日志暴露出四项级联缺陷:① 社区摘要A transaction is already begun on this Session异常使终态降级为completed_with_errors;② 75 条原始关系经 resolving 后 34 条端点 unresolved(45% 流失)集中指向被合并实体的 32-hex hash id(如entity:57cff7c895...出现 15 次);③build_run_updated日志run_id=<UUID PK>与外层run_id=build-xxx-ts双身份割裂;④ Cloudflare 502 错误返回retry_after: 60但应用层只按指数退避 1.1s 立刻重试。核心变更
service.pyB3 阶段从shared_session剥离,使用独立AsyncSessionLocal()会话承载 corpus 配置查询与 summarizer 调用;community_summarizer.py移除三处内部db.commit(),事务边界回归调用方("事务边界单一来源"原则)。entity_resolver.pyResolutionResult新增id_merge_map: dict[str, str](old_entity_id → surviving_entity_id),Exact / Token / ANN 三阶段同步维护;ANN 阶段返回签名变更为tuple[set[int], dict[str, str]],命中 DB 既有实体时记录跨表 UUID 映射;新增_flatten_chain工具函数统一展平 label / id 两条链路的传递映射;service.py_resolve_ref重写为"ID 直查 → 已是存留 id → 标签级 fallback"四级优先级,删除旧的len(ref) == 32 and all hex启发式分支。repository.pyupdate_build_run新增可选human_run_id形参;build_run_created/build_run_updated/build_run_update_skipped_by_state_guard三处日志统一输出run_uuid=<DB PK> + run_id=<人类可读>双字段,串联跨日志条目语义;service.py全部update_build_run调用点传入human_run_id=run_id。extractors.py新增_extract_retry_after_seconds同时支持 JSON body('retry_after': N)与 HTTP header(Retry-After: N)双源解析;_compute_retry_backoff仅对 502 / 503 / 429 / bad gateway / rate limit / too many requests 等瞬时故障启用 retry_after,叠加 floor(≥ 默认指数退避防反向加速)与 cap(≤ 120s 防超长阻塞)+ jitter(防羊群),参考 RFC 9110 §10.2.3。风险与回滚
AsyncSessionLocal配置异常(连接池耗尽 / DSN 不可达),独立 session 创建可能失败——已通过try/except包裹整个 B3 阶段并回退到 warning + 兜底 shared_session.rollback() 兜底;id_merge_map成为关系端点的权威映射;若 EntityResolver 三 stage 之一漏建 id 映射,会回到旧的标签级 fallback(保留原行为)但不会引入新的 unresolved;retry_after提示的 502/503/429 类错误生效,普通错误退避不变。git revert 04e88d57单 commit 即可还原全部四项修复;ISSUE-085 详细记录每项的根因 / 修复 / 测试,便于二次审视。验证证据
test_extractors_retry_backoff.py18 例:覆盖 JSON / HTTP header 解析、502+retry_after 尊重、429 cap 120s、retry_after=1 不加速(floor 防御)、400 非瞬时故障不被错误延长、524 优先级高于 retry_after;test_entity_resolver.py::TestEntityResolverIdMergeMap6 例:Exact / Token / ANN 三 stage 各自 id 映射 + ANN→DB UUID 跨表 + 多跳传递链展平 + 空输入 / 无合并空字典;TestFlattenChain5 例:单跳 / 二跳 / 三跳 / 环路防御 / 空字典;test_resolve_ref.py结构断言(id_merge_map 引用 + 移除relation_endpoint_hash_unresolved断言)与test_entity_resolver_token_overlap.py3-tuple 解包;test_extraction_llm_plan::test_build_llm_invocation_plan_returns_none_when_serialization_fails与本次修复完全无关,已通过git stash验证)。unresolved_endpoints / raw_countcommunity_summary_failed警告statuscompleted_with_errorscompletedbuild_run_updated字段一致性run_id=UUIDrun_uuid+run_id影响范围
apps/negentropy/src/negentropy/knowledge/graph/五个核心模块(service / entity_resolver / community_summarizer / repository / extractors);tests/unit_tests/knowledge/四个测试文件 + 一个新增;ABCGraphRepository.update_build_run签名扩展(向后兼容,新增可选形参)。docs/issue.md新增 ISSUE-085 完整摘要(表因 / 根因 / 处理方式 / 验证证据 / 后续防范 / 同类问题影响),符合 CLAUDE.md 工程纪律。Next Best Action
43bacd7e)重建,对照"指标对比"表验证unresolved_endpoints占比 < 5% 与终态completed;按gh pr review走 ultrareview。/build-runs/latest轮询替换为 SSE / EventSource,消除构建期 ~100 次/run 的 DB 压力;③ Stage 2 (ANN) borderline LLM 验证实装(entity_resolver.py:364 注释明确"留给后续迭代")。🤖 Generated with Claude Code