feat(knowledge-graph): 引入实体抽取密度上限与类型纠偏防御层 - #535
Merged
Conversation
针对 Harness Engineering 语料构建中暴露的两类缺陷:单 chunk 实体过度抽取(如 1137 字符产出 16 实体)与类型误分类(Claude 被标为 person),按"宁缺毋滥" 原则补强 LLM 抽取链路: - 实体 Prompt 增加密度引导(每 200 字符约 1 个核心实体)、AI 产品/机构 vs 真人名的分类引导、Chain-of-Thought 推理步骤;关系 Prompt 增加 ⌈|E|×1.2⌉ 上限避免组合爆炸。 - 新增 extraction_validator 后置校验模块:known_entities 白名单覆盖 LLM 错判, AI 产品 regex 兜底未列型号变体,按 confidence 降序的密度截断(cap = max(3, chunk_len // 200)),全部信号回写到 ChunkExtractionStats。 - 新增 known_entities.yml 维护 AI 产品/机构白名单(Claude / GPT-4 / Anthropic 等),作为 SSOT 避免代码硬编码。 - 解析层接入 validator:被纠偏的实体在 metadata 标记 type_override_source 与 original_type,支持审计回滚。 - KgBuildMetrics 扩展 over_extraction_chunks / type_override_count / entity_density_p95 三个观测字段;service 层在每 chunk 抽取后聚合并按需 WARN。 - 配套 21 项单测覆盖白名单加载、类型重判、密度截断、边界与端到端集成。 参考:Martinez-Rodriguez 等 (Semantic Web J., 2018) 的 Schema-Guided Extraction 范式,以及 Neo4j LLM-Graph-Builder 的 schema-first 设计。 🤖 Generated with [Claude Code](https://github.com/claude), [CodeX](https://openai.com), [Gemini](https://github.com/apps/gemini-code-assist) Co-Authored-By: Aurelius Huang<threefish.ai@gmail.com>
依据代码审查意见落实两处修复: 1. `AI_PRODUCT_PATTERN` 尾部分组 `(?:[- ]?[\w.]+)*` 过于宽松,会把 `Claude Shannon` / `Claude Monet` / `Gemini Cricket` 等真人复合名 整体吞掉并误改为 product。现要求触发词后至少跟随一个“型号样式” 后缀(数字开头或 opus/sonnet/haiku/pro/ultra/mini/turbo 等已知规格 关键字),裸名继续由 known_entities 白名单覆盖。 2. `_parse_entity_response` 在 `entity_density_truncated` 日志中 `cap=len(results) + dropped - dropped` 是恒等式,简化为 `cap=len(results)`,避免读者怀疑特殊意图。 同步更新单元测试:删除“裸名命中正则”的过时断言,新增对真人复合名 (`Claude Shannon` 等)的负向断言以防回归。21 个用例全部通过。 🤖 Generated with [Claude Code](https://github.com/claude), [CodeX](https://openai.com), [Gemini](https://github.com/apps/gemini-code-assist) Co-Authored-By: Aurelius Huang<threefish.ai@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
核心变更
风险与回滚
验证证据
影响范围
Next Best Action