refactor(scripts): 正交分解 scripts/ 目录,提取共享模块消除三轴重复; - #549
Merged
Conversation
- 新建 scripts/_db.py:提取 DB 引擎生命周期(script_engine/script_connection) 与异步入口点(run_script),消除 4 个脚本中的连接样板重复 - 新建 src/negentropy/knowledge/cleanup.py:将聚类算法(cluster_by_time/ check_index_completeness)及 DB 操作(scan_source_uris/get_chunks_for_source/ delete_chunks)收敛为单一事实源 - 重构 cleanup_orphan_chunks.py:从 330 行减至 195 行,导入 cleanup 模块函数 - 重构 find_dup_arxiv_ids.py/force_reset_db.py/init_test_db.py:使用 _db 共享工具 - 更新 test_cleanup_orphans.py:删除 25 行内联函数,改用 cleanup 模块导入 - 消除 api.py 中 L5764-5792 的聚类/完整性检查内联实现,复用 cleanup 模块 🤖 Generated with [Claude Code](https://github.com/claude), [CodeX](https://openai.com), [Gemini](https://github.com/apps/gemini-code-assist) Co-Authored-By: Aurelius Huang<threefish.ai@gmail.com>
cluster_by_time() 返回 list[list[dict]],但 kept_ids 仍使用 ORM 属性语法 c.id 遍历 latest,运行时抛出 AttributeError,改为 dict 键访问 c["id"]。 🤖 Generated with [Claude Code](https://github.com/claude), [CodeX](https://openai.com), [Gemini](https://github.com/apps/gemini-code-assist) Co-Authored-By: Aurelius Huang<threefish.ai@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
apps/negentropy/scripts/目录中 4 个脚本(471 行)存在三轴正交重复——DB 引擎生命周期(create_async_engine+dispose在 4 个脚本中各重复一次)、异步入口点(asyncio.run+ try/except 样板重复 4 次)、清理算法(cluster_by_time/check_index_completeness在 CLI 脚本、API 端点、单元测试三处逐字重复)核心变更
scripts/_db.py(45 行):提取script_engine(async context manager,自动 dispose)、script_connection(引擎+连接快捷方式)、run_script(asyncio.run 封装 + 统一错误处理)三个共享工具src/negentropy/knowledge/cleanup.py(133 行):将聚类算法(cluster_by_time/check_index_completeness)及异步 DB 操作(scan_source_uris/get_chunks_for_source/delete_chunks)收敛为单一事实源,CLI 脚本、API 端点、单元测试均从此模块导入cleanup_orphan_chunks.py:从 330 行减至 195 行,删除 5 个本地函数定义,改为导入 cleanup 模块 +_db工具find_dup_arxiv_ids.py(79→55 行)、force_reset_db.py(39→22 行)、init_test_db.py(26→14 行):使用_db共享工具消除连接样板test_cleanup_orphans.py:删除 25 行内联函数副本,改为从negentropy.knowledge.cleanup导入api.pyL5764-5792 的重复算法:替换内联聚类循环和完整性检查为cluster_by_time/check_index_completeness调用风险与回滚
git revert单次提交即可完整回滚验证证据
pytest tests/unit_tests/knowledge/test_cleanup_orphans.py— 7/7 全部通过--help入口正常输出ruff check全部通过,pre-commit hooks(lint + format)通过影响范围
negentropy.knowledge.api中 cleanup-orphans 端点的聚类逻辑改为从模块导入(外部行为不变)Next Best Action
cleanup.py中的异步 DB 函数补充单元测试(当前测试仅覆盖纯函数)