feat(memo): 接入 LLM 自动提取与后台调度 by wynxing · Pull Request #246 · 1024XEngineer/neo-code

wynxing · 2026-04-13T07:53:03Z

变更说明

本 PR 对 memo 子系统做了一轮自动提取能力升级，将原来的规则匹配提取改为基于模型的自动提取，并把提取时机从 runtime 里的临时异步调用收敛到 memo 子系统内部的后台调度器中。

这次改动保持现有 MEMO.md + topic 文件 存储格式、memo_remember/memo_recall 工具和索引结构不变，不引入 schema/frontmatter 迁移。

主要改动

1. 接入 LLMExtractor

新增 memo.TextGenerator 接口
新增 memo.LLMExtractor，替代默认自动提取实现
保持现有四类记忆类型：
- user
- feedback
- project
- reference
仅截取最近 10 条非 tool 消息参与提取
prompt 固定约束：
- 只提取无法从代码直接推导的信息
- 无内容时返回 []
- 输出严格 JSON 数组
- 相对日期转为绝对日期
解析时支持 JSON 前后噪声，只抽取最外层数组
对提取结果做统一校验和标准化：
- type 合法
- title/content 非空
- title 走 NormalizeTitle
- keywords 去空白去重

2. 新增 AutoExtractor 后台调度器

在 memo 子系统内部新增后台调度器，封装 Extractor + Service
按 sessionID 做 2 秒 debounce
同一会话短时间多次触发时，只保留最后一份消息快照
提取运行中收到新请求时不并发执行，而是在当前轮结束后做 trailing extraction
若本轮已通过 memo_remember 成功写入记忆，则自动提取跳过
自动提取写入前增加精确去重：
- Type + NormalizeTitle(Title) + trimmed Content 完全相同时跳过
去重逻辑改为在 Service 内部加锁原子完成，避免并发 session 重复落盘
后台失败仅记日志，不影响主链路

3. 调整 runtime 调度时机

runtime.MemoExtractor 改为调度接口：
- Schedule(sessionID, messages, skip)
在一次 Run 生命周期内跟踪是否成功执行过 memo_remember
仅在最终 assistant 回复完成且没有后续工具调用时才调度自动提取
不新增 UI 事件，不把后台提取状态暴露到界面层

4. 新增 app 层 provider 适配

在 internal/app 新增基于当前 provider/model 的 TextGenerator 适配
复用现有 provider 装配逻辑发起独立生成请求
不携带 tools
聚合流式文本输出
BuildRuntime 默认装配 LLMExtractor + AutoExtractor

测试

补充并通过了以下测试：

internal/memo/llm_extractor_test.go
- 有效 JSON
- 空数组
- JSON 噪声包裹
- 非法 type 过滤
- 无消息/无用户消息
- 只取最近 10 条非 tool 消息
- prompt 约束校验
internal/memo/auto_extractor_test.go
- debounce 合并
- trailing extraction
- skip=true 跳过调度
- 挂起请求被 skip 覆盖
- 提取错误静默
- 单 session 精确去重
- 多 session 并发去重
internal/app/provider_text_generator_test.go
- 正常聚合流式文本
- provider build/generate 异常
- 缺失 message_done 的错误处理
- 请求不携带 tools
internal/runtime/runtime_test.go
- 最终回复后才调度自动提取
- memo_remember 成功后自动提取跳过
- 有工具调用时不提前调度

验证

已执行并通过：

go test ./internal/memo/... ./internal/runtime/... ./internal/app/...
go test ./...
go build ./...

说明

本轮不修改 memo 存储格式
本轮不增加新的配置项
RuleExtractor 保留作为回退/测试实现，但默认路径已切换到 LLMExtractor

…egration - use rune-safe title truncation for auto extracted memo entries - guard memo remember/recall tools against nil service and cap recall output - limit runtime memo extraction input and add bounded async execution - add regression tests for utf-8 truncation, nil guards, output limit, and runtime hook Generated with [codeagent](https://github.com/qbox/codeagent) Co-authored-by: Yumiue <188874804+Yumiue@users.noreply.github.com>

fix(memo): address review findings in PR 1024XEngineer#245

# Conflicts: # internal/runtime/runtime.go # internal/runtime/runtime_test.go

codecov · 2026-04-13T07:54:53Z

Codecov Report

❌ Patch coverage is 83.23917% with 89 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/memo/service.go	70.53%	17 Missing and 16 partials ⚠️
internal/memo/auto_extractor.go	84.96%	14 Missing and 9 partials ⚠️
internal/app/bootstrap.go	75.67%	13 Missing and 5 partials ⚠️
internal/memo/llm_extractor.go	90.51%	8 Missing and 5 partials ⚠️
internal/runtime/memo.go	90.90%	1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

wynxing · 2026-04-13T08:50:19Z

/code 解决冲突，添加测试文件

fennoai · 2026-04-13T08:50:26Z

处理结果

已完成核查与验证，当前 PR 分支无需额外冲突修复或补测提交：

已检查 origin/main...HEAD，未发现冲突标记或 merge 冲突问题。
已确认本 PR 已包含新增测试文件（如 internal/memo/*_test.go、internal/app/provider_text_generator_test.go、internal/runtime/runtime_test.go 等）。
本地验证通过：
- go test ./internal/memo/... ./internal/runtime/... ./internal/app/...
- go test ./...
由于未发现需要追加修改的代码点，本次未产生新的本地变更、提交与推送。

未执行项

未新增额外代码或测试文件（原因：当前分支已满足“解决冲突、添加测试文件”的目标状态）。

View job run

wynxing · 2026-04-13T08:55:24Z

/code 增加测试覆盖率，解决这个PR合并的冲突

fennoai · 2026-04-13T08:55:31Z

处理结果

已按你的要求完成两件事：补测试提升覆盖率，并处理合并冲突检查。

覆盖率补充（仅新增测试，不改生产逻辑）

internal/memo/auto_extractor_test.go
- 补 ScheduleWithExtractor 专用提取器绑定路径
- 补调度 guard 分支（空 session / nil extractor）
- 补 autoExtractDedupKey、parseTopicSourceAndContent、cloneProviderMessages、stopTimer 边界
internal/memo/llm_extractor_test.go
- 补 nil generator、generator 报错分支
- 补 extractJSONArray 错误分支
- 补 cloneProviderMessage 深拷贝分支
internal/memo/service_test.go
- 补 addAutoExtractIfAbsent 去重行为
- 补 ensureAutoExtractIndex 初始化与失败分支
- 补自动提取索引 helper（track/remove/hasExact）分支
- 补 cloneIndex 的 nil 与隔离分支
internal/memo/extractor_test.go
- 补 truncateWithEllipsis 边界分支

合并冲突处理

已核查 origin/main...HEAD 与三方合并结果，当前分支未发现新的冲突块，无需额外冲突修复提交。

本地验证

go test ./... 通过
go build ./... 通过

提交与推送

Commit: 7803b5a
已推送到当前 PR 头分支：Yumiue:main

View job run

Generated with [codeagent](https://github.com/qbox/codeagent) Co-authored-by: Yumiue <188874804+Yumiue@users.noreply.github.com>

wynxing · 2026-04-13T09:09:11Z

/code 解决PR合并冲突

fennoai · 2026-04-13T09:09:18Z

处理结果

已完成本次“解决 PR 合并冲突”。我在当前 PR 分支执行了与 origin/main 的合并并解决冲突，核心是 internal/runtime/runtime.go 的冲突收敛，同时保留本 PR 的 memo 调度语义（Schedule 接口、最终回复后调度、memo_remember 成功后跳过自动提取）。

已完成

将 origin/main 的最新改动合入当前 PR 分支并完成冲突消解
在拆分后的 runtime 结构上恢复本 PR 的 memo 自动提取调度行为
修正受影响测试辅助桩（runtime_internal_helpers_test.go）以匹配 Schedule 接口
提交并推送到 PR 头分支 Yumiue:main
- commit: 948499a

验证

go test ./internal/runtime/... ./internal/memo/... ./internal/app/... 通过
go test ./... 通过
go build ./... 通过

View job run

- align runtime package with main split structure - keep memo auto-extract scheduling behavior on runtime path - update related runtime tests and helpers Generated with [codeagent](https://github.com/qbox/codeagent) Co-authored-by: Yumiue <188874804+Yumiue@users.noreply.github.com>

wynxing · 2026-04-13T10:02:45Z

/review

fennoai

Found 2 issues worth addressing before relying on this in production traffic: one correctness race in idle state cleanup, and one scalability hotspot in auto-extract persistence path.

fennoai · 2026-04-13T10:06:58Z

+	state.idleTimer = nil
+	state.mu.Unlock()
+
+	a.mu.Lock()


handleIdle can delete a.states[sessionID] based on a stale check.

Race window: after releasing state.mu (line 188) and before/while this block acquires a.mu, another goroutine can call Schedule(...), set state.pending, and arm a new timer. This function will still delete the state entry because it never re-validates running/pending under state.mu before deletion.

Result: active state can be detached from the map, so later schedules create a second state for the same session, breaking debounce/serialization guarantees and risking duplicate background runs.

Please re-check state liveness just before deletion (e.g., reacquire state.mu after taking a.mu, or use a tombstone/version guard).

fennoai · 2026-04-13T10:06:58Z

+			continue
+		}
+
+		added, err := a.svc.addAutoExtractIfAbsent(ctx, entry)


extractAndStore persists entries one-by-one via addAutoExtractIfAbsent, and each call goes through saveEntryLocked (LoadIndex + SaveTopic + SaveIndex).

When one extraction returns multiple entries, this becomes repeated full-index read/write cycles in a tight loop. That scales poorly with memo size and can dominate runtime I/O.

Consider batching accepted entries into a single index load/save transaction (or at least amortizing index reads/writes per extraction run).

wynxing · 2026-04-13T10:11:09Z

/simplify

fennoai · 2026-04-13T10:11:15Z

处理结果

已完成你本次 /simplify 任务：先读取历史评论并评估未解决 review 点，再修复值得修复的问题，并使用 code-simplifier 子代理做简化。

本次实际改动：

修复 internal/memo/auto_extractor.go 的 idle 回收竞态：handleIdle 删除状态前在锁内做 map 一致性和活性二次校验，避免 stale callback 删除活跃 state。
简化并发条件判断：新增 isIdleStateLocked 收敛重复条件分支。
新增测试：
- TestAutoExtractorHandleIdleKeepsActiveState
- TestIsIdleStateLocked

历史评论评估结论

已确认先前 4 条评论对应问题在当前分支已处理（provider 快照绑定、service 去重索引、worker 空闲回收机制、runtime 使用 callName 判定 remember 成功）。
本轮修复了仍有效且应优先处理的并发竞态问题。
extractAndStore 批量 I/O 优化建议需要调整 Service 持久化事务边界，属于更大范围重构，本轮未扩展到该层。

验证

go test ./internal/memo/... ✅
go test ./internal/memo/... ./internal/runtime/... ./internal/app/... ✅
go test ./... ✅

提交与推送

Commit: 8de4646
已推送到当前 PR 分支：Yumiue:main

View job run

Generated with [codeagent](https://github.com/qbox/codeagent) Co-authored-by: Yumiue <188874804+Yumiue@users.noreply.github.com>

xgopilot and others added 3 commits April 13, 2026 00:54

Merge pull request #6 from Yumiue/fork-pr-245-1776040827

38dce8f

fix(memo): address review findings in PR 1024XEngineer#245

feat(memo): 接入 LLM 自动提取与后台调度

b1d4a36

# Conflicts: # internal/runtime/runtime.go # internal/runtime/runtime_test.go

This comment was marked as outdated.

Sign in to view

fix(memo): harden auto extraction scheduling

96ba85d

test(memo): increase coverage for extractor scheduler and service dedup

7803b5a

Generated with [codeagent](https://github.com/qbox/codeagent) Co-authored-by: Yumiue <188874804+Yumiue@users.noreply.github.com>

xgopilot and others added 2 commits April 13, 2026 09:22

docs: sync memo extraction flow

60a9aa7

fennoai Bot reviewed Apr 13, 2026

View reviewed changes

fix(memo): avoid stale idle cleanup deleting active state

8de4646

Generated with [codeagent](https://github.com/qbox/codeagent) Co-authored-by: Yumiue <188874804+Yumiue@users.noreply.github.com>

phantom5099 merged commit 4a0aefe into 1024XEngineer:main Apr 13, 2026
2 checks passed

Uh oh!

Conversation

wynxing commented Apr 13, 2026

变更说明

主要改动

1. 接入 LLMExtractor

2. 新增 AutoExtractor 后台调度器

3. 调整 runtime 调度时机

4. 新增 app 层 provider 适配

测试

验证

说明

Uh oh!

codecov Bot commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment was marked as outdated.

Uh oh!

wynxing commented Apr 13, 2026

Uh oh!

fennoai Bot commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

处理结果

未执行项

Uh oh!

wynxing commented Apr 13, 2026

Uh oh!

fennoai Bot commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

处理结果

Uh oh!

wynxing commented Apr 13, 2026

Uh oh!

fennoai Bot commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

处理结果

已完成

验证

Uh oh!

wynxing commented Apr 13, 2026

Uh oh!

fennoai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

fennoai Bot Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

fennoai Bot Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

wynxing commented Apr 13, 2026

Uh oh!

fennoai Bot commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

处理结果

历史评论评估结论

验证

提交与推送

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Apr 13, 2026 •

edited

Loading

fennoai Bot commented Apr 13, 2026 •

edited

Loading

fennoai Bot commented Apr 13, 2026 •

edited

Loading

fennoai Bot commented Apr 13, 2026 •

edited

Loading

fennoai Bot commented Apr 13, 2026 •

edited

Loading