DSH | dsh-memory | 结合 cc 的 auto-memory 与 codex 的 session 整理的长期记忆 (附 LoCoMo 测评结果) #5678
hr98w
started this conversation in
Show Your Plugins!
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
项目 / Repository: https://github.com/hr98w/dsh-memory
测评报告 / Interactive report: https://hr98w.github.io/dsh-memory/
中文
dsh-memory,给 DeepSeek Harness 加上跨 Session 的长期记忆。实现尽量简单:记忆就是本机的 Markdown,不需要向量数据库或独立记忆服务。
核心思路:Claude Code × Codex
Global 保存跨项目偏好,Workspace 保存项目事实和约定。日常 Agent 通过
memory_update写入记忆;新 Session 按需读取。模型提出修改,Host 负责校验、版本检查和写入。插件接入 DSH 的提示词、工具和 Settings 页面,支持中英文 UI。整理模型直接从 DSH 已配置的模型中选择。
LoCoMo-10 测评
也跑了一次完整测评,想看看这些 Markdown 能否帮助回答历史对话里的问题。流程参考了 OpenViking 的 Claude Code LoCoMo 设置,不作跨项目分数比较。
使用 DeepSeek V4 Flash,排除 Category 5 后共 1,540 题,用 LLM Judge 对照标准答案评分:
这说明在这套设置下,记忆能帮助回答历史问题,但不是与完整上下文或其他记忆系统的对比。整理并非每组都更好;两种记忆模式独立形成初始记忆,不能把分数差全归因于整理。
受成本限制,每种模式只保留一组完整结果,含开发过程中的重试和重跑;其中 Memory 的 conv-41 在观察到低分后重跑并替换,可能存在选择偏差。Ingest、QA、Judge 都有模型随机性。逐题回答、分类分数和成本已放在在线测评报告里。
English
dsh-memory adds long-term memory across Sessions to DeepSeek Harness. The implementation stays simple: memory is stored as local Markdown, with no vector database or separate memory service required.
Core idea: Claude Code × Codex
Global holds cross-project preferences; Workspace memory holds project facts and conventions. Agents write through
memory_updateand recall relevant files in later Sessions. The Host validates proposals, checks revisions, and commits changes.The plugin integrates with DSH prompts, tools, and Settings, with Chinese/English UI and a consolidation model selector using existing DSH model configuration.
LoCoMo-10 evaluation
I also ran a complete evaluation to see whether these Markdown memories help answer questions about past conversations. The workflow follows the approach of OpenViking's Claude Code LoCoMo setup, without comparing scores across projects.
Using DeepSeek V4 Flash, the evaluation covers 1,540 questions after excluding Category 5, with an LLM Judge scoring answers against the reference answers:
These results suggest memory helps with historical QA in this setup, not that it beats full context or other memory systems. Consolidation did not improve every conversation, and the independently built memories prevent a clean causal comparison.
Due to cost constraints, each mode retains only one complete result set, including development retries and reruns. Memory's conv-41 was rerun and replaced after a low score, introducing possible selection bias. Ingestion, QA, and judging all involve LLM variability. Per-question answers, category scores, and costs are available in the interactive evaluation report; further limitations are documented in the experiment retrospective (Chinese).
安装 / Install
安装后完全重启 dsh web。 / Fully restart dsh web after installation.
目前请让一个进程负责同一 DSH_HOME 的记忆写入,跨进程并发保护尚未实现。
Use one memory-writing process per DSH_HOME; cross-process write protection is not implemented yet.
截图 / Screenshot
All reactions