Skip to content

Refactor memory extract#952

Merged
qin-ctx merged 18 commits intomainfrom
refactor_memory_extract
Mar 25, 2026
Merged

Refactor memory extract#952
qin-ctx merged 18 commits intomainfrom
refactor_memory_extract

Conversation

@chenjw
Copy link
Collaborator

@chenjw chenjw commented Mar 25, 2026

Description

  1. 修复v2记忆没有embedding的问题。
  2. overview的格式(prompt)和从overview生成abstract的逻辑保持一致。
  3. 优化下bot的readme。

还遗留问题:

  1. 目前工作记忆提取(归档中的overview)还没有和老版本对齐。待实现。

Related Issue

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Screenshots (if applicable)

Additional Notes

chenjw and others added 15 commits March 18, 2026 00:45
…on design document

- Add bilingual (English/Chinese) design document for memory templating system
- Include YAML-based MemoryTypeRegistry with 8 built-in types
- Detail ReAct 3+1 phase flow with pre-fetch optimization
- Describe 3-operation Schema: write/edit/delete
- Document RoocodePatch SEARCH/REPLACE format
- Explain dual-mode design: simple mode vs template mode
- Cover pre-fetch optimization: ls directories + read .abstract.md/.overview.md + search once
- Include merge operations: patch, sum, avg, immutable
- Address #578: allow custom prompt template addition and specification

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add YAML-configurable memory schemas (cards, events, entities, etc.)
- Implement MemoryReAct with tool use (read/find/ls)
- Add schema-driven memory operations (write_uris/edit_uris/delete_uris)
- Implement memory patch handler for incremental updates
- Add comprehensive test suite
## Summary

Implement memory templating system (GitHub Issue #578) - a complete
rewrite of the memory extractor subsystem to support YAML-configurable
memory types instead of hardcoded categories.

## Key Changes

### Architecture
- Replace hardcoded 8 memory types with YAML-configurable schema system
- Add MemoryTypeRegistry to load memory type definitions from YAML files
- Dynamic Pydantic model generation from schema for type safety
- Field-level merge operations: PATCH, SUM, IMMUTABLE

### Memory Extraction Flow
- Implement ReAct orchestrator for single-pass memory updates
- MemoryUpdater for applying operations to storage
- Memory tools (read, search, ls) for ReAct loop
- Stable JSON parser with 5-layer fault tolerance

### File Naming & Storage
- Semantic filenames from template ({topic}.md instead of random IDs)
- Two memory modes: simple mode and template mode
- MEMORY_FIELDS HTML comment for structured metadata

### Configuration
- 9 YAML templates in openviking/prompts/templates/memory/
- memory_config.py for memory system configuration
- Dual-threshold compact upload mechanism in design doc

### Deletions
- Remove old memory_content.py, memory_data.py, memory_operations.py
- Remove memory_types.py, memory_utils.py, memory_patch.py
- Remove corresponding old test files

### Updated Components
- VLM backends (litellm, openai, volcengine) for new interfaces
- Session and service core integration
- Test suite for new architecture

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
## Summary

Fix missing parameters in commit_async() when calling extract_long_term_memories().
The synchronous commit() method correctly passes these parameters, but the async
version was missing them, causing memory extraction to be skipped.

## Changes

- Pass user=self.user, session_id=self.session_id, ctx=self.ctx
  in commit_async() when calling extract_long_term_memories()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
## Summary

Fix JSON serialization error by converting FindResult object to dict
using its to_dict() method before returning from MemorySearchTool.

## Changes

- In MemorySearchTool.execute(), return search_result.to_dict()
  instead of search_result directly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Also rename schema_models.py to schema_model_generator.py for clarity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…zation

- Add edit_overview_operations to MemoryUpdater for updating .overview.md files
- Optimize MemoryTypeRegistry initialization in SessionCompressorV2 (load once)
- Various memory templating system improvements

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Also fix duplicate line in system prompt.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- MemoryUpdater now vectorizes written/edited memory files after apply_operations
- MemoryReAct generates overview following semantic.overview_generation.yaml format
- Auto-extract and write .abstract.md from overview in memory_updater
- Fix import: VikingURI is from openviking_cli.utils.uri

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Change vikingbot gateway to openviking-server --with-bot
- Change vikingbot chat to ov chat
- Update --no-markdown to --no-format
- Remove --logs flag (not available)
- Simplify CLI Reference table
- Also update Chinese README_CN.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Convert from pytest to standalone script with:
- SyncHTTPClient instead of AsyncHTTPClient
- Rich for pretty console output
- Phase control (ingest/verify/all)
- Better error handling and progress display

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions
Copy link

Failed to generate code suggestions for PR

chenjw and others added 2 commits March 25, 2026 11:20
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@chenjw chenjw requested a review from qin-ctx March 25, 2026 03:39
@qin-ctx qin-ctx self-assigned this Mar 25, 2026
Copy link
Collaborator

@qin-ctx qin-ctx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这次 review 的结论是:修复方向基本对,但当前 v2 新增的 embedding 路径还有一个阻塞问题需要先处理,否则会把序列化后的 memory 文件直接送去向量化,影响检索语义。其余几个问题主要是测试退化和拷贝残留,建议这轮一起收掉。

另外两点建议同步处理:

  • PR description 还比较空,没有关联 issue,也没有明确测试说明。
  • 当前 CI 失败在 ruff format --check

- Remove duplicate logger and create_session_compressor in session/__init__.py
- Remove duplicate MemoryConfig import in open_viking_config.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@qin-ctx qin-ctx merged commit 55a0c0e into main Mar 25, 2026
11 of 12 checks passed
@github-project-automation github-project-automation bot moved this from Backlog to Done in OpenViking project Mar 25, 2026
@qin-ctx qin-ctx deleted the refactor_memory_extract branch March 25, 2026 05:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants