Added memory_search and Merged knowledge_extract and memory_operation

[RyanNg1403]

---

🧠 Memory System Refactor: Unified Operations & Search

📋 Overview

Major refactor consolidating memory extraction and operations into a single atomic tool while adding semantic search capabilities for improved knowledge management.

🔄 Key Changes

1. Merged extract_knowledge + memory_operation → cipher_extract_and_operate_memory

Benefits:

• ✅ Efficient Operations: Single step extraction and memory management
• ✅ Smart Filtering: Automatically excludes non-technical content

2. Added cipher_memory_search for Knowledge Retrieval

Features:

• 🔍 Semantic Search: Vector-based similarity matching
• 🎯 Configurable: Query, top_k (1-50), similarity_threshold (0.0-1.0), type filtering
• 📊 Rich Results: Detailed metadata and similarity scores
• 🚀 Performance Optimized: Efficient vector search with threshold filtering

🔧 Updated Workflow

1. User Input → Agent processes message and tools (including cipher_memory_search)
2. Response Generation → Agent generates response using available context
3. Automatic Memory Management → cipher_extract_and_operate_memory extracts and persists knowledge

📈 Improvements

• Searchability: Semantic retrieval of relevant past knowledge
• Enhanced Prompts: Improved filtering to extract important programming content only
• Non-Blocking: cipher_extract_and_operate_memory runs automatically after each interaction without blocking agent responses

Note: When running in CLI, the agent response is displayed at last (after cipher_extract_and_operate_memory is done, however, the response is already generated before this tool call. This is done because I want the response to be at the end of each interaction, without any tool usage log showing up after that. The display order can be easily switched as follows.

File: src/core/session/coversation-session.ts (lines ~60-65)

// Generate response
const response = await this.llmService.generate(input, imageDataInput, stream);

// PROGRAMMATIC ENFORCEMENT: Memory extraction runs in background (non-blocking)
this.enforceMemoryExtraction(input, response).catch(error => {
    logger.error('ConversationSession: Background memory extraction failed', { error });
});

return response; // CLI displays response IMMEDIATELY, memory extraction logs appear after

🛠️ Technical Details

• Consolidated Logic: Single tool handles extraction + operation
• Shared Utilities: Reuses functions from memory_operation.ts
• Error Handling: Graceful fallbacks and comprehensive logging
• Backward Compatibility: Deprecated tools preserved for reference

📋 Migration

• New Tools: Use cipher_extract_and_operate_memory and cipher_memory_search
• Deprecated: extract_knowledge and memory_operation (commented out)
• Config: SEARCH_MEMORY_TYPE environment variable controls search defaults

---

[ngduyanhece]

@RyanNg1403

got the test case failed! please kindly check