feat: add token-efficient recall and file compression tool

[rohitg00]

Add memory_recall format and token budget support, plus a new memory_compress_file tool with API/MCP wiring, tests, and updated tool/endpoint docs.

This pull request primarily updates documentation to reflect the addition of a new MCP tool (bringing the total to 44) and a reduction in the number of REST API endpoints (now 104). It also introduces a new assets/architecture.html file that visually explains the system architecture and the flow of context and memory within the agentmemory system.

Documentation updates:

• Updated tool and endpoint counts across README.md and AGENTS.md to show 44 MCP tools and 104 REST endpoints, including changes to badges, environment variable comments, and feature lists. [1] [2] [3] [4] [5] [6]

New system architecture visualization:

• Added assets/architecture.html, a comprehensive HTML file with diagrams and annotations illustrating how external context, memory, and context fragments interact with the computation box (model context window) in agentmemory.

Summary by CodeRabbit

• New Features

  • Added file compression capability for Markdown documents with automatic backup creation
  • Enhanced search functionality with format options (full, compact, narrative) and configurable token budgets

• Documentation

  • Added interactive architecture diagram and memory cycle visualization
  • Updated all documentation to reflect 44 MCP tools and 104 REST endpoints

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR introduces markdown file compression (mem::compress-file), extends the search function with format selection and token budgeting options, adds architectural documentation diagrams, and updates documentation/counts to reflect 44 MCP tools and 104 REST endpoints.

Changes

Cohort / File(s)	Summary
Documentation & Manifest Updates AGENTS.md, README.md, plugin/.claude-plugin/plugin.json	Updated MCP tool count from 43 → 44 and REST endpoint count from 103 → 104. Documented new memory tools (memory_compress_file, memory_patterns, memory_timeline, memory_relations) and updated configuration guidance.
Architecture Diagrams assets/architecture.html, assets/memory-cycle.html	Added self-contained static HTML pages rendering architectural diagrams: systems view with memory flow and computation boundary, and memory cycle workflow visualization with graph reshaping illustration.
Compression Feature src/functions/compress-file.ts, src/mcp/tools-registry.ts, src/mcp/server.ts, src/triggers/api.ts, src/index.ts	Introduced mem::compress-file MCP tool and api::compress-file REST endpoint for markdown compression with validation (preserves headings, URLs, code blocks), backup creation, and audit logging. Registered new function and updated tool definitions.
Search Enhancements src/functions/search.ts, src/mcp/tools-registry.ts, src/mcp/server.ts, src/triggers/api.ts	Extended mem::search with optional format parameter (full/compact/narrative) and token_budget for controlling response size. Added token estimation and result truncation logic. Updated memory_recall MCP tool and api::search endpoint to support new parameters.
Test Coverage test/compress-file.test.ts, test/search.test.ts, test/mcp-standalone.test.ts	Added test suites for compression function (path validation, summarization, backup creation, validation logic) and search enhancements (format variants, token budgeting, truncation). Updated core tools count assertion from 10 → 11.

Sequence Diagram(s)

sequenceDiagram
    actor User
    participant API as API Endpoint<br/>(api::compress-file)
    participant Func as Function<br/>(mem::compress-file)
    participant FS as File System
    participant Provider as Summarization<br/>Provider
    participant Audit as Audit Logger

    User->>API: POST /compress-file<br/>{filePath}
    API->>API: Validate filePath<br/>(non-empty string)
    API->>Func: Trigger mem::compress-file
    
    Func->>Func: Validate path<br/>(.md extension, not sensitive)
    Func->>FS: Read file contents
    FS-->>Func: Original content
    
    Func->>Func: Check if empty
    
    alt Not Empty
        Func->>Provider: summarize(content)<br/>with compression prompt
        Provider-->>Func: Compressed result
        
        Func->>Func: Strip code fence<br/>if present
        Func->>Func: Validate compression<br/>(headings, URLs, code blocks)
        
        alt Validation Passes
            Func->>FS: Write backup<br/>(filename.original.md)
            FS-->>Func: Backup created
            Func->>FS: Overwrite original<br/>with compressed content
            FS-->>Func: File updated
            Func->>Audit: Record audit event<br/>(paths, char counts)
            Audit-->>Func: Logged (errors ignored)
            Func-->>API: {success: true, ...sizes}
        else Validation Fails
            Func-->>API: {success: false, error, details}
        end
    else Empty File
        Func-->>API: {success: true, skipped}
    end
    
    API-->>User: HTTP 200 JSON response

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~90 minutes

Possibly related PRs

• docs: README polish — badges, cost table, architecture diagram #106: Modifies README.md documentation and adds/adjusts architecture diagram content alongside this PR's updates to stats and new architecture diagrams.
• feat: v0.7.0 — lessons, tool visibility, auto-consolidation, obsidian export, npx bootstrap #82: Extends MCP tools surface by modifying src/mcp/tools-registry.ts and src/mcp/server.ts to add and adjust tool definitions, paralleling this PR's tool additions.
• feat(search): add optional project/cwd filter to mem::search and /agentmemory/search #105: Modifies src/functions/search.ts and src/triggers/api.ts to extend input schemas with new optional parameters and validation logic, directly related to the search enhancements in this PR.

Poem

🐰 A file walks in, looking rather long,
Compression works—now it belongs!
With backups safe and URLs bright,
The memory cycle gleams with light! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately and concisely summarizes the two primary changes: adding token-efficient recall (via new format/token_budget parameters) and a file compression tool (mem::compress-file).

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/token-efficient-recall

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}