FOSS Memory Evaluation Framework

A strategic evaluation of leading open-source AI memory systems. This repository contains submodules for each system, a qualitative evaluation rubric, and a side-by-side comparison.

Evaluation Goals

• Architectural Maturity: How well-engineered is the system?
• Ease of Integration: How fast can a developer plug it in?
• Data Integrity: Does it preserve the original context and truth?
• Operational Simplicity: What is the cost of the required infrastructure?

Content

• MEMORY_SYSTEMS_COMPARISON.md: A detailed qualitative comparison and strategic scoring.
• RUBRIC.md: The standardized scoring framework (1-10) used for the evaluation.
• compare.html: Visual comparison of benchmarks, technical architecture, and strategic maturity.

Memory Systems (Submodules)

• claude-mem: Claude Code lifecycle plugin (TS/Node).
• mem0: Pluggable multi-provider memory layer (Python/TS).
• supermemory: High-performance fact extraction engine (TS/Bun).
• mempalace: Local-first, verbatim spatial memory (Python).

Getting Started

To explore the submodules and verify the findings:

git clone --recursive https://github.com/bensig/foss-memory-eval.git
cd foss-memory-eval

Refer to RUBRIC.md for instructions on how to independently verify our scores.