memory-hygiene-bench is an open benchmark that measures the cost of degraded AI agent memory and the return from consolidation: agent task success, token cost, and repeat-failure rate across three arms — no memory, degraded memory, and consolidated memory.
AI agent memory degrades over time — duplicates, contradictions, stale facts, and poisoned entries accumulate — and existing benchmarks do not price that degradation. This benchmark is an intervention axis, not another memory benchmark: existing benchmarks measure whether memory helps (STATE-Bench, MemoryArena, AMB) or whether agents resist stale memory (MemSyco-Bench); this one measures what memory degradation costs and what memory hygiene returns. It is designed to run on top of existing open environments (STATE-Bench, MemoryArena) rather than replacing them.
Every task suite runs three times with identical tasks and seeds:
- Arm A — no memory: the agent starts cold; baseline for what memory is worth at all.
- Arm B — degraded memory: the agent runs with accumulated dirty memory (duplicates, contradictions, stale and injected entries, seeded from a published error taxonomy with ground truth).
- Arm C — consolidated memory: the same memory after cleanup and consolidation.
Metrics per arm: task success rate, tokens per task, and repeat-failure rate (does the agent make the same mistake twice).
The benchmark's maintainers build a consolidation tool, so the rules are strict by construction:
- Open code, open data, fixed seeds.
- One-command replication of every reported number.
- External memory and consolidation systems are welcome and can be plugged into arms B/C.
- STATE-Bench — does memory help agents across sessions
- MemoryArena — agent memory in interdependent multi-session tasks
- AMB — agent memory benchmark and leaderboard
- MemSyco-Bench — memory-induced sycophancy, including stale-memory contamination
- Canonical recall datasets: LongMemEval, LoCoMo, HaluMem
Early development. This repository reserves the benchmark's public home; the task suites and harness follow.
- sleepsweet — engine + CLI for memory hygiene and audit
- auditable-memory-record — open specification for auditable agent memory records
- memory-hygiene-bench — this repository
Dual: code under Apache-2.0, data under CC BY 4.0.