Skip to content

About

Structured registry of agent memory benchmarks, with every field traced to its source

Resources

Stars

209 stars

Watchers

0 watching

Forks

Repository files navigation

memory-bench-registry

English | 中文

Browse the registry online → Search and filter by memory type, reproducibility level and year.

A structured registry of agent memory benchmarks: one YAML file per benchmark (benchmarks/<slug>.yaml). This README is generated by a script. Every field must be traceable to the paper or the official repository; where each value came from is recorded in the entry's sources. Anything that cannot be found is null.

207 entries. Reproducibility levels: unverified 0 · paper_only 37 · data_public 31 · official_script 139 · smoke_tested 0 · full_run 0

By memory type

A benchmark can belong to several types. Expand a type to see its entries, newest first.

Type What the benchmark tests
conversational_longterm Recall or reasoning over long, multi-session dialogue or chat histories (e.g. LoCoMo, LongMemEval, MSC)
personalization Using user-specific preferences, personas or profiles to tailor answers (e.g. PersonaMem, PrefEval)
episodic Recalling specific events and experiences with time, place or order ("what happened when")
semantic_knowledge_update Factual knowledge stored outside the weights (context or external store), updated correctly when facts change
procedural_workflow Distilling skills, workflows or tool use from earlier tasks and reusing them on different later tasks
coding_cross_session Coding agents that must retain information across independent sessions or tasks
parametric Memory in the model weights: knowledge editing, injection by fine-tuning, test-time training, memorization or unlearning
multi_agent_group Memory shared among several agents, or multi-party / group conversations
multimodal Remembered content includes non-text input (images, video, audio) that the system must process directly
embodied Agents act in a simulated or real environment; memory is scored through actions or task success
memory_security Attacks on or protection of memory: poisoning, injection, privacy leakage, whether deletion requests take effect

Single-input long-context tests (e.g. RULER, NIAH, LongBench) are not memory benchmarks and are not included.

conversational_longterm (81)
Name Date Links Scale Repro level
DyadMem 2026‑10 paper 3,065 conv.
61,210 Q
paper_only
RealCompanion 2026‑10 paper · code · data 10 conv.
3,235 Q
paper_only
CUE-Mem 2026‑09 paper · code · data 648 conv.
2,674 Q
official_script
DolphinBench 2026‑09 paper · code · data 3 conv.
600 Q
official_script
LoCoMo-Conv 2026‑09 paper · code · data 10 conv.
6,581 Q
official_script
LSREP 2026‑09 paper · code 4 conv.
219 Q
paper_only
MemProbe 2026‑09 paper · code · data 56 conv.
224 Q
official_script
PRAGMA 2026‑09 paper · code · data 100 conv.
400 Q
data_public
TWIST 2026‑09 paper · code · data 161 Q official_script
VoxMem 2026‑09 paper · code · data 3,196 conv.
3,196 Q
official_script
MemUse 2026‑08 paper · code · data 72 conv.
316 Q
official_script
MobileMem 2026‑08 paper · code · data 16 conv.
7,415 Q
official_script
StateMemBench 2026‑08 paper 234 conv.
322 Q
paper_only
TANGLE 2026‑08 paper 40 conv.
541 Q
paper_only
UTILMEM 2026‑08 paper · code · data 1,717 conv.
1,717 Q
data_public
Ground Truth First 2026‑07 paper · code 383 Q paper_only
InMind 2026‑07 paper · code 1 conv.
125 Q
data_public
MemOps 2026‑07 paper · code 403 conv.
2,006 Q
official_script
RUMBA 2026‑07 paper · code · data 85 conv.
1,543 Q
official_script
EvoArena 2026‑06 paper · code · data 10 conv.
505 Q
official_script
GateMem 2026‑06 paper · code · data 91 conv.
2,218 Q
official_script
H2HMem 2026‑06 paper · code · data 25 conv.
2,236 Q
official_script
HUSH-Bench 2026‑06 paper 10 conv.
2,400 Q
paper_only
MemAudit (formerly MEMPROBE) 2026‑06 paper · code 50 conv.
1,550 Q
official_script
MemoryDocDataSet 2026‑06 paper 50 conv.
1,000 Q
paper_only
MemTrace 2026‑06 paper · code 20 conv.
15,422 Q
official_script
SubtleMemory 2026‑06 paper · code · data 10 conv.
1,522 Q
official_script
EvoMemBench 2026‑05 paper · code 5,754 Q official_script
GroupMemBench 2026‑05 paper · code · data 4 conv.
745 Q
official_script
MemAudit 2026‑05 paper · code · data 87 Q official_script
MemEye 2026‑05 paper · code · data 8 conv.
371 Q
official_script
Momento 2026‑05 paper · code · data 161 Q official_script
RefMem-Bench 2026‑05 paper 35 conv.
20,279 Q
paper_only
RHELM 2026‑05 paper · code · data 10 conv.
1,305 Q
official_script
STALE 2026‑05 paper · code · data 400 conv.
1,200 Q
official_script
WorldMemArena 2026‑05 paper · code · data 461 conv.
24,258 Q
official_script
MemHome / MemHomeLife 2026‑04 paper 389 Q paper_only
StratMem-Bench 2026‑04 paper · code 657 Q data_public
AMB (Agent Memory Benchmark) 2026‑03 code · data — official_script
AMemGym 2026‑03 paper · code · data 20 conv.
200 Q
official_script
Persona-MME 2026‑03 paper · code · data 200 conv.
2,034 Q
official_script
RealPref 2026‑03 paper · code 100 conv.
1,300 Q
official_script
ES-MemEval 2026‑02 paper · code · data 18 conv.
1,209 Q
official_script
HLME 2026‑02 paper — paper_only
LoCoMo-Plus 2026‑02 paper · code 10 conv.
401 Q
official_script
Neuromem 2026‑02 paper · code 10 conv. official_script
CAME-Bench 2026‑01 paper · code · data 14 conv.
373 Q
official_script
EvolMem 2026‑01 paper · code 1,600 conv. paper_only
Mem-Gallery 2026‑01 paper · code · data 20 conv.
1,711 Q
official_script
Mem2ActBench 2026‑01 paper · code 2,029 conv.
400 Q
data_public
MemoryRewardBench 2026‑01 paper · code · data 2,400 Q official_script
RealMem 2026‑01 paper · code 10 conv.
1,415 Q
official_script
PersonaMem-v2 2025‑12 paper · code · data 195 conv.
5,000 Q
official_script
ConvoMem 2025‑11 paper · code · data 75,336 Q official_script
HaluMem 2025‑11 paper · code · data 20 conv.
3,467 Q
official_script
BEAM 2025‑10 paper · code · data 100 conv.
2,000 Q
official_script
MemoryBench 2025‑10 paper · code · data 4,063 Q official_script
ZH-4O 2025‑09 paper · code 28 conv.
1,068 Q
official_script
OdysseyBench 2025‑08 paper · code 602 conv.
602 Q
official_script
LOCCO 2025‑07 paper · code 100 conv.
2,981 Q
data_public
MemoryAgentBench 2025‑07 paper · code · data 2,071 Q official_script
MemBench 2025‑06 paper · code · data 820 conv.
820 Q
data_public
MS-TOD 2025‑05 paper · code 132 conv.
956 Q
official_script
PersonaMem 2025‑04 paper · code · data 60 conv.
2,727 Q
official_script
ImplexConv 2025‑03 paper · code · data 2,500 conv. data_public
MemoryCode 2025‑02 paper · code · data 360 conv. official_script
PersonaBench 2025‑02 paper · code 15 conv.
582 Q
official_script
PrefEval 2025‑02 paper · code · data 3,000 Q official_script
REALTALK 2025‑02 paper · code 10 conv.
728 Q
official_script
TReMu 2025‑02 paper 600 Q paper_only
LongMemEval 2024‑10 paper · code · data 500 conv.
500 Q
official_script
GoodAI LTM Benchmark 2024‑09 paper · code 1 conv.
33 Q
official_script
MADial-Bench 2024‑09 paper · code 160 conv. official_script
DialSim 2024‑06 paper · code · data 3 conv. official_script
TeaFarm 2024‑06 paper · code · data 100 conv.
200 Q
data_public
LoCoMo 2024‑02 paper · code 10 conv.
1,986 Q
official_script
PerLTQA 2024‑02 paper · code 3,409 conv.
8,593 Q
data_public
DMR (Deep Memory Retrieval) 2023‑10 paper · code · data 500 conv.
500 Q
data_public
MemoryBank 2023‑05 paper · code 15 conv.
194 Q
data_public
MemoEval (SCM Self-Controlled Memory evaluation dataset) 2023‑04 paper · code 18 conv.
105 Q
data_public
Multi-Session Chat (MSC) 2021‑07 paper · code · data 501 conv. official_script
personalization (68)
Name Date Links Scale Repro level
CMSC-E 2026‑10 paper · code · data — official_script
DyadMem 2026‑10 paper 3,065 conv.
61,210 Q
paper_only
RealCompanion 2026‑10 paper · code · data 10 conv.
3,235 Q
paper_only
ServeLearnBench 2026‑10 paper · code 3,210 Q official_script
CUE-Mem 2026‑09 paper · code · data 648 conv.
2,674 Q
official_script
DolphinBench 2026‑09 paper · code · data 3 conv.
600 Q
official_script
Lapse 2026‑09 paper 128 Q paper_only
MemCalib 2026‑09 paper · code · data 1,500 Q data_public
MemProbe 2026‑09 paper · code · data 56 conv.
224 Q
official_script
PairPref 2026‑09 paper 1,227 Q paper_only
PRAGMA 2026‑09 paper · code · data 100 conv.
400 Q
data_public
MemUse 2026‑08 paper · code · data 72 conv.
316 Q
official_script
MobileMem 2026‑08 paper · code · data 16 conv.
7,415 Q
official_script
PAST-Bench 2026‑08 paper · code 26 conv.
204 Q
official_script
TANGLE 2026‑08 paper 40 conv.
541 Q
paper_only
UTILMEM 2026‑08 paper · code · data 1,717 conv.
1,717 Q
data_public
CAPA 2026‑07 paper · code 60 conv.
300 Q
data_public
Ground Truth First 2026‑07 paper · code 383 Q paper_only
InMind 2026‑07 paper · code 1 conv.
125 Q
data_public
MemSyco-Bench 2026‑07 paper · code · data 1,550 conv.
1,550 Q
official_script
MERIT 2026‑07 paper · code — official_script
EvoArena 2026‑06 paper · code · data 10 conv.
505 Q
official_script
HUSH-Bench 2026‑06 paper 10 conv.
2,400 Q
paper_only
MemAudit (formerly MEMPROBE) 2026‑06 paper · code 50 conv.
1,550 Q
official_script
MIST 2026‑06 paper 598 conv.
598 Q
paper_only
StreamMemBench 2026‑06 paper · code 16,214 Q official_script
SubtleMemory 2026‑06 paper · code · data 10 conv.
1,522 Q
official_script
GroupMemBench 2026‑05 paper · code · data 4 conv.
745 Q
official_script
Momento 2026‑05 paper · code · data 161 Q official_script
RefMem-Bench 2026‑05 paper 35 conv.
20,279 Q
paper_only
RHELM 2026‑05 paper · code · data 10 conv.
1,305 Q
official_script
STALE 2026‑05 paper · code · data 400 conv.
1,200 Q
official_script
MemHome / MemHomeLife 2026‑04 paper 389 Q paper_only
AMB (Agent Memory Benchmark) 2026‑03 code · data — official_script
AMemGym 2026‑03 paper · code · data 20 conv.
200 Q
official_script
ATM-Bench 2026‑03 paper · code · data 1,038 Q official_script
LifeBench 2026‑03 paper · code · data 10 conv.
2,003 Q
official_script
Persona-MME 2026‑03 paper · code · data 200 conv.
2,034 Q
official_script
RealPref 2026‑03 paper · code 100 conv.
1,300 Q
official_script
ES-MemEval 2026‑02 paper · code · data 18 conv.
1,209 Q
official_script
LoCoMo-Plus 2026‑02 paper · code 10 conv.
401 Q
official_script
MemoryArena 2026‑02 paper · code · data 701 conv.
4,850 Q
official_script
PersistBench 2026‑02 paper · code 500 Q official_script
AndroidIntent 2026‑01 paper · code · data 91 conv.
990 Q
official_script
CloneMem 2026‑01 paper · code · data 10 conv.
1,183 Q
official_script
EvolMem 2026‑01 paper · code 1,600 conv. paper_only
KnowMe-Bench 2026‑01 paper · code · data 3 conv.
2,580 Q
official_script
Mem2ActBench 2026‑01 paper · code 2,029 conv.
400 Q
data_public
RealMem 2026‑01 paper · code 10 conv.
1,415 Q
official_script
RPEval 2026‑01 paper · code 953 Q official_script
PersonaMem-v2 2025‑12 paper · code · data 195 conv.
5,000 Q
official_script
BELA 2025‑11 paper · code · data 40 conv. official_script
ConvoMem 2025‑11 paper · code · data 75,336 Q official_script
HaluMem 2025‑11 paper · code · data 20 conv.
3,467 Q
official_script
BEAM 2025‑10 paper · code · data 100 conv.
2,000 Q
official_script
Stateful SWE-bench 2025‑10 paper · code · data 15 conv.
500 Q
official_script
ZH-4O 2025‑09 paper · code 28 conv.
1,068 Q
official_script
MPR 2025‑08 paper · code · data 12 conv.
108,000 Q
official_script
MemBench 2025‑06 paper · code · data 820 conv.
820 Q
data_public
MEMENTO 2025‑05 paper · code 438 Q official_script
PersonaMem 2025‑04 paper · code · data 60 conv.
2,727 Q
official_script
ImplexConv 2025‑03 paper · code · data 2,500 conv. data_public
PersonaBench 2025‑02 paper · code 15 conv.
582 Q
official_script
PrefEval 2025‑02 paper · code · data 3,000 Q official_script
LongMemEval 2024‑10 paper · code · data 500 conv.
500 Q
official_script
MemDaily 2024‑09 paper · code 2,954 conv.
2,954 Q
official_script
PerLTQA 2024‑02 paper · code 3,409 conv.
8,593 Q
data_public
Multi-Session Chat (MSC) 2021‑07 paper · code · data 501 conv. official_script
episodic (56)
Name Date Links Scale Repro level
DyadMem 2026‑10 paper 3,065 conv.
61,210 Q
paper_only
RealCompanion 2026‑10 paper · code · data 10 conv.
3,235 Q
paper_only
APM-Bench 2026‑09 paper · code · data 104 conv.
2,719 Q
paper_only
DolphinBench 2026‑09 paper · code · data 3 conv.
600 Q
official_script
EmbodiedMemory-Bench (EMem-Bench) 2026‑09 paper · code · data 2,554 conv.
2,554 Q
official_script
ICM-Bench 2026‑09 paper · code · data 839 conv.
1,217 Q
official_script
MemTransfer 2026‑09 paper 100 conv.
100 Q
paper_only
VoxMem 2026‑09 paper · code · data 3,196 conv.
3,196 Q
official_script
MobileMem 2026‑08 paper · code · data 16 conv.
7,415 Q
official_script
PAST-Bench 2026‑08 paper · code 26 conv.
204 Q
official_script
AgenticSTS 2026‑07 paper · code · data — official_script
Ground Truth First 2026‑07 paper · code 383 Q paper_only
MERIT 2026‑07 paper · code — official_script
RUMBA 2026‑07 paper · code · data 85 conv.
1,543 Q
official_script
DMV-Bench 2026‑06 paper · code · data 35,410 Q official_script
MemAudit (formerly MEMPROBE) 2026‑06 paper · code 50 conv.
1,550 Q
official_script
MemoryDocDataSet 2026‑06 paper 50 conv.
1,000 Q
paper_only
StreamMemBench 2026‑06 paper · code 16,214 Q official_script
EgoStream 2026‑05 paper 2,250 Q paper_only
GroupMemBench 2026‑05 paper · code · data 4 conv.
745 Q
official_script
LongMemEval-V2 2026‑05 paper · code · data 1,540 conv.
451 Q
official_script
RHELM 2026‑05 paper · code · data 10 conv.
1,305 Q
official_script
RoboMemArena 2026‑05 paper · code · data — official_script
SuperMemory-VQA 2026‑05 paper · code · data 4,853 Q data_public
WorldMemArena 2026‑05 paper · code · data 461 conv.
24,258 Q
official_script
AMB (Agent Memory Benchmark) 2026‑03 code · data — official_script
ATM-Bench 2026‑03 paper · code · data 1,038 Q official_script
LifeBench 2026‑03 paper · code · data 10 conv.
2,003 Q
official_script
AMA-Bench 2026‑02 paper · code · data 208 conv.
2,496 Q
official_script
ES-MemEval 2026‑02 paper · code · data 18 conv.
1,209 Q
official_script
MemoryArena 2026‑02 paper · code · data 701 conv.
4,850 Q
official_script
CloneMem 2026‑01 paper · code · data 10 conv.
1,183 Q
official_script
KnowMe-Bench 2026‑01 paper · code · data 3 conv.
2,580 Q
official_script
Mem-Gallery 2026‑01 paper · code · data 20 conv.
1,711 Q
official_script
RealMem 2026‑01 paper · code 10 conv.
1,415 Q
official_script
HaluMem 2025‑11 paper · code · data 20 conv.
3,467 Q
official_script
BEAM 2025‑10 paper · code · data 100 conv.
2,000 Q
official_script
MemTrack 2025‑10 paper · data 47 conv. data_public
TeleEgo 2025‑10 paper · code · data 5 conv.
3,291 Q
official_script
FlashAdventure 2025‑09 paper · code 34 Q official_script
M3-Bench 2025‑08 paper · code · data 1,020 conv.
4,490 Q
official_script
StuLife 2025‑08 paper · code · data 1,284 Q official_script
MemoryAgentBench 2025‑07 paper · code · data 2,071 Q official_script
StoryBench 2025‑06 paper 86 Q paper_only
MEMENTO 2025‑05 paper · code 438 Q official_script
LifeState-Bench 2025‑03 paper 311 Q paper_only
REALTALK 2025‑02 paper · code 10 conv.
728 Q
official_script
TReMu 2025‑02 paper 600 Q paper_only
Episodic Memory Benchmark 2025‑01 paper · code · data 686 Q official_script
LongMemEval 2024‑10 paper · code · data 500 conv.
500 Q
official_script
GoodAI LTM Benchmark 2024‑09 paper · code 1 conv.
33 Q
official_script
MADial-Bench 2024‑09 paper · code 160 conv. official_script
DialSim 2024‑06 paper · code · data 3 conv. official_script
LoCoMo 2024‑02 paper · code 10 conv.
1,986 Q
official_script
PerLTQA 2024‑02 paper · code 3,409 conv.
8,593 Q
data_public
MemoryBank 2023‑05 paper · code 15 conv.
194 Q
data_public
semantic_knowledge_update (74)
Name Date Links Scale Repro level
DyadMem 2026‑10 paper 3,065 conv.
61,210 Q
paper_only
Knowing the Store 2026‑10 paper · code · data 444 Q official_script
ServeLearnBench 2026‑10 paper · code 3,210 Q official_script
APM-Bench 2026‑09 paper · code · data 104 conv.
2,719 Q
paper_only
Controlled In-Context Memory 2026‑09 paper 1,200 conv.
1,200 Q
paper_only
EmbodiedMemory-Bench (EMem-Bench) 2026‑09 paper · code · data 2,554 conv.
2,554 Q
official_script
EvoPathBench 2026‑09 paper · code 1,800 conv.
3,600 Q
official_script
Impact Is Not Invalidation 2026‑09 paper 10,369 Q paper_only
Lapse 2026‑09 paper 128 Q paper_only
LSREP 2026‑09 paper · code 4 conv.
219 Q
paper_only
MemProbe 2026‑09 paper · code · data 56 conv.
224 Q
official_script
MemRiskBench 2026‑09 paper · code · data 120 conv.
120 Q
official_script
PRAGMA 2026‑09 paper · code · data 100 conv.
400 Q
data_public
VoxMem 2026‑09 paper · code · data 3,196 conv.
3,196 Q
official_script
agent-memory-bench 2026‑08 code · data 196 conv.
26 Q
official_script
DreamBench-SWE 2026‑08 paper · code 60 conv.
60 Q
data_public
MobileMem 2026‑08 paper · code · data 16 conv.
7,415 Q
official_script
PAST-Bench 2026‑08 paper · code 26 conv.
204 Q
official_script
StateMemBench 2026‑08 paper 234 conv.
322 Q
paper_only
TANGLE 2026‑08 paper 40 conv.
541 Q
paper_only
Ground Truth First 2026‑07 paper · code 383 Q paper_only
MemOps 2026‑07 paper · code 403 conv.
2,006 Q
official_script
MemSyco-Bench 2026‑07 paper · code · data 1,550 conv.
1,550 Q
official_script
MERIT 2026‑07 paper · code — official_script
PM-Bench 2026‑07 paper · code 1 conv.
81 Q
official_script
RUMBA 2026‑07 paper · code · data 85 conv.
1,543 Q
official_script
Continual Learning Bench (CL-Bench) 2026‑06 paper · code · data 6 conv.
301 Q
official_script
EvoArena 2026‑06 paper · code · data 10 conv.
505 Q
official_script
ForgetEval 2026‑06 paper · code 385 Q official_script
GateMem 2026‑06 paper · code · data 91 conv.
2,218 Q
official_script
H2HMem 2026‑06 paper · code · data 25 conv.
2,236 Q
official_script
MemoryDocDataSet 2026‑06 paper 50 conv.
1,000 Q
paper_only
MemTrace 2026‑06 paper · code 20 conv.
15,422 Q
official_script
SubtleMemory 2026‑06 paper · code · data 10 conv.
1,522 Q
official_script
EvoMemBench 2026‑05 paper · code 5,754 Q official_script
GroupMemBench 2026‑05 paper · code · data 4 conv.
745 Q
official_script
MemAudit 2026‑05 paper · code · data 87 Q official_script
MemEye 2026‑05 paper · code · data 8 conv.
371 Q
official_script
Momento 2026‑05 paper · code · data 161 Q official_script
PrecisionMemBench 2026‑05 paper · code · data 2 conv.
89 Q
official_script
RHELM 2026‑05 paper · code · data 10 conv.
1,305 Q
official_script
STALE 2026‑05 paper · code · data 400 conv.
1,200 Q
official_script
WorldMemArena 2026‑05 paper · code · data 461 conv.
24,258 Q
official_script
MemHome / MemHomeLife 2026‑04 paper 389 Q paper_only
AMB (Agent Memory Benchmark) 2026‑03 code · data — official_script
AMemGym 2026‑03 paper · code · data 20 conv.
200 Q
official_script
ATM-Bench 2026‑03 paper · code · data 1,038 Q official_script
LifeBench 2026‑03 paper · code · data 10 conv.
2,003 Q
official_script
Persona-MME 2026‑03 paper · code · data 200 conv.
2,034 Q
official_script
AMA-Bench 2026‑02 paper · code · data 208 conv.
2,496 Q
official_script
ES-MemEval 2026‑02 paper · code · data 18 conv.
1,209 Q
official_script
HLME 2026‑02 paper — paper_only
Neuromem 2026‑02 paper · code 10 conv. official_script
CAME-Bench 2026‑01 paper · code · data 14 conv.
373 Q
official_script
CloneMem 2026‑01 paper · code · data 10 conv.
1,183 Q
official_script
EvolMem 2026‑01 paper · code 1,600 conv. paper_only
Mem-Gallery 2026‑01 paper · code · data 20 conv.
1,711 Q
official_script
Mem2ActBench 2026‑01 paper · code 2,029 conv.
400 Q
data_public
RealMem 2026‑01 paper · code 10 conv.
1,415 Q
official_script
ConvoMem 2025‑11 paper · code · data 75,336 Q official_script
HaluMem 2025‑11 paper · code · data 20 conv.
3,467 Q
official_script
BEAM 2025‑10 paper · code · data 100 conv.
2,000 Q
official_script
MemTrack 2025‑10 paper · data 47 conv. data_public
MemoryAgentBench 2025‑07 paper · code · data 2,071 Q official_script
MemBench 2025‑06 paper · code · data 820 conv.
820 Q
data_public
PersonaMem 2025‑04 paper · code · data 60 conv.
2,727 Q
official_script
LifeState-Bench 2025‑03 paper 311 Q paper_only
MemoryCode 2025‑02 paper · code · data 360 conv. official_script
Minerva 2025‑02 paper · code 1,110 Q official_script
PersonaBench 2025‑02 paper · code 15 conv.
582 Q
official_script
EvoWiki 2024‑12 paper · code · data 10,264 Q paper_only
LongMemEval 2024‑10 paper · code · data 500 conv.
500 Q
official_script
GoodAI LTM Benchmark 2024‑09 paper · code 1 conv.
33 Q
official_script
MQuAKE 2023‑05 paper · code 3,000 Q official_script
procedural_workflow (53)
Name Date Links Scale Repro level
Learn2Play Bench 2026‑10 paper · code 20 Q data_public
MetaKernelBench 2026‑10 paper · code 74 Q official_script
MLE-InsightBench 2026‑10 paper 12 Q paper_only
ServeLearnBench 2026‑10 paper · code 3,210 Q official_script
ConsolidationBench 2026‑09 paper — paper_only
EconSkills 2026‑09 paper · data 360 Q data_public
EngramBench 2026‑09 paper · code 30 conv.
13 Q
official_script
EvoHarnessBench 2026‑09 paper · data 802 Q data_public
EvoPathBench 2026‑09 paper · code 1,800 conv.
3,600 Q
official_script
VibeMemBench 2026‑09 paper · code 3,634 conv.
111 Q
paper_only
ContinualSkillBench 2026‑08 paper · code 500 Q official_script
DreamBench-SWE 2026‑08 paper · code 60 conv.
60 Q
data_public
FinEvo-Bench 2026‑08 paper 120 Q paper_only
GDPevo 2026‑08 paper · code 240 Q official_script
MemTrapBench 2026‑08 paper · code 1,050 conv.
1,050 Q
official_script
PAST-Bench 2026‑08 paper · code 26 conv.
204 Q
official_script
PATH-Bench 2026‑08 paper 120 Q paper_only
S3Gym 2026‑08 paper — paper_only
AgenticSTS 2026‑07 paper · code · data — official_script
AgentStream 2026‑07 paper · code 300 Q official_script
EvoAgentBench 2026‑07 paper · code · data 267 Q official_script
SLVMBench 2026‑07 paper · data 2,261 Q data_public
AFTER 2026‑06 paper · code · data 382 Q data_public
AgentCL 2026‑06 paper · data — data_public
AhaBench 2026‑06 paper · code — official_script
Continual Learning Bench (CL-Bench) 2026‑06 paper · code · data 6 conv.
301 Q
official_script
EvoArena 2026‑06 paper · code · data 10 conv.
505 Q
official_script
FinEvolveBench 2026‑06 paper · code · data — official_script
MedEvoEval 2026‑06 paper 700 Q paper_only
OmniGameArena 2026‑06 paper · code · data — official_script
SEAGym 2026‑06 paper · code 55 Q official_script
BenchTrace 2026‑05 paper · code · data 1,821 conv.
4,391 Q
official_script
EvoMemBench 2026‑05 paper · code 5,754 Q official_script
LongMemEval-V2 2026‑05 paper · code · data 1,540 conv.
451 Q
official_script
SeqMem-Eval 2026‑05 paper · code 3,726 Q official_script
SkillEvolBench 2026‑05 paper · code · data 180 Q official_script
STATE-Bench 2026‑05 code 300 conv.
150 Q
official_script
SEA-Eval 2026‑04 paper · code 92 conv.
32 Q
data_public
SkillFlow 2026‑04 paper · code · data 166 Q official_script
SkillLearnBench 2026‑04 paper · code 100 Q official_script
MemoryArena 2026‑02 paper · code · data 701 conv.
4,850 Q
official_script
SkillCraft 2026‑02 paper · code 126 Q official_script
SWE-ContextBench 2026‑02 paper · code · data 376 Q official_script
Trust-Memevo 2026‑02 paper · code 2,329 Q paper_only
TraineeBench 2026‑01 paper · code 50 conv.
203 Q
official_script
BELA 2025‑11 paper · code · data 40 conv. official_script
Evo-Memory 2025‑11 paper 3,716 Q paper_only
MemoryBench 2025‑10 paper · code · data 4,063 Q official_script
StuLife 2025‑08 paper · code · data 1,284 Q official_script
EvaLearn 2025‑06 paper · code 182 conv.
648 Q
official_script
SWE-Bench-CL 2025‑06 paper · code · data 8 conv.
273 Q
official_script
LifelongAgentBench 2025‑05 paper · code · data 1,396 Q official_script
StreamBench 2024‑06 paper · code · data 9,702 Q official_script
coding_cross_session (17)
Name Date Links Scale Repro level
MetaKernelBench 2026‑10 paper · code 74 Q official_script
EngramBench 2026‑09 paper · code 30 conv.
13 Q
official_script
VibeMemBench 2026‑09 paper · code 3,634 conv.
111 Q
paper_only
agent-memory-bench 2026‑08 code · data 196 conv.
26 Q
official_script
DreamBench-SWE 2026‑08 paper · code 60 conv.
60 Q
data_public
CAPA 2026‑07 paper · code 60 conv.
300 Q
data_public
ChainSWE 2026‑07 paper · data 100 conv.
304 Q
data_public
EvoAgentBench 2026‑07 paper · code · data 267 Q official_script
Insecure Coding Preferences in Long-Term Memory (MemSecurity) 2026‑07 paper · code 194 Q official_script
AgentCL 2026‑06 paper · data — data_public
Continual Learning Bench (CL-Bench) 2026‑06 paper · code · data 6 conv.
301 Q
official_script
EvoArena 2026‑06 paper · code · data 10 conv.
505 Q
official_script
AMB (Agent Memory Benchmark) 2026‑03 code · data — official_script
SWE-ContextBench 2026‑02 paper · code · data 376 Q official_script
Stateful SWE-bench 2025‑10 paper · code · data 15 conv.
500 Q
official_script
SWE-Bench-CL 2025‑06 paper · code · data 8 conv.
273 Q
official_script
MemoryCode 2025‑02 paper · code · data 360 conv. official_script
parametric (27)
Name Date Links Scale Repro level
Cross-lingual Unlearning Tensor 2026‑09 paper · code 4,000 Q paper_only
IDUnlearn-Bench 2026‑09 paper · code · data 4,860 Q data_public
MemoReason 2026‑09 paper · code · data 1,200 Q official_script
MemToC 2026‑08 paper · code 6,504 Q data_public
S3Gym 2026‑08 paper — paper_only
ForgetBench 2026‑07 paper 6,431 Q paper_only
MemOps 2026‑07 paper · code 403 conv.
2,006 Q
official_script
MPR 2025‑08 paper · code · data 12 conv.
108,000 Q
official_script
LOCCO 2025‑07 paper · code 100 conv.
2,981 Q
data_public
OpenUnlearning 2025‑06 paper · code · data 400 Q official_script
UniEdit 2025‑05 paper · code · data 311,142 Q official_script
WikiDYK 2025‑05 paper · code · data 77,180 Q official_script
KUP (Knowledge Update Playground) 2025‑04 paper · code · data 6,260 Q official_script
TiC-LM 2025‑04 paper · code — official_script
LifeState-Bench 2025‑03 paper 311 Q paper_only
WikiBigEdit 2025‑03 paper · code · data 502,382 Q official_script
QAEdit 2025‑02 paper · code · data 19,249 Q official_script
Episodic Memory Benchmark 2025‑01 paper · code · data 686 Q official_script
EvoWiki 2024‑12 paper · code · data 10,264 Q paper_only
HalluEditBench 2024‑10 paper · code · data 6,738 Q official_script
CodeUpdateArena 2024‑07 paper · code · data 670 Q official_script
KnowEdit 2024‑01 paper · code · data 5,653 Q official_script
AToKe 2023‑12 paper · code 8,820 Q data_public
ConflictEdit / RoundEdit 2023‑10 paper · code 2,500 Q official_script
MMEdit 2023‑10 paper · code · data 3,093 Q official_script
RippleEdits 2023‑07 paper · code 4,000 Q official_script
MQuAKE 2023‑05 paper · code 3,000 Q official_script
multi_agent_group (8)
Name Date Links Scale Repro level
CoMemBench 2026‑09 paper 800 Q paper_only
Correlated Promotion Benchmark (CPB) 2026‑09 paper · code · data 100 conv. official_script
GateMem 2026‑06 paper · code · data 91 conv.
2,218 Q
official_script
H2HMem 2026‑06 paper · code · data 25 conv.
2,236 Q
official_script
MemoryDocDataSet 2026‑06 paper 50 conv.
1,000 Q
paper_only
GroupMemBench 2026‑05 paper · code · data 4 conv.
745 Q
official_script
LifeState-Bench 2025‑03 paper 311 Q paper_only
DialSim 2024‑06 paper · code · data 3 conv. official_script
multimodal (28)
Name Date Links Scale Repro level
APM-Bench 2026‑09 paper · code · data 104 conv.
2,719 Q
paper_only
CUE-Mem 2026‑09 paper · code · data 648 conv.
2,674 Q
official_script
EmbodiedMemory-Bench (EMem-Bench) 2026‑09 paper · code · data 2,554 conv.
2,554 Q
official_script
ICM-Bench 2026‑09 paper · code · data 839 conv.
1,217 Q
official_script
IDUnlearn-Bench 2026‑09 paper · code · data 4,860 Q data_public
Video2Skill 2026‑09 paper · data 172 conv.
1,671 Q
data_public
VoxMem 2026‑09 paper · code · data 3,196 conv.
3,196 Q
official_script
MobileMem 2026‑08 paper · code · data 16 conv.
7,415 Q
official_script
Sequential-EQA 2026‑07 paper · code 50 conv.
498 Q
official_script
SLVMBench 2026‑07 paper · data 2,261 Q data_public
DMV-Bench 2026‑06 paper · code · data 35,410 Q official_script
H2HMem 2026‑06 paper · code · data 25 conv.
2,236 Q
official_script
OmniGameArena 2026‑06 paper · code · data — official_script
EgoStream 2026‑05 paper 2,250 Q paper_only
LongMemEval-V2 2026‑05 paper · code · data 1,540 conv.
451 Q
official_script
MBench 2026‑05 paper · code · data 1,040 Q official_script
MemEye 2026‑05 paper · code · data 8 conv.
371 Q
official_script
RoboMemArena 2026‑05 paper · code · data — official_script
SuperMemory-VQA 2026‑05 paper · code · data 4,853 Q data_public
WorldMemArena 2026‑05 paper · code · data 461 conv.
24,258 Q
official_script
ATM-Bench 2026‑03 paper · code · data 1,038 Q official_script
Persona-MME 2026‑03 paper · code · data 200 conv.
2,034 Q
official_script
RMBench 2026‑03 paper · code · data — official_script
Mem-Gallery 2026‑01 paper · code · data 20 conv.
1,711 Q
official_script
TeleEgo 2025‑10 paper · code · data 5 conv.
3,291 Q
official_script
FlashAdventure 2025‑09 paper · code 34 Q official_script
M3-Bench 2025‑08 paper · code · data 1,020 conv.
4,490 Q
official_script
MMEdit 2023‑10 paper · code · data 3,093 Q official_script
embodied (12)
Name Date Links Scale Repro level
EmbodiedMemory-Bench (EMem-Bench) 2026‑09 paper · code · data 2,554 conv.
2,554 Q
official_script
MemTransfer 2026‑09 paper 100 conv.
100 Q
paper_only
Sequential-EQA 2026‑07 paper · code 50 conv.
498 Q
official_script
AgentCL 2026‑06 paper · data — data_public
OmniGameArena 2026‑06 paper · code · data — official_script
BenchTrace 2026‑05 paper · code · data 1,821 conv.
4,391 Q
official_script
EvoMemBench 2026‑05 paper · code 5,754 Q official_script
RoboMemArena 2026‑05 paper · code · data — official_script
SeqMem-Eval 2026‑05 paper · code 3,726 Q official_script
RMBench 2026‑03 paper · code · data — official_script
Evo-Memory 2025‑11 paper 3,716 Q paper_only
MEMENTO 2025‑05 paper · code 438 Q official_script
memory_security (18)
Name Date Links Scale Repro level
ConsolidationBench 2026‑09 paper — paper_only
Correlated Promotion Benchmark (CPB) 2026‑09 paper · code · data 100 conv. official_script
Cross-lingual Unlearning Tensor 2026‑09 paper · code 4,000 Q paper_only
IDUnlearn-Bench 2026‑09 paper · code · data 4,860 Q data_public
MemRiskBench 2026‑09 paper · code · data 120 conv.
120 Q
official_script
TWIST 2026‑09 paper · code · data 161 Q official_script
MemTrapBench 2026‑08 paper · code 1,050 conv.
1,050 Q
official_script
Ground Truth First 2026‑07 paper · code 383 Q paper_only
Insecure Coding Preferences in Long-Term Memory (MemSecurity) 2026‑07 paper · code 194 Q official_script
MemOps 2026‑07 paper · code 403 conv.
2,006 Q
official_script
MemSecBench 2026‑07 paper 310 Q paper_only
RUMBA 2026‑07 paper · code · data 85 conv.
1,543 Q
official_script
ForgetEval 2026‑06 paper · code 385 Q official_script
GateMem 2026‑06 paper · code · data 91 conv.
2,218 Q
official_script
HUSH-Bench 2026‑06 paper 10 conv.
2,400 Q
paper_only
MPBench 2026‑06 paper · code 3,240 Q data_public
PersonaMem-v2 2025‑12 paper · code · data 195 conv.
5,000 Q
official_script
OpenUnlearning 2025‑06 paper · code · data 400 Q official_script

All entries

Grouped by year, newest first. Click a name to see all fields and their sources.

2026 (143)

Name Month Types Links Scale Repro level
CMSC-E 10 personalization paper · code · data — official_script
DyadMem 10 conversational_longterm, personalization, episodic, semantic_knowledge_update paper 3,065 conv.
61,210 Q
paper_only
Knowing the Store 10 semantic_knowledge_update paper · code · data 444 Q official_script
Learn2Play Bench 10 procedural_workflow paper · code 20 Q data_public
MetaKernelBench 10 procedural_workflow, coding_cross_session paper · code 74 Q official_script
MLE-InsightBench 10 procedural_workflow paper 12 Q paper_only
RealCompanion 10 conversational_longterm, personalization, episodic paper · code · data 10 conv.
3,235 Q
paper_only
ServeLearnBench 10 procedural_workflow, semantic_knowledge_update, personalization paper · code 3,210 Q official_script
APM-Bench 09 episodic, semantic_knowledge_update, multimodal paper · code · data 104 conv.
2,719 Q
paper_only
CoMemBench 09 multi_agent_group paper 800 Q paper_only
ConsolidationBench 09 procedural_workflow, memory_security paper — paper_only
Controlled In-Context Memory 09 semantic_knowledge_update paper 1,200 conv.
1,200 Q
paper_only
Correlated Promotion Benchmark (CPB) 09 multi_agent_group, memory_security paper · code · data 100 conv. official_script
Cross-lingual Unlearning Tensor 09 parametric, memory_security paper · code 4,000 Q paper_only
CUE-Mem 09 conversational_longterm, personalization, multimodal paper · code · data 648 conv.
2,674 Q
official_script
DolphinBench 09 conversational_longterm, personalization, episodic paper · code · data 3 conv.
600 Q
official_script
EconSkills 09 procedural_workflow paper · data 360 Q data_public
EmbodiedMemory-Bench (EMem-Bench) 09 embodied, episodic, semantic_knowledge_update, multimodal paper · code · data 2,554 conv.
2,554 Q
official_script
EngramBench 09 procedural_workflow, coding_cross_session paper · code 30 conv.
13 Q
official_script
EvoHarnessBench 09 procedural_workflow paper · data 802 Q data_public
EvoPathBench 09 procedural_workflow, semantic_knowledge_update paper · code 1,800 conv.
3,600 Q
official_script
ICM-Bench 09 episodic, multimodal paper · code · data 839 conv.
1,217 Q
official_script
IDUnlearn-Bench 09 parametric, memory_security, multimodal paper · code · data 4,860 Q data_public
Impact Is Not Invalidation 09 semantic_knowledge_update paper 10,369 Q paper_only
Lapse 09 personalization, semantic_knowledge_update paper 128 Q paper_only
LoCoMo-Conv 09 conversational_longterm paper · code · data 10 conv.
6,581 Q
official_script
LSREP 09 conversational_longterm, semantic_knowledge_update paper · code 4 conv.
219 Q
paper_only
MemCalib 09 personalization paper · code · data 1,500 Q data_public
MemoReason 09 parametric paper · code · data 1,200 Q official_script
MemProbe 09 conversational_longterm, personalization, semantic_knowledge_update paper · code · data 56 conv.
224 Q
official_script
MemRiskBench 09 semantic_knowledge_update, memory_security paper · code · data 120 conv.
120 Q
official_script
MemTransfer 09 episodic, embodied paper 100 conv.
100 Q
paper_only
PairPref 09 personalization paper 1,227 Q paper_only
PRAGMA 09 conversational_longterm, personalization, semantic_knowledge_update paper · code · data 100 conv.
400 Q
data_public
TWIST 09 conversational_longterm, memory_security paper · code · data 161 Q official_script
VibeMemBench 09 coding_cross_session, procedural_workflow paper · code 3,634 conv.
111 Q
paper_only
Video2Skill 09 multimodal paper · data 172 conv.
1,671 Q
data_public
VoxMem 09 conversational_longterm, multimodal, semantic_knowledge_update, episodic paper · code · data 3,196 conv.
3,196 Q
official_script
agent-memory-bench 08 coding_cross_session, semantic_knowledge_update code · data 196 conv.
26 Q
official_script
ContinualSkillBench 08 procedural_workflow paper · code 500 Q official_script
DreamBench-SWE 08 coding_cross_session, semantic_knowledge_update, procedural_workflow paper · code 60 conv.
60 Q
data_public
FinEvo-Bench 08 procedural_workflow paper 120 Q paper_only
GDPevo 08 procedural_workflow paper · code 240 Q official_script
MemToC 08 parametric paper · code 6,504 Q data_public
MemTrapBench 08 procedural_workflow, memory_security paper · code 1,050 conv.
1,050 Q
official_script
MemUse 08 conversational_longterm, personalization paper · code · data 72 conv.
316 Q
official_script
MobileMem 08 conversational_longterm, personalization, episodic, semantic_knowledge_update, multimodal paper · code · data 16 conv.
7,415 Q
official_script
PAST-Bench 08 procedural_workflow, semantic_knowledge_update, personalization, episodic paper · code 26 conv.
204 Q
official_script
PATH-Bench 08 procedural_workflow paper 120 Q paper_only
S3Gym 08 procedural_workflow, parametric paper — paper_only
StateMemBench 08 conversational_longterm, semantic_knowledge_update paper 234 conv.
322 Q
paper_only
TANGLE 08 personalization, conversational_longterm, semantic_knowledge_update paper 40 conv.
541 Q
paper_only
UTILMEM 08 conversational_longterm, personalization paper · code · data 1,717 conv.
1,717 Q
data_public
AgenticSTS 07 procedural_workflow, episodic paper · code · data — official_script
AgentStream 07 procedural_workflow paper · code 300 Q official_script
CAPA 07 coding_cross_session, personalization paper · code 60 conv.
300 Q
data_public
ChainSWE 07 coding_cross_session paper · data 100 conv.
304 Q
data_public
EvoAgentBench 07 procedural_workflow, coding_cross_session paper · code · data 267 Q official_script
ForgetBench 07 parametric paper 6,431 Q paper_only
Ground Truth First 07 conversational_longterm, personalization, episodic, semantic_knowledge_update, memory_security paper · code 383 Q paper_only
InMind 07 personalization, conversational_longterm paper · code 1 conv.
125 Q
data_public
Insecure Coding Preferences in Long-Term Memory (MemSecurity) 07 coding_cross_session, memory_security paper · code 194 Q official_script
MemOps 07 conversational_longterm, semantic_knowledge_update, parametric, memory_security paper · code 403 conv.
2,006 Q
official_script
MemSecBench 07 memory_security paper 310 Q paper_only
MemSyco-Bench 07 personalization, semantic_knowledge_update paper · code · data 1,550 conv.
1,550 Q
official_script
MERIT 07 episodic, semantic_knowledge_update, personalization paper · code — official_script
PM-Bench 07 semantic_knowledge_update paper · code 1 conv.
81 Q
official_script
RUMBA 07 conversational_longterm, episodic, semantic_knowledge_update, memory_security paper · code · data 85 conv.
1,543 Q
official_script
Sequential-EQA 07 embodied, multimodal paper · code 50 conv.
498 Q
official_script
SLVMBench 07 procedural_workflow, multimodal paper · data 2,261 Q data_public
AFTER 06 procedural_workflow paper · code · data 382 Q data_public
AgentCL 06 procedural_workflow, coding_cross_session, embodied paper · data — data_public
AhaBench 06 procedural_workflow paper · code — official_script
Continual Learning Bench (CL-Bench) 06 procedural_workflow, coding_cross_session, semantic_knowledge_update paper · code · data 6 conv.
301 Q
official_script
DMV-Bench 06 episodic, multimodal paper · code · data 35,410 Q official_script
EvoArena 06 semantic_knowledge_update, procedural_workflow, coding_cross_session, personalization, conversational_longterm paper · code · data 10 conv.
505 Q
official_script
FinEvolveBench 06 procedural_workflow paper · code · data — official_script
ForgetEval 06 semantic_knowledge_update, memory_security paper · code 385 Q official_script
GateMem 06 multi_agent_group, memory_security, semantic_knowledge_update, conversational_longterm paper · code · data 91 conv.
2,218 Q
official_script
H2HMem 06 conversational_longterm, multimodal, multi_agent_group, semantic_knowledge_update paper · code · data 25 conv.
2,236 Q
official_script
HUSH-Bench 06 memory_security, personalization, conversational_longterm paper 10 conv.
2,400 Q
paper_only
MedEvoEval 06 procedural_workflow paper 700 Q paper_only
MemAudit (formerly MEMPROBE) 06 personalization, conversational_longterm, episodic paper · code 50 conv.
1,550 Q
official_script
MemoryDocDataSet 06 conversational_longterm, multi_agent_group, semantic_knowledge_update, episodic paper 50 conv.
1,000 Q
paper_only
MemTrace 06 conversational_longterm, semantic_knowledge_update paper · code 20 conv.
15,422 Q
official_script
MIST 06 personalization paper 598 conv.
598 Q
paper_only
MPBench 06 memory_security paper · code 3,240 Q data_public
OmniGameArena 06 procedural_workflow, embodied, multimodal paper · code · data — official_script
SEAGym 06 procedural_workflow paper · code 55 Q official_script
StreamMemBench 06 personalization, episodic paper · code 16,214 Q official_script
SubtleMemory 06 conversational_longterm, personalization, semantic_knowledge_update paper · code · data 10 conv.
1,522 Q
official_script
BenchTrace 05 procedural_workflow, embodied paper · code · data 1,821 conv.
4,391 Q
official_script
EgoStream 05 episodic, multimodal paper 2,250 Q paper_only
EvoMemBench 05 procedural_workflow, semantic_knowledge_update, embodied, conversational_longterm paper · code 5,754 Q official_script
GroupMemBench 05 multi_agent_group, conversational_longterm, semantic_knowledge_update, personalization, episodic paper · code · data 4 conv.
745 Q
official_script
LongMemEval-V2 05 episodic, procedural_workflow, multimodal paper · code · data 1,540 conv.
451 Q
official_script
MBench 05 multimodal paper · code · data 1,040 Q official_script
MemAudit 05 semantic_knowledge_update, conversational_longterm paper · code · data 87 Q official_script
MemEye 05 conversational_longterm, multimodal, semantic_knowledge_update paper · code · data 8 conv.
371 Q
official_script
Momento 05 conversational_longterm, personalization, semantic_knowledge_update paper · code · data 161 Q official_script
PrecisionMemBench 05 semantic_knowledge_update paper · code · data 2 conv.
89 Q
official_script
RefMem-Bench 05 conversational_longterm, personalization paper 35 conv.
20,279 Q
paper_only
RHELM 05 conversational_longterm, episodic, semantic_knowledge_update, personalization paper · code · data 10 conv.
1,305 Q
official_script
RoboMemArena 05 embodied, multimodal, episodic paper · code · data — official_script
SeqMem-Eval 05 procedural_workflow, embodied paper · code 3,726 Q official_script
SkillEvolBench 05 procedural_workflow paper · code · data 180 Q official_script
STALE 05 semantic_knowledge_update, conversational_longterm, personalization paper · code · data 400 conv.
1,200 Q
official_script
STATE-Bench 05 procedural_workflow code 300 conv.
150 Q
official_script
SuperMemory-VQA 05 episodic, multimodal paper · code · data 4,853 Q data_public
WorldMemArena 05 conversational_longterm, episodic, semantic_knowledge_update, multimodal paper · code · data 461 conv.
24,258 Q
official_script
MemHome / MemHomeLife 04 personalization, semantic_knowledge_update, conversational_longterm paper 389 Q paper_only
SEA-Eval 04 procedural_workflow paper · code 92 conv.
32 Q
data_public
SkillFlow 04 procedural_workflow paper · code · data 166 Q official_script
SkillLearnBench 04 procedural_workflow paper · code 100 Q official_script
StratMem-Bench 04 conversational_longterm paper · code 657 Q data_public
AMB (Agent Memory Benchmark) 03 conversational_longterm, personalization, episodic, semantic_knowledge_update, coding_cross_session code · data — official_script
AMemGym 03 conversational_longterm, personalization, semantic_knowledge_update paper · code · data 20 conv.
200 Q
official_script
ATM-Bench 03 personalization, episodic, multimodal, semantic_knowledge_update paper · code · data 1,038 Q official_script
LifeBench 03 episodic, personalization, semantic_knowledge_update paper · code · data 10 conv.
2,003 Q
official_script
Persona-MME 03 personalization, conversational_longterm, multimodal, semantic_knowledge_update paper · code · data 200 conv.
2,034 Q
official_script
RealPref 03 personalization, conversational_longterm paper · code 100 conv.
1,300 Q
official_script
RMBench 03 embodied, multimodal paper · code · data — official_script
AMA-Bench 02 episodic, semantic_knowledge_update paper · code · data 208 conv.
2,496 Q
official_script
ES-MemEval 02 conversational_longterm, personalization, episodic, semantic_knowledge_update paper · code · data 18 conv.
1,209 Q
official_script
HLME 02 conversational_longterm, semantic_knowledge_update paper — paper_only
LoCoMo-Plus 02 conversational_longterm, personalization paper · code 10 conv.
401 Q
official_script
MemoryArena 02 episodic, procedural_workflow, personalization paper · code · data 701 conv.
4,850 Q
official_script
Neuromem 02 conversational_longterm, semantic_knowledge_update paper · code 10 conv. official_script
PersistBench 02 personalization paper · code 500 Q official_script
SkillCraft 02 procedural_workflow paper · code 126 Q official_script
SWE-ContextBench 02 coding_cross_session, procedural_workflow paper · code · data 376 Q official_script
Trust-Memevo 02 procedural_workflow paper · code 2,329 Q paper_only
AndroidIntent 01 personalization paper · code · data 91 conv.
990 Q
official_script
CAME-Bench 01 conversational_longterm, semantic_knowledge_update paper · code · data 14 conv.
373 Q
official_script
CloneMem 01 personalization, episodic, semantic_knowledge_update paper · code · data 10 conv.
1,183 Q
official_script
EvolMem 01 conversational_longterm, semantic_knowledge_update, personalization paper · code 1,600 conv. paper_only
KnowMe-Bench 01 personalization, episodic paper · code · data 3 conv.
2,580 Q
official_script
Mem-Gallery 01 conversational_longterm, multimodal, episodic, semantic_knowledge_update paper · code · data 20 conv.
1,711 Q
official_script
Mem2ActBench 01 conversational_longterm, personalization, semantic_knowledge_update paper · code 2,029 conv.
400 Q
data_public
MemoryRewardBench 01 conversational_longterm paper · code · data 2,400 Q official_script
RealMem 01 conversational_longterm, semantic_knowledge_update, personalization, episodic paper · code 10 conv.
1,415 Q
official_script
RPEval 01 personalization paper · code 953 Q official_script
TraineeBench 01 procedural_workflow paper · code 50 conv.
203 Q
official_script

2025 (42)

Name Month Types Links Scale Repro level
PersonaMem-v2 12 personalization, conversational_longterm, memory_security paper · code · data 195 conv.
5,000 Q
official_script
BELA 11 personalization, procedural_workflow paper · code · data 40 conv. official_script
ConvoMem 11 conversational_longterm, personalization, semantic_knowledge_update paper · code · data 75,336 Q official_script
Evo-Memory 11 procedural_workflow, embodied paper 3,716 Q paper_only
HaluMem 11 conversational_longterm, semantic_knowledge_update, personalization, episodic paper · code · data 20 conv.
3,467 Q
official_script
BEAM 10 conversational_longterm, semantic_knowledge_update, episodic, personalization paper · code · data 100 conv.
2,000 Q
official_script
MemoryBench 10 procedural_workflow, conversational_longterm paper · code · data 4,063 Q official_script
MemTrack 10 semantic_knowledge_update, episodic paper · data 47 conv. data_public
Stateful SWE-bench 10 coding_cross_session, personalization paper · code · data 15 conv.
500 Q
official_script
TeleEgo 10 episodic, multimodal paper · code · data 5 conv.
3,291 Q
official_script
FlashAdventure 09 episodic, multimodal paper · code 34 Q official_script
ZH-4O 09 conversational_longterm, personalization paper · code 28 conv.
1,068 Q
official_script
M3-Bench 08 episodic, multimodal paper · code · data 1,020 conv.
4,490 Q
official_script
MPR 08 personalization, parametric paper · code · data 12 conv.
108,000 Q
official_script
OdysseyBench 08 conversational_longterm paper · code 602 conv.
602 Q
official_script
StuLife 08 episodic, procedural_workflow paper · code · data 1,284 Q official_script
LOCCO 07 conversational_longterm, parametric paper · code 100 conv.
2,981 Q
data_public
MemoryAgentBench 07 conversational_longterm, episodic, semantic_knowledge_update paper · code · data 2,071 Q official_script
EvaLearn 06 procedural_workflow paper · code 182 conv.
648 Q
official_script
MemBench 06 conversational_longterm, personalization, semantic_knowledge_update paper · code · data 820 conv.
820 Q
data_public
OpenUnlearning 06 parametric, memory_security paper · code · data 400 Q official_script
StoryBench 06 episodic paper 86 Q paper_only
SWE-Bench-CL 06 coding_cross_session, procedural_workflow paper · code · data 8 conv.
273 Q
official_script
LifelongAgentBench 05 procedural_workflow paper · code · data 1,396 Q official_script
MEMENTO 05 personalization, episodic, embodied paper · code 438 Q official_script
MS-TOD 05 conversational_longterm paper · code 132 conv.
956 Q
official_script
UniEdit 05 parametric paper · code · data 311,142 Q official_script
WikiDYK 05 parametric paper · code · data 77,180 Q official_script
KUP (Knowledge Update Playground) 04 parametric paper · code · data 6,260 Q official_script
PersonaMem 04 personalization, conversational_longterm, semantic_knowledge_update paper · code · data 60 conv.
2,727 Q
official_script
TiC-LM 04 parametric paper · code — official_script
ImplexConv 03 conversational_longterm, personalization paper · code · data 2,500 conv. data_public
LifeState-Bench 03 episodic, parametric, multi_agent_group, semantic_knowledge_update paper 311 Q paper_only
WikiBigEdit 03 parametric paper · code · data 502,382 Q official_script
MemoryCode 02 coding_cross_session, conversational_longterm, semantic_knowledge_update paper · code · data 360 conv. official_script
Minerva 02 semantic_knowledge_update paper · code 1,110 Q official_script
PersonaBench 02 personalization, conversational_longterm, semantic_knowledge_update paper · code 15 conv.
582 Q
official_script
PrefEval 02 personalization, conversational_longterm paper · code · data 3,000 Q official_script
QAEdit 02 parametric paper · code · data 19,249 Q official_script
REALTALK 02 conversational_longterm, episodic paper · code 10 conv.
728 Q
official_script
TReMu 02 conversational_longterm, episodic paper 600 Q paper_only
Episodic Memory Benchmark 01 episodic, parametric paper · code · data 686 Q official_script

2024 (13)

Name Month Types Links Scale Repro level
EvoWiki 12 semantic_knowledge_update, parametric paper · code · data 10,264 Q paper_only
HalluEditBench 10 parametric paper · code · data 6,738 Q official_script
LongMemEval 10 conversational_longterm, personalization, semantic_knowledge_update, episodic paper · code · data 500 conv.
500 Q
official_script
GoodAI LTM Benchmark 09 conversational_longterm, semantic_knowledge_update, episodic paper · code 1 conv.
33 Q
official_script
MADial-Bench 09 conversational_longterm, episodic paper · code 160 conv. official_script
MemDaily 09 personalization paper · code 2,954 conv.
2,954 Q
official_script
CodeUpdateArena 07 parametric paper · code · data 670 Q official_script
DialSim 06 conversational_longterm, multi_agent_group, episodic paper · code · data 3 conv. official_script
StreamBench 06 procedural_workflow paper · code · data 9,702 Q official_script
TeaFarm 06 conversational_longterm paper · code · data 100 conv.
200 Q
data_public
LoCoMo 02 conversational_longterm, episodic paper · code 10 conv.
1,986 Q
official_script
PerLTQA 02 personalization, episodic, conversational_longterm paper · code 3,409 conv.
8,593 Q
data_public
KnowEdit 01 parametric paper · code · data 5,653 Q official_script

2023 (8)

Name Month Types Links Scale Repro level
AToKe 12 parametric paper · code 8,820 Q data_public
ConflictEdit / RoundEdit 10 parametric paper · code 2,500 Q official_script
DMR (Deep Memory Retrieval) 10 conversational_longterm paper · code · data 500 conv.
500 Q
data_public
MMEdit 10 parametric, multimodal paper · code · data 3,093 Q official_script
RippleEdits 07 parametric paper · code 4,000 Q official_script
MemoryBank 05 conversational_longterm, episodic paper · code 15 conv.
194 Q
data_public
MQuAKE 05 parametric, semantic_knowledge_update paper · code 3,000 Q official_script
MemoEval (SCM Self-Controlled Memory evaluation dataset) 04 conversational_longterm paper · code 18 conv.
105 Q
data_public

2021 (1)

Name Month Types Links Scale Repro level
Multi-Session Chat (MSC) 07 conversational_longterm, personalization paper · code · data 501 conv. official_script

Reproducibility levels

Level Meaning
unverified No one has opened the paper and repository to check yet
paper_only Paper checked; no public data or code found
data_public Data is publicly available, but the official repository has no evaluation script
official_script The official repository has a script that runs the evaluation, and the data is available
smoke_tested We ran the official script on a small sample
full_run We ran the full evaluation

Contributing

  1. Add benchmarks/<slug>.yaml following docs/fields.md, and cite a source in sources for every non-null field.
  2. Run python scripts/validate.py and python scripts/build_readme.py.
  3. Do not edit the READMEs by hand; CI checks that they match the data.

License

  • Data (benchmarks/, docs/, schema/ and the READMEs): CC BY 4.0
  • Code (scripts/, templates/, site/, pending/tools/): MIT

Papers, code and datasets linked from the entries are third-party works under their own licenses, which this repository's licenses do not cover. Check the original source before use.

About

Structured registry of agent memory benchmarks, with every field traced to its source

Resources

Stars

209 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages