How does Mem0 handle memory deduplication and contradiction resolution at scale?

[DomLynch]

Been thinking about memory deduplication as a real challenge for long-running agents. A few questions after reading through the codebase:

1. When the same fact gets stored multiple times across sessions (slightly different wording, same semantic meaning), how does Mem0 detect and merge these? Is it embedding similarity above a threshold, or something more structured?

2. What happens when two stored memories directly contradict each other? For example the user says "I prefer Python" in session 1 and "I mostly use TypeScript now" in session 3. Does the newer one overwrite, both get kept, or is there a merge strategy?

3. At scale (say 10k+ memories per user), does retrieval quality degrade noticeably or does the deduplication layer keep things manageable?

Interested in the design decisions here, not just the API surface. Happy to dig into specifics if useful.

[barry166]

From the current OSS code, this is not a simple "embedding similarity above threshold = merge" flow.

In the newer additive path, Mem0 retrieves nearby existing memories and passes them into the extraction prompt as existing_memories. New memories are still emitted as ADD, but they can include linked_memory_ids pointing to related old memories. That means duplicate/updated facts are represented as linked memories rather than automatically collapsing into one canonical row.

So for your examples: "I prefer Python" and "I mostly use TypeScript now" should normally both remain available unless you explicitly update/delete one, but the newer memory can be linked to the older one as the same preference topic. At 10k+ memories, retrieval quality depends more on your vector store filters/scoping and over-fetch/rerank strategy than on a built-in global dedupe pass.

[simon9679]

barry166's explanation of linked_memory_ids matches what I found too — but linking isn't resolution. Both memories stay independently retrievable even when linked; nothing flags the pair for review.
I built a small external tool for this: reads your Mem0 memories through the standard SDK, finds duplicate/contradiction/stale candidates, reports them. Doesn't change anything automatically.
Would appreciate any feedback: https://github.com/simon9679/mem-audit

[23HughAI]

Great questions! These are exactly the pain points we focused on when building our memory layer.

1. Deduplication: We use a hybrid approach: structured entity extraction (to identify "who/what/where/when") + semantic similarity checks. This avoids both false merges and duplicate facts that slip through pure embedding matching.
2. Contradictions: We lean on temporal ordering (ZEP-style sequences) — the latest statement takes precedence, but we keep a history of prior facts for traceability. It’s simple but works really well in production.

FWIW, if you're looking for a drop-in alternative, our API is fully compatible and handles both out of the box.

[yashwanth123]

Answers based on the v3 ADD-only path (+ maintainer notes on #4896 / #4904):

1. Dedup — not cosine-threshold merging. Nearby memories go into the extraction prompt as existing_memories (LLM can skip semantic duplicates/set linked_memory_ids). After that, only exact MD5 hash dedupe runs. Slightly reworded facts usually stay as separate rows unless the LLM skips them.

2. Contradictions — both are kept by design. Newer memory can link to the older via linked_memory_ids; retrieval (recency/relevance/entity signals) is meant to surface the current truth, not overwrite/delete the old one.

3. At 10k+ — quality depends more on filters/scoping + retrieval/rerank than a global dedupe pass. Linking helps relatedness, but doesn’t collapse the store.

Happy to point at the prompt / main.py phases if useful.

[RemanenetSpy]

Worth noting how we approached this in Smriti: dedup/contradiction resolution is handled via explicit supersession — each event tracks what it replaces (superseded_by), so the query layer can filter to only the current belief instead of merging or scoring similarity between old and new. Avoids the 'which version wins' ambiguity that pure similarity-threshold dedup runs into at scale. Happy to compare notes if useful: smriti-kaal.vercel.app

[simon9679]

The answers above describe the design accurately — both memories are kept, linked_memory_ids relates them, nothing collapses the store. I want to add the part nobody has touched yet, which is DomLynch's third question: what this actually costs you, measured rather than reasoned about.

I built an external read-only auditor for exactly this and ran it against a synthetic store with known ground truth — 24 facts, 7 deliberately planted pairs (3 duplicates, 2 contradictions, 2 supersessions), one topically-related-but-distinct trap pair, and 8 unrelated facts as noise. Embedding pass for nearest neighbours, then an LLM judge on only those candidates.

What held up across four runs on two completely different embedding stacks:

• Recall was stable at 7/7. Every planted pair surfaced, every time. The trap pair and all 8 noise facts stayed clean.
• Type classification was not. Both supersession pairs ("barista" → "data analyst") were labelled contradictions rather than updates, consistently, on every configuration. That's a judge-side property, and it reproduces.
• False positives varied with the embedder, 1 to 3 depending on the stack — so that number tracks candidate selection, not the judge.

The practical read for anyone auditing their own store: finding the pairs is the reliable part; deciding what kind of conflict each one is still needs a human. Which is why the tool reports and stops rather than merging anything.

One thing worth flagging separately, because it bit me: the first measurement I published didn't reproduce months later, and I couldn't determine why — the reports hadn't recorded which models produced them, and by the time I re-ran, the embedding provider I'd used had been shut down entirely. If you're benchmarking memory behaviour, record the whole stack in the output. Write-up: https://github.com/simon9679/mem-audit/blob/main/docs/accuracy-postmortem.md

Tool is https://github.com/simon9679/mem-audit — read-only, talks to Mem0 through the standard SDK, works with whatever backend you already run. Embeddings can run fully local through Ollama, so an audit needs no cloud keys at all.

Caveats on the numbers: one 24-fact set, short English personal-memory-style facts, one measurement per configuration. Treat it as "the approach works as designed," not as a generalisation to a 10k store — which, to be clear, I have not measured.