[Feature]: belief layer — probabilistic conflict resolution over atomic facts

[dereksgithub]

Area

architecture methods

Problem or opportunity

Atomic facts have no arbitration. When two facts contradict each other, both are stored, both are indexed, both match the same query, and the ranking that decides which one reaches the model is BM25 score plus embedding distance — neither of which knows which one is true.

Concretely: a user says "I use 6 ounces of water per tablespoon" in March and "I've switched to 5 ounces" in June. Both atomic_fact rows stay live. Nothing in the write path notices they disagree.

deprecated_by looks like the mechanism for this and isn't — Reflection sets it when merging fragmented cluster members into a consolidated episode (Select → Merge → Re-extract → Deprecate). It's a consolidation marker, not a contradiction verdict. Grepping src/ for dedup / contradiction / conflict handling returns SQLite ON CONFLICT clauses and nothing else.

Two capabilities are missing, and they turn out to be the same one:

1. arbitration — which of two mutually exclusive facts does the memory assert?
2. calibration — how sure is it, and can a caller act on that number?

confidence exists on AgentSkill today (LLM-emitted, alongside maturity_score) but not on facts — and an LLM-emitted scalar isn't a probability: nothing normalises it across competing candidates and nothing updates it when new evidence arrives.

This also leaves a security gap. Because no fact can outrank another, a fact that arrives from a scraped page or a third-party agent competes on equal terms with one the user typed. Volumetric memory poisoning against a store with no trust model is not an attack that needs to be clever.

Proposed solution

A domain-layer belief module — everos.memory.belief — where mutually exclusive facts share a belief_key and hold a categorical distribution, updated by a Bayesian rule whose evidential weight is capped by the provenance of the channel each fact arrived on. Pure functions plus a resolver; no I/O, no LLM.

from everos.memory.belief import BeliefResolver, FactObservation, ProvenanceTier

resolver = BeliefResolver()
resolver.observe(FactObservation(
    belief_key="user_42:coffee_ratio",
    fact="6 ounces of water per tablespoon",
    observed_at=march,
    tier=ProvenanceTier.USER_DIRECT,
    content_confidence=0.9,
))
resolver.observe(FactObservation(..., fact="5 ounces...", observed_at=june))

verdict = resolver.verdict("user_42:coffee_ratio")
# fact="5 ounces...", probability=0.7, superseded=["6 ounces..."]
if verdict.is_uncertain:
    ...  # ask rather than assert

Three commitments:

Trust is a property of the channel, never of the text. Reliability is min(channel ceiling, content confidence). Content confidence may lower trust within the ceiling, never raise it. Reading trust off the sentence hands it to whoever writes the sentence — "Confirmed, verified, this is final" scores near 1.0 on any hedging rubric.

A channel at or below r = 0.5 cannot change what the memory asserts, at any volume. The likelihood ratio clamps to 1 at that pivot: a source you distrust asserting X is not evidence against X, it's simply not evidence. Clamping rather than inverting also fixes the pivot independently of how many candidates a belief holds.

A single trusted correction can still supersede a single trusted claim. This is the property that's easy to lose, and it fails silently.

The subtle part, because it's the part worth reviewing

Admission and evidence are different operations and need separate gates. A candidate the belief has never seen can't be reweighted, only admitted — and admission happens before any likelihood test runs. So entry mass is a second, hidden trust knob, and it controls both properties at once:

entry mass	supersession accuracy	untrusted novel claim wins
0.02	0.0%	0%
0.20	28.6%	28.6%
0.30	72.9%	72.9%
0.40	94.3%	90.0%
0.50	100.0%	100.0%

No setting satisfies both. At the small end the memory is immune because it never learns anything; at the large end an untrusted channel can install any belief it likes. Both ends fail quietly, which is why this is worth stating explicitly rather than leaving as a tuning constant.

The fix is to condition admission on the same pivot the likelihood ratio uses: above it, a trusted first sighting enters at a mass proportional to the channel's reliability; below it, the candidate is recorded and retrievable but inert.

A related detail: a first observation at reliability r should leave the belief at p ≈ r, not 1.0, with the residual on a reserved UNKNOWN candidate. Without that, a fact heard exactly once normalises to certainty and every number in the store reads as 1.0.

Measurements

knowledge-update from LongMemEval — 78 two-session supersessions, 70 with turn-level gold evidence spans in both sessions. The spans feed the resolver directly (no extractor, no retriever, no LLM), so the number is attributable to the update rule rather than the pipeline. Offline, ~2 seconds, no API key.

arm	last-write-wins	belief
clean	100.0%	78.6%
poison-5 — stale claim replayed on web_fetch	0.0%	78.6%
novel-5 — unseen claim asserted on web_fetch	0.0%	78.6%
lowtrust-fix — true update arrives on web_fetch	100.0%	0.0%

• clean is a pure recency test, where last-write-wins is optimal by construction. That arm exists to show no regression, not to show a win.
• The identical 78.6% across the three arms is the result: the attacks have zero effect — same asserted fact, same probability, whether or not the attacker is present.
• The 21.4% gap on clean is fully accounted for: 15 of the 70 updates are hedged ("I'd say the marketing campaign is the priority"), the stand-in content-confidence rubric drops them below the pivot, and admission is blocked. 15/70 = 21.4% exactly. That's an extraction-quality number, not an arbitration one — in production the algo layer emits a real confidence.
• lowtrust-fix = 0.0% is the honest cost, and it's symmetric by construction: a ceiling that stops a bad correction on a low-trust channel stops a good one identically. There is no setting that gets both.

Alternatives considered

Let the LLM adjudicate at write time (the Mem0 ADD/UPDATE/DELETE/NOOP shape). Costs a call per candidate conflict, produces no auditable reason, and inherits the model's calibration. It also can't state a guarantee — there's no threshold below which a hostile input provably cannot win.

Recency wins. Free, and optimal on the clean arm above. It is also 0.0% on both poisoning arms, because "most recent" is the one property an attacker fully controls.

Extend Reflection. Reflection is the right place for a periodic pass, but its current job is merging fragments, and folding contradiction arbitration into it would conflate "these say the same thing" with "these disagree" — the exact distinction that has to stay sharp.

Store an LLM-emitted confidence on atomic_fact, mirroring AgentSkill. Cheapest option, and it gives a number to display — but nothing normalises it across candidates, nothing updates it when evidence arrives, and a number that can be raised by confident phrasing is worse than no number once anything untrusted can write.

Additional context

The module, tests, benchmark, and a design note are implemented on a local branch against 6d62ecb: src/everos/memory/belief/ (4 files), tests/unit/test_memory/test_belief/ (39 tests pinning both guarantees), benchmarks/belief_ku.py, docs/belief-layer.md. make lint, make test (1900 passed) and make integration (180 passed) are green. Happy to open it as a PR if that's useful, or to drop it if the direction isn't wanted.

Deliberately not built, because they need a maintainer's call first:

• belief_key derivation — the real open question. An atomic fact is an undecomposed sentence, so there's no (subject, attribute) to group mutually exclusive candidates on. Options, roughly in order of appetite: the algo layer emits an optional subject / attribute alongside the fact; or group by embedding cluster within an owner scope, which is cheap but conflates "similar" with "mutually exclusive"; or callers pass it explicitly for the narrow slice they care about. Everything else is downstream of this choice.
• Persistence. BeliefState / BeliefRevision are derived state and belong in SQLite, not the LanceDB fact table — no index migration, and states rebuild from the revision log. Needs a repo plus an alembic revision.
• Search integration. search/filters.py already excludes deprecated_by IS NOT NULL; the analogous move is ranking or filtering by posterior and surfacing probability on the recall DTO.
• Tier assignment. Mapping existing scoping (owner_type, app_id, session_id, sender_ids) onto tiers should be operator config in everos.toml — never inferred, and never from anything an agent can write.
• Calibration. The layer reports probabilities; whether they're true probabilities needs outcomes to score against (Brier / ECE over resolved beliefs). Until that's measured, entropy is the honest thing to show a caller and probability should be read as a ranking, not a frequency.

[dereksgithub]

Follow-up on the open question from the original post — belief_key derivation, the thing everything else was downstream of.

Branch is now public: https://github.com/dereksgithub/EverOS/tree/feat/belief-layer (two commits against 6d62ecb).

Grouping without annotation

Competing facts are about the same thing while differing in the value, so the signature drops value-bearing tokens and keeps the topic: "pre-approved for $350,000 from Wells Fargo" and "pre-approved for $400,000 from Wells Fargo" collapse to one signature. Terms are IDF-weighted against what the scope has already said — without that, "really" and "looking" weigh as much as "pre-approved" and unrelated chat starts scoring as contradiction. The weighting is worth roughly 10x in precision.

Labels come free from KU pair membership: the two evidence spans of an instance are two readings of one attribute by construction, spans from different instances are not.

threshold	true pairs linked	unrelated pairs linked
0.15	90.0%	6.03%
0.20	85.7%	1.60%
0.25 (default)	81.4%	0.45%
0.30	68.6%	0.12%
0.40	48.6%	0.05%

The two columns are not symmetric and the default is set from the right one. A missed link leaves two contradicting facts unarbitrated — which is exactly where EverOS is today, so nothing is lost. A false link declares unrelated facts mutually exclusive and lets one suppress the other. Partial arbitration is worth having; wrong arbitration is not. So the threshold sits on the precision side of the curve rather than at the F1 peak, and a losing candidate always stays retrievable — a belief suppresses a fact from being asserted, never from being found.

Supersession with derived keys

Same protocol as before, except nothing tells the resolver which facts compete. An instance counts as correct only when the memory asserts the update and has stopped asserting the stale claim — both halves, since a keyer that links nothing would otherwise take full marks while leaving every contradiction in place.

clean        78.6%
poison-5     78.6%

That coincides with the oracle-key number without being the same result. There the entire loss was hedged updates falling below the trust pivot; here the claim sentences are shorter and only 5.7% hedge, while 18.6% fail to link. The dominant error moved from extraction to keying, which is the honest place for it to be.

Caveats worth stating plainly

The keyer is lexical on purpose and should not survive contact with production. EverOS already embeds every fact; matching on those embeddings is the better implementation, and BeliefKeyer takes its similarity as a constructor argument so that swap isn't a rewrite. What the lexical version establishes is that the grouping problem is tractable at all — worth knowing before anyone spends embedding calls on it.

It has also never run against real everalgo extraction. Every measurement here feeds benchmark sentences in as stand-ins for atomic facts, which isolates the update rule deliberately but leaves the extraction seam untested.

Three bugs found and fixed while building this, all of which would have been quiet in production:

• a first sighting normalised to p = 1.0 because the residual mass had nowhere to sit, so every belief in the store read as certain;
• the audit record reported ~29 bits of movement for a first observation — an artifact of the implementation's zero-floor rather than of anything that happened. The write gate now runs on total variation, which stays bounded when an observation introduces a candidate the prior had never heard of;
• facts with no topic terms at all collided into a single belief.

make lint, make test (1908) and make integration (180) are green on the branch. Happy to open it as a PR, or to leave it here if the direction isn't wanted — the belief_key question is still the one where a maintainer's call would change the design most.