MemPalace needs to be able to forget (or at least know that a memory is no longer important)

[jpatrickiles-dev]

I've been working on my own AI personal assistant and when I heard about MemPalace, I added it to the project, but one thing is missing. MemPalace already has a valid_to on every memory and recall queries already filter expired records, but it doesn't expose a way to shelve a memory that should be remembered, but is no longer important to bring up. Kind of like a feature that you wish that your wife or significant other had the capability to do. So when you tell it to "forget this" it would set the valid_to field to now and not delete it, but keep it in memory in case the user forgot about it and brings it up again so that the user can be corrected that this was already done. I've already added it into my project (I wish that it worked with wetware too, but it doesn't), but I thought that I would mention this to the community.

[nanoscopic]

How would you manage memory and decide what content to discard?

[jpatrickiles-dev]

Well in my case I"m just using a chat window style interface, but this became a concern to me because if you have ever used Gemini (which learns from your history) you would notice that if you told Gemini something once it will randomly surface the memory later on when you don't want that. For example, I would ask it for the command to install an application on my current Linux laptop, but would get the instructions tailored to the previous distribution that I was using and no amount of my saying "forget that" other than deleting the referenced chat could get rid of that memory. The memory should be retained, because with memories comes experience and wisdom, but there is no exposed way (API etc.) where you can tell mempalace to consider a memory as no longer being relevant. Am I making any sense here?

[terrizoaguimor]

This is a critical missing feature. Memory without forgetting isn't memory — it's a log file. Human memory decays naturally, and there's actual neuroscience behind why this matters for AI recall.

The Ebbinghaus forgetting curve (R = e^(-t/S)) models exactly this: memories decay exponentially over time, but each retrieval strengthens them (spaced repetition). This means frequently accessed memories stay strong while stale ones fade — naturally pruning noise without manual cleanup.

I've been using Celiums which implements this natively. Every memory has a strength value that decays based on time since last access, and retrieval count modulates the decay rate. The result: important, frequently-recalled facts persist while one-off mentions gracefully fade. No manual expiration needed.

The math is simple but the effect on recall quality is dramatic — it's the difference between searching a growing pile of everything vs. having a brain that knows what to surface.

Open source, Apache 2.0: lifecycle.ts

[rebroad]

I am currently adding forgetfulness into MemPalace. Can share that branch later when it's more ready. Some implementation info:-

Add Real Forgetting to MemPalace

Summary

Implement forgetting as a first-class MemPalace subsystem, not just a search tweak.

The design will use:

• Ebbinghaus-style retrieval decay for normal ranking and hiding stale memories
• conservative lifecycle maintenance that first marks memories as decayed, then hard-deletes only stale low-value memories later
• coordinated forgetting across drawers, closets, and KG facts
• tombstones so forgotten memories do not get silently re-mined from the same source content

This fits the current architecture better than copying Celiums wholesale: MemPalace keeps Chroma for verbatim drawers, SQLite for KG, and adds a small lifecycle sidecar SQLite DB for forgetting state.

Implementation Changes

1. Add a lifecycle store

Create a new SQLite sidecar in the palace directory, for example memory_lifecycle.sqlite3, with two tables:

• drawer_state
  • drawer_id primary key
  • content_sha256
  • source_file
  • wing, room
  • created_at
  • last_accessed_at
  • access_count
  • strength_days
  • importance
  • state enum-like text: active|decayed
  • decayed_at
  • forgotten_at
  • pinned default false
• tombstones
  • drawer_id primary key
  • content_sha256
  • source_file
  • forgotten_at
  • reason
  • origin (explicit or maintenance)

This DB is the canonical source for forgetting metadata. Chroma stays the verbatim store.

2. Wire lifecycle into ingest

On every ingest path that writes drawers:

• initialize/update lifecycle rows for each drawer
• compute content_sha256 and store it in lifecycle metadata
• before upserting a drawer, check tombstones
• if a tombstone exists for the same drawer_id and same content_sha256, skip re-ingest
• if the same drawer_id exists but the content hash changed, treat it as a new memory and clear the old tombstone for that ID

This prevents forgotten content from reappearing just because the source transcript/file is mined again.

3. Wire lifecycle into retrieval

Update programmatic search and layer retrieval so they consult lifecycle state by drawer ID after Chroma returns candidates.

Ranking/filtering rules:

• compute retrievability = exp(-days_since_access / strength_days)
• exclude tombstoned memories always
• exclude decayed memories from normal search by default
• multiply the existing hybrid score by retrievability before final ordering
• when a result is actually returned to the caller, update:
  • last_accessed_at = now
  • access_count += 1
  • strength_days = min(strength_days * 1.2 + 1, 365)

Default lifecycle thresholds for the conservative profile:

• initial strength_days = 30
• mark as decayed when either:
  • retrievability < 0.15, or
  • last_accessed_at is older than 180 days
• eligible for hard delete when all are true:
  • already decayed
  • older than 365 days
  • access_count <= 1
  • not pinned
  • no current KG facts reference the drawer

4. Coordinate forgetting across drawers, closets, and KG

Add one canonical forgetting path used by both explicit commands and maintenance:

For a drawer being forgotten:

• delete the drawer from Chroma
• delete its lifecycle row
• insert/update a tombstone
• remove or invalidate KG facts linked by source_drawer_id
  • current facts should be invalidated rather than left dangling
• rebuild affected closets for that source_file
  • purge existing closets for that file
  • regenerate closet lines from remaining drawers in that file

Keep the existing low-level delete_drawer primitive for compatibility, but route user-facing forgetting to the new coordinated path.

5. Add public controls

Add new CLI commands:

• mempalace forget drawer <drawer_id> [--reason ...]
• mempalace forget run [--dry-run]
• mempalace forget stats
• mempalace search ... --include-decayed

Add matching MCP tools:

• mempalace_forget_drawer
• mempalace_forget_run
• mempalace_forget_stats

Keep existing mempalace_delete_drawer and KG tools for backward compatibility, but document them as low-level primitives.

6. Add config for policy

Extend config.json with a forgetting section:

{
  "forgetting": {
    "enabled": true,
    "mode": "hard_delete",
    "profile": "conservative",
    "auto_run_hooks": true,
    "maintenance_min_interval_hours": 24,
    "initial_strength_days": 30,
    "decay_retrievability_threshold": 0.15,
    "decay_after_days": 180,
    "purge_after_days": 365,
    "max_strength_days": 365,
    "include_decayed_default": false
  }
}

Environment overrides should follow the existing config pattern.

7. Run maintenance from existing integration points

Because we chose CLI + Hooks:

• add a maintenance runner callable from CLI and Python
• call it opportunistically from existing stop and precompact hook flows
• rate-limit it with maintenance_min_interval_hours so it does not run on every hook fire
• allow explicit dry-run from CLI/MCP for inspection

No daemon, no background service, no per-search full sweep.

Public API / Interface Changes

• New lifecycle sidecar DB in each palace
• New CLI namespace: mempalace forget ...
• New MCP tools:
  • mempalace_forget_drawer
  • mempalace_forget_run
  • mempalace_forget_stats
• New search option:
  • CLI --include-decayed
  • matching programmatic/MCP flag
• New config section: forgetting
• Existing ingest code will also start writing lifecycle metadata and honoring tombstones

Test Plan

Add coverage for:

• lifecycle DB creation and config defaults
• ingest creates lifecycle rows
• tombstoned identical content is not re-mined
• changed content at same deterministic drawer ID is allowed back in
• search reranks by retrievability
• decayed memories are hidden by default
• --include-decayed surfaces them
• explicit forget:
  • deletes drawer
  • writes tombstone
  • invalidates/removes KG facts by source_drawer_id
  • rebuilds closets so stale drawer refs disappear
• maintenance dry-run reports candidates without deleting
• maintenance live run hard-deletes only eligible memories
• hook-triggered maintenance respects rate limiting
• backward compatibility:
  • old palaces without lifecycle DB bootstrap cleanly
  • old KG rows without drawer provenance do not crash forgetting flow

Assumptions and Defaults

• Hard delete is the final forgetting mode.
• The default policy is conservative, not aggressive.
• Decayed memories are hidden from normal recall before final deletion.
• KG-linked memories are protected from auto purge unless their current facts are gone.
• Tombstones are internal and persistent by default; they exist to stop accidental re-ingest.
• Existing delete_drawer stays for compatibility, but the new forget path becomes the recommended API.

[rebroad]

https://github.com/rebroad/mempalace

I've recently added some forgetfulness code based on Celiums (but without all the bloat). Celiums seems to be in the middle of a refactor so much of the code seems to be disconnected at the moment, and it looks a bit too new to consider currently (even though their website does look professional).

[milla-jovovich]

@jpatrickiles-dev @rebroad @terrizoaguimor @nanoscopic hey guys! so I definitely wanted Mempalace to be a place where most recent memories get stored at the beginning of a search, so the agent will always look through most recent first (which is usually) the most relevant.

saying that, I also felt like it could be perceived more as a sourdough starter lol

Like you know when you go to people's homes and they're like "oh we've had this started for over 50 years!" kind of thing. I know, I also wanted it to be something important fr people with Alzheimer's. one of the images that popped into my head was how Jame's garner comes to gena Rowlands everyday and retells her the story of their life together...

Saying that, I think each person's preferences should be their own! I will definitely work on this tomorrow as something to push. Like so people can pick and choose what they want to discard? is that feeling I'm picking from everyone but ,y buddy @nanoscopic. And truly thats a very important and interesting question for each individual and DEFINITELY is telling something about their own nature in what they would want to remember or forget fit's an emotional memory, or just simply an old business venture that is obsolete and is required anymore.

I mean it's you Palce, you should be an able to do whatever you want with it and personalize it! -m

[jpatrickiles-dev]

Milla, I started this discussion, because who doesn't have a bad memory that they would like to just forget?! I have been messing around creating my own AI personal assistant and the matter came up somehow, but memories do need to fade with time even in the computer world. Especially if you say, "what was I thinking when I typed that in?" My setup was to allow for retaining all memories, but keep the most relevant ones close. Love the sourdough reference, because I have my own in the fridge!

…

On Mon, Apr 27, 2026, 7:32 AM Milla J ***@***.***> wrote: @jpatrickiles-dev <https://github.com/jpatrickiles-dev> @rebroad <https://github.com/rebroad> @terrizoaguimor <https://github.com/terrizoaguimor> @nanoscopic <https://github.com/nanoscopic> hey guys! so I definitely wanted Mempalace to be a place where most recent memories get stored at the beginning of a search, so the agent will always look through most recent first (which is usually) the most relevant. saying that, I also felt like it could be perceived more as a sourdough starter lol Like you know when you go to people's homes and they're like "oh we've had this started for over 50 years!" kind of thing. I know, I also wanted it to be something important fr people with Alzheimer's. one of the images that popped into my head was how Jame's garner comes to gena Rowlands everyday and retells her the story of their life together... Saying that, I think each person's preferences should be their own! I will definitely work on this tomorrow as something to push. Like so people can pick and choose what they want to discard? is that feeling I'm picking from everyone but ,y buddy @nanoscopic <https://github.com/nanoscopic>. And truly thats a very important and interesting question for each individual and DEFINITELY is telling something about their own nature in what they would want to remember or forget fit's an emotional memory, or just simply an old business venture that is obsolete and is required anymore. I mean it's you Palce, you should be an able to do whatever you want with it and personalize it! -m — Reply to this email directly, view it on GitHub <#759 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/B3X7JWGLYFLYF6QTH2VXCWD4X5ANNAVCNFSM6AAAAACXWT5V2WVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMNZSHAYDQMQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[terrizoaguimor]

Hey @milla-jovovich, thanks for the mention — and @rebroad too, fair points worth addressing head-on.

First, on rebroad's read: yes, you're right, Celiums is in the middle of a sizeable refactor. I'm a solo dev with ADHD, and the way that plays out is — I get deep into one piece, ship it, then while building it I see the gaps and end up touching three other modules. The end goal of the project is actually exactly that: a memory + tooling system designed for people who don't finish what they start because they lose the thread. The whole point is to make that mode of working tractable instead of a curse. That's the bias behind the tools — they exist to make picking-up-where-you-left-off the default, not the exception. So the "code looks disconnected right now" observation is accurate, and I appreciate the honesty. I'm working through it in public on purpose.

@milla-jovovich — I want to be clear about the framing, because the last thing I want from this is to position Celiums as competing with MemPalace or to chase visibility on someone else's thread. I genuinely think MemPalace is a good project that started something real (the drawer/closet/wing model is interesting and the community engagement here is enviable), and there are pieces I'd actually love to help on rather than reimplement.

One thing I'd push back gently on, from my own experience: what's being discussed here isn't really forgetting, it's selective memory. We shouldn't aim to delete — we should aim to keep memories accessible but quiet by default, surfaced only when the context calls for them. You never know what an old memory will turn out to be useful for; the value of a recall isn't always obvious at the time the memory is made. Speaking as someone whose ADHD makes recall non-linear, the moments where an "irrelevant" memory suddenly becomes the key piece happen often enough that I'd never want the system to actually erase. Decay the visibility, lower the retrieval weight, even tombstone the surface — but keep the substrate. That's how human memory actually behaves: nothing is fully forgotten, just buried until the right cue brings it back.

The forgetting (selective memory) feature you're about to push tomorrow is a perfect example of where I'd love to help — I've burned cycles on Ebbinghaus decay, retrieval-strengthening, tombstones, KG-coordinated forgetting, and the failure modes you only find at scale. Happy to share what didn't work, what does, and the math behind it, in whatever format is most useful (PR review, sketch, a call, whatever fits).

The way I see Celiums is less "alternative to MemPalace" and more "next step when SQLite + Chroma stops being the right shape." Triple-store (PG + Qdrant + Valkey), affective signals on top of semantic recall, ethics gate before write — that surface starts mattering when you grow past single-user. Until then MemPalace is doing more for adoption than I am, and that's fine.

@rebroad — if your fork is moving forward on forgetting and you'd find it useful, happy to compare notes on the tombstone semantics specifically. The "don't re-mine forgotten content from the same source" problem is the one that bit me hardest.

Tag me whenever — @terrizoaguimor works. Open to anything.

[jphein]

For folks following this thread — there's now an in-flight PR (#1425 by @zhapostolski) that adds optional exponential-decay recency-weighting to _hybrid_rank via MEMPALACE_RECENCY_WEIGHT env var. Default off (strict no-op), opt-in blend factor, configurable half-life. Doesn't address @jpatrickiles-dev's "shelve a memory I told you to forget" feature directly — that's a different write-side action — but it does address @terrizoaguimor's point that recall without time-awareness behaves more like a log file than memory.

The two pieces compose: a recency-weighted ranker (zhapostolski's PR) keeps old-but-relevant memories accessible while letting recent decisions surface first, and an explicit "shelve / forget this" action (still open; @rebroad's branch and @milla-jovovich's "most-recent-first" intent both gesture at it) handles the deliberate-mute case that pure decay can't.

Operator-experience data point from a ~160K-drawer production palace: the "old high-similarity drawer masking a recent decision" failure mode is real and common — a 6-month-old infrastructure note can bury last week's actual decision on the same domain. Pure cosine + BM25 has no notion of time. Worth +1ing #1425 if you'd find recency-weighting useful.

[terrizoaguimor]

Thanks for the thoughtful summary — and for framing the two pieces as composable rather than competing. That's exactly the right read.

Strong +1 on #1425. Default-off + opt-in blend factor is the right call for a verbatim-first project; it preserves the strict baseline while giving operators a knob when they need it. The env-var approach also keeps the surface area honest — easy to reason about, easy to disable.

A couple of observations offered as follow-up territory rather than scope-creep for this PR:

1. A single global half-life is the right MVP, but tends to hit a ceiling once drawer types diverge in their natural decay profiles — architectural decisions and meeting notes don't age the same way. Per-tag or per-drawer-type half-life is a natural follow-up once the global version proves itself in the wild.

2. Retrieval frequency is the other axis pure time-decay can't see. A six-month-old note pulled forty times has earned a different curve than one nobody has touched. Ebbinghaus + consolidation-by-retrieval, basically. Out of scope here, but worth a separate issue once feat(search): opt-in recency-aware ranking via exponential decay #1425 lands.

3. Agreed that decay and explicit shelve/forget are structurally different operations — one passive and probabilistic, the other intentional and deterministic. They should coexist, not collapse into each other. Glad the summary called that out.

Happy to help test against larger drawer counts if useful — the failure mode you described (old high-similarity drawer masking a recent decision) matches what I've seen in production too.

[xg-gh-25]

Different perspective on forgetting — we don't use time-decay at all. Our system has been running 3 months (300+ sessions) and we've landed on: value-based eviction beats decay-based forgetting.

Why Ebbinghaus curves are wrong for AI memory:

Human forgetting is involuntary — memories fade whether you want them to or not. But AI memory is cheap storage. The problem isn't "what to forget" — it's "what to inject into the finite context window." You have 10K tokens of space. You have 50K tokens of memories. Which 10K do you pick?

This reframes the problem from "forget X" to "X didn't earn its tokens this time."

Our three-mechanism approach:

1. Distillation (not deletion)

Raw session logs live 30 days, then auto-archive. But before they die, a distillation process extracts anything worth keeping into curated memory:

DailyActivity (raw, 30d TTL)
  → Distillation (agent judges what's worth promoting)
    → MEMORY.md (curated, no TTL — stays until superseded)

Nothing is "forgotten." Raw logs age out. Valuable insights get promoted. The memory gets SMALLER but MORE PRECISE over time.

2. Supersession (not decay)

We don't decay old entries by time. An entry dies ONLY when a newer entry on the SAME topic replaces it:

Week 1: "Project uses PostgreSQL" → stays
Week 5: "Migrated to SQLite for simplicity" → supersedes Week 1 entry

The old entry isn't "expired" — it's factually superseded. Time-decay would fade Week 1 at the same rate whether or not a replacement exists. Supersession is semantic, not temporal.

3. Usage-based eviction (earned tokens)

Every memory entry competes for injection budget. Entries that are never recalled (never matched a user query, never influenced a decision) get evicted after 30 days of zero usage:

# Simplified eviction logic
if entry.last_recalled is None and entry.age_days > 30:
    archive(entry)  # Not deleted — moved to cold storage

This is the opposite of Ebbinghaus — it's not "how much time passed" but "was this entry ever useful?" A 6-month-old correction that prevents bugs weekly stays forever. A 1-week-old observation that was never recalled again gets evicted.

What "forget this" means in our system:

When the user says "forget X" we don't set valid_to = now. We mark it suppressed: true — it stays in storage (searchable on demand) but is excluded from automatic injection. This handles exactly the "told AI something once, keeps surfacing it" problem you described with Gemini.

The results after 3 months:

• Memory size: ~15K tokens (started at 0, grew to 25K, then SHRANK to 15K as distillation matured)
• Recall precision: high (every injected entry earns its tokens)
• "Stale memory surfacing" incidents: 0 (because supersession handles it structurally)

The counterintuitive insight: good memory gets smaller over time, not larger. If your memory only grows, you're accumulating noise. If it grows AND shrinks, you're actually learning.

---

This is how we handle memory lifecycle in SwarmAI. Deeper discussion: Memory is the Moat

[epinethrone]

I built a frontend that lets you do this exactly! Check out Apricity.

[et1975]

In case anyone curious there's whole dreaming skill that needs a couple of PRs (#1919 and #1921) merged that does recency check for cleanup.