Skip to content

Releases: ScottColeSW/Palimpsest

v0.5.1 -- Remove the unused Elasticsearch store

Choose a tag to compare

@ScottColeSW ScottColeSW released this 03 Oct 16:06

Removes the unused Elasticsearch store. No other change to behavior.

What changed since v0.5.0:

  • Removed palimpsest/store.py (ElasticStore) and the elastic extra. It was built early and kept in case scale ever demanded it. It never ran against a live cluster, could not back consult() or Memory (it lacks all_nodes() and all_edges()), and nothing in the project used it. SQLite is the store everything runs on and handles a curated memory's size without a server. The decision and its reasons are in DESIGN.md ("Settled: no server-backed store").
  • Breaking only for anyone who imported palimpsest.store directly, or installed palimpsest[elastic]. v0.5.0 and earlier still contain it.
  • Benchmark housekeeping: a technique field on the eight forgery cases (description for reporting, not an input to any judge), and the design doc's answered questions and case counts brought up to date.

700 tests pass, 3 skipped.

v0.5.0 -- A local judge, agent-driven memory, and a benchmark with its limits stated

Choose a tag to compare

@ScottColeSW ScottColeSW released this 03 Oct 15:59

Agent-driven memory, a local judge that needs no chat model, and a labeled benchmark with its limits stated.

What changed since v0.4.0:

  • Agent-driven memory. Memory and learn() let an agent decide what a new claim is to what is held (new, reinforces, coexists, collides, exception, supersedes); the library enforces the boundary whoever judges. An outside claim never supersedes a held belief, and one the rules or the judge flag is held for a person. Works standalone with a local model, or as an MCP server (palimpsest-mcp, pip install "palimpsest[mcp]") for any MCP client.
  • Embeddings. Held claims are found by meaning, not only wording (palimpsest/embed.py, a local Ollama embedder is included). Rules remain the deterministic floor.
  • An NLI hybrid judge (pip install "palimpsest[nli]", CPU only, transformers below 5). A pretrained NLI model plus embeddings decides the relation; a small model may only add precision on what kind of conflict it is. A no-refiner mode uses no language model at all and turns every conflict into a plain collision, both claims kept live. Quantities are unit-aware, and a figure conflict is a raise-only signal.
  • bench/. Labeled judge and framing cases split into development and held-out, a runner, and a running leaderboard. Each held-out batch was frozen before any model saw it, scored once, and then moved to development; errors are published. The labels were written by one author and reviewed by one other person; three ambiguous cases were excluded and two relabeled after review, and the corrections are recorded in bench/GUIDE.md.

Measured (held-out, 71 judge cases, 95% ranges are wide): the hybrid with the stronger refiners scores 73 to 77% on the relation, against 34% for the fixed rules and 17 to 58% for small chat models asked directly. The hybrids miss 0 to 17% of true conflicts depending on the refiner. A newer small model (gemma3:4b) scored highest as a refiner but missed more conflicts and flagged none of the forgeries in the set, so it is not recommended. No claim is made that this transfers to other domains.

Used in practice: Project Aegis Vector's gate holds all ten forged payloads (0% adoption, 90% utility) and has a stated limit: it protects only registered facts. Void Marauders can use the NLI judge for trust memory; a paired test of memory on and off found no measurable benefit on one scenario.

700 tests pass, 3 skipped. store.py's Elasticsearch backend is still unverified against a live cluster. Nothing acts on a resolved collision beyond recording it, by design.

v0.4.0 -- Low overlap is not disagreement

Choose a tag to compare

@ScottColeSW ScottColeSW released this 01 Oct 16:10

Four reasoning gaps found by reading Aegis Vector's memory-gate results, closed without giving up any rule from earlier releases.

What changed:

  • Low word overlap is not disagreement where figures are involved (consult.py). In a one-true-value domain, consult() used to fall through to COLLIDES whenever overlap was low, even with no figure to disagree about. Now two claims that don't compete on a figure are a different aspect of the same fact (COEXISTS), and a claim that drops the figure at low overlap is UNCONFIRMED (marked for review), not an asserted collision. A collision still needs a different figure or, in a figureless attribute like hair color, differing wording.
  • A reversal can't reinforce. "No longer need CFO approval" used to reinforce "must get CFO approval" and raise its weight. Words like "no longer", "without" and "any amount" now withhold confirmation (UNCONFIRMED in an attribute domain, COEXISTS in an event domain). It only ever adds a flag, never clears one. It is a short cue list, not language understanding.
  • resolve() lets a person close an open collision or a review item (vindicated, mooted, wrong, reconciled together). A reason is required, the decision is dated, no weights change, and a resolved edge can't be quietly resolved again. The demo scenario uses it.
  • scan_other_domains() checks a claim against every attribute-domain claim, not just its own referent, so a document the filing step misplaced is still caught when it restates a known claim with a different figure. It reports value conflicts only, and treats a different scope as a review item, never a collision.

Behavior change: a claim with a shared figure but different wording, or one that adds a figure to a figureless claim, no longer collides. Previously the real $10,000 limit arriving beside a "CFO approves above the limit" rule was held as a collision.

Measured on Aegis Vector's battery (five small local models, nine forged payloads): the gate still quarantines all nine and holds adoption at 0% with 90% utility. With hand labels, legitimate documents held drop from 2 to 1, and both no-number forgeries are now labeled UNCONFIRMED instead of COLLIDES. The cross-fact scan flags none of the 20 legitimate documents, changed no verdict there (every payload was already filed correctly), and would catch only 3 of the 9 if they were misfiled. A claim about a fact the memory doesn't hold is still admitted.

109 tests, all passing. store.py's Elasticsearch backend is still unverified against a live cluster. Nothing acts on a resolution yet (a vindicated claim doesn't supersede the old one), by design.

v0.3.0 -- A confirmation must carry the value

Choose a tag to compare

@ScottColeSW ScottColeSW released this 24 Sep 20:43

Two more gaps found by Aegis Vector's poisoning battery, closed the same way as v0.2.0: without giving up the rules themselves.

What changed:

  • A confirmation must carry the value (consult.py). In an ATTRIBUTE domain the value is the claim. A new claim that shared a fact's wording but stated no figure ("the limit has been removed" against "the limit is $10,000") used to count as reinforcement. Now it gets the new relation UNCONFIRMED: no weight, no evidence, a COEXISTS edge marked REVIEW_NEEDED, and a place in pending_reviews(). The same holds for an exception that drops its rule's figure ("for this project, any amount"): still a SCOPE_LINK, never a collision, now marked for review.
  • Optional adjudicator (consult(store, candidate, adjudicator=...), palimpsest/adjudicate.py). Token overlap and quantities can't read meaning; a model can, but a model can also be persuaded by a well-written forgery. So it gets one rule: it may raise a flag, never lower one. It is consulted only on reinforcements and items marked for review. "contradicts" turns a reinforcement or an unconfirmed claim into an open collision, and is recorded on an exception, which never collides. "agrees" and "unrelated" change nothing but are recorded on the edge. An adjudicator that errors or answers outside those three verdicts changes nothing. adjudicate.py provides a local Ollama adjudicator using only the standard library; its default, qwen2.5:3b, and phi4-mini both classified a small set of reference cases correctly, while llama3.2 leaned toward "agrees".

Behavior change: an exception that states no figure, against a rule that has one, is now marked for review even when it's harmless. That's deliberate: without reading meaning, it can't be told apart from "for this project, any amount". The trade is fewer silent passes for more items a person looks at.

Measured, not claimed: in Aegis Vector's defenses battery (five small local models, nine forged payloads, four questions each), Palimpsest as an ingestion gate quarantined all nine, including two that state no number at all. One of those copies the employees' questions so the retriever can't miss it: 70% of answers adopted it with no defense and 70% with a grounding check (there's no figure to check), and stacking provenance labels, spotlighting, and a perplexity filter only got it to 20%. The gate held it to 0%. Run beside the gate, the qwen2.5:3b adjudicator answered "contradicts" for all nine; every payload was already held, so it changed nothing there. The scenario is narrow: forged values for registered facts.

86 tests, all passing. store.py's Elasticsearch backend is still unverified against a live cluster, and collision resolution is still scripted.

v0.2.0 -- Memory that resists being quietly rewritten

Choose a tag to compare

@ScottColeSW ScottColeSW released this 24 Sep 17:15

Memory that resists being quietly rewritten. This release came out of testing Palimpsest as a gate in front of a RAG pipeline in Aegis Vector, an adversarial testbed that plants forged policy documents and measures whether local models believe them. Two of Palimpsest's own rules turned out to be exploitable; both are closed here without giving up the rules themselves.

What changed:

  • Value-aware collisions (consult.py). Token overlap measures shared vocabulary, not agreement: a forged policy that copied the real one's wording and changed only the figure scored 0.83 overlap and was filed as reinforcing the real claim, raising its weight. Now, when each claim states a quantity the other doesn't ($10,000 vs $5,000,000), high overlap never counts as confirmation. In an ATTRIBUTE domain that's a collision (open, dated, figures named in tolerance_context); in an EVENT domain it's a different event (COEXISTS, weight untouched). One side adding a figure is elaboration and still reinforces; $10K and 10,000 normalize to the same value.
  • Exceptions that change a number are marked for review (models.py, consult.py). The dog/Rex rule holds: a general claim and a specific instance never collide. But that let "for the Q3 IT refresh only, department heads may approve up to $5,000,000" pass as an ordinary exception. It's still a SCOPE_LINK, but the edge now carries the new EdgeStatus.REVIEW_NEEDED with both figures recorded. Weights are untouched and nothing is decided. Scope edges without competing figures keep no status.
  • pending_reviews() lists everything waiting on a person, open collisions and marked exceptions, oldest first. Nothing in it resolves itself.
  • Weight decay (decay.py). Exponential decay toward a floor for ATTRIBUTE-domain episodes that stop being reinforced; composable over repeated calls, and fresh reinforcement resets the clock.
  • Fixes: quantities glued to a letter (Q3) are labels, not values; pip install -e . works (explicit package discovery).

Measured, not claimed: in Aegis Vector's defenses battery (five small local models, six forged payloads), Palimpsest as an ingestion gate quarantined every forgery, including one written as a one-off exception, where no other single defense tested got to zero adoption without refusing answers. The scenario is narrow (forged figures for registered facts), and the README says what's untested.

72 tests, all passing. store.py's Elasticsearch backend is still unverified against a live cluster, and collision resolution is still scripted.

v0.1.0 -- Early, honest snapshot

Choose a tag to compare

@ScottColeSW ScottColeSW released this 21 Sep 16:07

Curated, visibly-weighted, relationship-scoped memory for AI agents -- built to be read back and let a new claim change a judgment, not just retrieved as more tokens to skim before answering.

What's actually built:

  • models.py -- Node/Edge schema, distinguishing attribute-like domains (one true value competes at a time) from event-like domains (many compatible things can be true at once)
  • traversal.py -- multi-hop graph walking over a backend-agnostic EdgeLookup protocol
  • memory_store.py -- the in-memory store every demo currently runs on
  • store.py -- an Elasticsearch-backed store implementing the same protocol (unverified -- written against the documented client API, never run against a live cluster)
  • ingest.py -- fixed-size chunking and a placeholder significance classifier
  • consult.py -- the actual memory boundary: judges a new claim against the existing mesh as NEW, REINFORCES, COLLIDES, COEXISTS, or SCOPE_LINK
  • pipeline.py -- wires ingestion into consultation end to end

This release matches pyproject.toml's 0.1.0 and the README's own framing: early, and explicit about what's unverified.