Skip to content

v3.6.0 — consolidation stops manufacturing work, and its counters stop lying

Choose a tag to compare

@acidkill acidkill released this 16 Aug 08:13
· 91 commits to main since this release
39e8570

Running smem consolidate twice in a row on an unchanged brain was not a no-op: it created new
data every time, and the report gave no way to tell that apart from useful work.

Consolidation stops duplicating and destroying data

  • summarize is idempotent. Each run used to re-materialise every tag cluster as a fresh
    concept neuron, summary fiber and up to ten synapses — and because summary fibers carry the
    union of their sources' tags, each run fed the previous run's output back into clustering.
    Clusters now have a stable identity and an existing summary is skipped.
  • merge creates before it destroys. Sources were deleted first, so a failing write left
    fibers gone with no successor.
  • merge no longer orphans the typed-memory layer. Memory type, priority, trust score and
    validity window used to be stranded on every merge, because delete_fiber has no cascade and
    a typed-memory row cannot be re-pointed in place.
  • merge preserves source metadata — essence, conductivity, coherence and the summary
    itself, which is what recall surfaces display.

Two blind spots closed

  • The dedup census window rotates. It took a fixed prefix of an id-ordered list that never
    shrinks, so the same anchors were compared every run and the rest were never compared at all.
  • Query-pattern mining uses a recent window instead of the oldest events in the brain's
    history.

Every counter states what it measured

Delete attempts are no longer reported as removals; drift reports detection and persistence
separately; a dead detector no longer reports a silent zero; and counters that were recorded but
never rendered — semantic-link failures, deferred compressions, replay, interference, schema
creation, auto-tiering, tool events — now appear when non-zero. Silent input truncation is
announced. The health delta's "Consolidation ratio" label, which sat under the merge counters
and read as a compression ratio, is now named for what it measures; the wire field is unchanged.

A new contract test scans the engine for every key written into the report and fails unless each
is rendered or explicitly listed as silent. It found six live defects the first time it ran.

Performance

Consolidation no longer reads the whole synapse table in one response — measured at six figures
of rows, twice per run, which is the shape that produces [Errno 104] Connection reset by peer
on the HTTP transport. Prune, infer, enrichment and dream now page.

Upgrade notes

Nothing is migrated or deleted by this release. Brains that accumulated duplicate summaries under
earlier versions keep them; scripts/cleanup_consolidation_debt.py measures and, with an explicit
--apply and a named brain, removes them. Run it only after upgrading and restarting the smem
services
— pipx replaces files, not running processes.

Reasoning-injection fix (#177, thanks @RobertSigmundsson)

get_reasoning_context() applies a session-scoped idempotency marker to every caller —
right for SessionStart/UserPromptSubmit, which race for the same session, but wrong
for a hypothetical per-subagent hook event: its payload carries the parent session's
id, so once the parent injects, every subagent after it would silently get "" forever.
SubagentStart-shaped events are now exempt from both reading and writing the marker.
This repo ships no SubagentStart hook today, so it's a no-op for a stock install and
purely forward-looking for anyone who wires that event themselves.