LCM-X

Lossless Context Memory eXtension

Lossless context memory for Hermes Agent.

Bounded context, unbounded memory. Nothing is ever lost.

LCM-X replaces one-shot active-context compression with a SQLite-backed, DAG-based context engine. It keeps the live prompt bounded, preserves raw messages, and gives the agent tools to recover exact detail after compaction.

The project name is LCM-X. From v0.24.0 the plugin is named hermes-lcm-x and the runtime context engine lcm-x (#471); this is a breaking change for existing installs — see Migrating from hermes-lcm. Unchanged: lcm.db, the lcm: config block, LCM_* environment variables, the lcm_* tools, /lcm, and the bundled skill name hermes-lcm.

Based on the LCM paper by Ehrlich & Blackman (Voltropy PBC, Feb 2026). Inspired by lossless-claw for OpenClaw. For an interactive visualization of the LCM idea, see losslesscontext.ai. Project site: losslesscontext.com.

What we are building and why: VISION.md. What ships when: ROADMAP.md and the tracker #658.

• What it does
• Project status
• LCM vs built-in compression
• Quick start
• Commands and tools
• Recall skill and policy
• Configuration
• Retrieval contract
• OpenClaw/lossless-claw import
• Troubleshooting
• Architecture
• How it works
• Documentation
• Development
• Contributing
• License

Hermes Agent's built-in compressor is a practical continuity layer: when the prompt crosses its configured threshold, it prunes older tool results, asks an auxiliary model to summarize the middle/older conversation, and rebuilds the active prompt from that summary plus a protected recent tail. The original session rows can still live in Hermes state.db and remain searchable through host tools such as session_search, but the model's active context no longer contains the compacted turns verbatim or a structured drill-down path back to them.

LCM-X instead:

1. Persists messages in a plugin-local SQLite store with FTS metadata.
2. Compacts older context into depth-aware summary nodes.
3. Condenses summaries into a hierarchical DAG as they accumulate.
4. Assembles active context from system prompt, highest-value summaries, and the protected fresh tail.
5. Provides recall tools so agents can search, inspect, and expand compacted material without flooding the main prompt.

Nothing is lost in normal operation. Raw messages stay recoverable in bounded pages, summaries retain source lineage, and oversized externalized payloads keep stable refs for later expansion.

Core capabilities:

• SQLite message store - preserves raw messages before compaction
• Summary DAG - builds depth-aware summary nodes over compacted history
• Bounded recovery - pages raw messages, child summaries, and externalized payloads instead of dumping everything into the prompt
• Agent tools - lcm_grep, lcm_recall, lcm_query_state, lcm_compute, lcm_compile_evidence, lcm_evidence_pack, lcm_retrieve, lcm_recent, lcm_load_session, lcm_describe, lcm_expand, lcm_expand_query, lcm_status, lcm_inspect, and lcm_doctor
• Source-aware retrieval - filters raw rows and summaries by descendant source lineage
• Session controls - ignore noisy sessions or keep sessions read-only with glob patterns
• Large payload controls - externalize oversized tool/media/raw payloads and protect SQLite from inline media-ish base64 blobs
• Sensitive-pattern controls - two independent layers: opt-in durable redaction of API keys, bearer tokens, passwords, and private keys before LCM stores or summarizes them (off by default — the durable store is lossless), and automatic protection of the copies sent to known cloud embedding providers
• Diagnostics - runtime health, database checks, optional /lcm slash commands, backup-first repair/rotate paths

Beyond the core loop, three feature families extend LCM from a compression layer into a memory system: large-output externalization and context-budget controls (giant tool results move to recoverable refs instead of crowding the prompt), temporal memory (day/week/month rollups plus natural-time recall through lcm_recent), and semantic retrieval (opt-in embedding-backed lcm_grep semantic/hybrid modes with Voyage, OpenAI-compatible endpoints, Ollama, or in-process FastEmbed). See the Feature overview for what each family does and why, and Agent configuration profiles for copy-paste setups per agent type. Externalization and temporal rollups are on by default.

The latest stable release is v0.28.1 at 77b2b4f795fb6a4f84c55abf6d0441da4469971e, released 2026-10-20 (plugin hermes-lcm-x, engine lcm-x). This patch covers session continuity, replay matching, doctor diagnostics, short-turn coverage after rollback and routed-profile settings. See project status for the release summary, disclosed qualification exceptions and current work. It requires Hermes 0.21.3 or newer (#1080). It carries the v0.23.2 lossless default (durable sensitive-pattern redaction is opt-in; cloud-embedding privacy is an independent flag that transforms only the provider-bound copy — see "Sensitive-pattern redaction" below), the PEM private-key redaction hardening, loud privacy-policy failures on the recall path, and the rc-first release gauntlet, while preserving the 15-tool surface, schema-5 core store, and default-off experimental features.

Stable release identity and the continuing main development line are separate proof planes; do not describe an arbitrary main checkout as the installed stable product.

The main line and v0.28.1 carry the hermes-lcm-x v0.28.1 identity. An identity is never a restamp of any past commit's own recorded identity (#385 fixed the earlier drift). The manifest declares 15 tools; runtime discovery advertises 13 by default, or 15 with both assertion and adaptive-retrieval flags on.

Every compaction LCM-X starts today runs on the turn thread, so the user waits for it. v0.25.0 makes a hidden backlog drain at turn pace (#597); v0.31.0 removes the wait by preparing summaries in the background and publishing them at the threshold (#787). Recall is re-baselined on the shipped default configuration. See VISION.md, the roadmap, the tracker #658, Current project state and Benchmark methodology for current identities, gaps, and proof boundaries.

OmniMemEval: contributor-run, accepted upstream. BEAM 100K 71.84: 1st of 12 scored backends; LongMemEval 87.00: 2nd of 13 scored backends. Results, configuration, evidence and disclosures.

Hermes core may persist original conversation history in state.db before built-in compression rewrites the active prompt. Built-in compression can still be lossy in the active context, but previous content may be recoverable later through host-level history tools such as session_search.

LCM-X is different because recall is part of the active context engine:

• plugin-local store and DAG built specifically for drill-down
• current-session retrieval through LCM tools, not an auxiliary cross-session search step
• explicit source-lineage and session-boundary rules

Position LCM around retrieval quality, autonomy, and drill-down behavior. Do not claim that Hermes core has no persisted record of pre-compression history.

• Hermes Agent 0.21.3 or later (requires_hermes in plugin.yaml; LCM-X needs the plugin register_hook API)
• Python 3.11+
• No required third-party runtime dependencies

tiktoken is used if available; otherwise LCM falls back to character-based token estimates. regex is used if available to apply timeouts to message ignore patterns; without it, message-level regex filtering is disabled with a warning rather than running unbounded stdlib re matches.

Canonical install path: clone LCM-X as a general user plugin into plugins/hermes-lcm-x.

git clone https://github.com/electricsheephq/lcm-x \
  "${HOME}/.hermes/plugins/hermes-lcm-x"

For a profile-specific install:

git clone https://github.com/electricsheephq/lcm-x \
  "${HOME}/.hermes/profiles/myprofile/plugins/hermes-lcm-x"

From an existing checkout, install a symlink:

./scripts/install.sh

# Optional profile-aware install:
HERMES_PROFILE=myprofile ./scripts/install.sh

Run scripts/install.sh even when the checkout already lives at the canonical plugin path. It leaves that checkout in place and exposes the bundled hermes-lcm skill in the matching global/profile skills/ directory. The installer preflights both paths and refuses conflicts before creating links. When it finds a pre-0.24 install (plugins/hermes-lcm, or a config.yaml that enables hermes-lcm / selects context.engine: lcm), it prints the migration steps below. It never edits config.yaml and never deletes the old directory.

Semantic retrieval is optional; its dependency and recovery after a host update are described in the embedding setup doc.

The plugin has two names:

• plugin manifest name: hermes-lcm-x
• runtime context engine name: lcm-x

Both must be configured:

plugins:
  enabled:
    - hermes-lcm-x

context:
  engine: lcm-x

Restart Hermes after changing plugin or context-engine config.

Run:

hermes plugins list

Expected signals:

• plugin list includes hermes-lcm-x
• selected context engine is lcm-x
• tool list includes lcm_grep, lcm_recall, lcm_recent, lcm_load_session, lcm_describe, lcm_expand, lcm_expand_query, lcm_compute, lcm_compile_evidence, lcm_evidence_pack, lcm_status, lcm_inspect, and lcm_doctor (13 by default)
• the normal available-skills index includes hermes-lcm; current hosts can also resolve the explicit plugin-qualified skill hermes-lcm-x:hermes-lcm

On the main line, output with both opt-in flags enabled is:

Plugins (1):
  ✓ hermes-lcm-x v0.28.1 (15 tools)

Provider Plugins:
  Context Engine: lcm-x

Older v0.23.x stable tags instead report hermes-lcm v0.23.x (15 tools) and engine lcm; verify the loaded commit before treating either string as release proof.

For source checkouts, lcm_status, /lcm status, lcm_inspect, lcm_doctor, and /lcm doctor also report the loaded plugin path and best-effort git identity: plugin_git_commit, plugin_git_branch, and plugin_git_dirty.

If startup logs say LCM tools are available through context-engine schemas or mention the Path B fallback, that is expected on older Hermes hosts such as Hermes Agent v0.16. The 13 default lcm_* tools and any flag-enabled tools remain available through the context-engine path; standalone plugin-registry registration is not required there.

If you cloned directly into the plugin directory:

cd "${HOME}/.hermes/plugins/hermes-lcm-x" && git pull --ff-only

For a profile-specific install:

cd "${HOME}/.hermes/profiles/myprofile/plugins/hermes-lcm-x" && git pull --ff-only

A clone from v0.23.x or earlier still lives at plugins/hermes-lcm. Stop Hermes, update that clone in place (Hermes matches the manifest name, not the directory), then make the config change in Migrating from hermes-lcm before you start Hermes again:

cd "${HOME}/.hermes/plugins/hermes-lcm" && git pull --ff-only
# profile-specific:
cd "${HOME}/.hermes/profiles/myprofile/plugins/hermes-lcm" && git pull --ff-only

If you installed a symlink from a separate checkout, update that checkout and rerun the installer (it is idempotent):

git -C /path/to/lcm-x pull --ff-only && /path/to/lcm-x/scripts/install.sh

A catalog install (hermes plugins install hermes-lcm-x) updates only through the reviewed catalog pin: hermes plugins update hermes-lcm-x. LCM-X ships no self-updater. An install pinned with --ref <sha> (outside the catalog) refuses hermes plugins update; move it to the new reviewed commit with hermes plugins install https://github.com/electricsheephq/lcm-x --force --ref <40-character commit SHA> (the refusal prints that command).

Restart Hermes after updating.

Before upgrading to v0.23.1, resolve the effective database path from LCM_DATABASE_PATH or lcm_status. For an online backup, set LCM_ENABLE_SLASH_COMMAND=true in the old runtime's launch environment, restart that runtime so /lcm is registered, then run /lcm backup. Otherwise stop every SQLite writer and copy that configured database with its WAL/SHM companions as one cold set. Check out the exact stable tag/SHA, restart Hermes, and verify the loaded path, version, database path, schema 5, and all 15 tools. No manual core migration is required for the upgrade itself. Existing cloud vectors from a pre-v0.23.1 identity require privacy-policy warmup, an identity check from the warmup/profile output, and the finite monotonic dry-run/apply loop in the operator guide before semantic coverage is complete.

Cloud embeddings are a separate opt-in. Provider-bound copies use the configured sensitive-pattern catalog by default, while durable messages and FTS remain raw. LCM_EMBEDDING_PRIVACY_ENABLED=false explicitly opts out and binds vectors to the distinguished privacy:off revision. When provider-copy privacy is on, the catalog must be nonempty and contain only supported pattern names (any unknown name fails warmup and dispatch loudly). Dry-run the backfill first; raw-chunk cloud backfill also requires explicit raw-text consent. See the operator upgrade and privacy notes before enabling embeddings.

BREAKING in v0.24.0. The plugin manifest is now hermes-lcm-x and the context engine lcm-x (#471). Hermes matches plugins.enabled against the manifest name, not the install directory, so a config that enables only hermes-lcm stops loading LCM-X after an update. Run exactly one LCM copy, and change the config while Hermes is stopped:

1. Stop Hermes (every process that uses this profile).

2. Update the code: git pull --ff-only in the existing clone (it may stay at plugins/hermes-lcm; see Update it), or install a new checkout with scripts/install.sh.

3. Config: in plugins.enabled, replace hermes-lcm with hermes-lcm-x, and set context.engine: lcm-x:

   plugins:
     enabled:
       - hermes-lcm-x
   context:
     engine: lcm-x

   • If a separate older copy is still installed (for example v0.23.x left at plugins/hermes-lcm next to a new plugins/hermes-lcm-x), do not keep both names enabled: both copies would load. LCM-X detects that state, logs Another LCM generation is already loaded, and stays inert so only one copy writes lcm.db, but LCM-X itself is then not running.
   • Keeping hermes-lcm listed next to hermes-lcm-x is harmless only when plugins/hermes-lcm is this same checkout (an in-place clone, or the link scripts/install.sh reuses): there is one physical copy.
   • The legacy context.engine: lcm still selects LCM-X through an alias, but logs a once-per-process DEPRECATED LCM-X config warning and reports an identity_migration field in lcm_status and a warn check in lcm_doctor naming the exact change.

4. Start Hermes and verify: hermes plugins list shows hermes-lcm-x as enabled and hermes-lcm as not enabled (an old directory kept for rollback may still be listed), and the log shows LCM plugin loaded — lossless context management active.

5. Later, by hand: after verifying, remove a separate old directory (and its skills/hermes-lcm link). Keep it until then for rollback (a rollback also needs the steps in the operator guide). The installer never edits config.yaml or deletes anything.

If the config is not updated: Hermes logs Context engine 'lcm' not found — falling back to built-in compressor and runs without LCM-X. The existing lcm.db is untouched, but turns handled while Hermes runs without LCM-X (built-in compressor) are not in lcm.db and their compacted content may not be recoverable. Therefore update the config before restarting Hermes after the update.

Managed fleets (PCS / managed-plugin payloads) stay pinned to v0.23.x until their config stages hermes-lcm-x in plugins.enabled. That staging is a separate fleet-migration change owned outside this repository.

The catalog build follows the plugin-catalog admission rules: no self-updater (scripts/update.sh was removed; catalog installs update only through hermes plugins update hermes-lcm-x), provides_hooks matches what register() registers (pre_llm_call, post_llm_call, subagent_start, subagent_stop, all through ctx.register_hook), and hermes plugins validate reports the install scanner verdict safe: zero dangerous and zero caution (high-severity) findings. Planted-secret fixtures build their PEM markers by string concatenation and the bench env captures avoid piping the env listing, so test data is not read as a real key or an environment dump.

The scanner still lists informational (medium/low) findings; none affects the verdict:

• embedded_private_key, hardcoded_secret, openai_key_leaked — planted, non-functional fixtures in tests/ that prove redaction and privacy controls.
• python_subprocess, hex_encoded_string, uv_run, path_traversal — benchmark and release-gauntlet harnesses under bench/ and benchmarks/ that drive local Hermes runs; not loaded by the plugin.
• git_clone, unpinned_pip_install, dump_all_env — install/contributor docs and CI (pip install pytest numpy), plus one doc describing an env dump; documentation only.
• eval_string, proc_access, system_passwd_access — test assertions that such strings are rejected or handled.
• oversized_file, oversized_bundle, too_many_files — the repository ships docs images, large test modules, and benchmark tooling with the plugin.
• agent_config_ref, hermes_config_ref, hardcoded_ip_port (low) — maintainer skills and config examples that reference AGENTS.md, config.yaml, or loopback endpoints.

hermes plugins validate also warns that the 15 provides_tools are not registered by register(): LCM-X serves its tools through the context-engine tool schemas (ContextEngine.get_tool_schemas), which is the intended path.

Use these tools for current-session recall after compaction. Use Hermes session_search for earlier separate sessions or broad cross-session history outside the LCM database.

Both host paths hide lcm_query_state until LCM_ASSERTIONS_ENABLED=true and lcm_retrieve until LCM_ADAPTIVE_RETRIEVAL_ENABLED=true. The other 13 schemas remain advertised by default. LCM_DISABLED_TOOLS hides any named tool, including a flag-enabled tool. Restart after changing the loaded flags. The manifest still declares all 15 tools.

LCM-X ships skills/hermes-lcm/SKILL.md plus progressive-disclosure references for configuration, architecture, diagnostics, recall routing, and session lifecycle. The installer links that directory into the active Hermes profile so it appears in ordinary skill discovery. On hosts with plugin skill registration, it is also available explicitly as hermes-lcm-x:hermes-lcm.

The canonical policy is distributed through that bundled skill. It is not injected through pre_llm_call: current Hermes persists hook context in the current user's api_content and replays it as user-authored history. Keeping policy out of that seam prevents attribution errors and one-copy-per-user-turn replay growth while preserving clean transcript and LCM ingest. The policy:

• treats summaries as recall cues rather than exact proof;
• prefers newer source-backed evidence and verifies contradictions;
• teaches narrow FTS query construction and bounded scope selection;
• routes current compacted, cross-conversation, and recent/time-bounded recall through the appropriate existing tools;
• does not force a tool call when the current context is already sufficient.

The canonical bytes live in skills/hermes-lcm/references/recall-policy.md. The active-LCM pre_llm_call hook remains available for bounded pre-answer evidence when that feature is explicitly enabled, but it never prepends the recall policy. Older hosts without skill or hook registration keep their existing schema-driven behavior.

Slash commands are disabled by default. Enable them only in trusted operator contexts:

export LCM_ENABLE_SLASH_COMMAND=1

Available commands:

• /lcm or /lcm status - current runtime/session status
• /lcm doctor - read-only health checks
• /lcm doctor clean - read-only scan for obvious junk/noise session candidates
• /lcm doctor clean apply - backup-first cleanup for safe pattern-matched candidates; requires LCM_DOCTOR_CLEAN_APPLY_ENABLED=true
• /lcm doctor repair - read-only SQLite/FTS repair diagnostics
• /lcm doctor repair apply - backup-first SQLite/FTS repair
• /lcm doctor source - read-only scan for legacy blank-source rows
• /lcm doctor source apply - backup-first normalization of legacy blank-source rows to unknown
• /lcm doctor retention - read-only retention analysis
• /lcm backup - timestamped SQLite backup. The snapshot holds the database only; copy the externalized payload directory named in the output alongside it when moving a backup to another host.
• /lcm rotate - read-only preview of an in-place tail-preserving compact of the active session
• /lcm rotate apply - refused on Hermes (#1141); use /compress here N
• /lcm help - command help

Apply paths are intentionally narrow and backup-first. Start with diagnostics before cleanup or repair.

/lcm rotate previews the tail-preserving frontier for the active session without changing session_id or conversation_id. It reports the configured fresh tail, pre-tail message count, current and planned lifecycle frontier, and rolling backup path. The preview refuses ignored or stateless sessions.

/lcm rotate apply is refused on Hermes (#1141): Hermes reloads its own transcript after a restart, so advancing the frontier without summaries leaves rows LCM-X cannot map. No backup is created and no lifecycle state is changed. Use /compress here N to summarise everything except the most recent N messages. If an earlier LCM-X already rotated this session and Hermes restarted since, compaction can stop on rows below that frontier and /compress here N will not repair it: start a new session with /new (raw rows stay recoverable via lcm_load_session; in-place fix tracked in #1184). Raw source rows remain recoverable through lcm_load_session and lcm_expand.

lcm_status and /lcm status still surface last_rotate_at and the rolling backup path for existing stores.

The generated scalar environment reference lists every scalar registry key, type, declared default, diagnostic range/choice, and deprecation release. The tables below provide behavior and tuning guidance.

Most installs only need plugins.enabled and context.engine: lcm-x.

LCM_SQLITE_MMAP_SIZE sets the SQLite memory-map size in bytes (default 268435456, unchanged). On SD-card or single-board hosts, set LCM_SQLITE_MMAP_SIZE=0: a storage read error on a mapped page kills the process with SIGBUS, while 0 returns a catchable I/O error. Invalid or negative values use the default with one WARNING per process. /lcm doctor shows the effective sqlite_mmap_bytes value; SQLite may cap it below the requested size.

When LCM_FRESH_TAIL_MAX_TOKENS is above 0, the protected suffix must satisfy both the message-count and token bounds. While LCM_FRESH_TAIL_COUNT is nonzero the newest message is never dropped (0 keeps no fresh tail), and a boundary that would begin inside an assistant tool-call/result group is moved back to that assistant even when doing so exceeds a configured bound.

Advanced compaction, assembly, and extraction knobs are defined in config.py.

LCM is lossless by default: durable messages, FTS rows, summaries, and payloads remain untouched unless LCM_SENSITIVE_PATTERNS_ENABLED=true is explicitly set. Cloud embedding privacy is separate and transforms provider-bound copies only; it defaults on for known cloud providers and can be explicitly opted out with LCM_EMBEDDING_PRIVACY_ENABLED=false, which uses the privacy:off vector revision and sends raw provider input. The nonempty known-pattern policy requirements for warmup, backfill, and semantic-query dispatch apply only when provider-copy privacy is ON — the explicit privacy:off opt-out bypasses catalog validation by design. Local providers receive byte-identical input and do not enter the cloud privacy path.

When LCM_SENSITIVE_PATTERNS_ENABLED=true, matched secret values are replaced with metadata-only placeholders before SQLite, FTS, summaries, active replay, and externalized payload JSON receive the content. This is intentionally not lossless for matching values: the raw matched secret is unrecoverable after redaction.

Supported named catalog entries are:

• api_key: api_key, api_token, access_token, secret_key, and client_secret assignments or JSON keys.
• bearer_token: Bearer ... strings and token-like JSON keys.
• password_assignment: password, passwd, pwd, and passphrase assignments or JSON keys, including quoted values with spaces.
• private_key: PEM private-key blocks.

Redaction is forward-only. Enabling it does not rewrite existing SQLite rows, FTS shadow tables, DAG summaries, or externalized payload JSON that were written before the setting was enabled. Non-password placeholders include a short truncated SHA-256 digest for correlation. password_assignment placeholders omit the digest to avoid making password-like values easier to dictionary-check. lcm_status, lcm_inspect, and lcm_doctor expose the enabled state, configured pattern names, unknown names, source, and placeholder format without exposing raw secret values.

Cloud embeddings add a separate provider-input-only privacy transform. It does not rewrite historical SQLite, FTS, summary, or payload data. It resolves ON automatically for known cloud providers and does not require LCM_SENSITIVE_PATTERNS_ENABLED. While it is on, cloud warmup, summary/chunk backfill, and semantic-query dispatch require a nonempty, known LCM_SENSITIVE_PATTERNS catalog; matching provider input becomes pattern-only placeholders, residual matches are rejected, and the transform version plus active-pattern-name hash bind into the vector profile revision. LCM_EMBEDDING_PRIVACY_ENABLED=false opts out explicitly and binds the distinguished privacy:off revision instead of blocking. Either way a posture or policy change refuses dispatch until /lcm embed warmup registers the new identity. FastEmbed is in-process. The shipped Ollama provider does not apply the cloud gate, so use it only with a trusted local/loopback endpoint; #337 tracks endpoint-aware locality for remote Ollama configurations.

When context.engine: lcm-x is active, LCM takes its compaction threshold from, in order: LCM_CONTEXT_THRESHOLD, then lcm.context_threshold in config.yaml, then the Hermes core compression.threshold raised to at least the LCM default (0.75). A bare Hermes install seeds compression.threshold: 0.50, so without an LCM setting LCM uses 0.75; /lcm status reports the source config_yaml:compression.threshold(floored) and a config warning when the inherited value was raised. To go lower, set LCM_CONTEXT_THRESHOLD or lcm.context_threshold. When Hermes core compression.enabled is false, LCM uses its own default; that flag is still the global gate that allows compaction, so leave it enabled when using LCM.

If LCM_ABSOLUTE_THRESHOLD_TOKENS is set to a positive integer, it overrides the ratio-derived trigger after window math runs. Use this when you want a fixed context-health setpoint across model switches (for example 130000 for coding agents) so a larger window does not silently delay compaction, lower recall, or let long sessions accumulate more noise before LCM intervenes. Leave it at 0 to keep ratio-based behavior. When the absolute override is active, Codex GPT-5.5 ratio auto-raise is suppressed so the absolute setpoint stays pinned.

A single fraction scales the trigger with each route's window. That is fine when you want each route to use the same share of its window, but on a profile that switches between windows of very different sizes it can put the trigger much later than you want. With LCM_CONTEXT_THRESHOLD=0.75, a 200k primary route triggers at 150k tokens, but a backup route that resolves to a 1,000,000-token window triggers at 750k, so compaction does not run there until a prompt reaches 750k tokens. If you want a smaller prompt budget on that route, either give it its own fraction (LCM_MODEL_THRESHOLDS="glm-5.3:0.115" gives about 115k on a 1M window and leaves every other route unchanged), or pin one trigger for every route with LCM_ABSOLUTE_THRESHOLD_TOKENS. The pin also applies to routes with a smaller window, so keep it below the smallest window you use. /lcm status reports the resolved context_length, threshold_tokens and context_threshold_source for the active route.

If startup/status output shows a host-side compression percentage that disagrees with LCM, trust live LCM status after a normal message has initialized the session.

For the 2026-08-31 gpt-5.4 Codex-auth retirement, migrate to gpt-5.6-sol (quality) or gpt-5.6-luna (cost); codex_gpt_long_context remains correct for their 272K effective window.

Long-context models change the tuning problem. A 1M-token model does not mean you always want to spend 750k prompt tokens before LCM starts compacting. Start with the active prompt budget you are willing to pay for, then tune the threshold around that budget. If the budget itself should stay fixed across models, set LCM_ABSOLUTE_THRESHOLD_TOKENS instead of recomputing a ratio per window.

compaction trigger = effective context window * LCM_CONTEXT_THRESHOLD
LCM_CONTEXT_THRESHOLD = desired compaction trigger / effective context window
# or, model-independent:
# LCM_ABSOLUTE_THRESHOLD_TOKENS = desired compaction trigger

Examples, as math rather than universal recommendations:

A reasonable first pass for a true 1M effective window is:

The default 0.75 suits windows up to about 272k tokens. On a true 1M window it starts compaction at 750000 prompt tokens, so pick a row above, or set LCM_MODEL_THRESHOLDS / LCM_ABSOLUTE_THRESHOLD_TOKENS, if that is later than you want.

Tune against your effective context_length if Hermes caps the provider's advertised window.

Start with LCM_CONTEXT_THRESHOLD, LCM_FRESH_TAIL_COUNT, and large output externalization. Only tune leaf chunking after checking lcm_status and understanding whether your workload is dominated by huge raw backlog passes.

LCM_THRESHOLD_FULL_SWEEP_ENABLED (default true) is a cache-shape policy. Once threshold pressure triggers compaction, the invocation keeps summarizing the oldest raw chunks outside the protected fresh tail even after pressure falls below the trigger. It then condenses the provider-visible summary frontier only when that frontier exceeds LCM_SUMMARY_PREFIX_TARGET_TOKENS (or one leaf budget, capped at a quarter of the compaction threshold, when the target is 0). One invocation is bounded to 12 total leaf plus condensation calls and 120 seconds between calls, persists each completed DAG pass, and publishes one newly assembled active context at the end. It is synchronous and independent of deferred/background maintenance.

LCM is cache-friendly, not fully cache-aware. It may avoid some follow-on condensation churn, but current cache usage counters are retrospective status data only; they do not tell the plugin whether the next prompt mutation will break a hot provider cache.

LCM_CRITICAL_BUDGET_PRESSURE_RATIO is a narrow escape hatch. It is disabled by default (0.0). When set, LCM compares prompt pressure against the context window and only at or above that ratio may bypass existing polite gates for bounded deferred maintenance catch-up and cache-friendly follow-on condensation. Revisit full cache-aware deferred compaction only after Hermes core exposes reliable cache state / cache-break signals.

Pattern matching checks multiple keys: raw session_id, platform, and platform:session_id.

• * matches within one colon-delimited segment
• ** can span across colons

Example: cron:* can match Hermes cron sessions, while exact raw session IDs still work.

LCM offers two layers of noise filtering:

• Session-level filters (LCM_IGNORE_SESSION_PATTERNS, LCM_STATELESS_SESSION_PATTERNS) catch noisy traffic that arrives as its own session or platform.
• Message-level patterns (LCM_IGNORE_MESSAGE_PATTERNS) catch cron alerts or other noise injected into a normal Telegram or WhatsApp conversation as ordinary visible messages.

Message-level patterns are comma-separated Python regex strings compiled once at engine start. They run against plain text first; structured multimodal payloads use concatenated text parts first, then normalized JSON fallback when there are no text parts. Matching messages are skipped before storage.

Example operator config:

LCM_IGNORE_MESSAGE_PATTERNS=^Cronjob Response:,^>>>Cronjob Response<<<:

Invalid regex entries are logged at warning level and dropped. Pattern matching uses a 50 ms per-pattern timeout when the optional regex package is installed. If regex is not installed, LCM logs a warning and disables message-level regex filtering rather than running unbounded stdlib re matches in the ingest path.

Known limitation: the filter runs at ingest time. When a matching message is part of the chunk summarized in the same turn it arrived, the text may appear inside the resulting summary node. The filtered message is still not written to the message store, so DAG lineage stays clean; only serialized summary text can carry it.

lcm_status surfaces the full filter contract under session_filters, including pattern sources, whether the current session is ignored/stateless, and a process-lifetime ignored_message_count.

Ignored/stateless sessions are a storage ownership boundary, not a context-window opt-out. LCM does not ingest raw messages or create DAG nodes for sessions that match LCM_IGNORE_SESSION_PATTERNS, LCM_STATELESS_SESSION_PATTERNS, or the in-process auxiliary/thread stateless marker. If those sessions cross the normal context threshold, LCM delegates the compaction call to Hermes' native ContextCompressor so the active request is still bounded before model overflow. If the native compressor is unavailable, LCM falls back to a deterministic head/tail trim as a last-resort safety net, still without writing the bypassed session to lcm.db.

Storage-boundary payload guard contract: LCM prevents media-ish inline payloads from being written into plugin-local SQLite rows at the storage boundary.

Externalization for ordinary large tool output is on by default. When enabled, oversized tool results are written to plugin-managed JSON files and referenced from summaries. They remain inspectable through lcm_describe(externalized_ref=...) and lcm_expand(externalized_ref=...).

Active-replay stubbing is a second replay policy, on by default. When both externalization and active-replay stubbing are enabled, newly ingested textual tool results above the first-sight token threshold (10,000) are durably externalized and replaced immediately in provider-visible replay, including results in the protected fresh tail. This lets a stub-only replay change converge even when no leaf is eligible for compaction. At a compaction, a historical assembly pass stubs older tool results from the aged threshold (2,000 tokens) before budgeting, while respecting the protected fresh tail. Tool-call ids and compatible structured text block types/keys are retained; raw SQLite rows and DAG lineage are not rewritten by the historical pass. Structured image/media results remain inline, preserving the provider replay contract established by upstream Hermes-LCM PR #226. If durable externalization cannot be confirmed, replay keeps the original payload inline. Results from lcm_describe and lcm_expand also remain inline so recovery does not recursively produce another ref.

An automatic compaction whose input reached the threshold first tries these free cuts (#671): it assembles the summary prefix, the externalized placeholders and the aged-tier stubs without a model call and, when the host-measured list is at or under min(threshold - LCM_LEAF_CHUNK_TOKENS, 0.95 x threshold), returns it with no leaf, no condensation and no summariser call. The rows no leaf summarised stay stored and are recorded as raw backlog; the stop reason stub_first_exit is a partial stop in lcm_status (last_stub_first_exit, and the threshold sweep record when the sweep is on), the doctor and the LCM compaction stop: line, with the tokens before and after, the target and the backlog rows left. On error stops, reason=error stays unchanged and a trailing failure= field names the known cause (#1180). Recovery attempts, forced overflow and calls below the threshold never take this exit; when the cuts miss the target the compaction runs as before.

LCM_STUB_FIRST_EXIT_TARGET_RATIO (default 0, env only) lowers that target to a share of the threshold: the exit is taken only when the list is at or under min(threshold - LCM_LEAF_CHUNK_TOKENS, 0.95 x threshold, int(ratio x threshold)), and cuts that stay over it take the leaf path as before. 0 keeps the built-in target and a value of 0.95 or more changes nothing; a negative value, a value over 1 or a non-number keeps the built-in target with a config warning. The target in last_stub_first_exit and on the stop line is the one applied. It matters on a large window: with a 750,000-token threshold the built-in target is 712,500, so an exit can leave a live list of 600,000 tokens that reaches the threshold again soon, while 0.2 takes the exit only at 150,000 tokens or under. The share follows the active model's threshold, so a smaller window gets a smaller target and takes the leaf path more often.

The storage-boundary payload guard is separate from that configurable policy. LCM always scans messages at the store boundary before writing messages.content or messages.tool_calls to SQLite. Inline data:*;base64,... payloads and conservative long base64-looking runs are replaced with compact placeholders and written to the same plugin-managed externalized-payload directory.

This avoids duplicating media-ish payload bytes into lcm.db, FTS shadow tables, WAL files, or ordinary SQLite backups while preserving lossless recovery via the placeholder ref and lcm_expand(externalized_ref=...). If externalization fails, LCM logs a warning and leaves the original text inline rather than dropping data.

lcm_doctor reports SQLite journal_mode, quick_check, database/WAL sizes, largest content/tool-call rows, suspicious inline payload rows, and aggregate externalized-payload stats. Doctor output is metadata-only for these scans.

This guard is scoped to LCM's own lcm.db write boundary. It does not prevent Hermes core, or any other host layer, from writing inline payloads to Hermes state.db, and it does not rewrite historical rows already present in lcm.db. If bytes already landed in Hermes state.db, that is upstream/outside LCM scope; use backup-first cleanup or migration procedures before mutating historical host rows.

Transcript GC is separate and opt-in. It only rewrites already-externalized, already-summarized tool-role rows to compact placeholders. It keeps the same store_id, keeps payload files, skips pinned messages, and preserves lossless recovery through externalized_ref.

LCM retrieval tools default to current-session scope. lcm_grep accepts session_scope='all' or session_scope='session' as an explicit opt-in for bounded archive search over rows already present in lcm.db (raw-message hits only). Once a session id is known, lcm_load_session can enumerate that session's raw transcript in chronological store_id pages without a search query. Use Hermes session_search for broad cross-session history outside the LCM database.

Within the current session, source filters raw rows directly and filters summary nodes by descendant raw-message source lineage. unknown is a real source value, not a wildcard. Legacy blank-source rows are treated as unknown. role, time_from, and time_to are raw-message filters applied in the message search query before result limiting. When a raw-message filter is active, lcm_grep returns raw rows only and reports summary_results_omitted.

Tool responses are bounded so one retrieval call cannot flood the main context.

Lossless recovery means raw content is stored with stable source lineage and can be recovered in deterministic pages:

• lcm_expand(node_id=...) pages immediate sources with source_offset and source_limit
• lcm_load_session(session_id=...) pages ordered raw session rows with after_store_id and next_cursor
• oversized raw messages continue with content_offset
• lcm_expand(externalized_ref=...) pages payload content with content_offset
• lcm_expand_query uses context_max_tokens for auxiliary context and reports truncation/pagination hints when needed

Carried-over summary nodes can become current-session content after /new, but their source eligibility still comes from descendant raw messages. Expanding a carried-over current-session node recovers the original raw message sources even when those sources still belong to a previous session.

LCM-X includes an opt-in operator script for backfilling OpenClaw history into the local LCM-X SQLite store. It supports two source shapes:

1. SQLite LCM database (--source-db) for migrations from an existing lossless-claw/OpenClaw lcm.db.
2. JSONL session exports (--source-jsonl / --source-jsonl-dir) for fresh installs, plugin-off catch-up, or migrations where no source SQLite database is available.

SQLite source example:

python scripts/import_lossless_claw.py \
  --source-db "${HOME}/.openclaw/path/to/lcm.db" \
  --target-db "${HOME}/.hermes/lcm.db" \
  --agent sammy

JSONL source example:

python scripts/import_lossless_claw.py \
  --source-jsonl "${HOME}/.openclaw/agents/sammy/sessions/session-a.jsonl" \
  --target-db "${HOME}/.hermes/lcm.db" \
  --agent sammy \
  --json

For a directory of session exports:

python scripts/import_lossless_claw.py \
  --source-jsonl-dir "${HOME}/.openclaw/agents/sammy/sessions" \
  --target-db "${HOME}/.hermes/lcm.db" \
  --agent sammy

The script is intentionally conservative:

• dry-run is the default; pass --apply to write
• run it against an explicit target DB path, preferably while Hermes is stopped for that profile
• writes create a timestamped target DB backup first when the target already exists
• SQLite imports can include OpenClaw summaries with --include-summaries; this migrates compatible summary rows into Hermes summary_nodes
• JSONL imports migrate raw message rows only; JSONL does not carry a summary DAG
• imported rows keep explicit provenance in session_id and source, for example openclaw-lcm:agent:sammy:<source-session> or openclaw-jsonl:agent:sammy:<source-session>
• the SQLite default provenance identity is the source conversations.session_id, preserving source session boundaries even when many conversations share one session_key
• pass --session-identity session_key only for SQLite imports when you intentionally want conversations with the same source session key grouped into one imported LCM session
• reruns are idempotent for the same --import-id; the default import_id is source-path-derived, so pass a stable --import-id if you may import the same copied DB or JSONL export set from different paths
• --json prints a reconciliation report with scanned, eligible, would_import, imported, skipped_existing, skipped_empty, invalid_rows, warnings, and summary counters
• changing --agent, --namespace, or --session-identity under the same --import-id is treated as the same import and will skip already-tracked source messages; use a new --import-id for a different mapping
• no OpenClaw config or separate secret tables are imported, but raw transcripts, summaries, and tool payloads may contain sensitive user data

This is a local archive migration path. It does not make LCM a general memory provider, and it does not change the current-session retrieval contract for agent tools.

For databases that already contain large historical tool rows, scripts/backfill_externalized_tool_outputs.py can pre-create native recovery sidecars without rewriting SQLite. It is dry-run by default, writes a scrubbed digest/count manifest, is idempotent on repeat apply, and supports guarded manifest-based rollback. See the operator guide.

If plugins.enabled contains hermes-lcm-x, context.engine: lcm-x is set, and the runtime exposes LCM tools, LCM is loaded. The lcm-x (not found) line is a Hermes host discovery/status mismatch, not an LCM storage or compaction failure.

Compatible Hermes gateway hosts expose task-local lane metadata while dispatching /lcm. Once that lane has constructed an agent, /lcm status resolves the same active LCM runtime as lcm_status, including its foreground view while an ignored or stateless side channel is bound. Before the first normal message constructs an agent, or in a genuinely sessionless process, session_id: (unbound) and threshold_tokens: (uninitialized) remain expected.

Run the read-only /lcm status and /lcm doctor diagnostics, or ask your agent to use lcm_status and lcm_doctor with default arguments if slash commands are unavailable. Slash commands are disabled by default; never run repair or apply modes to reproduce a bug. From the next release, ask your Hermes agent to "report a problem with LCM-X"; it shows a draft for your approval. Search existing issues, then use the bug report form. Include versions and short, redacted excerpts from hermes logs -n 300 | grep -i lcm. For security problems, use the private advisory form.

The engine sits between Hermes context assembly and the backing conversation store. It records raw messages, compacts old material into a summary DAG, and exposes retrieval tools that can drill back into exact stored sources.

1. Ingest - persist each message in SQLite with FTS metadata
2. Compact - summarize older messages outside the fresh tail into D0 leaf nodes
3. Condense - merge same-depth nodes into higher-depth summaries
4. Escalate - shrink oversize summaries from detailed to bullets to deterministic truncate
5. Assemble - combine system prompt, highest-depth summaries, and fresh tail
6. Retrieve - use LCM tools to drill into compacted history or synthesize from expanded context

• Feature overview — every feature family, what it does, why it exists, and the switch that enables it
• Agent configuration profiles — copy-paste env profiles: coding agent, long-horizon assistant, fully-local, cost-guarded
• Operator guide — install, activation, full configuration reference, diagnostics
• Retrieval tools reference — exact tool contracts
• Current project state — v0.27.0 stable baseline, separate main-development identity, active work, and proof boundaries
• Benchmark methodology and results — retrieval and judged-QA evaluation contracts, reproduction, and landed result index
• Embeddings setup — free-tier and local embedding providers, warmup, backfill
• LCM paper
• Architecture diagram
• Standard compression diagram
• LCM compression diagram
• Contributing guide
• Code of conduct
• Security policy
• Tags

Important files:

plugin.yaml      manifest
__init__.py      plugin registration and optional slash-command registration
engine.py        LCMEngine main orchestrator
store.py         SQLite message store and FTS
dag.py           summary DAG and FTS
config.py        env var defaults and overrides
command.py       /lcm command handlers
tools.py         lcm_grep, lcm_load_session, lcm_describe, lcm_expand, lcm_expand_query
schemas.py       tool schemas shown to the model
tests/           standalone pytest coverage

Run tests:

pip install pytest numpy
python -m pytest tests/ -v

No Hermes Agent checkout is required for the test suite; tests include a lightweight ABC stub.

Issues and PRs welcome. Bug fixes and correctness improvements are highest priority. New features should be scoped, backwards-compatible, and tested.

See CONTRIBUTING.md for branch, validation, and PR guidance. See CODE_OF_CONDUCT.md for project conduct expectations and SECURITY.md for vulnerability reporting. See the releases page, tags page, and CHANGELOG for version history. v0.28.1 is the latest stable GitHub Release; verify its exact SHA before installation.

MIT