Skip to content

Inspector

Sietse edited this page Sep 29, 2026 · 2 revisions

Memory Inspector

A read-only window into what Galahad remembered, and when.

Status ✅ Works. A separate program, never linked into your server
Verified the inspector cannot write to the directory it reads: the whole directory is identical before and after a full run
Runs galahad-inspector --data-dir /var/galahad --listen 127.0.0.1:8080 --token-file tokens.txt

In plain words

Galahad's memory is invisible. You cannot see what it kept, what it threw away, or what it was holding when something went wrong.

This is a small separate program that reads the saved records and shows them: a timeline of what happened, what each saved entry contains, and how one step led to another.

⭐ It only reads. It can run while your server is down, which is exactly when you most need to know what happened.

The everyday example

An agent gave a strange answer at 2 a.m. In the morning you open the inspector, find that moment on the timeline, and see which saved memory the model was using when it answered.


Running it

galahad-inspector --data-dir /var/galahad --listen 127.0.0.1:8080 --token-file tokens.txt
Option Meaning
--data-dir DIR the Galahad data directory to read
--listen HOST:PORT where to listen (default 127.0.0.1:8080)
--token-file PATH who may connect, and what they may do (see below)
--audit-log PATH write a security log of every request
--no-auth for development only; works on localhost only and warns on every request
--help list every option

⚠ Without --token-file (and without --no-auth), every route that needs a login returns 503.

⭐ It is a separate program and is never linked into your server. It opens no socket inside your server's process.

⚠ Bind it to localhost or put it behind your own authentication. It shows stored content.

⭐ It cannot change what it inspects. That is what makes it safe to point at production data.

The token file

One line per identity: <token> <tenant> <permissions>. Permissions are inspect, rewind, or both, separated by a comma. Lines starting with # are comments.

# token         tenant   permissions
s3cret-token-a  acme     inspect
s3cret-token-b  acme     inspect,rewind

Clients send the token as a bearer token. A tenant of * sees every tenant; the inspector warns about that at startup.


Multi tenant safety

If you run multiple tenants, the inspector keeps them apart:

a tenant reads its own entry 200
another tenant asks for it ⭐ 404, not 403
every sub resource (/tensors, /lineage) 404 cross tenant
a misspelled permission at startup exit 2: it refuses to start

⭐ 404 rather than 403: a 403 would confirm the entry exists.

See Tenant Isolation.


Rewind

Rewind happens in your server, through the C API:

merlin_set_rewind_permission(1);   /* default is DENIED */
merlin_rewind(...);

⭐ Permission defaults to denied. A host that never enables it cannot rewind.

⚠ merlin_rewind does not perform the restore. It checks permission and the record, then hands you a description of what to restore. Your host does the restore. Galahad never reaches into your inference context by itself.

⚠ Permission is checked first, so the answer cannot be used to discover whether an entry exists.

⭐ Bit identical after a rewind.

⚠ Rewind restores memory, not the world. The email was still sent, the card was still charged. Undoing side effects is out of scope.

⚠ The inspector's own rewind view does not rewind anything. It is a separate program with no inference context. It shows you what a rewind would target; your server performs it. Only a token with the rewind permission can use it.


What it does not do

❌ no natural language "what the AI was thinking" it shows what was stored, not an interpretation
❌ no token level attribution inside an entry that would be a guess presented as a fact
❌ no writes, ever

Troubleshooting

Symptom Cause Fix
refuses to start, exit 2 a permission name in the token file is misspelled, or a bad option fix the name; it will not drop a permission you think you granted
every request returns 503 no --token-file given start it with --token-file
--no-auth refuses to start it is bound to an address other than localhost use --listen 127.0.0.1:PORT, or use a token file
cross tenant request returns 404 working as designed that is isolation, not a missing entry
merlin_rewind returns denied permission defaults to off call merlin_set_rewind_permission(1)
merlin_rewind returns busy that entry is in use retry after the in-flight work finishes
rewind "worked" but the agent repeats itself you restored memory but not your own history rewind both, or neither

Related

Clone this wiki locally