Skip to content

Ground CLI agents in the real workspace list, and stop surfaces asserting more than they know - #311

Merged
ProfSynapse merged 7 commits into
mainfrom
claude/cli-workspace-hallucination-7zrzc6
Aug 6, 2026
Merged

Ground CLI agents in the real workspace list, and stop surfaces asserting more than they know#311
ProfSynapse merged 7 commits into
mainfrom
claude/cli-workspace-hallucination-7zrzc6

Conversation

@ProfSynapse

Copy link
Copy Markdown
Owner

Problem

CLI/MCP agents invented Nexus workspace names, failed loadWorkspace, and retry-looped on more invented names.

The stated ask was better error recovery, but the cause was upstream: at the moment an agent chooses a workspace, it was shown zero real names, so it inferred one from the user's phrasing. Recovery is the backstop; grounding is the fix.

Root cause

GetToolsTool built its workspace line from a boot-time snapshot that is empty unless isSQLiteReady() passes at agent-registration time. On desktop the storage adapter is created on a 3-second timer after the connector registers agents, so the snapshot is empty on essentially every boot. refreshSchemaData() existed to repair this and was never called by anything.

Two related defects made it worse: loadWorkspace dead-ended on a miss, and the description asserted [default] was the complete list — a confident falsehood on every vault that had workspaces.

What changed

  • Live grounding. getTools resolves workspace names at call time (5s cache) and returns them in data.workspaces. The provider awaits WorkspaceService and does not depend on the boot-time gates; the boot snapshot keeps its non-blocking behavior, where blocking startup is the real risk.
  • Miss recovery. A single confident name/id match auto-loads and reports the substitution in a resolution field; 2+ matches return a ranked shortlist plus the exact retry command; 0 matches return the real inventory. No branch dead-ends.
  • Readiness gating. No surface claims a list is exhaustive unless the cache says it is.
  • Trace attribution. Traces can no longer be filed under an unresolvable identifier.

Defects found by live testing

Unit tests could not see these; each was found by running the plugin against a real vault.

# Defect Impact
1 Partial list asserted as complete Cache had replayed 1 of 12 workspaces and getTools called them "the only workspaces that exist"
2 loadWorkspace declared a real workspace nonexistent Same cold-cache window
3 A 0.075 description-only match vetoed a 0.8 name match The auto-load this PR exists to provide never fired
4 resolution note silently dropped by the batch formatter An auto-resolved near-miss looked like an ordinary load; the agent was never told a different workspace was opened
5 Unquoted retry command --workspace Blog Testing Workspace — three arguments to any agent copying it
6 plugin.workspaceService does not exist on NexusPlugin The invalid-workspace validation branch was silently dead
7 UNIQUE-name recovery re-ran the lookup that had just failed Failed to persist session on the first CLI call after every reload
8 resolveWorkspaceId passed unvalidated strings through as workspace ids Every unmatched name minted a phantom event store; ws_--workspace/ had been accumulating since 2026-07-19

They are one disease: a surface stating something with more confidence than it had earned. The original hallucination bug is the same thing one level up.

Verification

Verified live against a real vault, before and after each fix:

  • getTools returns all real workspace names (previously: no workspaces field, description claiming [default])
  • Description self-heals after the first live lookup, so later tools/list reads are correct
  • Near-miss auto-loads and reports the substitution; ambiguous returns a ranked shortlist without loading; nonsense returns the real inventory
  • Cold cache reports INCOMPLETE and steers to retry the original name, rather than declaring the workspace missing
  • No phantom workspace directories created, under either the positional or flag form
  • Re-verified after rebasing onto main's nexus use argv rework

Full suite 4213 passed / 1 failed / 23 skipped. The single failure is LocalCliInstaller › on Windows › reports a namesake command that appears earlier on PATH, confirmed pre-existing by running it with these changes stashed. Build and lint clean.

Notes

Two adjacent issues surfaced and are not addressed here, deliberately:

  • The post-load cache rebuild window measured 18 seconds on the test vault, and every defect above lived inside it. Root causes are fixed; nothing tells a caller how long to wait.
  • logger.systemWarn and logger.systemLog are no-ops, which is why the workspace-lookup failure stayed invisible. Anything relying on them for diagnostics is silently doing nothing.

🤖 Generated with Claude Code

claude and others added 7 commits August 6, 2026 09:39
Agents driving the CLI invent a workspace name, fail to load it, and retry
on another invented name. Three causes, addressed here; paused mid-review
pending design confirmation.

1. Discovery shows an empty list. getTools bakes its "Workspaces: [...]"
   line into a description built from a boot-time snapshot that is only
   populated when SQLite is query-ready at agent registration, and
   ToolManagerAgent.refreshSchemaData() — which exists to repair that — is
   never called. Reported in the field as "Workspaces: [default]" on a vault
   with several workspaces. The description also only travels in tools/list,
   so the agent may never see it while choosing. getTools now resolves the
   list live at call time and returns it in the RESULT, which is a surface
   the caller definitionally reads.

2. loadWorkspace dead-ends on a miss. A bare "not found" leaves guessing
   again as the only available move. It now recovers: a single match on
   name/id auto-loads with a resolution report naming what was actually
   opened, several matches return a ranked shortlist with the retry command,
   and no match returns the real inventory. No branch dead-ends.

3. The schema text invited the guess ("you do not need to call
   list-workspaces just to find the UUID"). Rewritten to require a name the
   caller has actually seen.

Also: an invalid context.workspaceId now lists live names and the closest
match, instead of reporting "(none created yet)" off the same stale snapshot.

Note on scoring: the matcher charges for query tokens the target lacks, so
the common hallucination shape ("Research Notes" for "Research") scores
0.125, not 0.8. Auto-resolution keys on a single name/id match rather than a
high score, and strips the filler word "workspace" from the request.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C3HZr39c4UMCMAMNTf8BqZ
The prior WIP added a live workspace provider but routed it through
listWorkspaceSummaries(), which is gated on getServiceIfReady() plus
isSQLiteReady(). Those gates are correct for the boot snapshot and wrong
at call time, so the headline fix could still hand agents an empty list.

Split the two:

- listWorkspaceSummariesLive() awaits WorkspaceService (instantiating it
  if needed) and drops the SQLite gate, bounded by a 4s timeout so a
  wedged storage layer degrades discovery instead of hanging it. The
  boot snapshot keeps its non-blocking behavior.
- getTools no longer asserts "[default]" as the complete list when the
  snapshot is empty. That string was confidently false on every vault
  with workspaces, and is what sent agents off inventing a name.
- A live lookup now writes the real names back into the description, so
  later tools/list reads (a fresh CLI invocation, a reconnecting client)
  stop serving the empty boot-time list.
- ToolBatchExecutionService read plugin.workspaceService, which does not
  exist on NexusPlugin — the invalid-workspace validation branch was
  silently dead. Route through plugin.services / getService() instead.

Verified live against a vault with 12 workspaces: getTools previously
returned no workspaces field at all and described the vault as
"[default]"; it now returns all 12 by name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tain

Live testing on a real vault turned up four defects the unit tests could
not see, all of the same shape: a surface stating something with more
confidence than it had earned.

- The live lookup now waits for query-readiness before reporting. For a
  few seconds after load the SQLite cache is still replaying JSONL and
  listWorkspaces() returns a partial set — measured at 1 of 12. Callers
  present that list as "the only workspaces that exist", so a partial
  answer is the same falsehood in different words. It now reports none,
  and the description says the list is elsewhere.
- loadWorkspace no longer reports "does not exist and nothing resembles
  it" off a rebuilding cache. Observed live telling us a real workspace
  was missing. It now says the list is INCOMPLETE and to retry the
  original name.
- resolveWorkspaceIdentifier counts rivals by the same standard used to
  qualify a winner. "Blog Testing" matched the right workspace at 0.8 on
  its name, but an unrelated workspace scored 0.075 because its
  DESCRIPTION ended "...handle testing" — enough to veto the auto-load.
  Description-only hits were already declared insufficient to pick a
  workspace; they no longer block one either.
- The batch formatter kept only `data` and silently dropped sibling
  top-level fields, so loadWorkspace's `resolution` note never reached
  the caller: an auto-resolved near-miss looked like an ordinary load
  and the agent was never told a different workspace had been opened.
  Extras now merge in, with `data` spread last so no key changes value.

Also quotes the retry command in the shortlist branch — it emitted
`--workspace Blog Testing Workspace`, which an agent copying verbatim
would have passed as three arguments.

Verified live, all three recovery paths: near-miss auto-loads and
reports the substitution; an ambiguous fragment returns a ranked
shortlist without loading; a nonsense name returns the real inventory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ng row

createWorkspace already catches "UNIQUE constraint failed: workspaces.name"
for the default workspace and tries to reuse the row that must exist. But
the recovery re-ran the two lookups that had just returned null — that is
how the INSERT was reached in the first place — so during the post-load
cache rebuild it was guaranteed to fail and rethrow.

Observed on every reload: the first CLI call auto-creates a session,
addSession cannot see "Default Workspace" in the still-replaying cache,
inserts a duplicate, and the whole thing surfaces as "[SessionService]
Failed to persist session". That session's traces are lost.

The constraint is proof the row exists, so a null lookup means our view is
stale rather than the workspace being absent. Wait for query-readiness
before looking again, bounded at 5s so a stalled rebuild fails the write
instead of wedging it.

Also extracts withTimeout into src/utils/, now shared by WorkspaceService
and AgentInitializationService.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`memory load-workspace` takes its argument both positionally and as
`--workspace <value>`, but the trace extractor read token[2]
unconditionally. On the flag form that token IS "--workspace", so the
literal string became the trace's workspace id — creating a phantom
Nexus/data/workspaces/ws_--workspace/ event store and logging
"Workspace --workspace not found" on every such call.

Handles the positional form, `--workspace <value>`, `--workspace=value`,
and unrelated flags appearing first. Flag detection mirrors
ToolCliNormalizer's `!wasQuoted && startsWith("--")` rule exactly: the
extractor has to agree with the parser, or a trace gets attributed
differently than the call actually ran.

Note: `--workspace="X"` (quoted value on the = form) parses as a
positional system-wide because of that shared rule. Pre-existing
normalizer behavior, left alone deliberately rather than diverged from.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The workspace id doubles as an event-store key, so writing a trace to an
id that does not exist CREATES it. resolveWorkspaceId ended in
`workspace?.id || candidate` — handing the raw, unvalidated string
straight through whenever the lookup missed.

That is the actual root cause of the phantom directories. The previous
commit stopped "--workspace" becoming the handle, but the handle is still
the name the caller REQUESTED, and getWorkspaceByNameOrId is exact-match
— so every near-miss minted its own workspace. Verified by watching
Nexus/data/workspaces/ws_Blog Testing/ appear during testing, moments
after removing ws_--workspace/.

Now: resolve, and on a miss fall back to the session's workspace (only if
IT resolves) and finally 'default'. A trace may be attributed to a
broader workspace than intended; it can no longer invent one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The console reported "Workspace not found" for every failed resolution,
including the case where the workspace exists and the cache is simply
still rebuilding. Those are different problems and reporting them
identically sends debugging down the wrong path. Log the note the caller
was actually given.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ProfSynapse
ProfSynapse merged commit 51fa3e9 into main Aug 6, 2026
@ProfSynapse
ProfSynapse deleted the claude/cli-workspace-hallucination-7zrzc6 branch August 6, 2026 13:48
ProfSynapse pushed a commit that referenced this pull request Aug 8, 2026
…#317)

`validateWorkspaceId` tested the name and the id against different lists. The
name was only ever matched against `knownWorkspaces` — the boot-time SchemaData
snapshot, which this file's own comment notes is empty whenever SQLite was not
query-ready at agent registration — while the id was matched against the live
`listWorkspaces()`.

The error message was already built from the live list (#311), so on exactly the
vaults #311 targeted the guard could only ever suggest a name it had never
checked for acceptance, producing a self-refuting rejection:

    Invalid workspace "Desenvolvedor". Closest match: "Desenvolvedor". …

That contradicted the other half of the mandatory pair: getTools grounds the
caller with the live names and says to pass one verbatim, and useTools then
answered "do not infer a workspace name from the user's wording". An agent
following the grounding perfectly was told it invented the name. Every by-name
envelope was unusable on such a vault until callers switched to UUIDs — the
opposite of the grounding #311 introduced.

Match id OR name against the live list in one pass, so acceptance and suggestion
read the same source. The snapshot fast path stays (it avoids the async lookup
where the snapshot is populated) but is now documented as accept-only: a miss
falls through to the live list, so a stale or empty snapshot can never reject a
real workspace.

Reported with a diagnosis by @gcp007-ops; verified independently against the
code rather than applied as given.

Tests: tests/unit/EnvelopeWorkspaceValidation.test.ts, 7 cases, RED verified
first — the name cases failed while the id case and the legitimate rejection
passed, pinning the defect to the by-name path. The last case is a
self-contradiction lock: for every live workspace, a rejection may never name
that same value as its closest match, so any future divergence between the
accept list and the suggest list fails there in either direction. That lock plus
the rejection case also caught a fail-open regression mid-fix, when a dropped
`listWorkspaces()` line sent a ReferenceError into the catch and made the guard
accept everything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dqXjmfKkH27qMhVAtS6JH
ProfSynapse added a commit that referenced this pull request Aug 8, 2026
`validateWorkspaceId` tested the name against `knownWorkspaces` — the boot-time
snapshot that is empty whenever SQLite was not query-ready at registration — and
the id against the live `listWorkspaces()`. Since #311 already built the error
message from the live list, on exactly the vaults #311 targeted the guard could
only ever suggest a name it had never checked for acceptance:

    Invalid workspace "Desenvolvedor". Closest match: "Desenvolvedor". …

That contradicted the mandatory pair: getTools grounds the caller with the live
names and says to pass one verbatim, and useTools then answered "do not infer a
workspace name from the user's wording".

Match id OR name against the live list in one pass so acceptance and suggestion
read the same source. The snapshot fast path stays but is documented as
accept-only — a miss falls through to the live list, so a stale or empty
snapshot can never reject a real workspace.

Tests: EnvelopeWorkspaceValidation.test.ts, 7 cases, RED verified first, plus a
self-contradiction lock asserting a rejection may never name that same value as
its closest match. Suite 4257 passed.

Reported and diagnosed by @gcp007-ops (#317); verified independently against the
code rather than applied as given.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants