Agent Context

Agent Context

简体中文 · Home

Agent context helps the refinement model spell project-specific names, commands, paths, API names, and model IDs. It is optional and is sent only as part of LLM refinement.

What is sent

When both options are enabled:

[llm]
enabled = true
agent_context_enabled = true
agent_context_max_chars = 6000

Voice Input sends a JSON user message containing:

• the ASR transcript;
• trusted metadata naming the detected agent as Pi or Codex;
• a sanitized excerpt of the latest completed assistant message from the Pi or Codex session focused when dictation ends.

The system prompt explicitly labels the assistant message as untrusted, limits its purpose to terminology resolution, and tells the model not to follow instructions inside it or import facts the speaker did not say. This is a prompt-level defense, not a proof that every third-party model will behave perfectly.

Discovery requirements

Voice Input captures context when a manual or automatic stop is accepted, before final ASR and refinement. It proceeds only when the active Hyprland window class at that moment is kitty. An explicit record cancel skips discovery. Voice Input queries that Kitty OS window through:

unix:/tmp/kitty-<kitty-process-id>

The kitty @ … ls result must identify a focused foreground process whose executable basename is exactly pi or codex. The query has a one-second external timeout.

Your Kitty launcher must therefore enable remote control and create the expected per-process socket. One launcher pattern is:

#!/bin/sh
exec kitty -o allow_remote_control=yes \
  --listen-on "unix:/tmp/kitty-$$" "$@"

Because exec preserves the shell PID, the socket name matches the Kitty process. Adapt this to your desktop launcher and security policy. Verify the actual active Kitty PID and socket before enabling context.

Pi sessions

make install copies:

~/.pi/agent/extensions/voice-input-session-registry.ts

The extension publishes a mode-0600 registry file in:

$XDG_RUNTIME_DIR/voice-input/agent-sessions/pi-<pid>.json

It updates on Pi session lifecycle events and every five seconds, and removes its own registry on shutdown. Voice Input validates:

• registry schema version, PID, and Linux process start ticks;
• the canonical session path is below ~/.pi/agent/sessions;
• the session header ID matches the registry;
• process identity, file device, and inode still match when context is later loaded.

Pi must be restarted or its extensions reloaded after installation/update. An already running Pi process does not automatically load a newly copied extension.

For reference extraction, Voice Input scans at most the last 8 MiB of session JSONL and walks the active branch backward to the latest assistant message with stopReason = "stop". Tool-call blocks are ignored; only text blocks are collected.

Codex sessions

Codex needs no bundled extension. Voice Input:

1. reads CODEX_HOME from the focused Codex process, falling back to ~/.codex;
2. inspects that process's open file descriptors under Linux /proc;
3. accepts .jsonl files canonically below CODEX_HOME/sessions whose header says source = "cli" and thread_source = "user";
4. proceeds only when exactly one eligible session file is open;
5. takes the latest assistant final_answer, with task_complete.last_agent_message as a compatibility fallback.

Container boundaries, restricted /proc, multiple eligible files, or a non-CLI session cause context discovery to return no reference rather than guessing.

Validation, redaction, and truncation

The focused Pi or Codex process is captured synchronously when dictation stops. Session resolution then runs in parallel with final ASR, and the session is read immediately before refinement. Switching focus during final ASR therefore does not change the selected session. Voice Input rechecks process start time and the session file's device/inode to reduce PID reuse and file replacement races.

Before upload, it:

• replaces whole lines containing common markers such as authorization headers, API keys, passwords, secrets, tokens, private-key headers, or cookies;
• replaces token-shaped words with known prefixes and long JWT-like values;
• clamps the configured limit to 500–12,000 characters;
• when needed, retains roughly the first two thirds and last third with an ellipsis.

Redaction is heuristic. It cannot recognize every secret format or sensitive sentence. Do not enable agent context for sessions whose assistant output must not be sent to the configured LLM provider.

Failure behavior

Agent-context discovery is optional and fail-soft:

• unsupported focus at dictation stop, failed Kitty query, invalid registry, missing completed message, or validation failure produces transcript-only refinement;
• a recognized context-size/payload failure may retry without context, but the retry shares the original refinement deadline and requires at least one second remaining;
• authentication, rate-limit, transport, provider, timeout, and truncation errors do not trigger that context-free retry; refinement then fails open to ASR text.

No context lookup changes the ASR result directly. Context is used only by the optional refinement request.

Diagnose context selection

Follow daemon logs without copying session content into a bug report:

journalctl --user -u voice-input.service -f

Useful messages report whether a supported session was captured at dictation stop, whether a completed assistant message was usable, and the agent label plus character count. They do not print the excerpt itself.

Check Pi registry creation:

ls -l "$XDG_RUNTIME_DIR/voice-input/agent-sessions/"

Check the Kitty endpoint using the actual active-window PID:

hyprctl activewindow -j
kitty @ --to unix:/tmp/kitty-<pid> ls

Replace <pid> manually; do not publish the resulting session command lines or paths unless they are safe to disclose.

See also: Security and Privacy · Troubleshooting