Agent Service EN

🌐 Language / 语言: 中文 · English

Agent Service

AI Helper can optionally pair with a Node.js Agent service that provides file system, terminal commands, Skills, the MCP protocol, a local knowledge base (RAG), and file upload capabilities beyond the browser sandbox.

← Back to Wiki Home · Related: Built-in Tools Reference · Configuration · FAQ

---

Why You Need the Agent Service

Browser extensions run in a restricted sandbox and cannot directly read/write local files or run system commands. The Agent service is a separate process running on your machine; the extension talks to it via 127.0.0.1:18910, unlocking:

• Read/write local files, browse the working directory, upload/download files
• Run terminal commands (with a three-tier security policy)
• Use the Skill system
• Extend third-party tools via the MCP protocol
• Local knowledge base (RAG) retrieval-augmented Q&A
• Cross-session long-term memory

Optional: most web-page operations work without installing the Agent; file Q&A automatically falls back to browser-side extraction.

---

Installation & Startup

Global install (recommended)

npm install -g ai-helper-agent

Requires Node.js >= 18.0.0.

Startup

# Foreground (terminal shows live run logs; good for debugging)
ai-helper-agent start

# Background (daemon mode; terminal returns immediately)
ai-helper-agent start --background
ai-helper-agent start -b

# Specify working directory and port
ai-helper-agent start --workdir /path/to/your/project --port 18911

After startup, the terminal shows a 6-digit pairing code; enter it in the "Agent" tab of the extension options page to complete pairing.

Local development mode

cd agent
npm install
npm start
# Defaults to listening on 127.0.0.1:18910

Startup options

--background, -b    Run in the background (daemon mode)
--port <port>       Listening port, default 18910
--host <address>    Listening address, default 127.0.0.1
--workdir <dir>     Working directory (file read/write restricted to this scope)

---

CLI Commands

Command	Description
ai-helper-agent start / start -b	Start in foreground / background
ai-helper-agent stop	Stop the running Agent
ai-helper-agent restart / restart -b	Restart the service
ai-helper-agent status	View run status
ai-helper-agent paircode	View the pairing code
ai-helper-agent config	View the current configuration
ai-helper-agent rag status / rag install	Check RAG dependency availability / one-click install
ai-helper-agent help	Show help
ai-helper-agent --version / -v	Show the version number

aha is a shortcut alias for ai-helper-agent, e.g. aha start -b.

Foreground vs Background

Mode	Command	Terminal behavior	Use case
Foreground	start	Blocks the terminal, shows live run logs	Development debugging, troubleshooting
Background	start --background	Returns immediately, no terminal log output	Production, daily use

In foreground mode, all operation logs stream to the terminal in a formatted way:

[12:30:01] [INFO] [file:read] path=/project/src/index.js size=2048
[12:30:05] [INFO] [command:started] command="npm test" cwd=/project execId=a1b2c3d4
[12:30:12] [INFO] [command:completed] execId=a1b2c3d4 exitCode=0 killed=false
[12:31:00] [WARN] [security:exec_denied] command="rm -rf /" reason=high-risk command blocked

Process management

The Agent manages its process lifecycle via a PID file (~/.ai-helper-agent/agent.pid):

• The PID file is written automatically on startup
• stop first tries a graceful shutdown via the API, and falls back to killing the process via the PID file
• The PID file is cleaned up automatically on normal shutdown
• Full process-tree termination: stopping a command (/api/exec/stop) or hitting a timeout cleans up the entire process tree — grandchild processes spawned by npm etc. (download / build subtasks) are terminated together, never leaving orphans writing to disk; POSIX signals the process group, Windows uses taskkill /T

---

Security Mechanisms

File sandbox

All file operations are restricted to the whitelisted directories in allowedPaths. realpath resolution prevents symlink bypasses, ensuring physical path safety.

Note: the sandbox applies to file tools (read / write / delete / move). Command execution goes through the tiered lists below and does not pass the allowedPaths check.

Tiered command control

Blacklist — blocked directly, cannot be bypassed:

Type	Examples
Disk destruction	rm -rf /, mkfs.*, dd if=... of=/dev/...
Recursive delete of critical system dirs	rm -rf /usr, /var, /bin, /sbin, /lib, /boot, /opt, /root, /System, /Library
System file overwrite	> /etc/passwd, > /etc/shadow
Malicious pipe execution	curl ... | bash, git clone ... | sh
Fork bomb	:(){ :|:& };:

Graylist — requires user confirmation before execution:

Command pattern	Reason
sudo ...	Requires admin privileges
npm install -g ...	Global package install
pip install/uninstall	Python package management
chmod -R 777, chmod 000, chmod a-rwx	Permission change that renders a file unusable
chown ...	Change file owner
rm -rf ... (incl. rm -r -f, rm --recursive --force; relative and absolute paths alike)	Recursive forced deletion
dd of=... (writing a normal file)	Raw overwrite of file contents
shred, truncate -s 0 / --size=0	Destroy / empty a file
curl, wget, nc, telnet, ssh, scp	Outbound network connection / data-exfiltration surface
kill, pkill, killall	Terminate processes
systemctl, launchctl, crontab, sysctl	System service / scheduled jobs / kernel parameters
git push --force	Force push
shutdown/reboot	Shutdown / reboot
doas ..., runas /user: ...	Privilege escalation equivalents of sudo
Remove-Item -Recurse/-Force	PowerShell recursive / forced deletion
pwsh -EncodedCommand ...	Execute a base64-encoded PowerShell script
backticks, $() command substitution	Statically unclear where the substituted result flows

Whitelist — regular commands, allowed directly.

⚖️ Per-segment evaluation: the lists are not matched against the start of the command string only. Each command is first expanded into several "sub-command scopes" and checked segment by segment, keeping the strictest verdict: splitting on shell control operators (&&, ||, ;, pipes, &, newlines), stripping environment-variable prefixes and wrapper commands (nohup / time / env / timeout / nice / command / xargs, plus the privilege prefixes sudo / doas), recursively unwrapping interpreter payloads (bash -c, python -c, cmd /c, powershell -Command / -EncodedCommand), and pulling out the inside of $() and backticks. So cd /tmp && sudo apt install foo, echo 1234 | sudo -S ..., nohup sudo ... and bash -c "rm -rf /" cannot escape confirmation / blocking, while plain text inside quotes is not mistaken for a command (grep -c "sudo rm -rf /" notes.txt still runs directly); but a substitution nested inside quotes is unwrapped layer by layer (bash -c "echo \rm -rf /`", echo "text $(sudo id) text"` still hit the blacklist / graylist). Evaluation can only get stricter, never looser.

🔒 Privilege prefixes are not a downgrade pass: sudo / doas are peeled layer by layer before the check, so sudo rm -rf /usr, sudo mkfs, sudo dd of=/dev/sda, sudo sh -c "rm -rf /usr" and curl … | sudo sh stay deny (they no longer drop to confirmation just for wearing a sudo coat); peeling keeps the sudo … intermediate form, so nohup sudo apt install foo still needs confirmation. Redirects overwriting system files are no longer line-start-only (echo x > /etc/passwd is deny too); path spellings are normalized (rm -rf //usr, rm -rf /.//usr equal /usr, same deny). The blacklist "change root permissions / ownership" only targets the root / itself (chmod -R 777 /, chown -R root /); ordinary absolute paths (chmod 777 /tmp/x) fall back to graylist confirmation rather than a hard block.

🔍 Command substitution is blocked for "suspicious syntax", not for a "dangerous effect": this one feature carries a read-only exemption — when the head command is a non-writing read-only one like echo / printf / cat / ls / pwd / cd / whoami / test, it is not rejected wholesale just because a $(…) follows (pwd; echo $(date +%s), cd /tmp && echo \ls` run directly). A substitution on the right-hand side of an assignment (a=$(echo hi)), inside an interpreter payload (bash -c 'echo $(echo hi)') or inside a quoted argument (git commit -m "feat: $(date)") still requires confirmation; the commands **inside** a substitution are still sent through the black/gray lists, so echo $(sudo id)is confirmed andcd /tmp && echo `rm -rf /`` is still blocked. The exemption covers only this substitution feature — the blacklist and every other graylist entry get no exemption.

Known limits: variable-indirect execution (S=rm; $S -rf ./x), the body content of script files, and "read-equals-exfiltration" (cat /etc/passwd) cannot be classified statically and are not handled at this layer.

Authentication

On first pairing, the user enters the 6-digit pairing code shown in the terminal (the code rotates every 30 seconds). After successful pairing, subsequent requests are authenticated with an HMAC token stored in ~/.ai-helper-agent/pairings.json.

Other protections

• Environment-variable whitelist (~40 vars), with TERM=dumb disabling interactive commands
• Script protection: writing .sh / .py / .js files automatically strips execute permissions
• Size limits: request body 10MB, single file 50MB

---

Configuration File

Config file path: ~/.ai-helper-agent/config.json

{
  "port": 18910,
  "host": "127.0.0.1",
  "workdir": "/path/to/project",
  "allowedPaths": [],
  "pairCodeTTL": 30,
  "commandTimeout": 300000,
  "fileMaxSize": 52428800
}

Field	Description	Default
port	Listening port	18910
host	Listening address	127.0.0.1
workdir	Default working directory	current directory at startup
allowedPaths	Additional allowed directory list	[]
pairCodeTTL	Pairing code validity (seconds)	30
commandTimeout	Command execution timeout (ms)	300000 (5 min)
fileMaxSize	Max file read/write bytes	52428800 (50MB)

---

HTTP API

No authentication

Method	Path	Description
POST	/api/pair	Pairing authentication
GET	/api/status	Health check (version, platform info, search-tool availability)
POST	/api/shutdown	Shut down the Agent service (local access only)

Requires authentication (Bearer Token)

Method	Path	Description
POST	/api/fs/read	Read file content
POST	/api/fs/write	Write a file
POST	/api/fs/list	List a directory
POST	/api/fs/delete	Delete a file / directory (recycle bin by default)
POST	/api/fs/stat	File details (permissions, UID/GID, MIME, timestamps)
POST	/api/fs/upload-stream	Streaming upload
POST	/api/fs/download-stream	Streaming download
POST	/api/fs/preview-xlsx	Excel parse preview
POST	/api/files/upload	File upload
POST	/api/fs/search_files	Search files by name pattern (glob)
POST	/api/fs/search_content	Search file content (rg preferred, Node.js fallback)
POST	/api/exec	Run a system command
POST	/api/exec/stop	Stop command execution
GET	/api/exec/running	List running processes
GET	/api/status/detail	Detailed info (working directory, pairing code, search tools)
GET	/api/logs	Query audit logs (date/category/limit/offset supported)
GET	/api/logs/dates	Get available log date list
POST	/api/agent/restart	Restart the Agent online
POST	/api/agent/update	Update the Agent online
POST	/api/trash/restore	Restore a file from the recycle bin
GET	/api/trash/list	Recycle bin list

Long-term memory (/api/memory/*)

Memory extraction, storage and recall all happen inside the agent process (entries live in ~/.ai-helper-agent/memory/global-memory.json, derived vectors in memory-vectors.json); the extension's tool-memory.js only forwards requests. These also require the Bearer token.

Method	Path	Description
POST	/api/memory/store	Create / update / delete memory entries (add / update / delete) with type, tags and importance
POST	/api/memory/recall	Recall memories: keyword and semantic vector channels fused by rank; returns tier (none / remote / local) and vectorMatched
POST	/api/memory/manage	Memory governance: quality review (review), compaction and eviction of low-value entries (compact)
POST	/api/memory/inject	Assemble the memories injected into the current turn
POST	/api/memory/reindex	Rebuild every vector with the current embedding config; returns skipped: true when the vector tier is disabled
GET / POST	/api/memory/embedding-config	Read / save the semantic recall config; the plaintext API key is never echoed (only a hasApiKey flag); pass clearApiKey: true to clear it explicitly
POST	/api/memory/test-embedding	Connectivity probe (8-second timeout); the body may override unsaved draft values
GET	/api/memory/vector-status	Read-only status: indexed / total counts, vector fingerprint and staleness, active tier

For what each option means and where to set it, see Configuration → Memory (Vector Recall) Configuration.

Speech Recognition (/api/asr/*)

Local voice input does its inference, model management and dependency installation inside the Agent process (models under ~/.ai-helper-agent/asr/models/, config in asr-config.json); the final pass runs on a dedicated worker thread and never blocks other endpoints. Bearer Token required, like the rest of the API.

Method	Path	Description
GET	/api/asr/status	Dependency / model / switch / loaded engine state
POST	/api/asr/config	Update config (switch, language, model ids, idle unload)
GET	/api/asr/models	Scanned model packs (invalid ones include a reason)
POST	/api/asr/models/scan	Re-scan the model directory
POST	/api/asr/models/pick	Import a pack from a directory
GET	/api/asr/models/catalog	Downloadable model catalog
POST	/api/asr/models/download	Download (resumable + hash verified)
GET	/api/asr/models/download/status	Download progress
POST	/api/asr/models/download/cancel	Cancel a download
POST	/api/asr/transcribe	One-shot recognition of a WAV buffer
POST	/api/asr/warmup	Warm the engines (also runs at startup)
POST	/api/asr/deps/detect	Detect the sherpa-onnx dependency
POST	/api/asr/deps/install	Install it (allow-listed command)
GET	/api/asr/deps/install/status	Install progress

Setup steps and the full protocol → Voice Input (Local STT).

WebSocket

Path	Description
ws://127.0.0.1:18910/ws/exec/:execId	Real-time command execution output stream
ws://127.0.0.1:18910/ws/asr/:sessionId	Streaming speech session (PCM up, partial / final down)

---

Skill System

The Agent service has a built-in Skill system for turning operation workflows into reusable skills.

Skill types

Type	Definition format	Execution	Purpose
Workflow Skill	JSON / YAML	Direct execution	Automation workflows, executed step by step
Agent Skill	SKILL.md	AI autonomous invocation	Knowledge distillation, triggered in conversation

Directory structure

All Skills are stored under ~/.ai-helper-agent/skills/:

~/.ai-helper-agent/skills/
├── workflow-skill.json          # Workflow Skill (JSON format)
├── another-skill.yaml           # Workflow Skill (YAML format)
└── agent-skill/                 # Agent Skill (directory form)
    ├── SKILL.md                 # Skill definition file
    └── _meta.json               # Metadata (optional)

SKILL.md format

---
name: <skill-name>
description: "<concise description, including: (1) what the skill does, (2) when to trigger it>"
enabled: true
---

# <skill title>

## When to Use This Skill

- trigger condition 1
- trigger condition 2

## Core Capabilities

- capability 1
- capability 2

## Usage

### Step-by-Step

1. step 1
2. step 2

## Examples

[concrete examples]

## Source

Distilled from conversation, created: YYYY-MM-DD

Built-in skill

• skill-creator: a meta-skill for creating and updating other skills from conversation.

Skill API

Method	Path	Description
GET	/api/skill/list	Get the list of all Skills
GET	/api/skill/:name	Get a single Skill's full definition
POST	/api/skill/import	Import a new Skill
POST	/api/skill/:name/toggle	Toggle enable / disable
DELETE	/api/skill/:name	Delete a Skill
POST	/api/skill/:name/run	Run a Workflow Skill

---

MCP Protocol Extension

Supports the Model Context Protocol (MCP) to extend third-party tool capabilities.

MCP Server configuration

Config file path: ~/.ai-helper-agent/config.json

{
  "mcpServers": [
    {
      "id": "my-mcp-server",
      "name": "My MCP Server",
      "command": ["python", "-m", "my_mcp_server"],
      "enabled": true
    }
  ]
}

MCP API

Method	Path	Description
GET	/api/mcp/status	Get the status of all MCP Servers
POST	/api/mcp/:serverId/connect	Connect a specific MCP Server
POST	/api/mcp/:serverId/disconnect	Disconnect
GET	/api/mcp/tools	Get the list of all MCP tools

How it works

1. The Agent connects to all enabled MCP Servers on startup
2. JSON-RPC 2.0 communication is established over stdio
3. Tools provided by the MCP Server are auto-discovered
4. Tool-call requests are forwarded through the Agent to the MCP Server
5. Tool results are returned to the extension

You can also add and manage MCP Servers visually in the "Toolbox" tab of the extension options page.

---

Knowledge Base (RAG)

An optional local retrieval-augmented generation (RAG) capability: index documents into a searchable knowledge base so the extension can answer with citation sources.

• Optional dependency — availability is auto-detected at Agent startup and exposed via /api/rag/status; when dependencies are missing, RAG requests return 503, and the extension offers one-click install (whitelisted dependencies, progress pollable via /api/rag/install/status); installation is also available from the CLI: aha rag install
• Automatic recovery on startup — for users who installed the dependencies successfully before: npm -g upgrades prune manually-installed packages inside the package directory, so the Agent re-installs them automatically at startup (marked trigger=auto) with zero user action; users who never installed successfully are not triggered
• Data location — the knowledge base is stored in ~/.ai-helper-agent/rag/ (overridable with the AI_HELPER_RAG_ROOT env var)
• Multiple knowledge bases — create / update / delete / enable / disable knowledge bases, each maintaining its own vector config and statistics
• Multi-format import — .txt .md .json .csv .html / PDF / Word (mammoth) / PPT (officeparser) / Excel; supports text, file (base64), and URL import
• Hybrid retrieval — vector similarity + keyword-hit boosting (mixed Chinese/English keyword extraction); for cross-library search, the topK quota is split evenly across libraries
• Pluggable vectorization — a local model by default (Xenova/bge-small-zh-v1.5, 512-dim, run locally via @huggingface/transformers), or any OpenAI-compatible vector service
• Index rebuild — when a knowledge base's vector space changes, the index is rebuilt in the background (replay in a temp directory → atomic swap-in)

RAG API

The following endpoints are registered in the authenticated zone (Bearer Token required) and are loaded on demand only when RAG dependencies are available.

Method	Path	Description
POST	/api/rag/install	Trigger whitelisted RAG dependency install
GET	/api/rag/install/status	Install progress
POST	/api/rag/detect	Re-detect RAG dependency availability
GET	/api/rag/status	Capability and install status
POST	/api/rag/test-embedding	Test remote vector-service connectivity (returns actual dimension)
GET	/api/rag/collections	List knowledge bases
POST	/api/rag/collections	Create a knowledge base
PUT	/api/rag/collections/{id}	Update name / description / vector config
DELETE	/api/rag/collections/{id}	Delete a knowledge base
POST	/api/rag/collections/{id}/toggle	Enable / disable a knowledge base
GET	/api/rag/collections/{id}/stats	Document / chunk count statistics
POST	/api/rag/collections/{id}/ingest	Ingest text / file / URL
GET	/api/rag/collections/{id}/ingest/status	Ingestion progress snapshot
GET	/api/rag/collections/{id}/documents	Document list
DELETE	/api/rag/collections/{id}/documents/{docId}	Delete a document
POST	/api/rag/collections/{id}/search	Single-library search
POST	/api/rag/search	Cross-knowledge-base search

---

Long-term Memory

Cross-session long-term memory is owned by the Agent: the storage file, the read-modify-write lock, recall scoring, review/compaction and injection selection all run inside the Agent process. The extension only forwards agent_memory tool calls — no generic file-API round-trips, and no dependency on the extension Service Worker staying alive.

• Data location — ~/.ai-helper-agent/memory/global-memory.json (facts + summaries + stats)
• Record fields — id / type (fact or summary) / category / content / title / tags / importance (1–10) / accessCount / lastAccessAt / createdAt / updatedAt / sourceSessionId
• Three injection tiers — Tier 1/2 apply to fact entries only: importance >= 8 is always resident and the rest are injected by value within a 1500-character / 12-item budget; summary entries are never injected and, together with the remaining facts, surface only as tag index lines for the model to pull with a recall call
• Recall scoring — keyword hits give content +3 / title +2 / tags +2; a full query (≥ 2 characters) found inside content adds +5 and inside title adds +3; CJK runs of ≤ 4 characters stay whole, longer runs split into bigrams, latin/digit runs become word tokens, stop words filtered, capped at 30 keywords (zero external dependencies)
• Semantic vector channel — optional: once an embedding is configured and saved in the options page Memory tab, recall adds a vector similarity pass and the two channels fuse by rank (the semantic tier takes pure top-K with no absolute score floor). Tiers are none (vectors off, keywords only) / remote (OpenAI-compatible endpoint) / local (local model, requires the RAG dependency); when vectors are unusable recall silently falls back to keywords
• Derived vectors and rebuilds — vectors are cached in memory-vectors.json beside the memory file; they are derived data and can be dropped and rebuilt. They are filed under a fingerprint of source + model + dimensions + prefix, so changing the config requires a reindex call — otherwise vector-status reports staleness and semantic ranking silently degrades
• Value and eviction — value = importance × (1 + ln(accessCount + 1)) × timeDecay (≤ 7 days 1.0 / ≤ 30 days 0.8 / ≤ 90 days 0.5 / older 0.2); entries dropping below value 10 are evicted by manage with action: compact
• Capacity warning — meta.maxFacts defaults to 50 and maxSummaries to 20; hitting 80% makes the store response carry a near-limit warning (nothing is auto-deleted — cleanup requires an explicit compact)
• Where summaries come from — besides explicit agent_memory tool calls, the extension's session context compaction sedimentates summaries automatically: type: summary / category: decision / importance: 6 / tags compaction + session:<id>; re-compacting the same session issues an update against the same memoryId (no new entries), and a sedimentation failure is only logged — the session summary itself is unaffected
• Visual entry point — the entry manager at the bottom of the options page Memory tab invents no new endpoint; it reuses manage (review / compact) and store (update / delete): the list reads the side-effect-free review (no accessCount is recorded and the per-session dedup cache is untouched), edits go through update which re-embeds that entry in the background, and deletes plus compaction also dropVectors and evict the ID from the recall cache so no orphan vector points at a removed entry
• Per-session dedup — the Agent process remembers which memories this conversation already surfaced and flags repeated recall
• i18n — response text is rendered from the request's Accept-Language

Long-term memory requires an Agent build shipping the /api/memory/* endpoints. The extension deliberately does not fall back to the generic file API: a missing endpoint returns an upgrade hint, and memory is unavailable entirely when the Agent is not connected.

---

File Search

The Agent prefers the system's native search tools and falls back to a Node.js implementation when unavailable:

Engine	Purpose	Detection command
fd	File-name search (fast)	fd --version
rg (ripgrep)	File-content search (fast)	rg --version

---

Audit Logs

All operations are recorded to audit logs under ~/.ai-helper-agent/logs/.

Dual-channel output

• Terminal output (foreground mode): formatted, human-readable, written to stderr
• File output: JSON Lines format, written to log files (named per day agent-YYYY-MM-DD.log)

Log categories

category	Description	Operations included
auth	Authentication events	pairing success / failure
fs	File operations	read, write, list, delete, search_files, search_content
exec	Command execution	started, completed, stopped, error
security	Security events	deny, confirm, auth failure, path-escalation blocks
system	System events	server_start, server_stop, shutdown, server_error, etc.

Log query API

GET /api/logs?date=2026-01-15&category=security&limit=50&offset=0
GET /api/logs/dates

• date — date filter (YYYY-MM-DD), defaults to today
• category — category filter, returns all if omitted
• limit — max number of results, default 200
• offset — pagination offset

Auto-cleanup

• Keeps at most 30 log files
• Files over 10MB are deleted automatically

---

Tech Stack

• Node.js >= 18
• Native http module (HTTP service)
• ws library (WebSocket service)
• Zero external framework dependencies
• Optional dependencies: fd, rg (ripgrep) — to accelerate file search
• RAG optional dependencies: vectra (vector storage), @huggingface/transformers (local vectorization), pdf-parse / mammoth / officeparser / cheerio (document parsing)