Repository navigation
Core Features EN
🌐 Language / 语言: 中文 · English
AI Helper provides 34 core capabilities covering input, reasoning, collaboration, management, and reliability. This page introduces them grouped by topic so you can quickly understand what the extension can do.
← Back to Wiki Home · Related: Built-in Tools Reference · Architecture Overview
Attach images to a conversation and use a Vision API (OpenAI-compatible) for multimodal understanding and Q&A:
- Image compression: large images are automatically compressed (longest edge 1024px + JPEG 65%) to reduce token consumption
- Independent API config: a separate API Base / API Key / model can be configured for image recognition
- Global switch: image input can be enabled / disabled at any time
- Multiple images: attach several images at once for comparative analysis
Upload files directly in the browser and extract their content — no Agent service required:
| Format | Extraction engine | Notes |
|---|---|---|
| PDF.js (pdfjs-dist) | Full text extraction, multi-page support | |
| Word (.docx) | mammoth.js | Rich text to plain text |
| Excel (.xlsx/.xls) | SheetJS (xlsx) | Multi-sheet CSV export |
| Plain text | FileReader API | Auto-detects 50+ extensions |
- Agent-first upload: when connected to an Agent, files are uploaded to the working directory so the LLM can operate on the original files directly
- Browser fallback: automatically falls back to browser-side extraction when the Agent is unavailable
- File preview bar: shows file name, size, extraction status, and supports deletion
A complete built-in image annotation capability — edit images in the preview before sending:
- 6 annotation tools: brush (B), rectangle (R), ellipse (E), arrow (A), line (L), eraser
- Adjustable color / thickness / opacity
-
Undo support (up to 20 steps,
Ctrl+Z) -
Keyboard shortcuts:
Enterto confirm,Escto cancel - Auto-update after editing: annotations immediately update the attachment list
The project uses the ReAct (Reasoning + Acting) pattern as its core reasoning engine:
- Dynamic MCP tool injection: pulls the latest MCP tool list from the Agent before each reasoning round
- Tool pre-selection: before calling the main model, a lightweight API call decides which tools are needed, reducing 40+ built-in tools to 5–10 relevant ones and greatly cutting token consumption (simple questions can skip the loop and answer directly)
- Reasoning loop: LLM thinks → decides to call a tool → executes the tool → feeds back the result → continues reasoning
-
Token budget management: dynamically computes the available budget (80%) from the model's context window, truncates by token count, and preserves
tool_calls/toolmessage pairing integrity - Context pressure monitoring: three-level monitoring (safe / warning / critical) that automatically triggers summary compression
- Smart context compression: auto-generates summaries of long quoted content so it doesn't permanently occupy the context
- Tool result caching: parallel tool results are cached automatically (up to 30 entries)
-
Parallel tool execution: tools marked as parallel in the same round run concurrently via
Promise.all -
Task decomposition:
plan_tasksupports sequential, parallel, and conditional execution strategies; failed subtasks support retry / rollback / continue -
Subtask dispatch:
dispatch_taskdelegates subtasks to other Agents for parallel execution - Streaming responses: OpenAI streaming (SSE) and DeepSeek thinking supported; replies render word by word in real time
- Clarification mechanism: shows a clarification dialog when information is incomplete, auto-pauses the loop timer, and supports recommended options
- Multi-level timeouts: API timeout / tool timeout / overall loop timeout are all independently configurable
- Cancellation control: the user can cancel the reasoning loop at any time, isolated per session
- SW restart recovery: a Keepalive port monitors Service Worker silent restarts and automatically notifies the Side Panel to recover
- Checkpoint: auto-saved after each reasoning round, supporting resume after interruption
| Level | Description | Trigger condition |
|---|---|---|
| Tool-level reflection | Quickly evaluates whether a tool result is useful after execution | Tool returns error / empty result / oversized result (>50000 chars) / 3 consecutive failures |
| Subtask reflection | Evaluates subtask result completeness and relevance | Only for subtasks marked as complex (configurable) |
| Post reflection | 7-dimension quality scoring of the final answer | Runs automatically after each reasoning round |
Post-reflection scoring dimensions: completeness, accuracy, relevance, tool usage, clarity, safety, efficiency. Based on the score threshold it automatically decides: pass (≥7), revise (5–7), or re-run (<5).
- Context usage indicator: shows the live usage percentage below the input box (ring progress + safe / warning / danger color tiers); click to see the four-way breakdown (system prompt, tool definitions, history messages, current input)
- Manual context compaction: one click compacts history messages into an AI-structured summary with four fixed sections (task goal / completed & key conclusions / key identifiers / open questions & next steps); the instruction language follows the UI locale (persisted per session, undoable until a new message is sent; re-compacting merges the old summary with new messages)
- Semi-automatic threshold compaction (off by default): once enabled in the memory dropdown, a silent compaction runs before sending when context usage reaches the threshold (85% by default) and at least 4 messages have accumulated since the last compaction; two consecutive failures back off, and no failure ever blocks sending
-
Summary sedimentation into long-term memory (on by default): a successful compaction stores the summary as cross-session long-term memory (
type: summary, tagscompaction+session:<id>); re-compacting the same session updates the samememoryIdinstead of piling up entries; when the Agent is not connected this is skipped silently and the summary stays usable in the current session - Adaptive token estimation: Chinese ~1.5 chars/token, English ~4 chars/token
- Automatic context window detection: inferred from the model name (custom mapping supported; 256K default)
- Message budget = context window − system prompt − tool definitions − output reserve
- Three-level context pressure monitoring: safe / warning / critical
- Message summarization: auto-summarizes early messages when pressure reaches critical
- Citation compression: long quotes / selected content auto-compressed into summaries
- Token-level truncation: 70% head + 30% tail + truncation marker
When the conversation context exceeds the budget, old tool-call rounds are automatically summarized:
- Round extraction: auto-detects complete tool-call rounds (assistant tool_calls + all tool results)
- LLM summary generation: calls a lightweight API to condense each tool-call round into a one-sentence summary
- Fallback strategy: if the summary request fails, falls back to deleting old messages directly
- Batch processing: multiple rounds can be processed in one token-overrun cycle
Visualization of post-reflection results:
- Overall score: 0–10, color-coded (green / yellow / red) + emoji
- 7-dimension radar: an individual progress bar per dimension
- Issues found: specific problems and improvement suggestions
- Evaluation process: rounds, decisions, and reasoning details
Create and manage multiple custom AI assistants, each with its own system prompt and tool permissions:
- Built-in templates: default assistant, code review expert, web automation assistant, data analyst, document writer
- Custom Agents: create dedicated assistants with icon, name, system prompt, model, temperature, and tool permissions
-
Agent selector: quick switch at the top of the side panel, plus
@AgentNamequick switching - Tool filtering: each assistant can have its own toolset to avoid context bloat
-
Subtask dispatch:
dispatch_tasksupports parallel dispatch; sub-Agents execute independently and return results -
Agent persistence: based on
chrome.storage.local, survives restarts
Two types of skills cover both deterministic automation and AI-driven invocation:
- Workflow Skill: deterministic workflows defined in JSON/YAML, with parameter validation and execution-result display
-
Agent Skill: AI capability extensions based on
SKILL.md, invoked autonomously by the AI in conversation -
Built-in skill: the
skill-creatormeta-skill can create new skills from a conversation - Quick selector: a "Skills" tab in the input dropdown for fast search and selection
- Import methods: JSON files, direct Markdown authoring, Zip packages (with resources), and URL downloads
-
Skill editor: visual editing of
SKILL.mdcontent with description, version, and resource management - Global switch: enable / disable all Skills with one click in the options page toolbox
Supports the Model Context Protocol (MCP) to dynamically extend third-party tool capabilities:
- Dynamic MCP tool injection: pulls the latest MCP tool list from the Agent before each reasoning round and injects it
- MCP Client: JSON-RPC 2.0 communication over stdio transport
- MCP Server management: visually configure, connect, and disconnect in the options page "Toolbox"
- Environment variable support: each MCP Server can have its own env vars, with password inputs for sensitive values
- Multiple Server support: connect several MCP Servers at once; tools merge automatically and are grouped by Server
- Global switch: enable / disable all MCP services with one click
- Quick selector: an "MCP" tab in the input dropdown to quickly pick a specific MCP service
AI Helper has long-term memory that stores and retrieves user information across sessions:
-
Memory storage: the AI auto-detects important information in conversation (preferences, knowledge, decisions) and proactively calls
agent_memoryto store it - Memory types: fact memories and conversation summaries
- Smart retrieval: search historical memories by keyword, tag, and type (CJK text split into bigrams, zero external dependencies)
-
Semantic vector recall (off by default): configure an embedding in the options page Memory tab and a vector channel joins the keyword pass, the two fused by rank; when disabled or unusable recall silently falls back to keywords. Derived vectors are cached in
memory-vectors.jsonbeside the memory file — droppable and rebuildable, and a config change requires pressing Rebuild vector index - Three injection tiers: memories with importance >= 8 stay resident in the system prompt; the rest are injected by value within a 1500-character / 12-item budget; whatever is left surfaces only as tag index lines for the model to recall on demand
-
Memory management:
reviewscores entries and lists candidates by value;compactprunes low-value ones (value = importance x access frequency x time decay) -
Visual management: the bottom of the options page Memory tab lets you browse, search, sort, edit and delete entries, and compact low-value candidates in one click; the list reads the read-only
review, so it never inflates recall statistics - Tag categories: preferences, knowledge, decisions, custom
- Importance scoring: 1–10, helping the memory system prioritize
-
Storage location: the memory file and all processing logic live inside the local Agent process (
~/.ai-helper-agent/memory/global-memory.json), so long-term memory requires a connected Agent -
Capacity warning: max 50 facts and 20 summaries; writing past 80% returns a reminder, and nothing is auto-deleted — cleanup needs an explicit
compact
- Tab switching: horizontal tab bar that auto-restores message history, model, tools, and temperature config
- Session creation: one-click create with auto-generated title (from the first user message)
- Rename / delete: via the right-click menu
- Archive / restore: up to 20 archives kept and restorable at any time
- Cross-session message delivery: background tasks append results to the original session when done
-
Persistent storage: based on IndexedDB, auto-migrated from legacy
chrome.storage.local -
Export / import: batch-select sessions and export as
.aihelper.json
- Batch export: select multiple sessions in a dialog, with select-all / invert / current-only options
-
Export format:
.aihelper.json, including full message history, execution logs, reflection scores, HTML content, and Agent config -
Import compatibility: supports the new
.aihelper.jsonformat and the legacy message-array format - Smart file names: single sessions use the session name; multiple sessions use count + timestamp
- Natural-language chat: OpenAI-compatible API, DeepSeek V4 Pro by default
- Model switching: built-in DeepSeek series, with custom model add/remove and context-window mapping
- Temperature control: 4 presets (0.2 / 0.45 / 0.65 / 0.9) plus continuous fine-tuning
- Memory limit: configurable number of history messages sent to the LLM
- Isolated chat: a "don't reference prior context" mode
- Text-selection Q&A: selecting text inside the side panel pops up a quick menu
-
Prompt system: custom prompt CRUD,
/quick selection, drag-to-reorder - Input history: up/down arrows to re-fill, with automatic deduplication
- System prompt: auto-injects environment info (Chrome extension, OS, Agent platform) and appends task-planning rules
- Question list navigation: hover the bottom message hot zone, expand the "question list" on the right, click to jump to past questions
Hovering an assistant message generates a floating navigation menu:
- Auto-extracts H1–H6 headings
- Floating panel with smooth scrolling to the target
- 1.5s highlight flash effect
Full-text message search built into the side panel for quickly locating past conversations:
- Dual-mode search: global search (loads all sessions from IndexedDB) and current-session search (fast in-memory query)
- Incremental loading: searches the current session first for instant results, then searches other sessions one by one, updating as it goes
-
Advanced syntax:
&(AND) and|(OR) combined search, with&taking precedence over| - Navigation: clicking a result switches to the target session, scrolls to position, with a 2-second highlight animation
- Search history: up/down keys navigate history (up to 20 entries)
- Term highlighting: matched words highlighted with 30 chars of context before and after
One-click bookmarking of important AI answers:
- One-click bookmark: click the bookmark icon in the message action bar
- Pin support: important bookmarks can be pinned
- Search & filter: AND/OR advanced search syntax
- Grouped display: grouped by current session and other sessions, with matched-term highlighting
- Quick locate: click a bookmark to jump to it
- Auto-cleanup: orphaned bookmarks are removed when the message or session no longer exists
| Action | Description |
|---|---|
| Copy message | Copies the raw Markdown content |
| Edit & resend | Fills the message back into the input box |
| Quote & follow-up | Sets the assistant message as quoted context (auto-compressed summary) |
| Export Word | Markdown converted to a standard .docx download (OOXML format) |
| Export PDF | Exported via the browser print window |
| Export image | Message rendered to an image download |
| Export JSON | Full conversation history (including execution logs) as JSON download |
- Full Markdown support: code blocks (with line numbers, copy button, language label), tables (exportable to Excel / copy as Markdown), quotes, lists
- Three-phase placeholder strategy: Mermaid → code blocks → tables are extracted as placeholders in order, then restored after rendering
-
Mermaid diagrams: flowcharts, sequence diagrams, Gantt charts, etc., with zoom (
Ctrl+wheel), drag-pan, PNG download, copy to clipboard, and source view - Inline Markdown in table cells: bold, italic, code, strikethrough
-
Inline preview: iframe
srcdocrendering with auto-detection of full documents vs fragments -
Zoom control: 0.25x–2.0x,
Ctrl+wheel/Ctrl+0to reset - Prototype library: saved to IndexedDB with open / edit / delete support
-
Export: download as an
.htmlfile or open in a new tab - Continue optimizing: one click fills the optimization instruction into the input box
- Live mode: updates in real time during reasoning with a pulse animation indicating execution status
- Static mode: shows the full execution timeline after completion
- Task-group visualization: complex tasks grouped by task group, collapsible / expandable
- Node details: tool name, status, duration, input params, output content, API request details
- Summary stats: total nodes, success / failure counts, subtask progress
- Status filter: filter by success / failure / subtask
Selecting text on any web page pops up a frosted-glass floating toolbar:
- AI search: opens the side panel and starts a search
- Quick actions: explain, translate, summarize
- Custom tools: add custom-prompt tools with drag-to-reorder
- Follow-up input: inline input box for direct follow-up questions
- Result panel: draggable, resizable floating panel with Markdown rendering and a lock to prevent closing
- Suggested follow-ups: auto-generated follow-up buttons in AI responses
- Config options: icon-compact mode, direct count display, domain blocking, temporary hide
Recursive penetration of Shadow DOM and same-origin iframes for element lookup and operation:
-
Recursive penetration:
deepQuerySelector/deepQuerySelectorAllcan pierce multiple Shadow DOM layers (max depth 5) - iframe support: automatically searches inside same-origin iframes
- Visibility detection: strict visibility checks (display / visibility / opacity / size)
-
React component support:
keyboard_inputbypasses React's synthetic event system;fill_formsupports rich-text editors like contenteditable / prosemirror
Typing @ in the input box opens a selector with a new "Web" tab to quickly pick a current browser tab as conversation context:
- Tab list: lists all open tabs in the current window with favicon, title, and URL
- Current-page marker: highlights the active tab
- Keyword filter: quick filtering by title / URL
- Context injection: selecting a page injects its content as conversation context
- Dynamic counts: tab titles show live option counts (e.g. "Assistants (5)", "Web (12)")
After connecting the Agent service, browse and manage the local file system directly in the side panel:
- Directory tree browsing: breadcrumb navigation, go up a level, click to expand, sort by name / size / modification time
- Multi-select operations: checkbox batch selection with select-all / deselect-all, batch download as ZIP, batch delete
- Drag-and-drop move: drag files / directories within the panel to a target path, including onto the breadcrumb for quick moves
-
Keyboard navigation:
↑↓navigate,Enteropen,Spaceselect,Deletedelete,F2rename,Esccancel - Directory cache: LRU cache of the 20 most recent directories (30s TTL)
- Agent-switch awareness: cache resets and refreshes automatically when the Agent connection state changes
- File-based Q&A: select a file to attach its path as context to the chat input with one click
- Upload: button selection and drag-and-drop upload, streaming HTTP upload (up to 3 concurrent), with progress panel and cancel
- Download: streaming download with progress bar, cancel support, auto-ZIP for directories
- Virtual scrolling: auto-enabled when there are more than 200 file items, rendering only the visible area
Click a file in the working directory to preview it instantly without leaving the side panel:
| Type | Preview capability |
|---|---|
| Text / code | Syntax-highlighted preview with line numbers (limit 1MB / 10000 lines) |
| Built-in PDF.js preview with zoom, page navigation, fit-to-page | |
| Word (.docx) | mammoth.js HTML preview (limit 20MB) |
| Excel (.xlsx) | Parsed server-side and rendered as a table, first 2000 rows |
| Images | Built-in preview with wheel zoom + drag pan (limit 50MB) |
- File details: click the info button to show permissions (rwx + octal), UID/GID, MIME type, and timestamps
Create scheduled tasks that automatically run preset instructions at specified times and write results into a bound host session:
-
Three scheduling rules: one-time (after N minutes / at a specific time), interval (e.g.
30m/2h/1d), and Cron expressions (5-field) - Web context: bind a specific web page as context, with "extract body" or "URL only" modes
- Host session: results land in a host session; if the host session is deleted, a dedicated session is self-healed on execution
- Scheduling engine: based on Chrome Alarms; alarms are rebuilt automatically after a Service Worker restart
- Management & control: the management panel supports create / edit / delete / enable / disable / run-now
- Run history: records status, duration, and errors for each run (up to 50 entries)
- Desktop notification: task completion / failure triggers a desktop alert per the "Scheduled task notification" switch (on by default — especially important since background runs have no other alert channel); click the notification to open the side panel
- End conditions: interval tasks support "max run count" and "deadline" for auto-disable
- Auto-save: a Checkpoint is saved to IndexedDB after each ReAct reasoning round
-
SW restart recovery: state auto-recovers via
chrome.storage.sessionafter a silent Service Worker restart - Manual resume: interrupted task cards show a "Continue" button to append a description and resume
- Streaming output preserved: resumed streaming output is fully saved to message history and survives page refresh
- TTL expiry: Checkpoints are cleaned up automatically after 7 days
- Multi-connection management: IndexedDB supports safe close and rebuild of the SW + Side Panel dual connection
- Real-time stats: token consumption updates after every API call
- Session stats: cumulative consumption and average per round for the current session
- Today's stats: cumulative consumption and call count for today
- History: token usage history for the last 7 days
Know the outcome even when tasks run in the background (managed in the options page "Basic" → "Completion feedback" section):
- Three independent switches: task completion / failure notifications, scheduled-task notifications, and interaction reminders (when the AI is waiting for confirmation or clarification) — toggle any time in the options page with immediate effect
- Task completion / failure notifications: shown only when the side panel is not visible (no interruption when visible); success notifications auto-dismiss, failures stay until manually closed; clicking a notification opens the side panel and jumps to the corresponding session
- Scheduled-task notifications: scheduled tasks run in the background with no other alert channel, so they always pop up when the switch is on (on by default)
- Interaction reminders: when the reasoning loop is waiting on a sensitive-operation confirmation or clarification, a desktop reminder pops up so a task never gets stuck unnoticed; no interruption while the side panel is visible
- Multi-instance dedup: tab-bound mode can open multiple side-panel instances; notification creation is centralized in the Background (globally unique) and visibility is judged against the instance that started the task, so duplicates never pop up
- Completion sound & confetti: distinct sounds for success / failure and a success confetti animation, each independently switchable
The entire UI is built on semantic color tokens and supports three theme modes (light by default); switching takes effect instantly and stays in sync across every surface (storage key appTheme):
-
Three modes: System (flips live with the OS
prefers-color-scheme) / Light / Dark - Two entry points: the segmented control in Options → Basic → "Appearance" to pick directly; and the "Theme: " item in the side panel's top "More" menu, which cycles Light → Dark → System
- Cross-page sync: switching in the side panel, options page, or selection toolbar propagates to the others instantly without a reload
- No first-paint flash: the resolved theme is mirrored locally and applied to the root element before scripts run, so dark-mode users never see a white flash on open
- Fully tokenized: body text / Markdown / tables / code blocks / blockquotes / Mermaid diagrams follow the theme; PDF / Word / HTML exports stay a white-background, black-text light paper regardless of dark mode
- Dark-mode quality gate: dark mode is verified by a per-overlay contrast scan (WCAG AA) and large white-blob detection, eliminating any "mismatched" leftover colors
Complements the "Reflection System": reflection is the model's subjective self-scoring of answer quality, whereas this feature uses real page / command evidence to objectively verify whether the sub-tasks decomposed by plan_task were actually met, instead of trusting the model's self-report (fully byte-for-byte unchanged when left off):
-
Criteria declaration: each sub-task may carry
doneCriteriaacceptance checks supportingelement_exists/element_absent/text_contains/attribute_equals/url_matches -
Objective evidence gathering: the content side penetrates multi-layer Shadow DOM and same-origin iframes to locate targets; a tri-state verdict
satisfied/unmet/indeterminatenever reports an unavailable evidence channel as unmet - Optional auto-rerun loop: four gates (retryable / progress / budget / under-max) drive unmet sub-tasks back through re-execution; the rerun halts the moment the missing set stops shrinking monotonically, preventing oscillation
-
Inline, non-blocking: evidence is gathered live inside each sub-task's
onDone(best effort) with a per-item timeout, without interrupting the main loop - Toggles: the "Completion Verification" switch and the "auto-rerun" sub-switch (shown only when the former is on) live in the Options → ReAct tab
Say it instead of typing it, with audio that never leaves the machine: the optional Agent runs an ONNX recognizer locally — a streaming model writes the draft as you speak, then a final-pass model adds punctuation and normalizes numbers (off by default; needs the dependency and a model pack first):
-
Three-part enablement — optional dependency (
sherpa-onnx-node) + model packs (online/offline/vad, any subset) + the switch. Until all three are ready the 🎤 button stays hidden instead of failing after a click - Two tempos — the streaming model drafts roughly every 150 ms; the offline SenseVoice pass recomputes the whole take for punctuation and number normalization
- Taking over never loses text — once you edit the input by hand, later drafts only append what is genuinely new; a hypothesis that would rewrite rather than extend your text is skipped entirely
- Finalizing never sends — the result only fills the input; sending stays your call, and adding a sentence mid-generation does not turn Send into Stop
-
A recorder strip that holds up — 24 volume bars, timer, silence hint,
Escto cancel,Alt+Vthat also works while the focus is on the page, and Enter-to-send that stops and finalizes first - Bounded resources — 5-minute take limit, one concurrent session, engines unloaded after 10 idle minutes; the final pass runs on a worker thread so the Agent stays responsive, and near the RSS threshold it falls back to the draft
Full setup, model-pack layout and API → Voice Input (Local STT)
Agent-side file deletions default to soft delete — moved to the recycle bin instead of being physically removed:
-
Soft-delete protection: deleted files are auto-moved to
~/.ai-helper-agent/.trash/with the original path and time recorded - 7-day retention: expired files are auto-cleaned (opportunistic cleanup before operations + periodic cleanup every 6 hours)
-
One-click restore: restore to the original path via the
agent_trashtool -
API integration:
/api/fs/deleteuses the recycle bin by default;/api/trash/restoreand/api/trash/listprovide the interfaces
-
Online restart:
/api/agent/restartis a two-phase restart (spawn helper → wait for the old process to exit → start the new process) -
Auto-update:
/api/agent/updaterunsnpm install -g ai-helper-agent@latestand restarts automatically (90s timeout protection) - Update cooldown: repeated restart / update within 10 seconds is blocked to prevent accidental concurrency
- Failure fallback: a failed update does not roll back the old process; it returns manual-install instructions
- See all available tools → Built-in Tools Reference
- Learn about the local Agent and knowledge base → Agent Service
- Tune reasoning / reflection / chat parameters → Configuration
🌐 English Wiki: Home · Quick Start · Showcase · Core Features · Built-in Tools · Architecture · Agent Service · Configuration · FAQ
© AI Helper · GitHub · Discussions · Issues · MIT License
- 🏠 首页 Home
- 🚀 快速开始 Quick Start
- 🎬 实战案例 Showcase
- ✨ 核心功能 Core Features
- 🎙️ 语音输入 Voice Input
- 🧰 内建工具 Built-in Tools
- 🏗️ 架构与技术栈 Architecture
- 🤖 代理服务 Agent Service
- ⚙️ 配置参考 Configuration
- ❓ 常见问题 FAQ
English Wiki: Home · Quick Start · Showcase · Core Features · Voice Input · Built-in Tools · Architecture · Agent Service · Configuration · FAQ