Architecture EN

🌐 Language / 语言: 中文 · English

Architecture Overview

This page introduces AI Helper's overall architecture, core data flow, project structure, tech stack, and state-management design. It is intended for readers who want to understand the project deeply or do secondary development.

← Back to Wiki Home · Related: Core Features · Agent Service

---

Five-Layer Architecture

The project uses a five-layer architecture; each layer communicates through the Chrome Extension API message channel:

┌──────────────────────────────────────────────────────────────┐
│                   Side Panel (UI Layer)                      │
│  side_panel.html + src/side_panel/*.js                       │
│  Chat management | Multi-session tabs | Markdown/Mermaid     │
│  rendering | Tool panel | Prompt management | Selection Q&A   │
│  Input history | Execution log | Clarify/confirm dialogs     │
│  UI prototype preview | Quality evaluation display | TOC      │
│  Session archiving | Multi-assistant management | Token      │
│  stats panel | Agent selector | @ Agent/page switcher       │
│  Image recognition input | Image annotation editor | File     │
│  upload extraction | Session export/import | Skill selector  │
│  (Skill Tab) | MCP service selector (MCP Tab) | Chat export  │
│  (Word/PDF) | Breakpoint resume | Message copy | Workspace   │
│  directory management | File preview | Message search |      │
│  Message favorites                                          │
└──────────────┬──────────────────────────────┬────────────────┘
               │  chrome.runtime.sendMessage   │
               ▼                               ▼
┌──────────────────────────┐    ┌──────────────────────────────┐
│   Background Service      │    │     Options Page (Config)    │
│   Worker (Core logic)     │    │  options.html + src/options/  │
│                          │    │  API Key/Model/Tools/ReAct    │
│  src/background/          │    │  params | Reflection system  │
│  ├── index.js (msg route) │    │  Chat config | Toolbar       │
│  ├── react-loop.js (ReAct)│    │  Agent pairing management   │
│  ├── react-reflection.js  │    │  Toolbox (MCP + Skill mgmt)  │
│  ├── tool-executor.js     │    └──────────────────────────────┘
│  ├── tool-preselector.js  │
│  ├── local-agent-client.js│    ┌──────────────────────────────┐
│  ├── config.js            │    │    Agent Service (optional)  │
│  ├── state.js             │    │  agent/ (Node.js process)    │
│  ├── agent-dispatcher.js  │    │  HTTP REST + WebSocket       │
│  ├── stream-controller.js │    │  File I/O | Command exec |   │
│  └── token-recorder.js    │    │  Search | Skill system | MCP │
│                          │    │  protocol | Path sandbox |   │
│                          │    │  Security tiers | File upload│
│                          │    │  API                        │
└──────────────┬─────────────┘    └──────────────────────────────┘
               │  chrome.tabs.sendMessage
               ▼
┌──────────────────────────────────────────────────────────────┐
│              Content Script (Page tool execution)            │
│  src/content/*.js (injected into user web pages)             │
│  ├── index.js (message routing, page tools)                  │
│  ├── page-tools.js (page content extraction, a11y tree,      │
│  │                Markdown conversion)                       │
│  ├── page-interaction.js (interactive element query)         │
│  ├── interaction-tools.js (interaction ops, TTS, color       │
│  │                         picker)                           │
│  ├── advanced-tools.js (video control, performance audit,    │
│  │                      Shadow DOM, screenshots)             │
│  ├── shadow-dom-utils.js (Shadow DOM recursive penetration   │
│  │                        + iframe)                          │
│  └── selection-toolbar.js (floating selection toolbar)       │
└──────────────────────────────────────────────────────────────┘
               │
               ▼
┌──────────────────────────────────────────────────────────────┐
│              Offscreen Document (auxiliary layer)            │
│  src/offscreen/ (clipboard operation support)                │
│  └── offscreen.js (MV3-compatible copy/paste_from_clipboard) │
└──────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────┐
│                   Storage (persistence layer)                 │
│  src/storage/                                                 │
│  ├── db.js (IndexedDB wrapper: transaction retry, auto-       │
│  │          migration, v6 multi-connection management)       │
│  ├── session-store.js (session storage adapter)              │
│  └── token-store.js (token stats storage)                    │
└──────────────────────────────────────────────────────────────┘

---

Core Data Flow

User input → Side Panel (select Agent, optional image/file attachments,
  optional Skill/MCP, optional @page context)
  → chrome.runtime.sendMessage('CALL_API')
    → Background: MCP tool injection → tool pre-selection → ReAct reasoning loop
      → token budget management → context pressure monitoring → token stats
      → LLM API call (OpenAI-compatible, retry + exponential backoff,
        streaming + DeepSeek thinking)
        → If tools needed: tool confirmation check (sensitive ops) → execute tool
          ├── Background executes directly (tab management, bookmark search, etc.)
          ├── Delegate to Content Script (page interaction, content extraction)
          ├── Delegate to local Agent (file I/O, command execution, MCP tools)
          └── Offscreen document (clipboard read/write)
        → tool observation hint → result caching → feed back to LLM
      → subtask decomposition & parallel execution (plan_task integration,
        supports Agent subtask dispatch)
      → Checkpoint save → post-reflection: multi-dimension quality evaluation
        → pass / revise / retry
    → chrome.runtime.sendMessage('API_COMPLETE')
    → Task result reported → visibility check → desktop notification
      (popped when the initiating side panel is not visible)
  → Side Panel: Markdown rendering, Mermaid rendering, quality evaluation
    display, token stats update

Voice input path (side channel, outside the reasoning loop)

Side panel 🎤 → getUserMedia → AudioWorklet (voice-capture-worklet.js) → 16kHz PCM chunks
  → chrome.runtime.connect({name:'asr'}) port
    → Background: opens WebSocket /ws/asr/:sessionId with the Bearer token
      (the side panel never touches the token)
      → Agent asr/session.js: uplink start / audio / stop / cancel
        downlink ready / partial (streaming draft, throttled to 150ms) /
        endpoint / final / error
        → stop finalize: asr/offline-pool.js runs ONNX inference on a worker
          thread (keeps the Agent event loop free; the session slot is released
          first and the final frame is sent when the worker returns)
  → Side Panel: the draft is written into the input box, never sent automatically

The recorder bar (including its gating of dependencies × models × switch) and resampling fallbacks live in voice-input.js; this path does not go through CALL_API, so it triggers no tool calls and no reflection.

---

Project Structure

ai-helper/
├── agent/                               # Agent service (Node.js process)
│   ├── bin/agent.js                     # CLI startup script
│   ├── src/
│   │   ├── server.js                    # HTTP + WebSocket server
│   │   ├── executor.js                  # Command execution engine (stream/block)
│   │   ├── security.js                  # Path sandbox + command security tiers
│   │   ├── config.js                    # Agent config (disk persistence)
│   │   ├── auth.js                      # Pairing auth (6-digit dynamic code)
│   │   ├── search.js                    # File/content search (fd/rg accelerated)
│   │   ├── logger.js                    # Structured logging
│   │   ├── trash.js                     # File recycle bin (soft delete + 7-day cleanup)
│   │   ├── skill/                       # Skill system
│   │   ├── mcp/                         # MCP protocol support
│   │   └── asr/                         # Speech recognition (local STT)
│   │       ├── routes.js                # /api/asr/* HTTP endpoints
│   │       ├── session.js               # WebSocket session (/ws/asr/:sessionId)
│   │       ├── engine.js / offline-pool.js / offline-worker.js  # streaming + finalize inference (worker thread)
│   │       ├── model-pack.js / model-catalog.js / model-download.js  # model pack validation and download
│   │       └── detect.js / install.js / config.js  # dependency detection, install, config
│   └── package.json
├── icons/                               # Extension icons
├── libs/                                # Third-party deps (CDN/local)
│   ├── marked.min.js                    # Markdown rendering engine
│   ├── mermaid.min.js                   # Mermaid diagram rendering engine
│   ├── qrcode.min.js                    # QR code generation library
│   ├── pdf.worker.min.js                # PDF.js Worker
│   ├── html2canvas.min.js               # HTML Canvas screenshot library
│   ├── jspdf.min.js                     # jsPDF PDF generation library
│   └── github-markdown-light.min.css    # GitHub-style Markdown styles
├── scripts/                             # Build tooling scripts
│   ├── fix-build.js                     # Fixes @crxjs/vite-plugin build output
│   ├── silent-build.js                  # Silent build (CI-friendly)
│   ├── generate-icons.js                # Icon generation script
│   ├── deploy-pages.sh                  # Pages deployment script
│   └── split-tools.cjs                  # Tool definition splitting script
├── src/
│   ├── background/                      # Background Service Worker (core logic)
│   │   ├── index.js                     # Entry: message routing, session mgmt, Agent health
│   │   ├── react-loop.js                # ReAct reasoning loop (core engine, 3-tier reflection)
│   │   ├── react-reflection.js          # Three-tier reflection system
│   │   ├── context-summarizer.js        # Incremental context summarization
│   │   ├── context-compactor.js         # Manual compaction summary generation
│   │   ├── tool-executor.js             # Tool registration, execution scheduling, MCP injection
│   │   ├── tool-preselector.js          # Tool pre-selection
│   │   ├── local-agent-client.js        # Local Agent HTTP/WebSocket communication
│   │   ├── agent-dispatcher.js          # Agent subtask dispatcher
│   │   ├── stream-controller.js         # Streaming response controller (SSE + DeepSeek thinking)
│   │   ├── token-recorder.js            # Token usage stats recorder
│   │   ├── scheduler.js                 # Scheduled-task scheduling engine
│   │   ├── notifier.js                  # Desktop notification hub (task / schedule / interaction)
│   │   ├── panel-visibility.js          # Initiating-panel visibility check (notification gating)
│   │   ├── config.js / state.js         # Config read/write / multi-session cancel control
│   │   └── tools/                       # Tool definitions by directory
│   ├── content/                         # Page-injected scripts (page tool execution layer)
│   │   ├── index.js                     # Entry: message routing dispatch
│   │   ├── page-tools.js / page-extract.js
│   │   ├── page-interaction.js / interaction-tools.js
│   │   ├── advanced-tools.js            # Video, performance audit, Shadow DOM, screenshots
│   │   ├── shadow-dom-utils.js          # Shadow DOM recursive penetration + same-origin iframe
│   │   └── selection-toolbar.js         # Floating selection toolbar
│   ├── offscreen/                       # Offscreen document (clipboard operations)
│   ├── side_panel/                      # Side panel UI (UI layer)
│   │   ├── index.js                     # Entry: event binding, config, shortcuts
│   │   ├── chat-manager.js / chat-streaming.js
│   │   ├── chat-panels.js / chat-resume.js / chat-export.js / chat-copy.js
│   │   ├── markdown-render.js / tool-panel.js / prompt-manager.js
│   │   ├── agent-manager.js / agent-store.js / agent-at-selector.js
│   │   ├── page-selector.js / token-stats-panel.js
│   │   ├── session-manager.js / session-manager-ui.js
│   │   ├── clarify-dialog.js / confirm-dialog.js / interaction-gate.js / ui-prototype.js
│   │   ├── message-toc.js / input-history.js / image-preview.js
│   │   ├── file-extract.js / skill-selector.js / export-import.js
│   │   ├── execution-log-render.js / workspace-manager.js / workspace-panel.js
│   │   ├── voice-input.js / voice-capture-worklet.js  # voice input: recording, WS streaming, draft insertion
│   │   ├── bookmark-manager.js / bookmark-panel.js / search-panel.js
│   │   ├── schedule-panel.js / context-indicator.js / completion-feedback.js
│   │   └── state.js                     # Global state management (Proxy dual-export mode)
│   ├── options/                         # Extension options page (config layer)
│   │   ├── index.js / config-manager.js / config-io.js
│   │   ├── toolbar-config.js / toolbox-config.js / toolbox-shared.js
│   │   ├── toolbox-mcp.js / toolbox-skills.js / toolbox-rag.js
│   │   ├── knowledge-panel.js / save-bar.js / constants.js
│   ├── storage/                         # IndexedDB persistence layer
│   │   ├── db.js / session-store.js / token-store.js
│   ├── config/constants.js              # Storage keys, message types, etc.
│   └── shared/                          # Shared modules
│       ├── tools.js                     # Tool categories, temperature presets
│       ├── utils.js / token-counter.js / logger.js
│       └── agent-defaults.js            # Built-in Agent definitions and templates
├── manifest.json                        # Chrome extension config (Manifest V3)
├── side_panel.html                      # Side panel HTML
├── options.html                         # Options page HTML
├── vite.config.js / package.json
└── README.md

---

Tech Stack

Technology	Description
Vite + @crxjs/vite-plugin	Build toolchain, ES Modules, dev HMR
Manifest V3	Latest Chrome extension protocol
Service Worker	Background process for API calls and tool execution
Side Panel API	Chrome 114+ side panel
Content Script	Page injection, DOM operations
Offscreen Document	MV3 clipboard operation compatibility layer
IndexedDB	Session / prototype / token stats / bookmarks / scheduled tasks persistence
chrome.storage.local	Config storage, Agent definition storage
chrome.storage.session	Cross-restart message recovery, background task persistence
chrome.debugger API	CDP screenshots / PDF export / advanced page debugging
OpenAI Compatible API	LLM calls (incl. Vision), default DeepSeek V4, streaming support
marked.js	Markdown rendering engine
mermaid.js	Diagram rendering engine
QRCode.js	QR code generation (Canvas fallback)
pdfjs-dist	PDF text extraction
mammoth.js	Word .docx text extraction
SheetJS (xlsx)	Excel .xlsx/.xls text extraction
Web Speech API	Text-to-speech
EyeDropper API	Color picker
Navigation / Performance API	Performance audit
Node.js (Agent)	Local file / command service, Skill system, MCP protocol, local knowledge base (RAG)
WebSocket (Agent)	Real-time command output stream
MCP Protocol	Model Context Protocol, third-party tool extension

---

State Management Design

Dual-export Proxy pattern

src/side_panel/state.js uses a distinctive dual-export Proxy pattern:

// Both import styles point to the same data
import state from './state.js';               // state.messageHistory
import { messageHistory } from './state.js'; // direct destructuring

Through Proxy getters/setters delegated to top-level let bindings, all modules share the same state without introducing any framework. It supports 80+ state fields, including image input, file attachments, skill selection, MCP service selection, and more.

Data persistence strategy

Data type	Storage location	Description
Extension config	chrome.storage.local	API Key, ReAct params, models, image recognition, etc.
Chat sessions	IndexedDB (ai-helper-db)	Active sessions + archived sessions
UI prototypes	IndexedDB	Linked per session
Toolbar config	chrome.storage.local	Tool list, ordering, domain blocking
Input history	chrome.storage.local	Deduplicated, auto-managed
Cross-restart messages	chrome.storage.session	SW restart recovery, background task persistence

IndexedDB database design

ai-helper-db (v6), eight object stores:

Store	Purpose
sessions	Active sessions (index: updatedAt)
activeSession	Current active session ID
archivedSessions	Archived sessions (index: createdAt)
uiPrototypes	UI prototypes (index: createdAt, sessionId)
tokenStats	Token usage stats (index: timestamp, sessionId)
reactCheckpoints	ReAct checkpoints (index: updatedAt; 7-day TTL auto-expiry)
bookmarks	Message bookmarks (index: sessionId, createdAt)
scheduledTasks	Scheduled tasks (index: nextRunAt, enabled)

Supports automatic recovery from transaction failures and auto-migration from the legacy chrome.storage.local format.

---

Design Highlights

• Framework-free UI: both the side panel and options page use native JS + Proxy state management — small bundle, fast startup.
• MV3 compatibility: the Offscreen Document solves the unavailability of the clipboard API under MV3; a Keepalive port monitors Service Worker silent restarts and auto-recovers.
• Robustness: tool execution and API calls all have timeouts, retries, and cancellation control; background task state persists to chrome.storage.session.
• Extensibility: tool definitions are organized by directory, MCP tools are dynamically injected, and the Skill system supports hot-plugging.

---