Skip to content

Local AI Engine

Jump to bottom
DIIZZYFPS edited this page Mar 5, 2026 · 1 revision

Local AI Engine

sAIve distinguishes itself from other AI-powered finance trackers by running its Large Language Model (LLM) entirely on your local machine. No financial data is ever sent to OpenAI, Anthropic, or any remote server for inference.

How It Works

  1. The Model: By default, sAIve uses a highly compressed, INT4-quantized version of Google's Gemma 3 (270M parameters). Users can also select Llama 3.2 1B or Llama-3.2-1B-Instruct from the Settings menu.
  2. The Inference Engine: We use Hugging Face Transformers.js, which downloads the model weights directly from the HF Hub into the browser's persistent Cache Storage.
  3. Hardware Acceleration: Transformers.js leverages WebGPU. This allows the browser to utilize the user's dedicated or integrated graphics card to perform tensor math, resulting in highly performant generation speeds directly inside Electron.
  4. The Web Worker: To prevent the React UI from freezing while the model generates text, the entire generator loop runs inside a background Web Worker (ai.worker.ts). The worker streams text tokens back to the main thread piecemeal.

AI Features in sAIve

1. Smart Categorization

When you enter a new transaction, but before you hit save, the suggestCategory function fires. It sends the transaction recipient/description to the AI engine with a strict system prompt instructing it to return only one word from the accepted category whitelist (e.g., Food, Transportation, Subscriptions).

2. Conversational Finance (Agentic Chat)

The AiChat component allows you to converse with your data.

Context Window Injection: LLMs have no inherent memory. To give the AI knowledge of your finances, the React frontend silently injects your recent transaction history and current net worth into the System Prompt immediately before appending your typed message.

// Example of the prompt sent to the LLM backend
const messages = [
    { 
        role: "system", 
        content: `You are sAIve, a helpful personal finance assistant. 
                  The user's current net worth is $25,432.10. 
                  Here are their last 50 transactions: ...` 
    },
    { role: "user", content: "How much did I spend on food this month?" }
]

3. FastMCP (Advanced Data Tools)

Rather than relying purely on prompt-injected JSON (which uses up valuable context window tokens), sAIve integrates a native Model Context Protocol (MCP) server on the backend.

The AI is aware of specific tool functions (get_net_worth, log_transaction, get_expense_categories). The standard chat frontend doesn't directly run these yet, but the architecture allows an advanced agentic loop to recognize user intent (e.g., "Log $50 for groceries") and securely trigger the log_transaction execution tool on the backend without manual UI entry.

Model Selection

You can change the active AI model in the Settings page. Note that larger models require more RAM/VRAM and will take longer to download on first run.

Model Size Requirements Good For
Gemma 3 (270M) Int4 ~300MB Low (Any modern laptop) Quick categorization, basic chat, fast load times.
Gemma 1B / Llama 1B ~1-2GB Medium (Dedicated GPU recommended) Complex reasoning, conversational nuance.

Troubleshooting WebGPU

If the AI engine crashes or fails to load, it is usually because the underlying system lacks WebGPU support, or the Electron wrapper hasn't been granted hardware acceleration flags. Ensure you are running a recent version of Chrome/Edge natively, or that the Electron distribution is correctly built with --enable-features=Vulkan,UseSkiaRenderer.

Clone this wiki locally