-
Notifications
You must be signed in to change notification settings - Fork 0
AI Assistant and MCP
The AI Assistant is a chat interface with tool-calling access to your fleet — it can look up inventory, check guest/node status, and take actions you approve, using whichever model/provider is configured. It can use any OpenAI-chat-completions-compatible provider: OpenAI, Ollama, LM Studio, LocalAI, OpenRouter, or anything else speaking that API shape, configured under Settings > AI Providers. A provider can expose multiple models (e.g. one OpenAI provider with both gpt-4o-mini and gpt-4o), each with a friendly display name distinct from the exact model ID sent over the wire.
There's also an optional zero-config, no-API-key, fully local option backed by Needle 2 — a small (45M-parameter) tool-calling model that runs as a self-contained CLI subprocess, with no GPU and no network access required at inference time.
Needle 2 is Apache-2.0 licensed, so on Windows, Linux, and macOS (amd64 or arm64) Ferrum ships its official CLI binary baked into the ferrum binary itself — nothing to download, nothing to configure. On a fresh install (no AI provider configured yet), Ferrum extracts it to a cache file and registers it automatically as the default assistant the first time it starts. If you've already configured a provider, or want to add/re-add it yourself, use Settings > AI Providers > "Add provider" > the Needle 2 (built-in, local) preset.
On any other platform (32-bit, RISC-V, Windows/ARM64, ...) there's no bundled binary — Ferrum still never fetches executable content from the network on its own. To enable it there:
- Download the
needleCLI binary for your platform from the Needle 2 files. - Point Ferrum at it: set
FERRUM_NEEDLE_BIN=/path/to/needle(orneedleBinPathinconfig.yaml) before starting Ferrum. This also overrides the bundled binary on a supported platform, if you'd rather run a different build.
Ferrum starts the binary itself (as a local subprocess, 127.0.0.1-only) the first time it's used, and stops it on shutdown. If no binary is bundled for the platform and FERRUM_NEEDLE_BIN isn't set (or doesn't exist), this provider simply isn't usable — every other configured provider is unaffected.
Needle does a one-time "tool retrieval" pass over Ferrum's full tool catalog (~60 tools) the first time it's used, which briefly uses noticeably more memory and CPU than steady-state chat — Ferrum already waits up to 60s for that first startup. On a board with very little RAM this pass can get the subprocess killed by the kernel's OOM-killer instead; the error says needle process exited: signal: killed. If you hit that, set FERRUM_NEEDLE_MAX_TOOLS=<n> to cap how many tools Needle is given (e.g. 10 keeps just the original read-only lookup tools), trading away the newer create/manage tools for a much lighter embedding pass. If it still fails, the board likely doesn't have enough RAM to run even this small a model reliably — use a real OpenAI-compatible provider instead (a cheap/free hosted API, or Ollama on a separate, beefier machine).
Ferrum can also act as an MCP server, so any MCP-capable agent — Claude Code, Claude Desktop, or anything else speaking the protocol — can drive Ferrum's tool catalog directly, outside the built-in chat UI.
- An admin turns on the MCP endpoint under Settings > API & MCP (off by default), which also caps how many tool calls an agent loop can make per message.
- Any user generates an MCP-scoped API key under Profile > API Keys.
-
Profile > MCP integration has ready-to-paste config for Claude Code/Desktop pointing at
/mcpwith that key.
The general REST API (Authorization: Bearer <key> against /api/v1/...) is a separate, independently-toggleable surface for 3rd-party integrations and scripts — see Settings > API & MCP and Profile > API Keys.
Every mutating action — through the UI, the REST API, or MCP — is recorded with who, what, and when under Audit Log (admin-only), so AI-assistant and MCP-driven actions are traceable exactly like manual ones.