Skip to content

Local LLM Bonsai 27B Guide

Jump to bottom
Svwergio_mel edited this page Nov 13, 2026 · 3 revisions

Local LLM via LangChain (optional use case)

Filename note: this page keeps the historical slug Local-LLM-Bonsai-27B-Guide so older links work. Bonsai-27B was one maintainer example of a local OpenAI-compatible stack; the same pattern applies to LangChain MCP, any /v1 local server, or cloud MCP clients.

Product scope: Chat Export is a standalone stack — Chrome MV3 extension + chat-export-server (+ optional dashboard). It automates your logged-in provider tabs via the extension bridge. No Chrome DevTools MCP (CDT) and no vendored agent code. Everything below is an optional example of wiring an external MCP client (Cursor, Claude Desktop, LangChain, or a local OpenAI-compatible runtime) to http://127.0.0.1:8765/mcp.

Example: “local brain + web council”

A fast private local model (any OpenAI-compatible /v1 server — LangChain agents are the usual glue, or an optional stack such as Bonsai-27B on oMLX) can act as orchestrator while Chat Export handles browser-side list/export/ask on twelve web UIs (eleven via MCP; JanitorAI is extension-only). You bring your own agent runtime; Chat Export does not ship one.

Typical benefits of this pattern:

  1. No per-token vendor API for web consultations — prompts go through sessions already open in Chrome.
  2. Local synthesis — keep raw repo context on the local model; send distilled sub-questions to the web council; persist exports under ~/.chat-export/exports/.
  3. Multi-provider flows — use chat_ask, jobs_enqueue, and jobs_status instead of manual tab switching.

Architecture (example)

flowchart LR
  subgraph Local["Your machine"]
    LLM["Local LLM\n(OpenAI-compatible :8000/v1)"]
    Client["MCP client\n(Cursor, Claude, LangChain)"]
    Server["chat-export-server\n127.0.0.1:8765/mcp"]
  end

  subgraph Browser["Chrome (your session)"]
    Ext["Chat Export extension\nWebSocket ceb_"]
    Web["8 provider web UIs"]
  end

  LLM <-->|"Your agent runtime"| Client
  Client <-->|"Streamable HTTP MCP\nBearer cem_..."| Server
  Server <-->|"WebSocket RPC"| Ext
  Ext <-->|"In-page automation"| Web
Loading

Connect any MCP client

Cursor (example)

{
  "mcpServers": {
    "chat-export": {
      "url": "http://127.0.0.1:8765/mcp",
      "headers": {
        "Authorization": "Bearer cem_YOUR_MCP_TOKEN"
      }
    }
  }
}

Claude Desktop (example)

Add the same Streamable HTTP URL and Authorization: Bearer cem_… in Claude Desktop MCP settings (or your Claude extension manifest when distributing). Chat Export does not ship Claude config; you point Claude at http://127.0.0.1:8765/mcp after chat-export-server is running.

LangChain / local agent (example)

Register Chat Export as an MCP tool source in your LangChain (or LangGraph) agent config using the same URL and token. The local model’s OpenAI-compatible base URL stays in your agent repo — Chat Export only exposes browser automation on localhost.

Generic Streamable HTTP MCP

Point your client at the same URL. Example env-style variables for a local agent you maintain separately:

export CHAT_EXPORT_MCP_URL="http://127.0.0.1:8765/mcp"
export CHAT_EXPORT_MCP_TOKEN="cem_YOUR_MCP_TOKEN"
{
  "chat-export": {
    "transport": "streamable_http",
    "url": "http://127.0.0.1:8765/mcp",
    "headers": {
      "Authorization": "Bearer ${CHAT_EXPORT_MCP_TOKEN}"
    },
    "timeout_s": 180
  }
}

If your local model uses OpenAI-compatible tool calling, configure its base URL and model id in your agent — not inside Chat Export repos.

Suggested MCP tool subset for smaller local models

  • browsers_status — confirm Chrome bridge is online.
  • chats_list, chat_read / chats_get, chats_export
  • chat_ask — ask on a provider (claude, grok, chatgpt, etc.)
  • jobs_enqueue, jobs_status, jobs_list, jobs_cancel — non-blocking lanes

Set timeout_s=180 and extension rpcTimeoutS=180 for long-running provider replies, or use wait: false + job polling.

Example council workflow

  1. browsers_status — verify bridge.
  2. Enqueue or call chat_ask on two or three providers (e.g. Claude + Perplexity + ChatGPT).
  3. Poll jobs_status until done; chats_export each thread to Markdown.
  4. Let your local model read exports from disk and write a summary (Obsidian or notes — via your tools, not bundled in Chat Export).

Related

Clone this wiki locally