Myth — ReAct Agent Pipeline for Local LLMs

A dual-mode agent pipeline that automatically selects the best execution strategy for your model. Built for 10–16 GB RAM systems running Ollama models from 1.5B to 8B parameters.

---

Architecture

Myth uses a probe-first strategy: every request starts by testing whether the model supports native tool calling via the OpenAI-compatible /v1/chat/completions endpoint. The response determines which pipeline runs.

User Message
     │
     ▼
┌──────────────────┐
│  PROBE: /v1/     │──▶ 404 or error?
│  chat/completions │      │
│  + tools array    │      ▼
└────────┬─────────┘   LEGACY PIPELINE
         │             (Route → Extract → Execute → Synthesize)
    tool_calls?            │
    ┌──┴──┐                 │
   YES    NO                │
    │     │                 │
    ▼     ▼                 │
┌────────┐  Direct answer  │
│  REACT │  (streamed)      │
│  LOOP  │                  │
│ max 5  │                  │
│ iters  │                  │
└────────┘                  │
     │                      │
     ▼                      ▼
   Final Response

ReAct Mode (Native Tool Calling)

For models that support the OpenAI tools array (qwen2.5, llama3.2, mistral, hermes3, etc.):

1. Single call with all 11 tools in the tools parameter
2. Model decides: call a tool, or answer directly
3. If tool called: execute → feed result back → model decides again (loop)
4. Max 5 iterations, then forced synthesis
5. Final answer is streamed to the UI

Why this is better: The model handles routing and argument extraction in a single call. No separate routing step, no regex JSON parsing, no prompt bleed from schema examples.

Legacy Mode (4-Step Fallback)

For models that don't support native tool calling (base code models, very small SLMs):

ROUTE → EXTRACT → EXECUTE → SYNTHESIZE

• Keyword fallback when the model fails to route
• 3-attempt extraction cascade with correction prompts
• Anti-hallucination synthesis prompts

Automatically activated when /v1/ returns 404 or the model never emits tool_calls.

---

11 Real Tools

Tool	Input	What It Does	Context Protection
web_search	{ query }	Real web search via DuckDuckGo	Top 8 results
fetch_url	{ url }	Fetches a web page, strips HTML	Truncated to 4000 chars
get_weather	{ location }	Real-time weather via wttr.in	Compact JSON
execute_code	{ code, language? }	Runs code in subprocess (15s timeout)	3000 char output cap
calculate_math	{ expression }	Evaluates math (sqrt, sin, cos, log, pi, e, ^)	Single number result
read_file	{ path }	Reads files from project directory	Truncated to 8000 chars
list_directory	{ path? }	Lists files/folders in a directory	Max 100 entries
search_codebase	{ pattern, directory? }	Ripgrep search, top 15 results	File + line + 120 char preview
read_file_chunk	{ path, start_line, end_line }	Reads specific line range (max 100 lines)	Refuses full file reads
query_pdf	{ path, query }	PDF RAG: semantic search (top 3)	500-char chunks, keyword fallback
fetch_url_markdown	{ url }	Clean markdown via Readability + Turndown	Hard 3000 char truncation

---

SLM-Hardening Defenses

Defense	Problem	Solution
Native tool calling	Regex JSON parsing fails on SLMs	Model outputs structured tool_calls natively
z.coerce.number()	SLM outputs "45" instead of 45	Zod auto-converts strings to numbers
Abstract schema descriptions	SLM copies schema examples verbatim	No concrete examples in .describe() fields
Schema guardrail	Even with native calling, placeholder bleed can occur	Post-parse check replaces description-matching args with user message
Anti-hallucination synthesis	SLM fabricates data relationships	"ZERO INFERENCE" rule in synthesis prompt
Type coercion	SLM can't distinguish JSON types	z.coerce at the validation layer
Legacy fallback	Old Ollama versions lack /v1/	Automatic degradation to 4-step pipeline

Schema Guardrail

A deterministic, zero-LLM-call defense against prompt bleed. After parsing tool arguments:

1. Check if any arg value exactly matches the parameter description → replace with raw user message
2. Check if ≥3 words from the description appear in the arg value → replace with user message

This is especially effective for single-field tools like web_search where the model might output {"query": "Search query string"}.

---

Prerequisites

• Node.js 18+ and Ollama
• ripgrep — sudo apt install ripgrep

curl -fsSL https://ollama.com/install.sh | sh

# For ReAct mode (native tool calling):
ollama pull qwen2.5:latest

# For legacy mode (prompt-based):
ollama pull qwen2.5-coder:1.5b

---

Quick Start

npm install
npm run dev

Open http://localhost:3000. Enter your Ollama URL, connect, pick a model, and type.

Model Recommendations

Model	Mode	Notes
qwen2.5:latest	ReAct	Best balance of speed + tool calling
llama3.2:latest	ReAct	Strong tool calling, slightly slower
qwen2.5-coder:1.5b	Legacy	Fast but no native tool calling
qwen2.5-coder:7b	Either	Can work in both modes

Quick prompts to try

• "What's the weather in Tokyo?"
• "Calculate 2^10 + sqrt(144)"
• "Search the web for latest AI news"
• "Find the auth function and explain how it works"
• "Search codebase for API routes, then read the handler"

---

Project Structure

src/
├── app/
│   ├── api/
│   │   ├── chat/route.ts            # POST — SSE (auto-selects ReAct or legacy)
│   │   └── models/route.ts          # GET — proxy to Ollama /api/tags
│   ├── globals.css
│   ├── layout.tsx
│   └── page.tsx                      # Myth UI with debug panels
└── lib/
    ├── tools.ts                      # 11 tool definitions with Zod schemas
    ├── tool-schemas.ts               # Zod → OpenAI function format converter
    ├── ollama.ts                     # Dual-endpoint client (/api/ + /v1/)
    ├── pipeline.ts                   # ReAct loop + legacy fallback + guardrail
    └── prompts.ts                    # All system prompts

scripts/
└── auto.py                           # Standalone Python MCP client

---

API Reference

POST /api/chat

{
  "baseUrl": "http://localhost:11434",
  "model": "qwen2.5:latest",
  "message": "What's the weather in Tokyo?"
}

SSE events: debug, token, done

GET /api/models?baseUrl=http://localhost:11434

Returns model list, health status, and 11 tool names.

---

Design Decisions

Why probe-first instead of configuration? Users shouldn't need to know whether their model supports tool calling. The probe call takes ~1s and the pipeline adapts automatically.

Why ReAct instead of Plan-and-Execute? Plan-and-Execute requires the model to output a full plan JSON array upfront — difficult for 1.5B models. ReAct lets the model decide one step at a time, which is more natural and error-tolerant.

Why keep the legacy pipeline? Many users run small code models (qwen2.5-coder, deepseek-coder) that don't support native tool calling. The legacy 4-step pipeline with its keyword fallback and 3-attempt extraction cascade is specifically optimized for these models.

Why simulated streaming instead of SSE parsing? Parsing delta.tool_calls from an SSE stream is complex and Ollama's implementation may differ from OpenAI's. Non-streaming calls during tool turns + simulated chunks for the final answer is more reliable.

---

Troubleshooting

Model always gives direct answers, never uses tools

Your model likely doesn't support native tool calling. Pull qwen2.5:latest or llama3.2:latest. The pipeline will automatically fall back to legacy mode for non-tool-calling models.

"Legacy mode" in console

This means the /v1/chat/completions endpoint returned 404. Update Ollama: curl -fsSL https://ollama.com/install.sh | sh

"Disallowed characters in expression"

The math tool supports sqrt(), sin(), cos(), tan(), log(), log2(), log10(), abs(), pow(), floor(), ceil(), round(), pi, e, and ^.

Schema guardrail triggered

If you see [GUARDRAIL] in the console, the model copied a schema description as a tool argument. The guardrail automatically replaced it with the actual user message.