harnage

AI Model = Brain. Harness = Hands.

A good harness amplifies what a model can do — it doesn't automate the model away. Describe the agent you want; harnage builds you a fully-owned, Claude Code-level harness for it, tuned so even a small local model performs at its ceiling.

---

harnage is two things in one repo:

1. The builder — harnage init "<what you want>" turns a prompt into a complete, compiling TypeScript agent harness, bespoke to your domain: custom system prompt, tools, slash commands, and skills, generated by a build brain and verified with a compile-and-repair loop.
2. The reference harness — the chassis every generated harness ships with: a goal-driven loop engine, 9 tools, path-rule permissions, sandboxed bash, context compaction, layered long-term memory, eval-in-loop, audit trail, session resume (including mid-task crash recovery), sub-agents, and an Ink TUI with live slash-command menus.

You own everything it generates. No SaaS, no lock-in — the output is a plain Bun/TypeScript project that runs against your local Ollama model or your own API key.

Features

Area	What you get
Builder pipeline	Interview → plan → generate → assemble → verify → repair. LLM-driven when a build brain is configured; falls back to a fully offline keyword pipeline when it isn't.
Per-model scaffolding	Generated harnesses resolve the plugged-in model to a profile (tier, loop mode, tool budget, edit format, decoding params) instead of assuming a frontier model.
Memory	Semantic + episodic facts/events (local bun:sqlite), procedural skills, working-memory compaction — nothing leaves the machine.
Eval-in-loop	Deterministic quality checks on every run, optional LLM-as-judge, a trace command over the local audit log — terminal-first LLMops.
Session resume	Transcripts survive restarts; an interrupted task is offered for continuation on next start.
Permissions + sandbox	Path-rule policy (~/.harnage/permissions.json) plus a command/path-blocklist sandbox for bash.
MCP dual-mode	The harnage CLI serves an MCP server (--mcp) and consumes external MCP servers today. Generated harnesses can serve MCP; external-MCP consumption in generated harnesses is in progress.
Ink TUI	Live slash-command menu, streaming output, plus a classic readline REPL fallback.

Install

npm i -g harnage        # or: bunx harnage@latest  (no install, run once)

Requires bun ≥ 1.1 on your PATH — the npm package ships TypeScript source and runs it via bun directly (no separate build step). Don't want bun installed? Grab a prebuilt binary (darwin-arm64, linux-x64) from the GitHub releases page instead — download, chmod +x, and run it directly, no runtime needed.

Building from source instead:

git clone https://github.com/PDgit12/harnage
cd harnage
bun install
bun run build
./harnage init "..."

Quick Start

harnage init "an agent that reviews git diffs for bugs and posts a summary to Slack"

The builder picks a build brain automatically (your configured API key, falling back to local Ollama, falling back to a keyword pipeline that works fully offline). The generated harness lands in .harnage-build-<name>/ — cd in, bun install, and run it.

Provider resolution on first run:

1. ~/.harnage/config.json (created by the setup wizard; supports OpenRouter/Anthropic/OpenAI/Ollama + fallback models)
2. ANTHROPIC_API_KEY / OPENAI_API_KEY env vars
3. Running Ollama instance
4. Nothing found → setup wizard

Usage

Command	What it does
harnage	Interactive REPL with the goal-driven loop
harnage init "<description>"	Build a bespoke harness from a prompt
harnage studio	Guided interview → harness build
harnage --mcp	Run as an MCP server (connect from Claude Code, Cursor, …)

What a generated harness includes

• Loop engine tuned to the chosen model's tier: native tool-calling for strong models; grammar-forced constrained-JSON decisions and baked domain pipelines for small local models
• Bespoke layer: domain system prompt, custom tools, slash commands, and skills generated for your use case
• Layered memory: semantic + episodic store (bun:sqlite) with deterministic recall and post-run consolidation, plus procedural skills and working-memory compaction
• Eval-in-loop: deterministic checks on every run, optional LLM judge, trace command over the local audit trail
• Session resume: transcripts survive restarts; an interrupted task is offered for continuation on next start (--resume picks it up mid-task)
• Permissions + sandbox: path-rule policy (allow/deny by tool + glob pattern), command/path-blocklist sandbox for bash — no OS-level container yet (see Status)
• Ink TUI with a live slash-command menu, plus a classic REPL and an MCP server mode

Slash commands (inside the REPL/TUI)

/help · /config · /cost · /clear · /model · /doctor · /exit — plus the bespoke commands the builder generated for your domain.

Demo

Coming soon — asciinema/GIF walkthrough of harnage init → generated harness → first run.

Development

bun run typecheck   # tsc --noEmit
bun run test        # vitest — 49 files, 357 tests passing
bun run lint        # biome check src/
bun run build       # compile binary

Generated harnesses must themselves pass bun install && tsc --noEmit — the builder verifies this and runs an LLM repair loop on failures.

Acceptance — every harness is tested before you get it

You pick the model before the build, so harnage builds the harness for that model and then runs it. The build brain writes a short battery of real tasks in your harness's own domain, harnage executes the finished harness against them on your chosen model, and the result ships inside the harness as ACCEPTANCE.md + acceptance.json.

Acceptance: 6 tasks on qwen2.5:3b...
  PASS workflow:generate
  PASS workflow:read
  FAIL workflow:write-file — model described the file instead of writing it
  ...
Acceptance: 4/6 — bar 4/6 for small tier → MET

A low score does not fail the build — you still get the harness, plus the failing tasks and a stronger model to try, because a poor score is usually the model rather than the harness. Skip it with --no-acceptance.

The build brain never writes grader code. Tasks are structured data (a goal, fixture files, and a typed expectation), evaluated by harnage — and any task that would pass on an empty answer is rejected before it can score anything.

Tools: every harness gets all of them

Capability is not rationed by domain — withholding tools looks like specialisation but is guesswork, and an agent missing one tool is simply broken. What the domain changes is how tools are ranked and presented: a small model can only be shown a handful per turn, so a docs harness surfaces grep first where a code harness surfaces bash. Specialisation lives in the system prompt, the pipeline stages, bespoke tools and commands, and that ranking — not in taking capability away.

Evals

Two axes, both offline-capable. Results append to ~/.harnage/eval-results.jsonl.

bun run eval:build         # BUILD axis: 10 domain prompts must build, compile,
                           # import nothing undeclared, and emit a registry that resolves
bun run eval qwen2.5:3b --suite smoke        # RUNTIME axis: 13 fast tasks
bun run eval qwen2.5:3b --suite full         # all 89 tasks, 8 categories
bun run eval:dry           # validate the battery itself, no model needed

The battery spans code · edit · data · docs · multistep · tools · refusal · safety. Most tasks are graded by a deterministic check; the open-ended ones (summaries, refusals, injection resistance) are graded by an LLM judge against a written rubric. Where a task has both, the check gates and the judge grades on top.

Pick the judge with EVAL_JUDGE_MODEL: unset = deterministic only · config = use the build brain from ~/.harnage/config.json (nothing extra runs on your machine) · or any Ollama model name. The judge is first scored against a hand-labelled set and refused below 75% agreement rather than silently trusted.

Status

Published: harnage@0.6.0 on npm, FSL-1.1-ALv2 licensed, prebuilt binaries on GitHub releases. Working: builder end-to-end (API + local + offline paths), reference harness (including MCP-server-consumer mode), all 9 tools, path-rule permissions, memory, eval, TUI. In progress: external-MCP consumption in generated harnesses (the harnage CLI itself already consumes MCP servers). No interactive permission-approval dialog yet — a denied call surfaces as tool-result text the model adapts to. No OS-level sandbox container yet — bash is confined by a command/path blocklist, not a VM/container. Found something broken? Open an issue.