AnyHarness

Build SFT training trajectories from a coding agent's /v1/messages traffic.

AnyHarness is the architectural name for the layer between a harness (Claude Code) and the sampling endpoint / API server. It manages the harness↔upstream contract: the adapter exposes /v1/messages downstream to Claude Code, forwards upstream to a pluggable backend (sglang, vLLM /v1/completions, or any messages/chat/responses API), and captures each turn into a trajectory tree that an offline aggregator dumps as SFT data. The "harness in the loop" is that the adapter sits between the harness and the model — it is not fire-and-forget: every /v1/messages call is intercepted, matched into a per-session tree (tolerant of Claude Code's tool-result trimming), and linearized into training samples at finish_session.

A standalone adapter + sandbox-pluggable Claude Code harness + CLI that drives the agent through the adapter and dumps loss-masked SFT samples.

Layout

src/anyharness/
  trajectory.py        # TrajectoryManager, TurnRecord, SampledSequence, TopkLogprobs
  types.py             # Sample
  parsing.py            # parse_model_output (reasoning + tool-call parsers)
  adapters/             # MetaAdapter: /v1/messages downstream + 5 upstreams
                         #   (sglang /generate, vLLM /v1/completions, vLLM chat,
                         #    Responses, Mint asample)
  harness/
    base.py             # Harness ABC + RunConfig (shared subprocess)
    claude_code.py      # ClaudeCodeHarness (claude CLI)
    pi.py               # PiHarness (pi CLI)
    codex.py            # CodexHarness (codex CLI, /v1/responses)
    opencode.py         # OpencodeHarness (opencode CLI, /v1/chat/completions)
  cli.py                # main(): adapter-in-thread -> harness -> finish_session -> dump
  dump.py               # dump_samples -> trajectories.jsonl + .md (+ .pt)
  sharegpt_dump.py     # fold a tree into ShareGPT (reasoning + thinking_signature)

How it works

1. The CLI builds an MetaAdapter and serves its aiohttp app on ADAPTER_PORT (a background thread via aiohttp.web.AppRunner).
2. ClaudeCodeHarness launches claude -p <PROMPT> --output-format stream-json with ANTHROPIC_BASE_URL=<adapter> and ANTHROPIC_AUTH_TOKEN=<session_id>. Each /v1/messages call is rendered to tokens, sent upstream (sglang /generate with logprobs, or forwarded to another /v1/messages API), and recorded into a per-session trajectory tree.
3. finish_session linearizes the tree into Samples: tokens = full prompt+response sequence, loss_mask marks prompt=0 / response=1.
4. dump_samples writes trajectories.jsonl, trajectories.md, and (if torch is installed) trajectories.pt.

Install

pip install -e ".[sglang,test]"   # sglang mode + tests
# or messages mode (no tokenizer needed):
pip install -e ".[test]"

The claude CLI must already be on PATH (the harness runs it directly; this harness skips that step). Override with CLAUDE_BIN=/path/to/claude.

Usage

sglang mode (capture logprobs from a served model):

export UPSTREAM_MODE=sglang-slash-generate
export MODEL_PATH=Qwen/Qwen3.5-9B
export ANYHARNES_UPSTREAM_URL=http://localhost:30000
export PROMPT="Fix the failing test in tests/test_foo.py"
export OUTPUT_DIR=./out
anyharness

completions mode (vLLM /v1/completions, native token-in token-out — the cleanest path for vLLM, no chat-shape workarounds):

export UPSTREAM_MODE=vllm-slash-completions
export ANYHARNES_UPSTREAM_URL=http://localhost:30001   # vLLM OpenAI server
export ANYHARNES_UPSTREAM_MODEL=qwen3-9b
export ANYHARNES_LOGPROBS_OUTPUT_TOPK=5                 # top-k output logprobs (0=off)
export MODEL_PATH=Qwen/Qwen3.5-9B                       # tokenizer (optional if
                                                         #  the server exposes /tokenize)
export PROMPT="..."
anyharness

messages mode (forward to an existing /v1/messages endpoint):

export UPSTREAM_MODE=messages
export ANYHARNES_UPSTREAM_URL=https://api.example.com
export PROMPT="..."
anyharness

mint mode (token-in token-out against a Mint/Mint gateway):

export UPSTREAM_MODE=mint-slash-asample
export ANYHARNES_UPSTREAM_URL=http://your-mint-gateway:28000
export ANYHARNES_UPSTREAM_API_KEY=...                       # sent as Bearer
export ANYHARNES_UPSTREAM_BASE_MODEL=Qwen/Qwen3.6-35B-A3B   # or ANYHARNES_UPSTREAM_MODEL_ID=sess-xxx_0
export MODEL_PATH=/path/to/matching/tokenizer
export PROMPT="..."
anyharness

Pass-through args to the harness CLI

Flags after -- (and the HARNESS_ARGS env var) are forwarded verbatim to the harness CLI (claude / pi / codex), appended after AnyHarness's own flags + prompt. AnyHarness does not parse them — the harness CLI does.

# forward --model sonnet to the harness CLI
anyharness -- --model sonnet

# or via env (shlex-split)
HARNESS_ARGS="--model sonnet" anyharness

Both channels merge: -- args first, then HARNESS_ARGS. Use HARNESS=pi or HARNESS=codex to select the harness.

Token-level data and TITO

Token ids must come from whoever sampled them. Re-encoding assistant text — or looking a logprob's token string back up in a vocab — breaks the token-in-token-out invariant: tokenization is not injective, so the recovered ids can differ from what the policy actually produced, and training then optimizes tokens the model never emitted. Where each mode stands:

Mode	Token ids from	Logprobs
mint-slash-asample	ids in, ids out (/api/v1/asample)	yes, paired by the server
vllm-slash-completions	vLLM choice.token_ids (/v1/completions)	yes — ids + logprobs from one response
sglang-slash-generate	native /generate (meta_info)	yes
chat	upstream return_token_ids (vLLM >= 0.10.2)	only if the upstream supplies ids
responses	not available	no — the Responses API exposes no ids
messages	n/a (message-level SFT)	no

When ids aren't available the adapter drops the logprobs and logs why, rather than reconstruct them. Message-level SFT is better than token-level data that is subtly wrong.

In mint-slash-asample and sglang-slash-generate mode the chat template is rendered locally, so MODEL_PATH must point at the served model's tokenizer. A mismatched tokenizer produces a prompt the server misreads — usually visible as the model echoing your prompt back — and the adapter warns when it detects one.

Multi-turn token drift

Each turn's prompt is rendered from the full message list, so it will not always reproduce the ids the policy sampled last turn. For a thinking model it cannot: Qwen3's template emits <think>...</think> while an assistant message is the final message and strips it once anything follows, so the re-render diverges across the whole response, every turn.

The trajectory builder classifies that divergence:

• CLEAN — the prompt extends the held tokens; append the tail.
• REALIGN — divergence covers only untrained tokens; heal in place and stay contiguous.
• FORK — divergence would overwrite trained tokens; close the sample and open a new one instead.

Trained tokens are never overwritten to preserve contiguity. Forking costs only contiguity — every sampled token still trains, spread across several Samples that each re-emit the shared prefix as loss_mask=0 context. That re-emission is not free: a 16-turn thinking rollout yields 16 samples and ~10x the tokens of one contiguous sample. The alternative — carrying sampled ids forward verbatim and appending only the new messages — keeps one sample but changes what the model conditions on (its own <think> blocks stay in context), and only works in modes where we send token ids rather than messages. It is not implemented.

Environment

One unified ANYHARNES_* set (the old SLIME_* / MINT_* / TINKER_* names are not recognized — see the migration table below).

Var	Default	Meaning
UPSTREAM_MODE	sglang-slash-generate	sglang-slash-generate, vllm-slash-completions, mint-slash-asample, tinker-slash-asample, chat, responses, or messages
ANYHARNES_UPSTREAM_URL	—	upstream base URL (all modes; was SLIME_SGLANG_URL / SLIME_*_BASE_URL / MINT_BASE_URL / TINKER_BASE_URL / SLIME_MESSAGES_UPSTREAM_URL)
ANYHARNES_UPSTREAM_API_KEY	—	upstream auth key (was SLIME_*_API_KEY / MINT_API_KEY / TINKER_API_KEY)
ANYHARNES_UPSTREAM_MODEL	—	served model name (was SLIME_*_MODEL / SLIME_MESSAGES_MODEL)
ANYHARNES_UPSTREAM_MODEL_ID	—	training-step model_id (mint/tinker only; was MINT_MODEL_ID / TINKER_MODEL_ID)
ANYHARNES_UPSTREAM_BASE_MODEL	—	base model (mint/tinker; was MINT_BASE_MODEL / TINKER_BASE_MODEL)
ANYHARNES_LOGPROBS_OUTPUT_TOPK	0	output-side top-k logprobs per position (0=off; was SLIME_TOP_LOGPROBS / SLIME_*_TOP_LOGPROBS + the *_LOGPROBS bool gate)
ANYHARNES_LOGPROBS_PROMPT_TOPK	0	prompt-side top-k (mint/tinker asample only; 0=off; was MINT_TOPK_PROMPT_LOGPROBS / TINKER_TOPK_PROMPT_LOGPROBS + bool gate)
ANYHARNES_UPSTREAM_TIMEOUT	900	retrieve_future + aiohttp sock_read timeout, seconds (was MINT_FUTURE_TIMEOUT / TINKER_FUTURE_TIMEOUT + hardcoded 900)
ANYHARNES_CHAT_ARGS_AS_DICT	0	1 keeps chat tool-call arguments as a dict (was SLIME_CHAT_ARGS_AS_DICT)
ANYHARNES_STREAM_DIR	—	streaming tree-storage dir (per-session .stream.jsonl); crash-safe replay/resume
MODEL_PATH	—	HF tokenizer path (required in sglang/mint/tinker; optional elsewhere)
ADAPTER_PORT	18080	port the adapter listens on
CLAUDE_MODEL	slime-actor	model name advertised to the CLI
PROMPT	—	the task prompt (required)
OUTPUT_DIR	./trajectories	where samples are written
TIME_BUDGET_SEC	600	harness time budget
CLAUDE_BIN	claude	path to the claude CLI
TOOL_PARSER / REASONING_PARSER	—	sglang parser names
FORK_THRESHOLD_TOKENS	—	trajectory fork threshold

Migration (old → new):

Old	New
SLIME_SGLANG_URL / SLIME_*_BASE_URL / MINT_BASE_URL / TINKER_BASE_URL / SLIME_MESSAGES_UPSTREAM_URL	ANYHARNES_UPSTREAM_URL
SLIME_*_API_KEY / MINT_API_KEY / TINKER_API_KEY	ANYHARNES_UPSTREAM_API_KEY
SLIME_*_MODEL / SLIME_MESSAGES_MODEL	ANYHARNES_UPSTREAM_MODEL
MINT_MODEL_ID / TINKER_MODEL_ID	ANYHARNES_UPSTREAM_MODEL_ID
MINT_BASE_MODEL / TINKER_BASE_MODEL	ANYHARNES_UPSTREAM_BASE_MODEL
SLIME_TOP_LOGPROBS / SLIME_*_TOP_LOGPROBS (+ the *_LOGPROBS bool gate)	ANYHARNES_LOGPROBS_OUTPUT_TOPK (0=off, N=top-N)
MINT_TOPK_PROMPT_LOGPROBS / TINKER_TOPK_PROMPT_LOGPROBS (+ bool gate)	ANYHARNES_LOGPROBS_PROMPT_TOPK (0=off)
MINT_FUTURE_TIMEOUT / TINKER_FUTURE_TIMEOUT	ANYHARNES_UPSTREAM_TIMEOUT
SLIME_CHAT_ARGS_AS_DICT	ANYHARNES_CHAT_ARGS_AS_DICT

Tests

pytest tests/test_harness_smoke.py

The smoke test exercises the trajectory layer end to end (no model, sglang, or claude binary): it builds a TrajectoryManager, feeds hand-built turns, and asserts loss masks (prompt=0, response=1).