Skip to content

AINode 0.5.19

Choose a tag to compare

View all tags
@webdevtodayjason webdevtodayjason released this 16 Sep 22:16
· 77 commits to main since this release
v0.5.19
f00c0ff
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

The fleet endpoint forwards the Anthropic Messages API; Claude Code is a harness bench adapter; Qwen3.8-Flash-Next entry pinned and proven.

Fixed

  • The fleet endpoint forwards the Anthropic Messages API. vLLM serves POST /v1/messages natively alongside its OpenAI paths, but AINode's port 3000 registered only /v1/chat/completions and /v1/completions, so /v1/messages was a 404 on every node: a client that speaks Messages and nothing else (Claude Code, the Anthropic SDKs) had to be aimed at one engine's :8000 and lost the model-id lookup, capability-aware ordering and failover that every other protocol gets for free. /v1/messages and /v1/messages/count_tokens now go through proxy_to_vllm, the same handler as chat completions, so routing on the body's model, transport failover across every node serving it, the 404 for a model nobody serves, SSE streaming and header passthrough (x-api-key, anthropic-version, anthropic-beta reach the engine untouched) are the code that already existed rather than a second implementation. is_multimodal_request now also reads the Messages API's own spelling of media (an {"type": "image"} or {"type": "document"} block, including one nested in a tool_result), so an image sent over /v1/messages is ordered onto an instance that accepts images instead of being routed on the model id alone. The proxy also stops dropping the caller's query string (path_qs, not path): Claude Code posts to /v1/messages?beta=true. The Server view's endpoint list no longer calls /v1/messages planned.

Added

  • Harness bench adapter: claude, Claude Code driven against a served model. The first harness here that does not speak the OpenAI protocol, and the reason the proxy fix above exists. --harness claude runs claude -p "<prompt>" --model <id> --dangerously-skip-permissions --output-format json --max-turns 12 with ANTHROPIC_BASE_URL set to --endpoint minus its trailing /v1 (Claude Code appends /v1/messages itself, so the /v1 form would produce /v1/v1/messages); --endpoint stays the /v1 form on the command line for every harness. The placeholder key goes in both ANTHROPIC_API_KEY and ANTHROPIC_AUTH_TOKEN, the served model fills both the main and small-fast model slots, and telemetry and non-essential traffic are off. CLAUDE_CONFIG_DIR points at the run's own scratch directory so a bench run never reads or writes the operator's Claude Code profile (settings, hooks, MCP servers, sessions, credentials), which is both hygiene and honesty, since a personal settings.json changes what the agent does; it needs no seed file (verified: claude 2.1.272 creates the whole tree itself from a directory that never existed and never prompts for a login). is_error: true in the result JSON is recorded as a crash even when the exit code is 0, because Claude Code reports a run it could not finish and still exits cleanly. Verified end to end on claude 2.1.272: 6/6 hidden tests on isogram driving Qwen3.8 27B NVFP4 on the fleet, 294 s over 6 turns. Known gap, documented rather than fixed: a CLAUDE.md in a parent of the working directory is still discovered, so a --work-dir inside a checkout that has one puts that file in the run.

Image: ghcr.io/getainode/ainode:0.5.19

Assets 2
Loading