v0.2.0

Switchyard v0.2.0

Switchyard v0.2.0 is a substantial redesign of the project around a native Rust server and the new libsy library architecture. Across 193 commits, this release separates orchestration, provider-neutral protocols, translation, model transport, and serving into focused components that can be used together or embedded independently.

Switchyard remains pre-alpha software. APIs and configuration may change before a stable release.

libsy Redesign

switchyard-libsy provides a provider-neutral framework for multi-LLM orchestration. Its Algorithm abstraction decides which semantic model targets to call, in what order, and how to combine their results.

Algorithms do not own provider SDKs or an HTTP stack. They yield model-call steps back to their host, allowing the same algorithm to run inside the Switchyard server, another proxy, an agent runtime, or an application with custom model clients. The driver supports both buffered and streaming execution, concurrent model calls, routing decisions, request processors, classifiers, and session-scoped state.

The redesign establishes clear crate boundaries:

• switchyard-protocol owns normalized request, response, streaming, routing, tool, usage, error, and metadata contracts.
• switchyard-translation encodes and decodes OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages.
• switchyard-libsy owns orchestration algorithms and their execution model.
• switchyard-llm-client supplies translated HTTP model calls and a ready-made libsy driver.
• switchyard-server composes these crates into a standalone proxy.

libsy includes typed errors, streamed algorithm output, concurrent calls, affinity-aware state, context-window recovery, classifier contracts, per-target prompts, handoff notes, escalation policies, and OpenTelemetry instrumentation.

Native Server

The new switchyard-server serves all three supported APIs from one explicit TOML deployment:

• OpenAI Chat Completions at /v1/chat/completions
• OpenAI Responses at /v1/responses
• Anthropic Messages at /v1/messages
• Anthropic token counting at /v1/messages/count_tokens
• Model discovery at /v1/models
• Health checks, Prometheus metrics, and JSON routing statistics

TOML configuration declares LLM clients, credentials, wire formats, targets, and algorithms explicitly. The server includes TLS, graceful shutdown, bounded upstream retries, model discovery, token counting, optional durable routing logs, and context-window overflow recovery.

Responses now consistently report the model target that actually served the request rather than the client-facing route alias. /v1/models also reports declared capabilities and Codex metadata.

Routing and Orchestration

The native algorithm set now includes:

• LLM task classification using a dedicated judge target.
• Capability, escalation, custom-schema, and multi-target classification policies.
• Signal-driven stage routing based on coding-agent tool use and progress.
• Per-target system prompts and handoff notes.
• Optional LLM fallback when stage-routing signals are inconclusive.
• Session affinity and sub-agent-aware target selection.
• Context-window eviction and rerouting with session-isolated history.
• Probability-based capability policies and configurable classifier prompts.

Classifier history preserves tool-call and tool-result pairs, malformed classifier scores fail safely, and streamed judge responses are fully consumed before a routing decision is made.

Translation and API Compatibility

Translation now preserves substantially more information across API boundaries:

• Buffered and streaming responses.
• Tool definitions, tool calls, and tool results.
• Responses reasoning and final-answer items.
• Usage details, cached-token counts, and cache-creation counts.
• Provider stream events for compatible same-format replay.
• Max-token truncation and incomplete Responses status.
• Prompt-cache metadata across Anthropic-compatible providers.

Known request fields are validated before translation rather than being silently coerced or omitted. Anthropic endpoints return Anthropic error envelopes, accept compatible stream terminators, filter incompatible beta headers and OpenAI-only fields, and omit unsigned thinking blocks rejected by Anthropic-compatible upstreams.

Anthropic prompt caching is enabled by default for translated calls. Streaming now terminates after an in-band upstream error instead of forwarding unrelated trailing events.

Observability

The native path includes:

• Prometheus metrics at /metrics.
• OpenTelemetry spans using GenAI semantic attributes.
• Structured request, target, latency, retry, token, cache, and error telemetry.
• Aggregate statistics at /v1/stats.
• Statistics reset through /v1/stats/reset.
• Optional per-session routing statistics at /v1/routing/session-stats.
• Optional durable JSONL routing logs for evaluation workloads.

Python and Coding-Agent Launchers

Python users can run native algorithms through switchyard.libsy and host the Rust server in-process through switchyard_rust.server.Server.

Claude Code, Codex CLI, and OpenClaw launchers now use the native server, TOML routes, translation, statistics, and OpenTelemetry paths. A packaged OpenRouter deployment provides the default launcher configuration, while --config accepts custom TOML deployments.

An experimental LiteLLM integration is also included.

Install

Python CLI and bindings:

uv tool install --python 3.12 "nemo-switchyard[cli,server]==0.2.0"

Native Rust server:

cargo install --locked switchyard-server --version 0.2.0

Published Packages

Python:

• nemo-switchyard 0.2.0

Rust:

• switchyard-protocol 0.2.0
• switchyard-translation 0.2.0
• switchyard-libsy 0.2.0
• switchyard-llm-client 0.2.0
• switchyard-server 0.2.0

Deprecations and Removals

The Python switchyard serve path, Python server, YAML route bundles, and profile APIs remain available for transition purposes but are deprecated. New deployments should use switchyard-server, native TOML configuration, and libsy algorithms.

The legacy Rust core, components-v2 stack, PyO3 profile bindings, plan-and-execute routing, RouteLLM integration, external router plugin, latency-aware router, and Intake-specific sink have been removed.

The CLI is now focused on switchyard launch and switchyard serve. The previous configure, verify, status, settings, smoke-test, and TUI workflows have been removed. The former cascade route is now named stage_router.

Known Issues

• Buffered upstream work may continue after a client disconnects and still incur provider cost.
• Some fallback and escalation decisions lack routing-target attribution in /v1/stats and /metrics.
• The retry-recovery counter remains zero after a successful upstream retry.
• x-switchyard-session-id is not yet included in native session statistics.
• The native server does not yet forward X-Switchyard-Version upstream.

See the getting-started guide, known issues, and full changelog.

v0.1.0

Switchyard v0.1.0

Switchyard v0.1.0 was the first public release. It introduced the Python proxy, OpenAI and Anthropic format translation, YAML route bundles, initial routing strategies, coding-agent launchers, and request metrics.

This release is retained for historical reference. New users should start with v0.2.0.

• nemo-switchyard 0.1.0 on PyPI
• Full v0.1.0 changelog

v0.0.1

build: set release test version to 0.0.1