Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ algorithm you write yourself.

Choose the launcher path to run Claude Code, Codex CLI, or OpenClaw through
Switchyard. Choose the server path to run Switchyard as a standalone proxy.
Choose the library path to embed routing in your own Rust application.

### Launcher Path

Expand Down Expand Up @@ -92,6 +93,36 @@ curl http://localhost:4000/health
For a complete configuration and a test request, follow
[Getting Started](docs/getting_started.md).

### Library Path

`switchyard-libsy` embeds the routing algorithms in your own Rust application.
It never calls a model itself: an algorithm decides which target to use and
hands every model call back to you, so it drops into an existing proxy, gateway,
or agent runtime without owning an HTTP stack. Pair it with
`switchyard-llm-client` when you want the calls made for you.

```toml
[dependencies]
switchyard-libsy = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" }
switchyard-protocol = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" }
```

See [Getting Started](docs/getting_started.md#library-path) for setup and the
algorithm list, or the [`switchyard-libsy`](crates/libsy/README.md) crate docs.

## Routing Strategies

| Strategy | Use it when | Route `type` |
|---|---|---|
| [LLM Classifier](docs/routing_algorithms/llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` |
| [Stage Router](docs/routing_algorithms/stage_router_routing.md) | Signals already in the conversation, such as tool results and errors, should route most turns without an extra model call. | `stage_router` |
| [Escalation Router](docs/routing_algorithms/escalation_router_routing.md) | Every turn runs on the weak tier first, and a judge reads that answer to decide whether to send the same request to the strong tier. | `llm_classifier` with `mode = "escalation"` |
| [Random](docs/routing_algorithms/random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` |

A `passthrough` route registers one target under one model ID with no routing
decision. See the [Routing Overview](docs/routing_algorithms/overview.md) for
the common route shape and self-hosted targets.

## Architecture

```mermaid
Expand Down
49 changes: 47 additions & 2 deletions docs/getting_started.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
# Getting Started with Switchyard

Switchyard has two native Rust execution paths:
Switchyard has three native Rust execution paths:

- **Launcher path:** install the Python-distributed CLI and launch Claude Code,
Codex, or OpenClaw through the packaged Rust server binding.
- **Server path:** build and run the standalone Rust server for API clients and
custom deployments.
- **Library path:** embed the routing algorithms directly in your own Rust
application with `switchyard-libsy`.

## Launcher Path

Expand Down Expand Up @@ -233,7 +235,50 @@ export SWITCHYARD_TELEMETRY_OPT_OUT=1

---

### Next steps
## Library Path

Use this path when you want routing inside your own Rust application rather than
behind a proxy. `switchyard-libsy` never calls a model itself: an algorithm
picks a target and hands the model call back to you.

### Add the dependencies

```toml
[dependencies]
async-trait = "0.1"
futures = "0.3"
switchyard-libsy = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" }
switchyard-protocol = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" }
tokio = { version = "1", features = ["macros", "rt"] }
```

### Choose an algorithm

| Type | Purpose |
|---|---|
| `LlmTaskClassifier` | Ask a judge model to choose an efficient or capable target. |
| `StageRouter` | Route from signals already in the conversation, such as tool results and errors, with an optional judge fallback. |
| `LlmTaskClassifier` with escalation | Every turn runs on the efficient target first, and a judge reads that answer to decide whether to send the same request to the capable target. |
| `Random` | Select among any number of targets, uniform or weighted. |

These are the same strategies the server exposes as route types, so a deployment
can move between the server and library paths without changing routing
behaviour.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

### Drive the algorithm

An algorithm yields a stream of steps. Each `Step::CallLlm` is a model call your
host performs over its own transport, and the run ends with
`Step::ReturnToAgent` carrying the final response. Serving those calls yourself
is what lets libsy embed in a host that already owns its HTTP stack, retries,
and credentials.

For the request, response, and streaming types the steps carry, see
[`switchyard-protocol`](../crates/protocol/README.md).

---

## Next steps

- [Core Concepts](core_concepts.md): LLM clients, targets, and routes
- [`switchyard-server`](../crates/switchyard-server/README.md): server configuration,
Expand Down
Loading