From e3e895aca059772b7bdd5518b660ff642b779032 Mon Sep 17 00:00:00 2001 From: Ryan Lempka Date: Fri, 7 Aug 2026 12:38:02 -0500 Subject: [PATCH 1/7] docs: add routing strategies table and the libsy library path Signed-off-by: Ryan Lempka --- README.md | 31 ++++++++++++++++++++++++++ docs/getting_started.md | 49 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 78 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index d84cae242..868663259 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,7 @@ algorithm you write yourself. Choose the launcher path to run Claude Code, Codex CLI, or OpenClaw through Switchyard. Choose the server path to run Switchyard as a standalone proxy. +Choose the library path to embed routing in your own Rust application. ### Launcher Path @@ -92,6 +93,36 @@ curl http://localhost:4000/health For a complete configuration and a test request, follow [Getting Started](docs/getting_started.md). +### Library Path + +`switchyard-libsy` embeds the routing algorithms in your own Rust application. +It makes no network calls: an algorithm decides which target to use and hands +every model call back to you, so it drops into an existing proxy, gateway, or +agent runtime without owning an HTTP stack. Pair it with +`switchyard-llm-client` when you want the calls made for you. + +```toml +[dependencies] +switchyard-libsy = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" } +switchyard-protocol = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" } +``` + +See [Getting Started](docs/getting_started.md#library-path) for a worked +example, or the [`switchyard-libsy`](crates/libsy/README.md) crate docs. + +## Routing Strategies + +| Strategy | Use it when | Route `type` | +|---|---|---| +| [Random](docs/routing_algorithms/random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` | +| [LLM Classifier](docs/routing_algorithms/llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` | +| [Stage Router](docs/routing_algorithms/stage_router_routing.md) | Tool-result and agent-progress signals should route most turns without an extra classifier call. | `stage_router` | +| [Escalation Router](docs/routing_algorithms/escalation_router_routing.md) | Start every task on the weak tier and escalate to strong when an LLM judge detects trouble. | `llm_classifier` with `escalation` | + +A `passthrough` route registers one target under one model ID with no routing +decision. See the [Routing Overview](docs/routing_algorithms/overview.md) for +the common route shape and self-hosted targets. + ## Architecture ```mermaid diff --git a/docs/getting_started.md b/docs/getting_started.md index 7519bada6..13a0be304 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -1,11 +1,13 @@ # Getting Started with Switchyard -Switchyard has two native Rust execution paths: +Switchyard has three native Rust execution paths: - **Launcher path:** install the Python-distributed CLI and launch Claude Code, Codex, or OpenClaw through the packaged Rust server binding. - **Server path:** build and run the standalone Rust server for API clients and custom deployments. +- **Library path:** embed the routing algorithms directly in your own Rust + application with `switchyard-libsy`, with no server process. ## Launcher Path @@ -233,7 +235,50 @@ export SWITCHYARD_TELEMETRY_OPT_OUT=1 --- -### Next steps +## Library Path + +Use this path when you want routing inside your own Rust application rather than +behind a proxy. `switchyard-libsy` makes no network calls of its own: an +algorithm picks a target and hands the model call back to you. + +### Add the dependencies + +```toml +[dependencies] +async-trait = "0.1" +futures = "0.3" +switchyard-libsy = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" } +switchyard-protocol = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" } +tokio = { version = "1", features = ["macros", "rt"] } +``` + +### Choose an algorithm + +| Type | Purpose | +|---|---| +| `Passthrough` | Always call one configured target. | +| `Random` | Select among any number of targets, uniform or weighted. | +| `LlmTaskClassifier` | Ask a judge model to choose an efficient or capable target. | +| `StageRouter` | Route coding-agent turns from tool and progress signals, with an optional judge fallback. | + +These are the same strategies the server exposes as route types, so a deployment +can move between the server and library paths without changing routing +behaviour. + +### Drive the algorithm + +An algorithm yields a stream of steps. Each `Step::CallLlm` is a model call your +host performs over its own transport, and the run ends with +`Step::ReturnToAgent` carrying the final response. + +If you would rather not drive the stream yourself, `switchyard-llm-client` +provides a ready-made consumer that performs the calls over HTTP. See the +[`switchyard-llm-client`](../crates/libsy-llm-client/README.md) quickstart for a +complete buffered and streaming example. + +--- + +## Next steps - [Core Concepts](core_concepts.md): LLM clients, targets, and routes - [`switchyard-server`](../crates/switchyard-server/README.md): server configuration, From 15c27f5b3b5e96907593028aab4f9a7fe309aa6f Mon Sep 17 00:00:00 2001 From: Ryan Lempka Date: Fri, 7 Aug 2026 12:38:53 -0500 Subject: [PATCH 2/7] docs: order routing tables by production relevance Signed-off-by: Ryan Lempka --- README.md | 2 +- docs/getting_started.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 868663259..d2bda1fa3 100644 --- a/README.md +++ b/README.md @@ -114,10 +114,10 @@ example, or the [`switchyard-libsy`](crates/libsy/README.md) crate docs. | Strategy | Use it when | Route `type` | |---|---|---| -| [Random](docs/routing_algorithms/random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` | | [LLM Classifier](docs/routing_algorithms/llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` | | [Stage Router](docs/routing_algorithms/stage_router_routing.md) | Tool-result and agent-progress signals should route most turns without an extra classifier call. | `stage_router` | | [Escalation Router](docs/routing_algorithms/escalation_router_routing.md) | Start every task on the weak tier and escalate to strong when an LLM judge detects trouble. | `llm_classifier` with `escalation` | +| [Random](docs/routing_algorithms/random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` | A `passthrough` route registers one target under one model ID with no routing decision. See the [Routing Overview](docs/routing_algorithms/overview.md) for diff --git a/docs/getting_started.md b/docs/getting_started.md index 13a0be304..cd57b4ec8 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -256,10 +256,10 @@ tokio = { version = "1", features = ["macros", "rt"] } | Type | Purpose | |---|---| -| `Passthrough` | Always call one configured target. | -| `Random` | Select among any number of targets, uniform or weighted. | | `LlmTaskClassifier` | Ask a judge model to choose an efficient or capable target. | | `StageRouter` | Route coding-agent turns from tool and progress signals, with an optional judge fallback. | +| `Passthrough` | Always call one configured target. | +| `Random` | Select among any number of targets, uniform or weighted. | These are the same strategies the server exposes as route types, so a deployment can move between the server and library paths without changing routing From 7aae4e74da361a65befb8a5beedda0f2a360002f Mon Sep 17 00:00:00 2001 From: Ryan Lempka Date: Fri, 7 Aug 2026 12:39:24 -0500 Subject: [PATCH 3/7] docs: drop passthrough from the libsy algorithm table Signed-off-by: Ryan Lempka --- docs/getting_started.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting_started.md b/docs/getting_started.md index cd57b4ec8..aa6ffe11c 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -258,7 +258,7 @@ tokio = { version = "1", features = ["macros", "rt"] } |---|---| | `LlmTaskClassifier` | Ask a judge model to choose an efficient or capable target. | | `StageRouter` | Route coding-agent turns from tool and progress signals, with an optional judge fallback. | -| `Passthrough` | Always call one configured target. | +| `LlmTaskClassifier` with escalation | Start every task on the efficient target and escalate when a judge detects trouble. | | `Random` | Select among any number of targets, uniform or weighted. | These are the same strategies the server exposes as route types, so a deployment From 2e7f90a00e325563e4cc023907ff48772c3a59e8 Mon Sep 17 00:00:00 2001 From: Ryan Lempka Date: Fri, 7 Aug 2026 12:46:37 -0500 Subject: [PATCH 4/7] docs: tighten routing strategy descriptions Signed-off-by: Ryan Lempka --- README.md | 10 +++++----- docs/getting_started.md | 10 +++++----- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index d2bda1fa3..94d31bda9 100644 --- a/README.md +++ b/README.md @@ -96,9 +96,9 @@ For a complete configuration and a test request, follow ### Library Path `switchyard-libsy` embeds the routing algorithms in your own Rust application. -It makes no network calls: an algorithm decides which target to use and hands -every model call back to you, so it drops into an existing proxy, gateway, or -agent runtime without owning an HTTP stack. Pair it with +It never calls a model itself: an algorithm decides which target to use and +hands every model call back to you, so it drops into an existing proxy, gateway, +or agent runtime without owning an HTTP stack. Pair it with `switchyard-llm-client` when you want the calls made for you. ```toml @@ -115,8 +115,8 @@ example, or the [`switchyard-libsy`](crates/libsy/README.md) crate docs. | Strategy | Use it when | Route `type` | |---|---|---| | [LLM Classifier](docs/routing_algorithms/llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` | -| [Stage Router](docs/routing_algorithms/stage_router_routing.md) | Tool-result and agent-progress signals should route most turns without an extra classifier call. | `stage_router` | -| [Escalation Router](docs/routing_algorithms/escalation_router_routing.md) | Start every task on the weak tier and escalate to strong when an LLM judge detects trouble. | `llm_classifier` with `escalation` | +| [Stage Router](docs/routing_algorithms/stage_router_routing.md) | Signals already in the conversation, such as tool results and errors, should route most turns without an extra model call. | `stage_router` | +| [Escalation Router](docs/routing_algorithms/escalation_router_routing.md) | Every turn runs on the weak tier first, and a judge reads that answer to decide whether to send the same request to the strong tier. | `llm_classifier` with `escalation` | | [Random](docs/routing_algorithms/random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` | A `passthrough` route registers one target under one model ID with no routing diff --git a/docs/getting_started.md b/docs/getting_started.md index aa6ffe11c..c552bb496 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -7,7 +7,7 @@ Switchyard has three native Rust execution paths: - **Server path:** build and run the standalone Rust server for API clients and custom deployments. - **Library path:** embed the routing algorithms directly in your own Rust - application with `switchyard-libsy`, with no server process. + application with `switchyard-libsy`. ## Launcher Path @@ -238,8 +238,8 @@ export SWITCHYARD_TELEMETRY_OPT_OUT=1 ## Library Path Use this path when you want routing inside your own Rust application rather than -behind a proxy. `switchyard-libsy` makes no network calls of its own: an -algorithm picks a target and hands the model call back to you. +behind a proxy. `switchyard-libsy` never calls a model itself: an algorithm +picks a target and hands the model call back to you. ### Add the dependencies @@ -257,8 +257,8 @@ tokio = { version = "1", features = ["macros", "rt"] } | Type | Purpose | |---|---| | `LlmTaskClassifier` | Ask a judge model to choose an efficient or capable target. | -| `StageRouter` | Route coding-agent turns from tool and progress signals, with an optional judge fallback. | -| `LlmTaskClassifier` with escalation | Start every task on the efficient target and escalate when a judge detects trouble. | +| `StageRouter` | Route from signals already in the conversation, such as tool results and errors, with an optional judge fallback. | +| `LlmTaskClassifier` with escalation | Every turn runs on the efficient target first, and a judge reads that answer to decide whether to send the same request to the capable target. | | `Random` | Select among any number of targets, uniform or weighted. | These are the same strategies the server exposes as route types, so a deployment From f099a19a598b9336cd2eda9216af1dfc20386332 Mon Sep 17 00:00:00 2001 From: Ryan Lempka Date: Fri, 7 Aug 2026 12:55:09 -0500 Subject: [PATCH 5/7] docs: drop the version-sensitive consumer reference Signed-off-by: Ryan Lempka --- docs/getting_started.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/getting_started.md b/docs/getting_started.md index c552bb496..d178d9df3 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -271,10 +271,8 @@ An algorithm yields a stream of steps. Each `Step::CallLlm` is a model call your host performs over its own transport, and the run ends with `Step::ReturnToAgent` carrying the final response. -If you would rather not drive the stream yourself, `switchyard-llm-client` -provides a ready-made consumer that performs the calls over HTTP. See the -[`switchyard-llm-client`](../crates/libsy-llm-client/README.md) quickstart for a -complete buffered and streaming example. +For the request, response, and streaming types the steps carry, see +[`switchyard-protocol`](../crates/protocol/README.md). --- From 8d41c35b7603302db16f7574ea2c488b10fc2f02 Mon Sep 17 00:00:00 2001 From: Ryan Lempka Date: Fri, 7 Aug 2026 13:00:13 -0500 Subject: [PATCH 6/7] docs: say why the host serves the calls Signed-off-by: Ryan Lempka --- docs/getting_started.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/getting_started.md b/docs/getting_started.md index d178d9df3..b737cf01d 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -269,7 +269,9 @@ behaviour. An algorithm yields a stream of steps. Each `Step::CallLlm` is a model call your host performs over its own transport, and the run ends with -`Step::ReturnToAgent` carrying the final response. +`Step::ReturnToAgent` carrying the final response. Serving those calls yourself +is what lets libsy embed in a host that already owns its HTTP stack, retries, +and credentials. For the request, response, and streaming types the steps carry, see [`switchyard-protocol`](../crates/protocol/README.md). From 22e6f2662402dca727c9b59fdccc36d43278dbff Mon Sep 17 00:00:00 2001 From: Ryan Lempka Date: Fri, 7 Aug 2026 13:04:24 -0500 Subject: [PATCH 7/7] docs: name the escalation mode key and drop the worked-example claim Signed-off-by: Ryan Lempka --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 94d31bda9..45cf3cbde 100644 --- a/README.md +++ b/README.md @@ -107,8 +107,8 @@ switchyard-libsy = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" } switchyard-protocol = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" } ``` -See [Getting Started](docs/getting_started.md#library-path) for a worked -example, or the [`switchyard-libsy`](crates/libsy/README.md) crate docs. +See [Getting Started](docs/getting_started.md#library-path) for setup and the +algorithm list, or the [`switchyard-libsy`](crates/libsy/README.md) crate docs. ## Routing Strategies @@ -116,7 +116,7 @@ example, or the [`switchyard-libsy`](crates/libsy/README.md) crate docs. |---|---|---| | [LLM Classifier](docs/routing_algorithms/llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` | | [Stage Router](docs/routing_algorithms/stage_router_routing.md) | Signals already in the conversation, such as tool results and errors, should route most turns without an extra model call. | `stage_router` | -| [Escalation Router](docs/routing_algorithms/escalation_router_routing.md) | Every turn runs on the weak tier first, and a judge reads that answer to decide whether to send the same request to the strong tier. | `llm_classifier` with `escalation` | +| [Escalation Router](docs/routing_algorithms/escalation_router_routing.md) | Every turn runs on the weak tier first, and a judge reads that answer to decide whether to send the same request to the strong tier. | `llm_classifier` with `mode = "escalation"` | | [Random](docs/routing_algorithms/random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` | A `passthrough` route registers one target under one model ID with no routing