Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,14 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

- **NeMo Relay native plugin** — a dynamically loaded integration that runs
libsy's weighted-random and LLM-classifier algorithms in process while
Switchyard owns provider HTTP dispatch, credentials, translation, retries,
and fallback. Managed calls require NeMo Relay 0.7 or newer and do not depend
on `switchyard-server`.

### Removed

- **Latency-aware router** — the `latency_service` route type and its
Expand Down
150 changes: 150 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ members = [
"crates/switchyard-server",
"crates/switchyard-skill-distillation",
"crates/switchyard-translation",
"crates/switchyard-nemo-relay-plugin",
]

[workspace.package]
Expand All @@ -23,6 +24,7 @@ repository = "https://github.com/NVIDIA-NeMo/Switchyard"
rust-version = "1.96.1"

[workspace.dependencies]
async-channel = "2"
async-stream = "0.3"
async-trait = "0.1"
futures = "0.3"
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ algorithm you write yourself.
- **Protocol Translation**: convert between OpenAI Chat, Anthropic Messages, and OpenAI Responses formats
- **Multi-Backend Routing**: random routing, LLM-as-classifier routing, signal-driven stage-router, or your own algorithm
- **Operational Metrics**: Prometheus metrics cover requests, errors, latency, tokens, and routing overhead
- **NeMo Relay Plugin**: run random or LLM-classifier routing in Relay while Switchyard owns provider HTTP dispatch

## Quick Start

Expand Down Expand Up @@ -119,6 +120,7 @@ configured LLM client selects one upstream format.
- **[`switchyard-libsy`](crates/libsy/README.md)**: embed routing algorithms in a Rust application
- **[`switchyard-protocol`](crates/protocol/README.md)**: provider-neutral request, response, and streaming types
- **[`switchyard-translation`](crates/switchyard-translation/README.md)**: request, response, and stream translation
- **[`switchyard-nemo-relay-plugin`](crates/switchyard-nemo-relay-plugin/README.md)**: install Switchyard as a native NeMo Relay plugin

## Community

Expand Down
32 changes: 32 additions & 0 deletions crates/switchyard-nemo-relay-plugin/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

[package]
name = "switchyard-nemo-relay-plugin"
version = "0.1.0"
description = "Switchyard-owned HTTP routing plugin for NeMo Relay"
authors.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
rust-version.workspace = true
publish = false

[lib]
crate-type = ["cdylib"]

[dependencies]
async-channel.workspace = true
async-trait.workspace = true
futures-util.workspace = true
http = "1"
# Use the published release candidate during development. This becomes `0.7.0`
# before the integration lands, after the stable crate is published.
nemo-relay-plugin = "=0.7.0-rc.5"
serde.workspace = true
serde_json.workspace = true
switchyard-libsy.workspace = true
switchyard-llm-client.workspace = true
switchyard-protocol.workspace = true
switchyard-translation.workspace = true
tokio.workspace = true
Loading
Loading