Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 22 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ SPDX-License-Identifier: Apache-2.0

## What Is NeMo Relay?

NeMo Relay is a portable execution runtime for agent systems that already have a
NVIDIA NeMo Relay is a portable execution runtime for agent systems that already have a
framework, model provider, policy layer, or observability backend. It gives those
systems one consistent way to describe, control, and observe what happens when an
agent crosses a request, tool, or LLM boundary.
Expand All @@ -43,7 +43,7 @@ consistent across frameworks and languages.
observability payloads, transform requests, or wrap execution without
rewriting every call site.
- 📡 **Emit one lifecycle stream**: Subscribers consume canonical runtime events
in-process or export them as [ATIF v1.6](https://github.com/harbor-framework/harbor/blob/main/rfcs/0001-trajectory-format.md)
in-process or export them as [ATIF v1.7](https://github.com/harbor-framework/harbor/blob/main/rfcs/0001-trajectory-format.md)
trajectories, OpenTelemetry traces, or OpenInference-compatible traces.
- 🧩 **Integrate without a framework migration**: NeMo Relay can sit below NeMo
ecosystem components, third-party agent frameworks, provider adapters, or
Expand All @@ -67,6 +67,10 @@ consistent across frameworks and languages.
- ✅ **Built-in observability plugin**: Configure Agent Trajectory Observability
Format (ATOF), ATIF, OpenTelemetry, and OpenInference exporters without
registering subscribers by hand.
- ✅ **Non-blocking subscriber delivery**: Keep managed execution moving while
subscriber callbacks and exporters drain in the background. Flush subscribers
before relying on callback side effects or exported files in tests and
shutdown paths.
- ✅ **Extension points for framework authors**: Wrap stable tool and provider
callbacks while preserving framework-owned scheduling, retries, memory, and
result handling.
Expand Down Expand Up @@ -108,12 +112,22 @@ uv add nemo-relay
npm install nemo-relay-node
```

The Node.js package requires Node.js 24 or newer.

### CLI Installation

The NeMo Relay CLI is offered as a separate crate:

```bash
cargo install nemo-relay-cli
```

If `cargo-binstall` is available on your machine:

```bash
cargo binstall nemo-relay-cli
```

For source builds, testing, and contribution workflow, see [CONTRIBUTING.md](CONTRIBUTING.md).

## Documentation
Expand Down Expand Up @@ -148,10 +162,10 @@ Below is our support matrix for agent harnesses.

| Agent | Observability | Security | Optimization | Notes |
|:--|:--:|:--:|:--:|:--|
| Claude Code | ✅ Yes | ❌ No | ❌ No | Observability only; no known issues |
| Codex | ✅ Yes | ❌ No | ❌ No | Observability only; missing some necessary hooks for full features |
| Hermes Agent | ✅ Yes | ❌ No | ❌ No | Observability only; no known issues |
| Cursor | ✅ Yes | ❌ No | ❌ No | Observability only; not feature-rich, missing hooks under `cursor-agent` |
| Claude Code | ✅ Yes | ⚠️ Partial | ⚠️ Partial | Tool guardrail support is wired up. LLM optimization is in place. |
| Codex | ✅ Yes | ⚠️ Partial | ⚠️ Partial | Tool guardrail support is wired up. LLM optimization is in place. Missing some necessary hooks for full feature parity. |
| Hermes Agent | ✅ Yes | ⚠️ Partial | ⚠️ Partial | Tool guardrail support is wired up. LLM optimization is in place. |
| Cursor | ✅ Yes | ⚠️ Partial | ⚠️ Partial | Tool guardrail support is wired up. LLM optimization is in place. Not feature-rich, missing hooks under `cursor-agent` |

## Third-Party Integrations

Expand All @@ -171,7 +185,7 @@ Below is the support matrix for our public API integrations.
| LangChain | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped Tool and LLM calling |
| LangGraph | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped Tool and LLM calling |
| Deep Agents | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped Tool and LLM calling |
| OpenClaw | ✅ Yes | ❌ No | ❌ No | Observability support; missing middleware for wrapped execution |
| OpenClaw | ✅ Yes | ⚠️ Partial | ❌ No | Hook-backed telemetry with pre-tool guardrails. Managed execution rewrites require the patch-based integration. |

#### LangChain

Expand Down Expand Up @@ -218,4 +232,4 @@ The following roadmap outlines planned features and integrations for upcoming re

## License

NeMo Relay is licensed under the [Apache License 2.0](LICENSE).
NVIDIA NeMo Relay is licensed under the Apache License 2.0.
2 changes: 1 addition & 1 deletion docs/about-nemo-relay/concepts/subscribers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ shape for their target backend.

Native subscribers are invoked by one process-wide worker thread in FIFO event
order and subscriber snapshot order. WebAssembly keeps its current synchronous
delivery behavior because this ticket does not add thread-based dispatch there.
delivery behavior because it does not use thread-based dispatch.

## Waiting for Delivery

Expand Down
67 changes: 64 additions & 3 deletions docs/about-nemo-relay/release-notes/highlights.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,70 @@ position: 1
{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0 */}

This page summarizes the notable capabilities in the current release documentation set.
This page summarizes the notable capabilities in the current release
documentation set.

## NeMo Relay 0.3

This release of NeMo Relay release introduces several new components and capabilities.
The complete changelog and release notes can be viewed on [GitHub](https://github.com/NVIDIA/NeMo-Relay/releases).
NeMo Relay 0.3 adds first-party guardrails, richer trajectory export, adaptive
plugin workflows, and clearer integration support for agent runtimes.

### Breaking Changes

- The project was renamed to NeMo Relay across documentation, package guidance,
and CLI-facing surfaces.
- The core runtime registry surface was narrowed. Applications should use the
documented middleware, intercept, subscriber, and plugin APIs instead of broad
registry access.
- Native subscriber delivery is now non-blocking. Event construction remains
synchronous, but subscriber callbacks and exporter work are queued on a
process-wide background dispatcher. Applications and tests that depend on
subscriber side effects must call the subscriber flush API before reading
captured events, files, or exported trace output.

### Guardrails

- Added the built-in `nemo_guardrails` plugin contract for installing NeMo
Guardrails behavior through NeMo Relay plugin configuration.
- Added a remote NeMo Guardrails backend for deployments that call an external
guardrails service.
- Added CLI editor support for guardrails plugin configuration.
- Added guardrail scopes for conditional guardrails so trace output shows
guardrail execution boundaries more clearly.
- Integrated security guardrails around managed agent calls.

### Observability

- Exposed canonical ATOF event JSON across bindings so applications can consume
the same event shape from Rust, Python, and Node.js.
- Upgraded ATIF exporters to ATIF v1.7 with nested subagent support.
- Added first-class S3-compatible storage export for ATIF traces.
- Added streaming LLM chunk marks for more precise streaming trace inspection.
- Fixed LLM start event ordering so managed LLM start events are emitted before
execution intercepts.
- Fixed ATIF tool-observation correlation.

### Plugins And Adaptive Runtime

- Enabled the adaptive plugin for the CLI and OpenClaw workflows.
- Added CLI editor support for adaptive plugin configuration.
- Added a Python context manager for plugin initialization and teardown.

### Integrations

- Improved LangChain serialization for wrapped requests and responses.
- Preserved OpenClaw tool call replay visibility with the upgrade to OpenClaw 2026.5.26
- Updated the launch banner to use `NEMO RELAY`.

### Documentation And Tooling

- Switched the documentation site to Fern and consolidated Fern publishing.
- Added broken-link validation for Fern documentation.
- Added agent runtime primer, trace incident runbook, plugin-building, migration,
and adaptive tuning guidance.
- Added built-in guardrails plugin documentation.
- Added CI path filters, CLI draft release assets, dependency updates, and ATIF
S3 storage test coverage.

The complete changelog and release artifacts can be viewed on
[GitHub Releases](https://github.com/NVIDIA/NeMo-Relay/releases).
54 changes: 52 additions & 2 deletions docs/about-nemo-relay/release-notes/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,58 @@ position: 5
{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0 */}

This page summarizes the current documentation-visible release state. GitHub Releases remain the source of truth for complete release history and tag-specific notes.
This page summarizes the current documentation-visible release state. GitHub
Releases remain the source of truth for complete release history and
tag-specific notes.

## Current Release

Use this page as the release-notes entry point. Use the child pages for current highlights, known issues, and related release resources.
NeMo Relay 0.3 focuses on the renamed NeMo Relay runtime, plugin-driven agent
observability, guardrails integration, adaptive behavior for coding agents, and
ATIF v1.7 trace export.

The most important compatibility notes are:

- The project and package-facing documentation now use the NeMo Relay name.
- The core runtime registry surface has been narrowed. Code that registered
runtime behavior through older broad registry entry points should move to the
documented middleware, subscriber, and plugin APIs.
- Node.js 24 or newer is now the minimum supported Node.js version.
- Native subscriber delivery is now non-blocking. Code that depends on
subscriber callback side effects, exporter output, or deterministic test
assertions must call the subscriber flush API.

Use the child pages for release highlights and support notes. For the complete
PR-by-PR changelog, release artifacts, and tag-specific history, use
[GitHub Releases](https://github.com/NVIDIA/NeMo-Relay/releases).

## Release Scope

This release includes:

- Built-in NeMo Guardrails plugin support, including a remote backend and CLI
editor support.
- Canonical ATOF event JSON exposure across Rust, Python, and Node.js bindings.
- ATIF v1.7 exporter updates for nested subagents, tool-observation
correlation, and first-class S3-compatible storage export.
- Adaptive plugin support for the CLI and OpenClaw integration, including
editor support for adaptive configuration.
- Streaming LLM chunk marks and improved guardrail scope emission.
- Python plugin context-manager ergonomics.
- Non-blocking native subscriber delivery with flush APIs across Rust, Python,
Node.js, FFI/Go, and WebAssembly parity.
- LangChain serialization fixes and OpenClaw trace replay visibility fixes.
- Fern documentation publishing, link validation, and updated guidance for
agent runtime concepts, trace incident response, plugin building, and adaptive
tuning.

## Feature Documentation

For the major 0.3 additions, start with:

- [Observability Plugin](/observability-plugin/about)
- [Agent Trajectory Interchange Format (ATIF)](/observability-plugin/atif)
- [NeMo Guardrails Plugin](/nemo-guardrails-plugin/about)
- [Adaptive Plugin](/adaptive-plugin/about)
- [OpenClaw Plugin Guide](/supported-integrations/openclaw-plugin)
- [LangChain Integration Guide](/supported-integrations/langchain)
40 changes: 34 additions & 6 deletions docs/about-nemo-relay/release-notes/known-issues.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,45 @@ position: 2
{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0 */}


This page lists current limitations and support notes for the release documentation set.
This page lists current limitations and support notes for the release
documentation set.

## NeMo Relay 0.3

These notes apply to the NeMo Relay 0.3 Release.
These notes apply to the NeMo Relay 0.3 release. The following known issues
and limitations apply to NeMo Relay 0.3:

- Go, WebAssembly, and the raw C FFI surface are experimental and source-first.
- Generated API pages cover Rust, Python, and Node.js. Experimental bindings do not yet have the same generated documentation depth.
- The NeMo Relay CLI is experimental. Coding agent observability support varies due to capabilities of hooks. Any encountered problems should be filed as bugs.
- Generated API pages cover Rust, Python, and Node.js. Experimental bindings do
not yet have the same generated documentation depth.
- The NeMo Relay CLI is experimental. Coding agent observability support varies
due to capabilities of hooks. Any encountered problems should be filed as
bugs.
- Node.js 24 or newer is required for Node.js binding and package workflows.
- OpenClaw support uses public hook-backed telemetry with partial security and
optimization support. Security is limited to pre-tool conditional guardrails,
and optimization is limited to adaptive telemetry unless the integration owns
a managed execution path.
- The NeMo Guardrails plugin remote backend depends on the availability,
latency, and policy behavior of the configured remote service.
- S3-compatible ATIF export requires valid storage credentials and endpoint
configuration in the runtime environment.
- `LLMRequest` objects in the Python binding should be treated as immutable.
Request middleware that changes content should return a new request object.
- Native subscriber callbacks are delivered asynchronously. Flush subscribers
before relying on callback side effects, captured event lists, files, or
exporter output. Deregistering a subscriber affects future emissions, but
callbacks from already-queued event snapshots may still run.

### Fixed in NeMo Relay 0.3

- Managed LLM start events are emitted before execution intercepts.
- Coding-agent trace scopes are aligned with NeMo Relay agent scope semantics.
- ATIF tool observations are correlated with their matching tool calls.
- OpenClaw tool call replay visibility is preserved.
- LangChain serialization handles wrapped integration payloads more reliably.

### Fixed in Earlier Releases

### Fixed issues from NeMo Relay 0.1:
- Enabled TLS support for OTLP HTTP export.
- Preserved Go scope stacks across OS threads.
12 changes: 0 additions & 12 deletions docs/about-nemo-relay/release-notes/related-topics.mdx

This file was deleted.

12 changes: 7 additions & 5 deletions docs/nemo-relay-cli/about.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,16 @@ If you are instrumenting an application or framework directly, use

## Agent Harness Support

NeMo Relay CLI support is experimental and observability-focused.
NeMo Relay CLI support is experimental. Observability is supported, while
security and optimization support are partial and depend on the hooks each
agent exposes.

| Agent | Observability | Security | Optimization | Notes |
| --- | --- | --- | --- | --- |
| Claude Code | ✅ Yes | ❌ No | ❌ No | Observability only; no known issues. |
| Codex | ✅ Yes | ❌ No | ❌ No | Observability only; some hooks needed for full feature coverage are missing. |
| Hermes Agent | ✅ Yes | ❌ No | ❌ No | Observability only; no known issues. |
| Cursor | 🚧 Partial | ❌ No | ❌ No | Observability only; highly experimental |
| Claude Code | ✅ Yes | ⚠️ Partial | ⚠️ Partial | Pre-tool hook responses are supported. LLM optimization uses gateway-routed traffic; full coverage depends on loaded Claude Code hooks. |
| Codex | ✅ Yes | ⚠️ Partial | ⚠️ Partial | Hook forwarding and gateway-routed LLM optimization are supported. Codex hook activation and the missing session-end hook limit full coverage. |
| Hermes Agent | ✅ Yes | ⚠️ Partial | ⚠️ Partial | Hook forwarding and Hermes API-request telemetry are supported. Guardrail and optimization support depend on Hermes shell and API-request hook coverage. |
| Cursor | 🚧 Partial | ⚠️ Partial | ❌ No | Hook forwarding is supported. Gateway observability is limited. Missing hooks under `cursor-agent` limit full feature coverage. |

## Guides

Expand Down
4 changes: 3 additions & 1 deletion docs/reference/performance.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ paths.
Use these practices when applying the concept in application or integration code.

- Prefer scope-local middleware for request-specific behavior so cleanup happens when the scope closes.
- Keep subscriber callbacks lightweight or move expensive export work out of the hot path.
- Keep subscriber callbacks lightweight. Expensive export work no longer blocks
managed execution directly, but it can still delay queued subscriber delivery,
flushes, and shutdown.
- Use execution intercepts when you need to wrap real execution and sanitize guardrails when you only need to change emitted observability payloads.
- Use binding-native typed wrappers and codecs when provider payload conversion would otherwise be repeated at many call sites.

Expand Down
2 changes: 1 addition & 1 deletion docs/resources/support-and-faqs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ plugins.
### Why The Name "NeMo Relay"?

"NeMo" places the project in the NVIDIA NeMo ecosystem. "Relay" refers to the
runtime flow of agent work through scopes, middleware, events, subscribers,
runtime relay of agent work through scopes, middleware, events, subscribers,
plugins, and exporters.

The name is about the execution path NeMo Relay makes visible and controllable;
Expand Down
2 changes: 1 addition & 1 deletion docs/supported-integrations/about.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ security middleware, and optimization features.
| LangChain | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped tool and LLM calling. |
| LangGraph | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped tool and LLM calling. |
| Deep Agents | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped tool and LLM calling. |
| OpenClaw | ✅ Yes | ❌ No | ❌ No | Observability support; missing middleware for wrapped execution. |
| OpenClaw | ✅ Yes | ⚠️ Partial | ❌ No | Hook-backed telemetry with pre-tool guardrails. Managed execution rewrites require the patch-based integration. |

## Guides

Expand Down
20 changes: 11 additions & 9 deletions docs/supported-integrations/openclaw-plugin.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,11 @@ Agent Trajectory Interchange Format (ATIF) JSON, OpenTelemetry spans,
OpenInference/Phoenix spans, and adaptive telemetry inputs.

This public OpenClaw plugin uses OpenClaw public hooks. It can initialize
generic NeMo Relay plugin components such as `observability` and `adaptive`, but
hook-backed mode does not rewrite OpenClaw tool execution, provider routing, or
model requests. For middleware-backed behavior that changes execution, use the
patch-based OpenClaw integration from the NeMo Relay repository.
generic NeMo Relay plugin components such as `observability` and `adaptive`, and
it can run pre-tool conditional guardrails when OpenClaw invokes the before-tool
hook. Hook-backed mode does not rewrite provider routing or model requests. For
middleware-backed behavior that changes execution, use the patch-based OpenClaw
integration from the NeMo Relay repository.

Use this guide to install the plugin, enable it in OpenClaw, configure telemetry
outputs, verify exported traces, and understand current LLM replay fidelity.
Expand All @@ -26,7 +27,7 @@ outputs, verify exported traces, and understand current LLM replay fidelity.

Required:

- OpenClaw `2026.5.6` or newer.
- OpenClaw `2026.5.26` or newer.
- The OpenClaw CLI available as `openclaw`.

Optional:
Expand Down Expand Up @@ -248,11 +249,12 @@ reason when present.

## Runtime Mapping

The plugin maps supported OpenClaw hook events into NeMo Relay telemetry and
adaptive inputs without changing OpenClaw execution behavior.
The plugin maps supported OpenClaw hook events into NeMo Relay telemetry,
adaptive inputs, and pre-tool conditional guardrail checks.

It does not change OpenClaw tool execution, provider routing, policy decisions,
or provider request payloads.
It does not rewrite provider routing or provider request payloads. Hook-backed
adaptive telemetry can learn from replayed events, while request-rewrite
features such as adaptive hints require a managed execution path.

| OpenClaw hook | NeMo Relay behavior |
| --- | --- |
Expand Down
4 changes: 4 additions & 0 deletions fern/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ global-theme: nvidia
announcement:
message: "🔔 NVIDIA NeMo Relay is <strong>beta software</strong>. APIs and behavior may change with each release."

redirects:
- source: /nemo/relay/about-nemo-relay/release-notes/related-topics
destination: /nemo/relay/about-nemo-relay/release-notes

experimental:
mdx-components:
- ./components
Expand Down
Loading
Loading