Skip to content

NeMo Relay 0.7.0

Choose a tag to compare

View all tags
@willkill07 willkill07 released this 05 Aug 16:22
· 313 commits to main since this release
Immutable release. Only release title and notes can be modified.
0.7.0
This tag was signed with the committer’s verified signature.
willkill07 Will Killian
SSH Key Fingerprint: SEpGTTIpz+IbOGCXzB7ZJ2jSuZKvjS/W2ElmHVlZWc8
Verified
Learn about vigilant mode.
326cf39
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

We are proud to release NeMo Relay 0.7.0!

Notable New Features:

  • 🛡️ Codec-aware sanitization and PII protection: LLM sanitizers receive the active per-call codec across Rust, Python, Node.js, native plugins, and workers, enabling provider-correct redaction for mixed OpenAI Chat, OpenAI Responses, Anthropic Messages, runtime, and opaque codec traffic.
  • 📊 Unified OpenTelemetry export: one typed configuration now supports independent full, gen_ai, and openinference endpoints, standardized GenAI semantic conventions, endpoint-local credentials, and multi-endpoint fan-out.
  • ⚡ Asynchronous middleware: Rust middleware can await work directly, while Python callbacks may return awaitables and Node.js callbacks may return Promises. Scope and mark publication remains synchronous at the API boundary and is delivered through an ordered background queue.
  • 🔀 Cross-boundary context propagation: Relay context can be carried between processes and runtimes, and Python applications can explicitly fork task-local scope stacks for concurrent asyncio work.
  • 💾 Opt-in exact-match LLM response caching: the Adaptive plugin can serve eligible repeated calls from in-memory or Redis-backed caches without invoking the provider. Caching remains disabled unless explicitly configured.
  • 🔧 Configuration, packaging, and operations: a structured nemo-relay config edit workflow, consistent layered configuration precedence, size-based operational-log rotation, environment-driven binding logging, offline doctor probes, package size reduction, and PyPI-packaged CLI binaries improve deployment and diagnostics.

Breaking Changes:

  • Middleware callback migration: Rust middleware callbacks now return futures. Native plugins must be rebuilt against ABI v3, and native and worker plugins must be rebuilt with the matching 0.7 SDK and protocol definitions. Python and Node.js synchronous callbacks remain supported.
  • LLM sanitizer contract: request and response sanitizers now use (payload, directional_context) -> optional payload. Returning None, null, or undefined omits the governed observability payload; there is no one-argument compatibility adapter.
  • Observability configuration version 3: migrate standalone OpenTelemetry and OpenInference configuration to typed opentelemetry.endpoints. Standalone OpenInference subscriber APIs and the otel and openinference Cargo features are removed.
  • Configuration and Rust source compatibility: top-level component lists and observability destination lists now concatenate across layers with higher-precedence entries first. Exhaustive FlowError matches must also handle the new callback-exception variant.
  • Switchyard deprecation: the experimental Relay-owned Switchyard plugin remains supported in 0.7 but is deprecated ahead of removal and replacement in 0.8.

Non-Exhaustive Fixes:

  • Privacy and enforcement safety: redact known LLM credential headers, reject malformed PII target paths, fail closed on sanitizer errors, and make generated coding-agent enforcement hooks fail closed when Relay cannot start, authenticate, evaluate, or deliver a response.
  • Observability correctness: prefer response-side model attribution, avoid duplicate ATIF continuation steps, export system prompts consistently, preserve callback exception details, and map structured failures to useful OpenTelemetry error types.
  • Exporter reliability: detect colliding or equivalent OTLP destinations, resolve bare OTLP/HTTP trace endpoints, report dropped spans, validate attribute mappings, and make failed ATIF remote deliveries visible, drainable, and recoverable.
  • Configuration and diagnostics: preserve explicit editor targets, apply configuration layers consistently, prevent disabled or inherited plugin configuration from masking programmatic settings, report logging errors, and expose effective upstream authentication presence without leaking credentials.
  • Python and Node.js reliability: preserve Node asynchronous context and flush semantics, isolate subscriber callback failures, make Python plugin teardown safe inside asyncio, report cancelled scopes as errors, and avoid completion callbacks targeting closed event loops.
  • Gateway resilience: preserve streaming provider error responses and make transparent-gateway shutdown bounded and interruptible.

What's Changed

💥 Breaking Changes

  • feat(sanitizers)!: expose per-call codec capabilities by @willkill07 in #546
  • feat(observability)!: unify OpenTelemetry exporters by @willkill07 in #556
  • feat(middleware)!: make middleware async across primary bindings by @willkill07 in #571

✨ New Features

⚡ Improvements

🐛 Bug Fixes

✅ Tests

🧰 Maintenance

📝 Documentation Updates

New Contributors

Full Changelog: 0.6.0...0.7.0

Contributors

cypres, willkill07, and 8 other contributors
Assets 40
Loading