Repository navigation
·
33 commits
to main
since this release
Immutable
release. Only release title and notes can be modified.
We are proud to release NeMo Relay 0.11.0!
This release includes changes since 0.10.0 and fixes shipped in the 0.10 patch releases. Highlights focus on diagnostics, framework integrations, and more reliable request handling.
Notable New Features
- 📊 Diagnostics: Inspect runtime counters, recent events, workers, and diagnostic memory across CLI, daemon, bindings, and plugin SDKs. Collection is opt-in with bounded storage.
- 🔗 Integrations: Add Python Google ADK support, including supported LiteLLM models, and configurable middleware placement for Deep Agents.
- 🔭 Observability: Configure OTLP custom certificate authorities and mutual TLS, Python trace-file sinks, and smaller ATIF exports. Doctor checks more exporter and privacy settings.
- 🔌 CLI Improvements Run full LLM middleware in daemon workers, supply API keys per request attempt, and use request codecs in manual LLM calls.
- 🧭 Scopes and coding agents: Inspect visible scope snapshots across bindings and plugin SDKs, and continue launcher traces through Codex and Claude Code hooks.
- ⚙️ Configuration: Configure worker callback deadlines, message-size limits, and optional operational log files. Expanded quick starts and Kubernetes guidance simplify deployment.
Breaking Changes
- Coding-agent hooks: Refresh installed integrations and restart affected processes. Local hooks verify TLS; tool checks default to fail-closed, and delivery failures return exit code
2. Generated hooks usePreToolUsewithout a separatePermissionRequesthook. - Provider access and policy ordering: Relay checks client access before lending provider credentials. Workers now load layered user and system configuration. Response caches default to priority
150; review custom priorities so policy checks run first. - Python and framework results: Object reconstruction requires explicit
allowed_types. LangChain and LangGraph tool results use the typednemo.relay.tool_result@1wrapper; update middleware and subscribers that inspect results. - Streaming and Node.js: Node.js requires version 24.13.1 or later. Streams expose provider, collector, and finalizer failures; handle errors and await
stream.close()during early cleanup. - Observability and request output: Automatic remote OTLP logs and metrics require HTTPS unless explicitly allowed. Fatal TLS errors stop HTTP trace exporters. CLI turn names inherit agent identity, and adaptive hints now honor configured injection settings.
- Rust and plugin SDKs: Update direct streaming and logging struct construction and exhaustive worker-error matches. Scope snapshots require native ABI v8 or the worker extension; existing v7 callback layouts remain compatible. Failed plugin integrity or signature checks prevent activation.
- Worker transport: Messages default to a 4 MiB limit each. Upgrade hosts and worker SDKs together before using callback-specific limits.
Non-Exhaustive Fixes
- Streaming and codecs: Preserve fragmented Unicode, Anthropic tool-input shapes, Gemini function arguments, Azure stream metadata, and OpenAI Responses usage. Retain failures through stream cleanup.
- Frameworks and coding agents: Preserve tool schemas, typed results, handled errors, and cache-replay content. Recover missing Codex MCP tool completions when matching results reach Relay.
- Observability and pricing: Correct ATIF content, trace parentage, exporter diagnostics, and token-cost accounting, including Gemini thinking and tool-use tokens.
- Daemon and plugins: Reduce startup memory and routing overhead; improve cancellation, session cleanup, configuration loading, and HTTP error propagation.
- Bindings and configuration: Preserve Python exceptions, isolate throwing Node.js codec callbacks, correct Go scope ownership, retain configured log sinks, and expose missing editor settings.
What's Changed
💥 Breaking Changes
- fix(python)!: require explicit codec reconstruction types by @willkill07 in #1352
- fix(node)!: surface stream provider, collector, finalizer errors by @greysonlalonde in #1287
- fix(cli)!: remove redundant PermissionRequest hooks by @willkill07 in #1388
- feat!: integrate bounded dynamic plugin diagnostics by @bbednarski9 in #1450
✨ New Features
- feat: support full LLM middleware in daemon workers by @willkill07 in #1281
- feat: make the Codex ChatGPT upstream configurable by @samratjha96 in #1319
- feat: add per-request upstream API key helpers by @willkill07 in #1342
- feat: expose consistent scope stack snapshots across bindings and plugin SDKs by @ericevans-nv in #1366
- feat: make short worker callback deadlines configurable by @willkill07 in #1371
- feat(observability): add insecure_http_hosts allowlist by @marcusds in #1367
- feat(core): recognize load_skill as a skill tool by @greysonlalonde in #1432
- feat(observability): add OTLP TLS and mTLS configuration by @willkill07 in #1422
- feat: add request codecs to manual LLM calls by @willkill07 in #1441
- feat: allow credential-free HTTP model upstreams on trusted hosts by @AjayThorve in #1429
- feat: add bounded runtime diagnostics foundation by @bbednarski9 in #1448
- feat: integrate bounded CLI diagnostics by @bbednarski9 in #1449
- feat: integrate bounded daemon diagnostics by @bbednarski9 in #1453
- feat: expose bounded diagnostics in language bindings by @bbednarski9 in #1455
- feat: unify diagnostics read contracts and conformance by @bbednarski9 in #1456
- feat: inspect diagnostic memory and correlate resource samples by @bbednarski9 in #1457
- feat: integrate native plugin SDK diagnostics by @bbednarski9 in #1467
- feat(python): add google adk integration by @greysonlalonde in #1459
- feat: support optional operational file log sinks by @bbednarski9 in #1484
- feat(python): support google adk litellm models by @greysonlalonde in #1540
⚡ Improvements
- perf: reduce hook client startup and delivery overhead by @willkill07 in #1379
- perf: reduce daemon routing and transport overhead by @willkill07 in #1392
- perf: reduce worker gateway and runtime overhead by @willkill07 in #1394
- perf: add borrowing APIs for stream collectors and codecs by @willkill07 in #1393
- perf: reuse owned payloads in built-in request codecs by @willkill07 in #1493
- enhancement: reduce ATIF output size by @willkill07 in #1520
- refactor(python): type google adk class swaps by @greysonlalonde in #1505
- refactor: address Sonar findings by @willkill07 in #1530
🐛 Bug Fixes
- fix: report OpenTelemetry file sink activation by @mnajafian-nv in #1194
- fix: support host-confined Windows workers and private state ownership by @bbednarski9 in #1256
- fix: align gateway hook timeout with daemon execution by @willkill07 in #1288
- fix: block on fail-closed hook delivery errors by @willkill07 in #1289
- fix: block fail-closed daemon hooks without credentials by @willkill07 in #1290
- fix: preserve multiple normalized request edits by @willkill07 in #1296
- fix: emit schema-valid ATIF content parts by @fernandol-nvidia in #1301
- fix: bound Anthropic streaming content block storage by @willkill07 in #1303
- fix: restrict typed reconstruction to registered types by @willkill07 in #1304
- fix: retain Go native handles throughout FFI calls by @willkill07 in #1302
- fix: authenticate pi hook forwarding and enforce session ownership by @willkill07 in #1306
- fix: skip dynamic plugins when trust verification fails by @willkill07 in #1305
- fix: load verified native plugin artifacts by @willkill07 in #1307
- fix: confine pi named upstreams to public and loopback addresses by @bbednarski9 in #1314
- fix: propagate LangChain tool schema edits to model calls by @ericevans-nv in #1316
- fix: emit text-only ATIF step messages as strings by @fernandol-nvidia in #1321
- fix: preserve plugin and remote HTTP error statuses by @willkill07 in #1317
- fix(pii): preserve bounded trajectory status metadata by @j-nolan in #1339
- fix: restore layered configuration for daemon workers by @willkill07 in #1344
- fix: validate upstream URLs and clarify daemon forwarding errors by @willkill07 in #1345
- fix: deliver managed daemon hooks without the system proxy by @tianyangc-nvidia in #1351
- fix: protect gateway dispatch and guardrails cache ordering by @willkill07 in #1353
- fix: keep gateway credentials and prompts on intended upstreams by @willkill07 in #1354
- fix: require client access before lending provider credentials by @willkill07 in #1355
- fix: keep Relay credentials off unmanaged provider routes by @willkill07 in #1358
- fix: secure hook delivery and fail-closed defaults by @willkill07 in #1356
- fix: preserve Responses usage and flag early stream endings by @willkill07 in #1361
- fix(codec): pass request codecs to daemon telemetry by @willkill07 in #1362
- fix: recover missing Codex MCP tool completions by @willkill07 in #1360
- fix: preserve file sinks with logging environment overrides by @willkill07 in #1373
- fix(cli): expose missing config.toml editor settings by @willkill07 in #1374
- fix: cancel provider work when worker calls end by @willkill07 in #1370
- fix: reduce daemon and worker startup memory by @willkill07 in #1375
- fix: include callback context in worker RPC timeout errors by @willkill07 in #1372
- fix: reclaim unused session locks by @willkill07 in #1381
- fix: name the trajectory in the ATIF delivery-failure warning by @phermosomore in #1413
- fix: let hooks verify a daemon that runs under another account by @tianyangc-nvidia in #1407
- fix: preserve explicit errors in manual LLM completion by @tianx-dev in #1405
- fix: preserve framework tool content and handled error status by @willkill07 in #1416
- fix(python): keep a pending exception through native stream wakes by @greysonlalonde in #1421
- fix(core): bill gemini thinking tokens when attaching model pricing by @greysonlalonde in #1427
- fix: use managed auth for OpenAI and Anthropic model discovery by @willkill07 in #1434
- fix: complete config editor schema support by @willkill07 in #1420
- fix(core): count gemini tool-use prompt tokens by @greysonlalonde in #1428
- fix(core): drain OTLP batch queue before accepting spans and logs by @greysonlalonde in #1445
- fix(node): preserve stream errors when close races a poll by @willkill07 in #1451
- fix(core): patch gemini text slots and merge streamed call args by @greysonlalonde in #1454
- fix: preserve framework tool results during cache replay by @willkill07 in #1440
- fix(diagnostics): diagnostics memory pagination hardening by @bbednarski9 in #1466
- fix: parent hook sessions to the launcher's W3C trace context by @rmalani-nv in #1463
- fix: preserve streaming Unicode and harden gateway transport by @bbednarski9 in #1470
- fix: wait for native and worker stream cleanup by @willkill07 in #1475
- fix(node): require node 24.13.1 or newer by @greysonlalonde in #1482
- fix: validate TLS files before exporter startup by @willkill07 in #1478
- fix: validate observability plaintext destinations during preflight by @ericevans-nv in #1477
- fix: report TLS trace failures and stop further sends by @willkill07 in #1480
- fix: identify invalid mTLS client credentials by @willkill07 in #1479
- fix: preserve readable ATIF user text with inline images by @bbednarski9 in #1483
- fix: handle raw Gemini and OCI requests in ATIF export by @bbednarski9 in #1486
- fix: report skipped automatic OpenTelemetry exporters by @ericevans-nv in #1487
- fix: record skipped automatic exporters in typed diagnostics by @bbednarski9 in #1489
- fix(core): bill uncached input tokens in model pricing by @greysonlalonde in #1500
- fix(codec): preserve input shape for streamed Anthropic tools by @Somay-kousis in #1509
- fix: accept config level names in binding log helpers by @Somay-kousis in #1504
- fix: keep real stream metadata after an empty Azure filter chunk by @Somay-kousis in #1508
- fix: ignore empty or relative XDG base directories by @Somay-kousis in #1513
- fix: add Python OpenTelemetry plugin file sinks by @willkill07 in #1518
- fix: make worker gRPC message limits configurable by @vantu-fit in #1499
- fix: reject BigInt in Gemini response decoding by @willkill07 in #1521
- fix: enforce transport policy for automatic OTLP logs and metrics by @willkill07 in #1523
- fix: classify malformed Gemini responses as invalid arguments by @willkill07 in #1522
- fix: accept large Gemini token counts across bindings by @willkill07 in #1524
- fix: honor adaptive_hints inject_header and inject_body_path by @Somay-kousis in #1525
- fix: warn on unknown keys in [gateway], [upstream] and [agents] by @Somay-kousis in #1511
- fix(node): isolate throwing codec callbacks by @greysonlalonde in #1529
- fix: preserve worker streaming errors and routes by @ericevans-nv in #1536
- fix: keep caller metadata when adaptive hints target a metadata path by @Somay-kousis in #1531
- fix: unify diagnostics HTTP errors and worker inspection by @bbednarski9 in #1537
- fix(go): recover callback panics and keep stream finalizer output by @greysonlalonde in #1294
- fix: support outer middleware placement for Deep Agents by @willkill07 in #1542
- fix: report OTLP readiness and privacy settings in doctor by @willkill07 in #1543
- fix(go): pin goroutine to thread while a scope is open by @greysonlalonde in #1546
- fix(python): record LangChain tool result values with types by @greysonlalonde in #1548
- fix: inherit CLI turn identity from owning Agent scope by @mnajafian-nv in #1547
✅ Tests
- test(core): roll back native registration entry-point leaks by @greysonlalonde in #1368
- test(node): make the collected stream END test event driven by @greysonlalonde in #1369
- test: address main Sonar findings by @willkill07 in #1382
- test: make Node stream rejection checks independent of order by @willkill07 in #1424
- test: reduce test scope for google-adk integration by @greysonlalonde in #1471
- test: enable Google ADK OpenAI and LiteLLM coverage by @willkill07 in #1476
🧰 Maintenance
- build: inherit example SDK versions for prerelease packaging by @willkill07 in #1235
- ci: make lychee pre-commit hook cross-platform by @pasmud in #1264
- chore: forward-merge release/0.10 into main by @willkill07 in #1270
- chore: bump main to 0.11.0 and reset release docs by @willkill07 in #1279
- chore: replace deprecated YAML dependency by @willkill07 in #1282
- ci: update nvskills-ci workflow to latest upstream main by @willkill07 in #1285
- chore: bump rustls to 0.23.46 by @willkill07 in #1340
- chore: resolve
cargo auditfindings by @willkill07 in #1386 - chore: bump aiohttp to 3.14.4 by @willkill07 in #1399
- chore: update Hono to clear npm audit by @willkill07 in #1419
- chore: update daemon guidance on macOS; address audit failures by @willkill07 in #1423
- chore: update markdown-it to clear npm audit by @greysonlalonde in #1430
- chore: update release 0.10 virtualenv for security advisories by @willkill07 in #1446
- ci: retain only the five newest documentation versions by @willkill07 in #1437
- chore: resolve npm audit advisory by @willkill07 in #1464
- chore: support type checks without Google ADK dependencies by @willkill07 in #1473
- build: require patched Tokio across the workspace by @willkill07 in #1481
- ci: publish CLI binaries to Artifactory for non-alpha tags by @willkill07 in #1488
- chore: update dependencies flagged by security audits by @bbednarski9 in #1490
- chore: update dependencies flagged by security audits by @willkill07 in #1494
- ci: prune old nightly tags and restrict upstream workflows by @willkill07 in #1497
- chore: upgrade ty to 0.0.91 by @greysonlalonde in #1498
- chore: handle ty suppressions for optional integration types by @willkill07 in #1519
- chore: address Zstandard security advisories on release/0.10 by @willkill07 in #1526
- chore: explicitly enable uv project management by @willkill07 in #1528
- chore: update Fern to 5.168.0 by @dagardner-nv in #1538
- ci: validate Quick Start contracts (RELAY-895) by @exactlyallan in #1336
📝 Documentation Updates
- docs: add 0.10.1 release and migration notes by @willkill07 in #1299
- docs: establish Getting Started quick start navigation (RELAY-895) by @exactlyallan in #1323
- docs: add Python quick start (RELAY-895) by @exactlyallan in #1324
- docs: add Node.js quick start (RELAY-895) by @exactlyallan in #1325
- docs: add Rust quick start (RELAY-895) by @exactlyallan in #1326
- docs: add 0.10.2 and 0.10.3 release and migration notes by @willkill07 in #1385
- docs: add Hermes quick start (RELAY-895) by @exactlyallan in #1327
- docs: await subscribers.flush_async() in instrumentation examples by @Letcise in #1396
- docs: add Claude Code quick start (RELAY-895) by @exactlyallan in #1328
- docs: add Codex quick start (RELAY-895) by @exactlyallan in #1329
- docs: add pi quick start (RELAY-895) by @exactlyallan in #1330
- docs: add LangChain quick start (RELAY-895) by @exactlyallan in #1332
- docs: add LangGraph quick start (RELAY-895) by @exactlyallan in #1333
- docs: add Deep Agents quick start (RELAY-895) by @exactlyallan in #1334
- docs: support native Windows in the Hermes quick start by @exactlyallan in #1403
- docs: explain ATIF HTTP timeout sizing and failures by @willkill07 in #1418
- docs: organize plugin usage and add Switchyard setup (RELAY-895) by @exactlyallan in #1335
- docs: add Kubernetes daemon deployment guidance by @willkill07 in #1425
- docs: clarify observability transport requirements by @willkill07 in #1438
- docs: explain streaming failures and cleanup by @willkill07 in #1442
- docs: complete 0.11 upgrade guidance by @willkill07 in #1443
- docs: validate shared Rust documentation examples by @willkill07 in #1444
- docs: clarify provider credentials and voice routing by @willkill07 in #1439
- docs: consolidated documentation updates for runtime diagnostics stack by @bbednarski9 in #1458
- docs: simplify Google ADK integration guidance by @willkill07 in #1472
- docs: fill 0.11 release and migration gaps by @willkill07 in #1474
- docs: define pull request etiquette by @willkill07 in #1539
- docs: clarify dynamic plugin configuration and code examples by @willkill07 in #1544
- docs: complete 0.11 release and migration guidance by @willkill07 in #1545
🧩 Skills Updates
- skills: refresh install and quick-start guidance by @exactlyallan in #1346
New Contributors
- @samratjha96 made their first contribution in #1319
- @j-nolan made their first contribution in #1339
- @tianyangc-nvidia made their first contribution in #1351
- @marcusds made their first contribution in #1367
- @Letcise made their first contribution in #1396
- @phermosomore made their first contribution in #1413
- @tianx-dev made their first contribution in #1405
- @rmalani-nv made their first contribution in #1463
- @Somay-kousis made their first contribution in #1509
- @vantu-fit made their first contribution in #1499
Full Changelog: 0.10.0...0.11.0