Request: Sanitized NeMo Agent Toolkit OTLP trace samples for compatibility testing

[trangocomputedev]

I’m developing an evidence-based NeMo Agent Toolkit compatibility profile for ContextIQ’s OTLP Agent Trace Inspector.

I’m looking for sanitized or synthetic OTLP/HTTP proto-JSON traces generated by a current NeMo Agent Toolkit workflow. I am not requesting customer or production telemetry.

Ideally, the sample set would include:

1. A successful agent workflow with one tool call
2. A workflow with nested tool calls
3. Multiple LLM calls with model and token attributes
4. A failed tool or workflow span
5. The NeMo Agent Toolkit and OpenTelemetry Collector versions used

The preferred format is an OTLP export object containing resourceSpans, suitable for testing a third-party parser.

I would also appreciate clarification on:

• Whether instrumentation scope nat is intended to be a stable NeMo-origin signal
• Which nat.* attributes are intended for external consumers
• Whether openinference.span.kind and llm.* are the recommended semantic fields
• Whether a canonical sanitized OTLP fixture already exists

If no fixture is available, is the documented simple_calculator_observability workflow with the otelcollector exporter the recommended reference for generating one?

Any sample would be kept version-specific, and ContextIQ would preserve the original attributes and distinguish observed evidence from derived mappings.

[IlyaKhalafi]

I checked the NVIDIA NeMo Agent Toolkit export format with manual synthetic spans. I have not generated real workflow fixtures.

The telemetry reference prescribes nvidia-nat[opentelemetry,profiler] for LLM/tool instrumentation. I did not test that complete setup. Start this local contrib Collector first, using otelcol-contrib --config collector.yml from an owned directory

receivers:
  otlp:
    protocols:
      http:
        endpoint: 127.0.0.1:41789
exporters:
  file:
    path: ./trace-fixtures.jsonl
    append: true
service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [file]

Copy the calculator file-tracing config to capture.yml. Its built-in file exporter writes intermediate steps, not OTLP. Replace only general.telemetry.tracing, keeping the other blocks

general:
  telemetry:
    tracing:
      otlp:
        _type: otelcollector
        project: fixture-check
        endpoint: http://127.0.0.1:41789/v1/traces

With your model configured, run nat run --config_file capture.yml --input "What is 2 * 4?". Let the workflow flush, then stop the Collector cleanly before reading. Require resourceSpans batches with scope nat and the expected LLM/tool spans before treating the capture as a fixture.

I tested Collector 0.163.1 with the pinned converter and toolkit exporter 1.9.0, using manual export and shutdown. The batch preserved parent links and error status. The merged workflow, profiler instrumentation and real tool failures were not run. The file exporter is alpha and writes one JSON batch per line. Its linked component README is 0.163.0; use a fresh file per fixture because this config appends.

At develop commit 26410de, the converter sets scope nat with literal version 1.0.0, unrelated to the toolkit release. NAT_SPAN_PREFIX can change the nat.* keys. The span model defines llm.model_name and llm.token_count.*; the converter adds openinference.span.kind. Check your installed version.

Use synthetic inputs and inspect input.value, output.value and resource attributes before sharing. A canonical fixture and stable attribute contracts remain unverified.