test: add ATIF trace correlation coverage#126
Conversation
Signed-off-by: Will Killian <wkillian@nvidia.com>
WalkthroughNew unit tests verify lineage correlation between ATIF trajectory data and OpenInference/OpenTelemetry span attributes through shared ChangesATIF Lineage Correlation with Tracing Exporters
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
github-actions
Bot
added
size:M
Signed-off-by: Will Killian <wkillian@nvidia.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/core/tests/unit/observability/openinference_tests.rs`:
- Around line 881-887: The test currently picks the first agent-sourced step
with trajectory.steps.iter().find(|step| step.source == "agent"), which is
brittle; instead iterate the steps, parse each step.extra into AtifStepExtra and
select the step whose extra.ancestry.function_id equals llm_uuid. Concretely,
replace the find by attempting serde_json::from_value on each step.extra (handle
the Option), match the parsed AtifStepExtra.ancestry.function_id against
llm_uuid, and unwrap the found step into agent_step and then parse extra as
AtifStepExtra as before.
In `@crates/core/tests/unit/observability/otel_tests.rs`:
- Around line 626-632: Replace the brittle source == "agent" selection with a
UUID-driven match: iterate trajectory.steps, parse each step.extra into
AtifStepExtra (using serde_json::from_value) and select the step whose
AtifStepExtra.ancestry.function_id equals llm_uuid; update the variable formerly
named agent_step to be found by this ancestry check (and keep existing
unwraps/asserts as needed) so the test reliably targets the agent step by its
ancestry UUID rather than its source string.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 08d0614e-fab7-44b2-a9cb-7df36d37da7f
📒 Files selected for processing (6)
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rsdocs/plugins/observability/about.mddocs/plugins/observability/atif.mddocs/plugins/observability/openinference.mddocs/plugins/observability/opentelemetry.md
📜 Review details
🧰 Additional context used
📓 Path-based instructions (29)
**/*.{md,rst,html,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
**/*.{md,rst,html,txt}: Always spellNVIDIAin all caps. Do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun because the name starts with an 'en' sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company.
Use trademark symbols with product names only when the document type or legal guidance requires them.
Verify official capitalization, spacing, and hyphenation for product names.
Precede NVIDIA product names withNVIDIAon first mention when it is natural and accurate.
Do not rewrite product names for grammar or title-case rules.
Preserve third-party product names according to the owner's spelling.
Include the company name and full model qualifier on first use when it helps identify the model.
Preserve the official capitalization and punctuation of model names.
Use shorter family names only after the full name is established.
Spell out a term on first use and put the acronym in parentheses unless the acronym is widely understood by the intended audience.
Use the acronym on later mentions after it has been defined.
For long documents, reintroduce the full term if readers might lose context.
Form plurals of acronyms withs, not an apostrophe, such asGPUs.
In headings, common acronyms can remain abbreviated. Spell out the term in the first or second sentence of the body.
Common terms such asCPU,GPU,PC,API, andUIusually do not need to be spelled out for developer audiences.
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
**/*.{md,rst,html}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
Link the first mention of a product name when the destination helps the reader.
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
**/*.{md,rst,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,rst,txt}: SpellNVIDIAin all caps. Do not useNvidia,nvidia, orNV.
Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text. Avoid raw URLs and weak anchors such as 'here' or 'read more.'
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative steps. Keep steps parallel and split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce.
Preferrefer tooverseewhen the wording points readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical documentation.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values.
Use numerals for 10 or greater and include commas in thousands.
Do not add trademark symbols to learning-oriented documentation unless the source, platform, or legal guidance explicitly requires them.
Do not add trademark symbols to NeMo Flow learning documentation by default.
Do not rewrite API names, package names, command flags, or code literals for style reasons.
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
**/*.{md,markdown,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-technical-docs.md)
**/*.{md,markdown,rst}: Use title case consistently in technical documentation headings
Avoid quotation marks, ampersands, and exclamation marks in headings
Keep product, event, research, and whitepaper names in their official title case
Use title case for table headers
Do not force social-media sentence case into technical docs
Use monospace formatting for code elements, commands, parameters, package names, and expressions
Use monospace formatting for directories, file names, and paths
Use angle brackets inside monospace for variables inside paths, such as/home/<username>/.login
Use quotation marks for error messages and strings in documentation
Use bold formatting for UI buttons, menus, fields, and labels in documentation
Use angle brackets between UI labels for menu paths, such as File > Save As
Use italics for new terms on first use in documentation
Use italics for publication titles in documentation
Use plain text formatting for keyboard shortcuts in documentation
Prefer[NVIDIA/NeMo](link)format for GitHub repository references over generic phrases like 'the GitHub repo'
Introduce every code block with a complete sentence
Do not make a code block complete the grammar of the previous sentence
Do not continue a sentence after a code block
Use syntax highlighting when the format supports it for code blocks
Avoid the word 'snippet' unless the surrounding docs already use it as a term of art
Keep inline method, function, and class references consistent with nearby docs, omitting empty parentheses for prose readability when no call is shown
Use descriptive anchor text that matches the destination title when possible for links
Avoid raw URLs in running text in documentation
Avoid generic link anchors such as 'here,' 'this page,' and 'read more' in documentation
Include the acronym in link text if a linked term includes an acronym
Do not link long sentences or multiple sentences in documentation
Avoid links that pull readers away from a procedure unles...
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
**/*.{html,md}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Include SPDX license headers in HTML and Markdown files using HTML comment syntax
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
docs/**/*.md
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run
./scripts/build-docs.shfor documentation site changes
docs/**/*.md: Relevant getting-started or reference docs must be updated when examples change
Release-policy docs must point to GitHub Releases as the only release-history source of truth
docs/**/*.md: Use title case for headings in technical documentation
Introduce code blocks, tables, and lists with complete lead-in sentences in documentation
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
**/*.md
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run Markdown link checking via
lycheeforREADME.md,CONTRIBUTING.md, anddocs/through pre-commit hooks
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
**/*.{md,markdown,py,sh,bash,js,ts,java,cpp,go,rust}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep package names, repo references, and build commands current in documentation
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
{RELEASING.md,CHANGELOG.md,docs/**/*.md}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep release-process and release-notes guidance in repo-maintainer docs such as RELEASING.md, not as user-facing docs pages or CHANGELOG.md
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
**/*.{md,markdown,py,sh,bash}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep stable user-facing wrappers at scripts/ root in docs and examples; only point at namespaced helper paths when documenting internal maintenance work
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
**/*.{md,markdown,py,sh,bash,js,ts,example}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Example commands must match current package names and paths
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
{scripts/*.sh,docs/**/*.md}
📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)
Use root
./scripts/*.shcommands in docs and contributor guidance as documented, with implementations underscripts/third-party/
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
{docs/**,examples/**,crates/adaptive/**,python/nemo_flow/**,go/nemo_flow/**,**/{example,component}.{ts,tsx,js,rs,py,go}}
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Any new adaptive component kind must have documentation, examples, and binding coverage across all supported languages
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
{README*,CHANGELOG*,docs/**/*.{md,rst,txt},examples/**/*,*.md}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update documentation, examples, and getting-started guides with new package/module/crate names after rename operations
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
**/*.{md,txt,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
**/*.{md,txt,rst}: Ensure commands, package names, file paths, and APIs in documentation are correct and not stale; flag incorrect or outdated information as blocking issues
Ensure examples and procedures in documentation will execute successfully with current APIs and commands
Use consistent user-facing terminology throughout documentation that matches current repo terminology
Capitalize NVIDIA correctly in all documentation and public-facing text
Format code, commands, paths, and filenames as inline code (monospace) in documentation
Use descriptive anchor text for links instead of bare URLs or weak labels like 'here' in documentation
Prefer active voice, present tense, short sentences, and plain English in documentation
Structure documentation procedures as imperative steps that are easy to scan and not too long for a single sequence
Prefer 'after' instead of 'once' for temporal references in documentation
Use 'can' instead of 'may' when describing possibility (rather than permission) in documentation
Avoid ambiguous numeric dates and ordinal dates in documentation body text
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
{README.md,docs/**/*.md,examples/**/*.{js,ts,py,go,rs}}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Keep documentation and examples synchronized with current install, import, and build commands
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
**/*.{py,js,ts,tsx,go,rs,md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Format changed files with the language-native formatter before the final lint/test pass
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.mdcrates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rs
{README.md,CONTRIBUTING.md,docs/**/*.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
For docs-only changes, run targeted checks only if commands, package names, or examples changed. Use
just docsfor docs-site builds andjust docs-linkcheckwhen links changed
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.
Files:
docs/plugins/observability/atif.mddocs/plugins/observability/opentelemetry.mddocs/plugins/observability/openinference.mddocs/plugins/observability/about.md
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-ffi-surface/SKILL.md)
**/*.rs: Runcargo fmt --allfor FFI work as it is Rust work
Runjust test-rustfor FFI validation
Runcargo clippy --workspace --all-targets -- -D warningsto enforce warnings-as-errors linting
**/*.rs: Runcargo fmt --allfor Rust code formatting
Runcargo clippy --workspace --all-targets -- -D warningsto enforce Rust linting with no warnings
Runjust test-rustas the shared-runtime build/test wrapper for Rust changesUse Rust snake_case naming convention for Rust code
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor Rust code formatting when Node changes touch Rust files
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting when Rust files changed as part of Node work
**/*.rs: Always runjust test-rustwhen any Rust code changes
Always runcargo fmt --allwhen any Rust code changes
Always runcargo clippy --workspace --all-targets -- -D warningswhen any Rust code changesIf any Rust files changed as part of the Python work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rs
**/*.{rs,go,js,ts,tsx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Include SPDX license headers in all Rust, Go, JavaScript, and TypeScript source files using C-style comment syntax
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rs
**/*.{rs,py,go,js,ts,tsx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use
SONAR_IGNORE_START/SONAR_IGNORE_ENDmarkers only for documented false positives that cannot be resolved in code; keep ignored blocks small, add explanatory comments, and require reviewer sign-off
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rs
**/*.{js,ts,tsx,jsx,py,rs,go,java,c,cpp,h,cc,cxx,cs,rb,php,swift,kt}
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changed files must be formatted with the language-native formatter
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rs
crates/{core,adaptive}/**
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rs
{crates/core,crates/adaptive}/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-wasm-binding/SKILL.md)
If the change touched shared runtime semantics in
crates/coreorcrates/adaptive, also usevalidate-change
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rs
crates/core/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
When
crates/corechanges, run the full validation matrix across Rust, Python, Go, Node.js, and WebAssembly
crates/core/**/*.rs: UseJson = serde_json::Valuein Rust-facing runtime APIs where the existing code expects JSON payloads.
UseResult<T>withFlowErrorin core runtime paths. Keep errors explicit and binding-appropriate at the wrapper layer.
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rs
**/*.{rs,py,js,ts,tsx,go}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
During iteration, prefer
uv run pre-commit run --files <changed files...>for targeted validation
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rs
crates/{core,adaptive}/**/*.rs
⚙️ CodeRabbit configuration file
crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rs
{crates/**/tests/**,python/tests/**,go/nemo_flow/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_flow/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
crates/core/tests/unit/observability/openinference_tests.rscrates/core/tests/unit/observability/otel_tests.rs
🔇 Additional comments (6)
crates/core/tests/unit/observability/openinference_tests.rs (1)
18-18: LGTM!crates/core/tests/unit/observability/otel_tests.rs (1)
17-17: LGTM!docs/plugins/observability/about.md (1)
63-75: LGTM!docs/plugins/observability/atif.md (1)
63-67: LGTM!docs/plugins/observability/openinference.md (1)
69-76: LGTM!docs/plugins/observability/opentelemetry.md (1)
68-72: LGTM!
|
/merge |
2 participants
Overview
Adds regression coverage and plugin documentation proving ATIF lineage can be correlated with exported OpenTelemetry and OpenInference spans through the shared NeMo Flow UUID attributes.
Details
nemo_flow.uuidandnemo_flow.parent_uuidmatch ATIFsession_idandstep.extra.ancestry.trace_idandspan_idvalues are not written into ATIF; the supported join keys are NeMo Flow UUID attributes.Validation run:
cargo test -p nemo-flow atif_lineage_correlates_with_otel_span_attributescargo test -p nemo-flow observability::otel::testscargo test -p nemo-flow atif_lineage_correlates_with_openinference_span_attributescargo test -p nemo-flow observability::openinference::testscargo fmt --alljust test-rustcargo clippy --workspace --all-targets -- -D warningsjust docsWhere should the reviewer start?
Start with
docs/plugins/observability/about.mdfor the documented correlation contract, then compare the assertions incrates/core/tests/unit/observability/otel_tests.rsandcrates/core/tests/unit/observability/openinference_tests.rs.Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Summary by CodeRabbit
New Features
Documentation