Observability user experiences fixes

[dnandakumar-nv]

Description

Improve export task handling and update LangSmith docs

Handle race conditions during export task creation, closing coroutines and logging warnings instead of propagating RuntimeErrors. Enhance test coverage for handling shutdown scenarios.

Additionally, update documentation to clarify LangChain dependencies, provide usage of --override flag for configuration, and correct environment variable naming for LangFuse.

By Submitting this PR I confirm:

• I am familiar with the Contributing Guidelines.
• We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  • Any contribution which contains commits that are not Signed-Off will not be accepted.
• When the PR is ready for review, new or existing tests cover these changes.
• When the PR is ready for review, the documentation is up to date with these changes.

Summary by CodeRabbit

• Documentation

  • Renamed step reference to LangChain and updated install instructions; corrected Langfuse env var name.
  • Added command-line override examples and a note describing the override flag behavior.
  • Expanded structured evaluation experiments with prerequisites and guidance.

• Bug Fixes

  • Improved observability export task shutdown/error handling to suppress coroutine warnings and log non-fatal issues.

• Tests

  • Added/updated tests to verify error handling, warning logs, and coroutine/resource cleanup.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Walkthrough

Renames LangSmith references to LangChain in docs, adds CLI --override examples and evaluation prerequisites, corrects a Langfuse env var in an example README, and updates ProcessingExporter to close coroutines when not running and log (not raise) on RuntimeError from task creation; tests updated accordingly.

Changes

Cohort / File(s)	Summary
Documentation docs/source/run-workflows/observe/observe-workflow-with-langsmith.md, examples/observability/simple_calculator_observability/README.md	Renamed Step 1 to reference the LangChain subpackage and updated installation command; added CLI --override dot-notation usage and note; added Structured Evaluation Experiments prerequisites/notes; corrected Langfuse env var from LANGFUSE_HOST to LANGFUSE_BASE_URL.
Observability Exporter Implementation packages/nvidia_nat_core/src/nat/observability/exporter/processing_exporter.py	In _create_export_task, explicitly close the coroutine when exporter is not running; catch RuntimeError from asyncio.create_task (e.g., loop shutdown), close coroutine, and log a warning instead of raising.
Observability Exporter Tests packages/nvidia_nat_core/tests/nat/observability/exporter/test_processing_exporter.py	Updated tests to assert coroutine closure when exporter not running and to expect a warning (not an exception) when asyncio.create_task fails due to loop shutdown; adjusted log/message expectations and resource-cleanup assertions.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

---

Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'Observability user experiences fixes' is vague and generic, using non-descriptive terms that don't convey specific information about the changes.	Make the title more specific by describing the main change, such as 'Fix observability export task handling and race conditions' or 'Handle coroutine cleanup and loop shutdown in observability exporter'.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[dnandakumar-nv]

/ok to test 5464fdd

[mnajafian-nv]

LGTM! Conditional approval upon addressing the inline NITs

[dnandakumar-nv]

/ok to test 29196a3

[dnandakumar-nv]

/ok to test a40a28e

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In
`@packages/nvidia_nat_core/tests/nat/observability/exporter/test_processing_exporter.py`:
- Around line 1113-1123: Replace the Mock() coroutine placeholders in the
failing tests with a real coroutine object (e.g. create a simple async def
_dummy(): await asyncio.sleep(0) and instantiate it with _dummy()) before
passing it to processing_exporter._create_export_task; keep the
asyncio.create_task patch that raises RuntimeError so the internal exception
path is exercised, then assert the coroutine was properly closed by checking
cr.cr_frame is None (instead of only verifying .close() was called). Apply this
change to both test cases that call _create_export_task (the shutdown case and
the "not running" case) so the test verifies actual coroutine cleanup rather
than just a Mock.close call.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 11bb6114-d879-43aa-b832-d55c774f4a50

📥 Commits

Reviewing files that changed from the base of the PR and between 29196a3 and a40a28e.

📒 Files selected for processing (1)

• packages/nvidia_nat_core/tests/nat/observability/exporter/test_processing_exporter.py

[dnandakumar-nv]

/merge