Skip to content

feat(core): add opt-in provider hooks for generated ids and timestamps - #2114

Merged
rapids-bot[bot] merged 3 commits into
NVIDIA:developfrom
DABH:deterministic-id-time-providers
Jul 18, 2026
Merged

feat(core): add opt-in provider hooks for generated ids and timestamps#2114
rapids-bot[bot] merged 3 commits into
NVIDIA:developfrom
DABH:deterministic-id-time-providers

Conversation

@DABH

@DABH DABH commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Description

The runtime currently stamps workflow runs, intermediate steps, spans, and function invocations with uuid.uuid4() and time.time() values called inline at each site (runtime/runner.py, builder/context.py, data_models/intermediate_step.py, data_models/span.py). Two otherwise identical runs therefore never produce the same identifiers or timings. That gets in the way of several third-party and testing use cases:

  • Record or replay style testing, where a test harness replays a captured run and asserts on stable step identifiers.
  • Golden-file trace comparison and reproducible traces in CI.
  • Integrations with runtimes that re-execute workflow code (for example plugins that relocate or instrument function execution) and need identifiers to remain stable across re-executions, so steps and spans correlate across systems.

This PR adds a small additive module, nat.utils.providers, with process-wide, opt-in hooks:

  • set_id_provider / get_id_provider — a zero-argument callable returning a new identifier as a canonical UUID string (default: uuid.uuid4).
  • set_time_provider / get_time_provider — a zero-argument callable returning the current time in fractional seconds (default: time.time).
  • Derivation helpers used by the runtime: generate_id, generate_trace_id (128-bit), generate_span_id (64-bit), current_time, and current_time_ns. Integer identifiers are derived by parsing the id provider's UUID string, so the default distribution is identical to uuid.uuid4().int.

The existing call sites are routed through these hooks:

  • Runner workflow run id, trace id, and workflow step id (single and streaming paths).
  • Context.push_active_function invocation id.
  • IntermediateStepPayload UUID and event_timestamp default factories.
  • Span/SpanContext/SpanEvent trace ids, span ids, and start/end/event timestamps.

Pydantic default_factory sites reference the module-level functions, so the installed provider is resolved lazily at model construction time rather than captured at import.

Zero behavior change by default: when no provider is installed, the defaults are uuid.uuid4 and time.time, exactly as before. Setters return the previously installed provider so callers can restore it. The hook is documented briefly in the observability guide, and the module is picked up by the auto-generated API reference.

No tracking issue exists for this yet; happy to file one if the team prefers.

Testing

  • New packages/nvidia_nat_core/tests/nat/utils/test_providers.py (8 tests): default provider behavior, setter install/restore semantics, UUID-string contract for integer id derivation, lazy resolution through Pydantic default factories, span model injection, push_active_function injection, and an end-to-end minimal Runner run (single and streaming) asserting deterministic run ids, trace ids, step UUIDs, and timestamps.
  • Full core suite: uv run pytest packages/nvidia_nat_core — 2725 passed, 54 skipped (defaults unchanged).
  • pre-commit run yapf|ruff-check --files <touched files>, python ci/scripts/copyright.py --verify-apache-v2, vale on the touched Markdown, and python ci/scripts/path_checks.py all pass.

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
    • Any contribution which contains commits that are not Signed-Off will not be accepted.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

Summary by CodeRabbit

  • New Features

    • Added process-wide, opt-in hooks to customize workflow/function identifiers and timestamps.
    • Workflow runs, spans, intermediate steps, and active function invocations now use the installed providers for deterministic observability values.
    • Added validation to reject provider outputs that produce invalid/zero-derived trace and span identifiers.
  • Documentation

    • Documented how to configure, restore, and override default ID/timestamp providers, including deterministic examples and ID derivation behavior.
  • Tests

    • Added deterministic test coverage for default behavior, provider installation/restoration, validation failures, and end-to-end runtime output.

@copy-pr-bot

copy-pr-bot Bot commented Jul 14, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4ac30522-0657-4fe6-b259-7389d7ef7739

📥 Commits

Reviewing files that changed from the base of the PR and between b4b72a8 and 7f819a4.

📒 Files selected for processing (1)
  • packages/nvidia_nat_core/src/nat/utils/providers.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/nvidia_nat_core/src/nat/utils/providers.py

Walkthrough

Process-wide providers now control runtime-generated identifiers and timestamps. Workflow execution, contexts, intermediate-step payloads, and spans use these helpers, with deterministic behavior documented and covered by provider propagation tests.

Changes

Deterministic runtime providers

Layer / File(s) Summary
Provider contract and documentation
packages/nvidia_nat_core/src/nat/utils/providers.py, docs/source/run-workflows/observe/observe.md
Adds configurable ID/time providers, UUID-derived trace/span helpers with zero-value validation, nanosecond conversion, restoration APIs, and usage documentation.
Runtime identifier and timestamp integration
packages/nvidia_nat_core/src/nat/builder/context.py, packages/nvidia_nat_core/src/nat/data_models/intermediate_step.py, packages/nvidia_nat_core/src/nat/data_models/span.py, packages/nvidia_nat_core/src/nat/runtime/runner.py
Routes function, workflow, intermediate-step, trace, span, and timestamp generation through the provider helpers.
Provider propagation tests
packages/nvidia_nat_core/tests/nat/utils/test_providers.py
Tests default and custom providers, invalid derived IDs, lazy model defaults, context and span propagation, and deterministic runner payloads.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Runner
  participant Providers
  participant IntermediateStepPayload
  participant Context
  Runner->>Providers: generate workflow ID and trace ID
  Providers-->>Runner: return configured identifiers
  Runner->>IntermediateStepPayload: create workflow start/end payloads
  IntermediateStepPayload->>Providers: generate payload ID and timestamp
  Runner->>Context: store workflow run and trace identifiers
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise, descriptive, and uses imperative mood to summarize the provider-hook changes.
Docstring Coverage ✅ Passed Docstring coverage is 91.67% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

The runtime stamps workflow runs, intermediate steps, spans, and
function invocations with uuid.uuid4() and time.time() values called
inline at each site. That makes two otherwise identical runs produce
different identifiers and timings, which hurts record/replay style
testing, golden-file trace comparison, and integrations with runtimes
that re-execute workflow code and need identifiers to remain stable
across re-executions.

Add nat.utils.providers with process-wide, opt-in id and time provider
hooks defaulting to uuid.uuid4/time.time, and route the runtime call
sites through them:

* Runner workflow run id, trace id, and workflow step id (single and
  streaming paths)
* Context.push_active_function invocation id
* IntermediateStepPayload UUID and event_timestamp default factories
* Span, SpanContext, and SpanEvent trace ids, span ids, and timestamps

Pydantic default factories reference the module-level functions so the
installed provider is resolved lazily at model construction time.
Behavior is unchanged when no provider is installed.

Signed-off-by: David Hyde <DABH@users.noreply.github.com>
@DABH
DABH force-pushed the deterministic-id-time-providers branch from 057e30b to f4fafaf Compare July 14, 2026 06:00
@DABH
DABH marked this pull request as ready for review July 14, 2026 06:31
@DABH
DABH requested a review from a team as a code owner July 14, 2026 06:31

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/nvidia_nat_core/tests/nat/utils/test_providers.py (1)

42-51: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use the required explicit fixture name and fixture suffix.

Rename the implementation to restore_providers_fixture and set name="restore_providers" on the decorator. As per coding guidelines, “Pytest fixtures should define the name argument” and fixture functions must use a fixture_ prefix or _fixture suffix.

Proposed fix
-@pytest.fixture(autouse=True)
-def restore_providers():
+@pytest.fixture(name="restore_providers", autouse=True)
+def restore_providers_fixture():
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/nvidia_nat_core/tests/nat/utils/test_providers.py` around lines 42 -
51, Rename the fixture function from restore_providers to
restore_providers_fixture and set the decorator’s explicit name to
restore_providers, preserving its autouse behavior and provider restoration
logic.

Sources: Coding guidelines, Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/nvidia_nat_core/src/nat/utils/providers.py`:
- Around line 108-121: Update generate_trace_id and generate_span_id to compute
their derived integer IDs, reject a zero result by raising ValueError, and
return only non-zero IDs. Add coverage for the nil UUID and UUIDs whose high
word is zero, preserving the existing provider-based generation behavior.

---

Nitpick comments:
In `@packages/nvidia_nat_core/tests/nat/utils/test_providers.py`:
- Around line 42-51: Rename the fixture function from restore_providers to
restore_providers_fixture and set the decorator’s explicit name to
restore_providers, preserving its autouse behavior and provider restoration
logic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b11fc0b7-2353-42b9-a001-54a9fbe2894b

📥 Commits

Reviewing files that changed from the base of the PR and between e8692d0 and f4fafaf.

📒 Files selected for processing (7)
  • docs/source/run-workflows/observe/observe.md
  • packages/nvidia_nat_core/src/nat/builder/context.py
  • packages/nvidia_nat_core/src/nat/data_models/intermediate_step.py
  • packages/nvidia_nat_core/src/nat/data_models/span.py
  • packages/nvidia_nat_core/src/nat/runtime/runner.py
  • packages/nvidia_nat_core/src/nat/utils/providers.py
  • packages/nvidia_nat_core/tests/nat/utils/test_providers.py

Comment thread packages/nvidia_nat_core/src/nat/utils/providers.py Outdated
@willkill07 willkill07 added feature request New feature or request non-breaking Non-breaking change labels Jul 15, 2026
@willkill07

Copy link
Copy Markdown
Member

@DABH coderabbit feedback seems reasonable here. No need to address yet.

Otherwise, I'll kick off CI to see how things go. We can defer updates based on the initial CI trigger.

@willkill07

Copy link
Copy Markdown
Member

/ok to test f4fafaf

@willkill07

Copy link
Copy Markdown
Member

@DABH in the documentation can you rephrase "record/replay" to "record or replay"?

Also, i do think addressing the coderabbit feedback should be done.

@DABH

DABH commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

Thanks so much for taking a look!! Yes - will address that change and the coderabbit feedback shortly. Thanks again!

DABH added 2 commits July 15, 2026 19:24
generate_trace_id and generate_span_id now raise ValueError when the
installed id provider yields a UUID whose derived 128-bit trace ID or
64-bit span ID is zero (the nil UUID, or any UUID with an all-zero high
word for span ids), upholding the non-zero contract enforced by
SpanContext instead of propagating an invalid telemetry id. Adds tests
for both shapes, renames the test fixture to follow the fixture naming
guideline, and rephrases record/replay wording in the observability
guide (which also satisfies the docs path checker).

Signed-off-by: David Hyde <DABH@users.noreply.github.com>
Signed-off-by: David Hyde <DABH@users.noreply.github.com>
@DABH

DABH commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

@willkill07 Should be all set - thanks again for taking a look!

@willkill07

Copy link
Copy Markdown
Member

/ok to test 7f819a4

@willkill07

Copy link
Copy Markdown
Member

/merge

@rapids-bot
rapids-bot Bot merged commit c03f3aa into NVIDIA:develop Jul 18, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature request New feature or request non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants