Skip to content

Conversation

@willkill07
Copy link
Member

@willkill07 willkill07 commented Oct 17, 2025

Description

Update all reasoning model references to Nemotron.

Also clean up the prompt descriptions for the reasoning agent documentation.

Workaround for #1034

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
    • Any contribution which contains commits that are not Signed-Off will not be accepted.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

Summary by CodeRabbit

  • Documentation

    • Updated reasoning agent documentation with clarified requirements for LLM thinking capabilities and restructured prompt templates with parameterized placeholders.
  • New Features

    • Enhanced example configurations with reasoning capabilities and expanded test coverage for reasoning workflows.
  • Chores

    • Updated example configurations across multiple projects to use updated model references and enabled thinking support.
    • Cleaned up commented configuration options in example files.

Signed-off-by: Will Killian <wkillian@nvidia.com>
@willkill07 willkill07 requested a review from a team as a code owner October 17, 2025 22:07
@willkill07 willkill07 added feature request New feature or request non-breaking Non-breaking change labels Oct 17, 2025
@coderabbitai
Copy link

coderabbitai bot commented Oct 17, 2025

Walkthrough

The PR systematically migrates reasoning agent examples from DeepSeek R1 model to NVIDIA Nemotron model across multiple configuration files, updates the related documentation with reworked prompt templates using structured placeholders, and expands test coverage to include new reasoning workflow configurations.

Changes

Cohort / File(s) Summary
Documentation updates
docs/source/workflows/about/reasoning-agent.md
Updates reasoning agent documentation: swaps YAML example from deepseek_r1_model to nemotron_model, adds requirement that LLM must support thinking tags, reworks reasoning_prompt_template and instruction_prompt_template to use structured placeholders ({augmented_function_desc}, {input_text}, {tools}, {plan}, {reasoning_output}).
Configuration migrations to Nemotron
examples/agents/react/configs/config-reasoning.yml, examples/agents/tool_calling/configs/config-reasoning.yml, examples/evaluation_and_profiling/email_phishing_analyzer/src/nat_email_phishing_analyzer/configs/config-reasoning.yml, examples/getting_started/simple_calculator/src/nat_simple_calculator/configs/config-reasoning.yml
Renames r1_model to nemotron_model in llms section, updates model_name from deepseek-ai/deepseek-r1 to nvidia/llama-3.3-nemotron-super-49b-v1, adds thinking: true flag, and updates workflow llm_name reference to nemotron_model.
Configuration cleanup
examples/HITL/por_to_jiratickets/src/nat_por_to_jiratickets/configs/config.yml
Removes commented alternative model_name entries under extract_llm and agent_llm; active model_name values remain unchanged.
Test updates
examples/agents/tests/test_agents.py
Adds config-reasoning.yml configurations for react and tool_calling workflows to parametrized test; expands test IDs from ["mixture_of_agents", "react", "tool_calling"] to ["mixture_of_agents", "react", "react-reasoning", "tool_calling", "tool_calling-reasoning"].

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

The changes follow a consistent, repetitive pattern across multiple configuration files (model replacement and flag addition), but the documentation rework requires careful review of the new prompt template structure and parameterization approach. The configuration changes are homogeneous and low-complexity, while the documentation changes introduce new structured placeholders that warrant closer attention.

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The pull request title "feat: switch to nemotron reasoning models" directly and accurately describes the main change in the changeset. The PR systematically updates multiple configuration files and documentation to replace DeepSeek R1 model references with Nemotron model references, and the title captures this core objective. The title is concise at 41 characters (well under the ~72 character limit), uses imperative mood with the verb "switch," and maintains clear descriptive language about what is being changed. The title is fully related to the primary purpose of the changeset.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 143757f and 77873a4.

📒 Files selected for processing (7)
  • docs/source/workflows/about/reasoning-agent.md (2 hunks)
  • examples/HITL/por_to_jiratickets/src/nat_por_to_jiratickets/configs/config.yml (0 hunks)
  • examples/agents/react/configs/config-reasoning.yml (2 hunks)
  • examples/agents/tests/test_agents.py (1 hunks)
  • examples/agents/tool_calling/configs/config-reasoning.yml (2 hunks)
  • examples/evaluation_and_profiling/email_phishing_analyzer/src/nat_email_phishing_analyzer/configs/config-reasoning.yml (2 hunks)
  • examples/getting_started/simple_calculator/src/nat_simple_calculator/configs/config-reasoning.yml (1 hunks)
💤 Files with no reviewable changes (1)
  • examples/HITL/por_to_jiratickets/src/nat_por_to_jiratickets/configs/config.yml
🧰 Additional context used
📓 Path-based instructions (10)
**/*.{yaml,yml}

📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)

In workflow/config YAML, set llms.._type: nat_test_llm to stub responses.

Files:

  • examples/evaluation_and_profiling/email_phishing_analyzer/src/nat_email_phishing_analyzer/configs/config-reasoning.yml
  • examples/getting_started/simple_calculator/src/nat_simple_calculator/configs/config-reasoning.yml
  • examples/agents/tool_calling/configs/config-reasoning.yml
  • examples/agents/react/configs/config-reasoning.yml
**/*.{py,yaml,yml}

📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)

**/*.{py,yaml,yml}: Configure response_seq as a list of strings; values cycle per call, and [] yields an empty string.
Configure delay_ms to inject per-call artificial latency in milliseconds for nat_test_llm.

Files:

  • examples/evaluation_and_profiling/email_phishing_analyzer/src/nat_email_phishing_analyzer/configs/config-reasoning.yml
  • examples/agents/tests/test_agents.py
  • examples/getting_started/simple_calculator/src/nat_simple_calculator/configs/config-reasoning.yml
  • examples/agents/tool_calling/configs/config-reasoning.yml
  • examples/agents/react/configs/config-reasoning.yml
**/configs/**

📄 CodeRabbit inference engine (.cursor/rules/general.mdc)

Configuration files consumed by code must be stored next to that code in a configs/ folder

Files:

  • examples/evaluation_and_profiling/email_phishing_analyzer/src/nat_email_phishing_analyzer/configs/config-reasoning.yml
  • examples/getting_started/simple_calculator/src/nat_simple_calculator/configs/config-reasoning.yml
  • examples/agents/tool_calling/configs/config-reasoning.yml
  • examples/agents/react/configs/config-reasoning.yml
**/*

⚙️ CodeRabbit configuration file

**/*: # Code Review Instructions

  • Ensure the code follows best practices and coding standards. - For Python code, follow
    PEP 20 and
    PEP 8 for style guidelines.
  • Check for security vulnerabilities and potential issues. - Python methods should use type hints for all parameters and return values.
    Example:
    def my_function(param1: int, param2: str) -> bool:
        pass
  • For Python exception handling, ensure proper stack trace preservation:
    • When re-raising exceptions: use bare raise statements to maintain the original stack trace,
      and use logger.error() (not logger.exception()) to avoid duplicate stack trace output.
    • When catching and logging exceptions without re-raising: always use logger.exception()
      to capture the full stack trace information.

Documentation Review Instructions - Verify that documentation and comments are clear and comprehensive. - Verify that the documentation doesn't contain any TODOs, FIXMEs or placeholder text like "lorem ipsum". - Verify that the documentation doesn't contain any offensive or outdated terms. - Verify that documentation and comments are free of spelling mistakes, ensure the documentation doesn't contain any

words listed in the ci/vale/styles/config/vocabularies/nat/reject.txt file, words that might appear to be
spelling mistakes but are listed in the ci/vale/styles/config/vocabularies/nat/accept.txt file are OK.

Misc. - All code (except .mdc files that contain Cursor rules) should be licensed under the Apache License 2.0,

and should contain an Apache License 2.0 header comment at the top of each file.

  • Confirm that copyright years are up-to date whenever a file is changed.

Files:

  • examples/evaluation_and_profiling/email_phishing_analyzer/src/nat_email_phishing_analyzer/configs/config-reasoning.yml
  • examples/agents/tests/test_agents.py
  • examples/getting_started/simple_calculator/src/nat_simple_calculator/configs/config-reasoning.yml
  • docs/source/workflows/about/reasoning-agent.md
  • examples/agents/tool_calling/configs/config-reasoning.yml
  • examples/agents/react/configs/config-reasoning.yml
examples/**/*

⚙️ CodeRabbit configuration file

examples/**/*: - This directory contains example code and usage scenarios for the toolkit, at a minimum an example should
contain a README.md or file README.ipynb.

  • If an example contains Python code, it should be placed in a subdirectory named src/ and should
    contain a pyproject.toml file. Optionally, it might also contain scripts in a scripts/ directory.
  • If an example contains YAML files, they should be placed in a subdirectory named configs/. - If an example contains sample data files, they should be placed in a subdirectory named data/, and should
    be checked into git-lfs.

Files:

  • examples/evaluation_and_profiling/email_phishing_analyzer/src/nat_email_phishing_analyzer/configs/config-reasoning.yml
  • examples/agents/tests/test_agents.py
  • examples/getting_started/simple_calculator/src/nat_simple_calculator/configs/config-reasoning.yml
  • examples/agents/tool_calling/configs/config-reasoning.yml
  • examples/agents/react/configs/config-reasoning.yml
**/*.py

📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)

**/*.py: Programmatic use: create TestLLMConfig(response_seq=[...], delay_ms=...), add with builder.add_llm("", cfg).
When retrieving the test LLM wrapper, use builder.get_llm(name, wrapper_type=LLMFrameworkEnum.) and call the framework’s method (e.g., ainvoke, achat, call).

**/*.py: In code comments/identifiers use NAT abbreviations as specified: nat for API namespace/CLI, nvidia-nat for package name, NAT for env var prefixes; do not use these abbreviations in documentation
Follow PEP 20 and PEP 8; run yapf with column_limit=120; use 4-space indentation; end files with a single trailing newline
Run ruff check --fix as linter (not formatter) using pyproject.toml config; fix warnings unless explicitly ignored
Respect naming: snake_case for functions/variables, PascalCase for classes, UPPER_CASE for constants
Treat pyright warnings as errors during development
Exception handling: use bare raise to re-raise; log with logger.error() when re-raising to avoid duplicate stack traces; use logger.exception() when catching without re-raising
Provide Google-style docstrings for every public module, class, function, and CLI command; first line concise and ending with a period; surround code entities with backticks
Validate and sanitize all user input, especially in web or CLI interfaces
Prefer httpx with SSL verification enabled by default and follow OWASP Top-10 recommendations
Use async/await for I/O-bound work; profile CPU-heavy paths with cProfile or mprof before optimizing; cache expensive computations with functools.lru_cache or external cache; leverage NumPy vectorized operations when beneficial

Files:

  • examples/agents/tests/test_agents.py
examples/*/tests/**/*.py

📄 CodeRabbit inference engine (.cursor/rules/general.mdc)

Example tests live under examples//tests/

Files:

  • examples/agents/tests/test_agents.py
{tests/**/*.py,examples/*/tests/**/*.py}

📄 CodeRabbit inference engine (.cursor/rules/general.mdc)

{tests/**/*.py,examples/*/tests/**/*.py}: Use pytest (with pytest-asyncio for async); name test files test_*.py; test functions start with test_; extract repeated code into fixtures; fixtures must set name in decorator and be named with fixture_ prefix
Mock external services with pytest_httpserver or unittest.mock; do not hit live endpoints
Mark expensive tests with @pytest.mark.slow or @pytest.mark.integration

Files:

  • examples/agents/tests/test_agents.py
docs/source/**/*.md

📄 CodeRabbit inference engine (.cursor/rules/general.mdc)

docs/source/**/*.md: Use the official naming throughout documentation: first use “NVIDIA NeMo Agent toolkit”, subsequent “NeMo Agent toolkit”; never use deprecated names (Agent Intelligence toolkit, aiqtoolkit, AgentIQ, AIQ/aiq)
Documentation sources are Markdown files under docs/source; images belong in docs/source/_static
Keep docs in sync with code; documentation pipeline must pass Sphinx and link checks; avoid TODOs/FIXMEs/placeholders; avoid offensive/outdated terms; ensure spelling correctness
Do not use words listed in ci/vale/styles/config/vocabularies/nat/reject.txt; accepted terms in accept.txt are allowed

Files:

  • docs/source/workflows/about/reasoning-agent.md
docs/source/**/*

⚙️ CodeRabbit configuration file

This directory contains the source code for the documentation. All documentation should be written in Markdown format. Any image files should be placed in the docs/source/_static directory.

Files:

  • docs/source/workflows/about/reasoning-agent.md
🪛 LanguageTool
docs/source/workflows/about/reasoning-agent.md

[style] ~56-~56: Consider replacing this word to strengthen your wording.
Context: ...son on. The function should be an agent and must be defined in the config YAML. * ...

(AND_THAT)

🔇 Additional comments (10)
examples/evaluation_and_profiling/email_phishing_analyzer/src/nat_email_phishing_analyzer/configs/config-reasoning.yml (1)

63-79: Model migration to Nemotron is correct and well-aligned.

The transition from DeepSeek R1 to NVIDIA Nemotron is cleanly implemented:

  • New nemotron_model LLM entry properly configured with thinking: true flag for reasoning capability
  • Model name format is consistent with NIM conventions
  • Workflow reference correctly updated to use the new model
  • Apache License header and copyright year (2024-2025) are current
  • File location in configs/ subdirectory complies with guidelines
examples/agents/tool_calling/configs/config-reasoning.yml (1)

23-28: LGTM! Clean migration to Nemotron model.

The model configuration correctly updates to the NVIDIA Nemotron reasoning model with the thinking: true flag enabled, and the workflow reference is properly updated to match.

examples/agents/tests/test_agents.py (1)

70-82: LGTM! Test coverage appropriately expanded.

The parametrization correctly includes the new reasoning configuration files, and the test IDs clearly distinguish between standard and reasoning variants.

docs/source/workflows/about/reasoning-agent.md (3)

40-46: LGTM! Example updated to reference Nemotron model.

The configuration example correctly reflects the migration from DeepSeek to Nemotron for reasoning workflows.


52-52: LGTM! Important requirement documented.

The addition clarifies that the LLM must support thinking tags, which is essential for reasoning agents.


89-101: LGTM! Instruction prompt template properly structured.

The updated prompt template uses clear placeholders and maintains the requirement for direct user-facing responses.

examples/agents/react/configs/config-reasoning.yml (2)

23-28: LGTM! Consistent migration to Nemotron model.

The configuration correctly updates to match the Nemotron reasoning model pattern used across other reasoning configs.


49-53: LGTM! Workflow reference properly updated.

The reasoning agent workflow correctly references the renamed nemotron_model LLM.

examples/getting_started/simple_calculator/src/nat_simple_calculator/configs/config-reasoning.yml (2)

65-70: LGTM! Simple calculator example updated consistently.

The Nemotron model configuration follows the same pattern established in the other reasoning configs.


72-76: LGTM! Workflow configuration properly updated.

The reasoning agent workflow correctly references the nemotron_model LLM.

@willkill07
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit 191af04 into NVIDIA:release/1.3 Oct 17, 2025
17 checks passed
bbednarski9 pushed a commit to bbednarski9/NeMo-Agent-Toolkit that referenced this pull request Oct 20, 2025
Update all reasoning model references to Nemotron.

Also clean up the prompt descriptions for the reasoning agent documentation.

Workaround for NVIDIA#1034

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.




## Summary by CodeRabbit

* **Documentation**
  * Updated reasoning agent documentation with clarified requirements for LLM thinking capabilities and restructured prompt templates with parameterized placeholders.

* **New Features**
  * Enhanced example configurations with reasoning capabilities and expanded test coverage for reasoning workflows.

* **Chores**
  * Updated example configurations across multiple projects to use updated model references and enabled thinking support.
  * Cleaned up commented configuration options in example files.

Authors:
  - Will Killian (https://github.com/willkill07)

Approvers:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

URL: NVIDIA#1036
@willkill07 willkill07 deleted the wkk_update-reasoning-models-to-nemotron branch October 23, 2025 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature request New feature or request non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants