Skip to content

fix: Ensure console front-end validation is called#844

Merged
rapids-bot[bot] merged 1 commit intoNVIDIA:developfrom
willkill07:wkk_front-end-call-pre-check
Sep 24, 2025
Merged

fix: Ensure console front-end validation is called#844
rapids-bot[bot] merged 1 commit intoNVIDIA:developfrom
willkill07:wkk_front-end-call-pre-check

Conversation

@willkill07
Copy link
Member

@willkill07 willkill07 commented Sep 24, 2025

Description

Running nat run without specifying --input or --input_file results in a long error being printed. It turns out we never called pre_run() anywhere, so let's call it before constructing the workflow.

Closes #843

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
    • Any contribution which contains commits that are not Signed-Off will not be accepted.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

Summary by CodeRabbit

  • Bug Fixes
    • Enforced exactly one input source: users must provide either --input or --input_file, not both. Clear, actionable error messages are shown for both the “both provided” and “neither provided” cases.
    • Pre-run checks now execute before the workflow starts, providing immediate validation feedback and preventing workflows from launching with invalid inputs.

Signed-off-by: Will Killian <wkillian@nvidia.com>
@willkill07 willkill07 self-assigned this Sep 24, 2025
@willkill07 willkill07 added the bug Something isn't working label Sep 24, 2025
@willkill07 willkill07 requested a review from a team as a code owner September 24, 2025 14:17
@willkill07 willkill07 added the non-breaking Non-breaking change label Sep 24, 2025
@coderabbitai
Copy link

coderabbitai bot commented Sep 24, 2025

Walkthrough

Added a pre_run invocation at the start of the base front-end run method and refined console plugin input validation to require exactly one of --input or --input_file, with distinct errors for both-provided and neither-provided cases.

Changes

Cohort / File(s) Summary of Changes
Front-end base run flow
src/nat/front_ends/simple_base/simple_front_end_plugin_base.py
Ensures run awaits pre_run before entering WorkflowBuilder and executing the workflow. Control flow now: pre_run → build workflow → run.
Console input validation
src/nat/front_ends/console/console_front_end_plugin.py
Updated pre_run validation to enforce mutual exclusivity between --input and --input_file; added explicit error when both are set and when neither is set; clarified error messages.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant CLI as CLI
  participant FE as FrontEndPlugin.run()
  participant PR as pre_run()
  participant WB as WorkflowBuilder (context)
  participant WF as run_workflow()

  CLI->>FE: invoke run()
  FE->>PR: await pre_run()
  Note right of PR: Validate inputs (exactly one of --input or --input_file)
  PR-->>FE: return or raise error

  alt Validation passes
    FE->>WB: enter context
    WB-->>FE: workflow built
    FE->>WF: run_workflow(session_manager)
    WF-->>FE: complete
  else Validation fails
    PR-->>CLI: error surfaced (no workflow run)
  end
Loading
sequenceDiagram
  autonumber
  participant User as User
  participant PR as ConsoleFrontEnd.pre_run
  Note over PR: Input validation branches

  User->>PR: Provide flags (--input / --input_file)
  alt Both provided
    PR-->>User: Error: provide one input, not both
  else Neither provided
    PR-->>User: Error: must provide either --input or --input_file
  else Exactly one provided
    PR-->>User: OK
  end
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title "fix: Ensure console front-end validation is called" is concise, imperative, and accurately summarizes the primary change of invoking pre_run so console validation occurs; it stays well under the ~72 character guideline and directly relates to the modifications in simple_front_end_plugin_base.py and console_front_end_plugin.py.
Linked Issues Check ✅ Passed The PR calls await self.pre_run() before building the workflow and tightens console input validation (mutual-exclusivity and clearer messages), which directly addresses issue #843 by ensuring pre_run runs and produces meaningful errors instead of the reported AssertionError.
Out of Scope Changes Check ✅ Passed The modifications are limited to two front-end files and are focused on invoking pre_run and improving input validation; there are no edits to unrelated modules or broader refactors that would indicate out-of-scope changes.
✨ Finishing touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
src/nat/front_ends/simple_base/simple_front_end_plugin_base.py (2)

33-36: Add type hints and brief docstrings to public methods.

These are public API hooks; annotate returns and add docstrings per repo guidelines.

-    async def pre_run(self):
-        pass
+    async def pre_run(self) -> None:
+        """Hook for front-end pre-run validation and setup. Override in subclasses."""
+        pass
-
-    async def run(self):
+    async def run(self) -> None:
+        """Build the workflow from config and delegate execution to `run_workflow`."""

55-56: Annotate run_workflow return type and document purpose.

Keep API surface consistently typed and documented.

-    async def run_workflow(self, session_manager: SessionManager):
-        pass
+    async def run_workflow(self, session_manager: SessionManager) -> None:
+        """Execute the workflow using the provided `SessionManager`. Must be implemented by subclasses."""
+        pass
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 483d30c and 9d1fd01.

📒 Files selected for processing (2)
  • src/nat/front_ends/console/console_front_end_plugin.py (1 hunks)
  • src/nat/front_ends/simple_base/simple_front_end_plugin_base.py (1 hunks)
🧰 Additional context used
📓 Path-based instructions (6)
**/*.{py,yaml,yml}

📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)

**/*.{py,yaml,yml}: Configure response_seq as a list of strings; values cycle per call, and [] yields an empty string.
Configure delay_ms to inject per-call artificial latency in milliseconds for nat_test_llm.

Files:

  • src/nat/front_ends/console/console_front_end_plugin.py
  • src/nat/front_ends/simple_base/simple_front_end_plugin_base.py
**/*.py

📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)

**/*.py: Programmatic use: create TestLLMConfig(response_seq=[...], delay_ms=...), add with builder.add_llm("", cfg).
When retrieving the test LLM wrapper, use builder.get_llm(name, wrapper_type=LLMFrameworkEnum.) and call the framework’s method (e.g., ainvoke, achat, call).

**/*.py: In code comments/identifiers use NAT abbreviations as specified: nat for API namespace/CLI, nvidia-nat for package name, NAT for env var prefixes; do not use these abbreviations in documentation
Follow PEP 20 and PEP 8; run yapf with column_limit=120; use 4-space indentation; end files with a single trailing newline
Run ruff check --fix as linter (not formatter) using pyproject.toml config; fix warnings unless explicitly ignored
Respect naming: snake_case for functions/variables, PascalCase for classes, UPPER_CASE for constants
Treat pyright warnings as errors during development
Exception handling: use bare raise to re-raise; log with logger.error() when re-raising to avoid duplicate stack traces; use logger.exception() when catching without re-raising
Provide Google-style docstrings for every public module, class, function, and CLI command; first line concise and ending with a period; surround code entities with backticks
Validate and sanitize all user input, especially in web or CLI interfaces
Prefer httpx with SSL verification enabled by default and follow OWASP Top-10 recommendations
Use async/await for I/O-bound work; profile CPU-heavy paths with cProfile or mprof before optimizing; cache expensive computations with functools.lru_cache or external cache; leverage NumPy vectorized operations when beneficial

Files:

  • src/nat/front_ends/console/console_front_end_plugin.py
  • src/nat/front_ends/simple_base/simple_front_end_plugin_base.py
src/**/*.py

📄 CodeRabbit inference engine (.cursor/rules/general.mdc)

All importable Python code must live under src/ (or packages//src/)

Files:

  • src/nat/front_ends/console/console_front_end_plugin.py
  • src/nat/front_ends/simple_base/simple_front_end_plugin_base.py
src/nat/**/*

📄 CodeRabbit inference engine (.cursor/rules/general.mdc)

Changes in src/nat should prioritize backward compatibility

Files:

  • src/nat/front_ends/console/console_front_end_plugin.py
  • src/nat/front_ends/simple_base/simple_front_end_plugin_base.py

⚙️ CodeRabbit configuration file

This directory contains the core functionality of the toolkit. Changes should prioritize backward compatibility.

Files:

  • src/nat/front_ends/console/console_front_end_plugin.py
  • src/nat/front_ends/simple_base/simple_front_end_plugin_base.py
{src/**/*.py,packages/*/src/**/*.py}

📄 CodeRabbit inference engine (.cursor/rules/general.mdc)

All public APIs must have Python 3.11+ type hints on parameters and return values; prefer typing/collections.abc abstractions; use typing.Annotated when useful

Files:

  • src/nat/front_ends/console/console_front_end_plugin.py
  • src/nat/front_ends/simple_base/simple_front_end_plugin_base.py
**/*

⚙️ CodeRabbit configuration file

**/*: # Code Review Instructions

  • Ensure the code follows best practices and coding standards. - For Python code, follow
    PEP 20 and
    PEP 8 for style guidelines.
  • Check for security vulnerabilities and potential issues. - Python methods should use type hints for all parameters and return values.
    Example:
    def my_function(param1: int, param2: str) -> bool:
        pass
  • For Python exception handling, ensure proper stack trace preservation:
    • When re-raising exceptions: use bare raise statements to maintain the original stack trace,
      and use logger.error() (not logger.exception()) to avoid duplicate stack trace output.
    • When catching and logging exceptions without re-raising: always use logger.exception()
      to capture the full stack trace information.

Documentation Review Instructions - Verify that documentation and comments are clear and comprehensive. - Verify that the documentation doesn't contain any TODOs, FIXMEs or placeholder text like "lorem ipsum". - Verify that the documentation doesn't contain any offensive or outdated terms. - Verify that documentation and comments are free of spelling mistakes, ensure the documentation doesn't contain any

words listed in the ci/vale/styles/config/vocabularies/nat/reject.txt file, words that might appear to be
spelling mistakes but are listed in the ci/vale/styles/config/vocabularies/nat/accept.txt file are OK.

Misc. - All code (except .mdc files that contain Cursor rules) should be licensed under the Apache License 2.0,

and should contain an Apache License 2.0 header comment at the top of each file.

  • Confirm that copyright years are up-to date whenever a file is changed.

Files:

  • src/nat/front_ends/console/console_front_end_plugin.py
  • src/nat/front_ends/simple_base/simple_front_end_plugin_base.py
🧬 Code graph analysis (1)
src/nat/front_ends/simple_base/simple_front_end_plugin_base.py (1)
src/nat/front_ends/console/console_front_end_plugin.py (1)
  • pre_run (57-61)
🪛 Ruff (0.13.1)
src/nat/front_ends/console/console_front_end_plugin.py

59-59: Avoid specifying long messages outside the exception class

(TRY003)


61-61: Avoid specifying long messages outside the exception class

(TRY003)

🔇 Additional comments (2)
src/nat/front_ends/console/console_front_end_plugin.py (1)

58-61: Pre-run validation correctly enforces mutual exclusivity.

LGTM. This prevents the assertion path and yields clear CLI errors when both/neither inputs are provided.

Please confirm we have CLI tests covering:

  • both --input and --input_file provided (expect UsageError)
  • neither provided (expect UsageError)
src/nat/front_ends/simple_base/simple_front_end_plugin_base.py (1)

38-39: Calling pre_run() before building the workflow is correct.

This aligns validation with the intended flow and fixes the unexpected assertion.

@willkill07
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit b3a964c into NVIDIA:develop Sep 24, 2025
17 checks passed
yczhang-nv pushed a commit to yczhang-nv/NeMo-Agent-Toolkit that referenced this pull request Sep 24, 2025
Running `nat run` without specifying `--input` or `--input_file` results in a long error being printed. It turns out we never called `pre_run()` anywhere, so let's call it before constructing the workflow.

Closes NVIDIA#843

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.




## Summary by CodeRabbit

* **Bug Fixes**
  * Enforced exactly one input source: users must provide either --input or --input_file, not both. Clear, actionable error messages are shown for both the “both provided” and “neither provided” cases.
  * Pre-run checks now execute before the workflow starts, providing immediate validation feedback and preventing workflows from launching with invalid inputs.

Authors:
  - Will Killian (https://github.com/willkill07)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#844
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
@willkill07 willkill07 deleted the wkk_front-end-call-pre-check branch October 23, 2025 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nat run reaches an unexpected state: "AssertionError: Should not reach here. Should have been caught by pre_run"

2 participants