Avoid Pydantic serialization warning triggered by tests#925
Avoid Pydantic serialization warning triggered by tests#925rapids-bot[bot] merged 10 commits intoNVIDIA:release/1.3from
Conversation
…to david-test-warnings Signed-off-by: David Gardner <dagardner@nvidia.com>
…to david-test-warnings Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
dagardner-nv
added
improvement
WalkthroughAdds new pytest fixtures for tests and plugins, marks a websocket test as slow and enables running slow tests in CI, configures setuptools-scm versioning in an example project, and refactors an example e2e test to use a fixture-provided OpenSearch URL and direct run_workflow invocation. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Pytest as pytest
participant FX as opensearch_url fixture
participant Test as test_full_workflow_e2e
participant Cfg as load_config
participant WF as run_workflow
Pytest->>FX: Resolve OpenSearch URL
alt URL available
FX-->>Pytest: opensearch_url: str
else Connection/Env missing
FX-->>Pytest: skip or raise (handled by pytest)
end
Pytest->>Test: Inject opensearch_url
Test->>Cfg: load_config(...)
Cfg-->>Test: config
Test->>Test: set config.workflow.opensearch_url
Test->>WF: run_workflow(config)
WF-->>Test: result
Test-->>Pytest: assert result type
note over Test,WF: Direct invocation replaces prior dynamic loading/context pattern
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Signed-off-by: David Gardner <dagardner@nvidia.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/conftest.py (1)
16-25: Remove conflicting proprietary license header.Lines 16-25 contain a proprietary license header (
LicenseRef-NvidiaProprietary) that conflicts with the Apache-2.0 license declared at the top of the file (lines 1-14). According to the coding guidelines, all code should be licensed under Apache License 2.0.Apply this diff to remove the conflicting license header:
-# limitations under the License. -# SPDX-FileCopyrightText: Copyright (c) 2024-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: LicenseRef-NvidiaProprietary -# -# NVIDIA CORPORATION, its affiliates and licensors retain all intellectual -# property and proprietary rights in and to this material, related -# documentation and any modifications thereto. Any use, reproduction, -# disclosure or distribution of this material and related documentation -# without an express license agreement from NVIDIA CORPORATION or -# its affiliates is strictly prohibited. -
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
packages/nvidia_nat_test/src/nat/test/functions.py(1 hunks)packages/nvidia_nat_test/src/nat/test/llm.py(0 hunks)tests/conftest.py(1 hunks)tests/nat/front_ends/auth_flow_handlers/test_websocket_flow_handler.py(1 hunks)
💤 Files with no reviewable changes (1)
- packages/nvidia_nat_test/src/nat/test/llm.py
🧰 Additional context used
📓 Path-based instructions (8)
**/*.{py,yaml,yml}
📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)
**/*.{py,yaml,yml}: Configure response_seq as a list of strings; values cycle per call, and [] yields an empty string.
Configure delay_ms to inject per-call artificial latency in milliseconds for nat_test_llm.
Files:
tests/nat/front_ends/auth_flow_handlers/test_websocket_flow_handler.pypackages/nvidia_nat_test/src/nat/test/functions.pytests/conftest.py
**/*.py
📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)
**/*.py: Programmatic use: create TestLLMConfig(response_seq=[...], delay_ms=...), add with builder.add_llm("", cfg).
When retrieving the test LLM wrapper, use builder.get_llm(name, wrapper_type=LLMFrameworkEnum.) and call the framework’s method (e.g., ainvoke, achat, call).
**/*.py: In code comments/identifiers use NAT abbreviations as specified: nat for API namespace/CLI, nvidia-nat for package name, NAT for env var prefixes; do not use these abbreviations in documentation
Follow PEP 20 and PEP 8; run yapf with column_limit=120; use 4-space indentation; end files with a single trailing newline
Run ruff check --fix as linter (not formatter) using pyproject.toml config; fix warnings unless explicitly ignored
Respect naming: snake_case for functions/variables, PascalCase for classes, UPPER_CASE for constants
Treat pyright warnings as errors during development
Exception handling: use bare raise to re-raise; log with logger.error() when re-raising to avoid duplicate stack traces; use logger.exception() when catching without re-raising
Provide Google-style docstrings for every public module, class, function, and CLI command; first line concise and ending with a period; surround code entities with backticks
Validate and sanitize all user input, especially in web or CLI interfaces
Prefer httpx with SSL verification enabled by default and follow OWASP Top-10 recommendations
Use async/await for I/O-bound work; profile CPU-heavy paths with cProfile or mprof before optimizing; cache expensive computations with functools.lru_cache or external cache; leverage NumPy vectorized operations when beneficial
Files:
tests/nat/front_ends/auth_flow_handlers/test_websocket_flow_handler.pypackages/nvidia_nat_test/src/nat/test/functions.pytests/conftest.py
tests/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Unit tests reside under tests/ and should use markers defined in pyproject.toml (e.g., integration)
Files:
tests/nat/front_ends/auth_flow_handlers/test_websocket_flow_handler.pytests/conftest.py
⚙️ CodeRabbit configuration file
tests/**/*.py: - Ensure that tests are comprehensive, cover edge cases, and validate the functionality of the code. - Test functions should be named using thetest_prefix, using snake_case. - Any frequently repeated code should be extracted into pytest fixtures. - Pytest fixtures should define the name argument when applying the pytest.fixture decorator. The fixture
function being decorated should be named using thefixture_prefix, using snake_case. Example:
@pytest.fixture(name="my_fixture")
def fixture_my_fixture():
pass
Files:
tests/nat/front_ends/auth_flow_handlers/test_websocket_flow_handler.pytests/conftest.py
{tests/**/*.py,examples/*/tests/**/*.py}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
{tests/**/*.py,examples/*/tests/**/*.py}: Use pytest (with pytest-asyncio for async); name test files test_*.py; test functions start with test_; extract repeated code into fixtures; fixtures must set name in decorator and be named with fixture_ prefix
Mock external services with pytest_httpserver or unittest.mock; do not hit live endpoints
Mark expensive tests with @pytest.mark.slow or @pytest.mark.integration
Files:
tests/nat/front_ends/auth_flow_handlers/test_websocket_flow_handler.pytests/conftest.py
**/*
⚙️ CodeRabbit configuration file
**/*: # Code Review Instructions
- Ensure the code follows best practices and coding standards. - For Python code, follow
PEP 20 and
PEP 8 for style guidelines.- Check for security vulnerabilities and potential issues. - Python methods should use type hints for all parameters and return values.
Example:def my_function(param1: int, param2: str) -> bool: pass- For Python exception handling, ensure proper stack trace preservation:
- When re-raising exceptions: use bare
raisestatements to maintain the original stack trace,
and uselogger.error()(notlogger.exception()) to avoid duplicate stack trace output.- When catching and logging exceptions without re-raising: always use
logger.exception()
to capture the full stack trace information.Documentation Review Instructions - Verify that documentation and comments are clear and comprehensive. - Verify that the documentation doesn't contain any TODOs, FIXMEs or placeholder text like "lorem ipsum". - Verify that the documentation doesn't contain any offensive or outdated terms. - Verify that documentation and comments are free of spelling mistakes, ensure the documentation doesn't contain any
words listed in the
ci/vale/styles/config/vocabularies/nat/reject.txtfile, words that might appear to be
spelling mistakes but are listed in theci/vale/styles/config/vocabularies/nat/accept.txtfile are OK.Misc. - All code (except .mdc files that contain Cursor rules) should be licensed under the Apache License 2.0,
and should contain an Apache License 2.0 header comment at the top of each file.
- Confirm that copyright years are up-to date whenever a file is changed.
Files:
tests/nat/front_ends/auth_flow_handlers/test_websocket_flow_handler.pypackages/nvidia_nat_test/src/nat/test/functions.pytests/conftest.py
packages/*/src/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Importable Python code inside packages must live under packages//src/
Files:
packages/nvidia_nat_test/src/nat/test/functions.py
{src/**/*.py,packages/*/src/**/*.py}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
All public APIs must have Python 3.11+ type hints on parameters and return values; prefer typing/collections.abc abstractions; use typing.Annotated when useful
Files:
packages/nvidia_nat_test/src/nat/test/functions.py
packages/**/*
⚙️ CodeRabbit configuration file
packages/**/*: - This directory contains optional plugin packages for the toolkit, each should contain apyproject.tomlfile. - Thepyproject.tomlfile should declare a dependency onnvidia-nator another package with a name starting
withnvidia-nat-. This dependency should be declared using~=<version>, and the version should be a two
digit version (ex:~=1.0).
- Not all packages contain Python code, if they do they should also contain their own set of tests, in a
tests/directory at the same level as thepyproject.tomlfile.
Files:
packages/nvidia_nat_test/src/nat/test/functions.py
🧬 Code graph analysis (1)
tests/conftest.py (2)
src/nat/builder/builder.py (1)
Builder(68-290)src/nat/data_models/function.py (1)
EmptyFunctionConfig(58-59)
🪛 Ruff (0.13.3)
tests/conftest.py
594-594: Unused function argument: config
(ARG001)
594-594: Unused function argument: builder
(ARG001)
🔇 Additional comments (3)
tests/nat/front_ends/auth_flow_handlers/test_websocket_flow_handler.py (1)
180-180: LGTM!Appropriately marks this OAuth2 error-handling test as slow, consistent with the coding guidelines for marking expensive tests.
tests/conftest.py (1)
587-599: Fixture correctly addresses the Pydantic serialization warning issue.The session-scoped autouse fixture properly registers
EmptyFunctionConfigto prevent serialization warnings when tests use the defaultConfig.workflowvalue. The no-op implementation is appropriate since this is only needed to satisfy the registration requirement.As per Python convention and static analysis hints, consider prefixing unused parameters with underscore:
@register_function(config_type=EmptyFunctionConfig) -async def empty_function(config: EmptyFunctionConfig, builder: Builder): +async def empty_function(_config: EmptyFunctionConfig, _builder: Builder): async def inner(*_, **__): return yield innerpackages/nvidia_nat_test/src/nat/test/functions.py (1)
75-96: Removed functions have no active references
All matches forConstantFunctionConfig,constant_function,StreamingConstantFunctionConfig, andstreaming_constant_functionappear only in comments; it’s safe to finalize their removal.
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
packages/nvidia_nat_test/src/nat/test/plugin.py (1)
164-173: Consider adding return type hint.The fixture implementation is correct and follows the existing pattern. However, consider adding a return type hint (
-> dict[str, str]) to match the coding guidelines requirement for type hints on all public APIs.Apply this diff to add the type hint:
-def serperdev_api_key_fixture(fail_missing: bool): +def serperdev_api_key_fixture(fail_missing: bool) -> dict[str, str]:Note: Other API key fixtures in this file also lack return type hints and could benefit from the same improvement.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
ci/scripts/github/tests.sh(1 hunks)examples/frameworks/haystack_deep_research_agent/pyproject.toml(1 hunks)examples/frameworks/haystack_deep_research_agent/tests/test_haystack_deep_research_agent.py(1 hunks)packages/nvidia_nat_test/src/nat/test/plugin.py(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- examples/frameworks/haystack_deep_research_agent/pyproject.toml
🧰 Additional context used
📓 Path-based instructions (8)
**/*.{py,yaml,yml}
📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)
**/*.{py,yaml,yml}: Configure response_seq as a list of strings; values cycle per call, and [] yields an empty string.
Configure delay_ms to inject per-call artificial latency in milliseconds for nat_test_llm.
Files:
examples/frameworks/haystack_deep_research_agent/tests/test_haystack_deep_research_agent.pypackages/nvidia_nat_test/src/nat/test/plugin.py
**/*.py
📄 CodeRabbit inference engine (.cursor/rules/nat-test-llm.mdc)
**/*.py: Programmatic use: create TestLLMConfig(response_seq=[...], delay_ms=...), add with builder.add_llm("", cfg).
When retrieving the test LLM wrapper, use builder.get_llm(name, wrapper_type=LLMFrameworkEnum.) and call the framework’s method (e.g., ainvoke, achat, call).
**/*.py: In code comments/identifiers use NAT abbreviations as specified: nat for API namespace/CLI, nvidia-nat for package name, NAT for env var prefixes; do not use these abbreviations in documentation
Follow PEP 20 and PEP 8; run yapf with column_limit=120; use 4-space indentation; end files with a single trailing newline
Run ruff check --fix as linter (not formatter) using pyproject.toml config; fix warnings unless explicitly ignored
Respect naming: snake_case for functions/variables, PascalCase for classes, UPPER_CASE for constants
Treat pyright warnings as errors during development
Exception handling: use bare raise to re-raise; log with logger.error() when re-raising to avoid duplicate stack traces; use logger.exception() when catching without re-raising
Provide Google-style docstrings for every public module, class, function, and CLI command; first line concise and ending with a period; surround code entities with backticks
Validate and sanitize all user input, especially in web or CLI interfaces
Prefer httpx with SSL verification enabled by default and follow OWASP Top-10 recommendations
Use async/await for I/O-bound work; profile CPU-heavy paths with cProfile or mprof before optimizing; cache expensive computations with functools.lru_cache or external cache; leverage NumPy vectorized operations when beneficial
Files:
examples/frameworks/haystack_deep_research_agent/tests/test_haystack_deep_research_agent.pypackages/nvidia_nat_test/src/nat/test/plugin.py
**/*
⚙️ CodeRabbit configuration file
**/*: # Code Review Instructions
- Ensure the code follows best practices and coding standards. - For Python code, follow
PEP 20 and
PEP 8 for style guidelines.- Check for security vulnerabilities and potential issues. - Python methods should use type hints for all parameters and return values.
Example:def my_function(param1: int, param2: str) -> bool: pass- For Python exception handling, ensure proper stack trace preservation:
- When re-raising exceptions: use bare
raisestatements to maintain the original stack trace,
and uselogger.error()(notlogger.exception()) to avoid duplicate stack trace output.- When catching and logging exceptions without re-raising: always use
logger.exception()
to capture the full stack trace information.Documentation Review Instructions - Verify that documentation and comments are clear and comprehensive. - Verify that the documentation doesn't contain any TODOs, FIXMEs or placeholder text like "lorem ipsum". - Verify that the documentation doesn't contain any offensive or outdated terms. - Verify that documentation and comments are free of spelling mistakes, ensure the documentation doesn't contain any
words listed in the
ci/vale/styles/config/vocabularies/nat/reject.txtfile, words that might appear to be
spelling mistakes but are listed in theci/vale/styles/config/vocabularies/nat/accept.txtfile are OK.Misc. - All code (except .mdc files that contain Cursor rules) should be licensed under the Apache License 2.0,
and should contain an Apache License 2.0 header comment at the top of each file.
- Confirm that copyright years are up-to date whenever a file is changed.
Files:
examples/frameworks/haystack_deep_research_agent/tests/test_haystack_deep_research_agent.pyci/scripts/github/tests.shpackages/nvidia_nat_test/src/nat/test/plugin.py
examples/**/*
⚙️ CodeRabbit configuration file
examples/**/*: - This directory contains example code and usage scenarios for the toolkit, at a minimum an example should
contain a README.md or file README.ipynb.
- If an example contains Python code, it should be placed in a subdirectory named
src/and should
contain apyproject.tomlfile. Optionally, it might also contain scripts in ascripts/directory.- If an example contains YAML files, they should be placed in a subdirectory named
configs/. - If an example contains sample data files, they should be placed in a subdirectory nameddata/, and should
be checked into git-lfs.
Files:
examples/frameworks/haystack_deep_research_agent/tests/test_haystack_deep_research_agent.py
{scripts/**,ci/scripts/**}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Shell or utility scripts belong in scripts/ or ci/scripts/ and must not be mixed with library code
Files:
ci/scripts/github/tests.sh
packages/*/src/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Importable Python code inside packages must live under packages//src/
Files:
packages/nvidia_nat_test/src/nat/test/plugin.py
{src/**/*.py,packages/*/src/**/*.py}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
All public APIs must have Python 3.11+ type hints on parameters and return values; prefer typing/collections.abc abstractions; use typing.Annotated when useful
Files:
packages/nvidia_nat_test/src/nat/test/plugin.py
packages/**/*
⚙️ CodeRabbit configuration file
packages/**/*: - This directory contains optional plugin packages for the toolkit, each should contain apyproject.tomlfile. - Thepyproject.tomlfile should declare a dependency onnvidia-nator another package with a name starting
withnvidia-nat-. This dependency should be declared using~=<version>, and the version should be a two
digit version (ex:~=1.0).
- Not all packages contain Python code, if they do they should also contain their own set of tests, in a
tests/directory at the same level as thepyproject.tomlfile.
Files:
packages/nvidia_nat_test/src/nat/test/plugin.py
🪛 Ruff (0.13.3)
examples/frameworks/haystack_deep_research_agent/tests/test_haystack_deep_research_agent.py
27-27: Audit URL open for permitted schemes. Allowing use of file: or custom schemes is often unexpected.
(S310)
29-29: Do not catch blind exception: Exception
(BLE001)
32-32: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
🔇 Additional comments (3)
ci/scripts/github/tests.sh (1)
33-33: LGTM!The addition of the
--run_slowflag enables slow tests to run in CI, which aligns with the PR objectives. The implementation is correct and consistent with the slow test marker infrastructure defined in the test plugin.packages/nvidia_nat_test/src/nat/test/plugin.py (1)
156-156: LGTM!The docstring clarification distinguishes this fixture from the newly added
serperdevfixture, improving clarity for test authors.examples/frameworks/haystack_deep_research_agent/tests/test_haystack_deep_research_agent.py (1)
36-52: Test implementation looks good, pending fixture fix.The refactored test implementation correctly uses fixtures, loads configuration, and invokes the workflow directly. However, the test will fail due to the bug in the
opensearch_urlfixture (which returns a boolean instead of a string). Once that fixture is fixed, this test implementation should work correctly.After fixing the
opensearch_urlfixture as noted in the previous comment, verify that this test passes with the correct workflow execution.
examples/frameworks/haystack_deep_research_agent/tests/test_haystack_deep_research_agent.py
Show resolved
Hide resolved
…to david-test-warnings Signed-off-by: David Gardner <dagardner@nvidia.com>
|
/merge |
2 participants
Description
Config.workflowisEmptyFunctionConfig()however this config is never registered and thus triggers a serialize warning if someone callsmodel_dumpon the config. Since a config with a workflow set to the empty function config is not valid, this is something that happens only in tests and does not come up in routine NAT usage.--run_slowflag in GitHub CI.test_websocket_oauth2_flow_error_handlingas slow as this test is taking about 5 minutes to run.e2emark ontest_haystack_deep_research_agent, update fixtures to match the rest of NAT.By Submitting this PR I confirm:
Summary by CodeRabbit
Chores
Tests
No user-facing features or behavior changes.