Skip to content

Add unit test suite and code coverage enforcement#135

Merged
mmccarty merged 6 commits intorapidsai:mainfrom
mmccarty:testing
Feb 12, 2026
Merged

Add unit test suite and code coverage enforcement#135
mmccarty merged 6 commits intorapidsai:mainfrom
mmccarty:testing

Conversation

@mmccarty
Copy link
Contributor

@mmccarty mmccarty commented Feb 11, 2026

Summary

  • Add comprehensive unit test suite (53 tests) covering all modules: CLI entry points, config, doctor checks (GPU, CUDA, memory, NVLink), debug commands, and doctor orchestration
  • Add pytest-cov dependency and configure 95% minimum coverage threshold with automatic reporting
  • Fix pytest test path configuration ([tool.pytest][tool.pytest.ini_options], testsrapids_cli/tests)
  • Add CLAUDE.md with project architecture, development commands, code style, and testing guidance
  • Add SPDX license headers to dependencies.yaml

Test plan

  • pytest passes all 53 tests
  • Coverage meets 95% threshold
  • pre-commit run --all-files passes

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

  • Documentation

    • Added developer guidance for the repository.
  • Tests

    • Expanded test coverage for CLI commands, configuration validation, debug output, doctor checks, GPU detection, memory diagnostics, and NVLink status.
  • Chores

    • Updated project dependencies and testing configuration with coverage reporting and metrics.

@mmccarty mmccarty requested review from a team as code owners February 11, 2026 22:10
@mmccarty mmccarty requested a review from jameslamb February 11, 2026 22:10
@mmccarty mmccarty mentioned this pull request Feb 11, 2026
2 tasks
@jacobtomlinson
Copy link
Member

@coderabbitai full review

@coderabbitai
Copy link

coderabbitai bot commented Feb 12, 2026

✅ Actions performed

Full review triggered.

@coderabbitai
Copy link

coderabbitai bot commented Feb 12, 2026

Walkthrough

This PR adds comprehensive test coverage infrastructure to the RAPIDS CLI project, including pytest-cov dependency addition, coverage configuration to pyproject.toml, seven new test modules covering CLI/config/debug/doctor/GPU/memory/NVLink functionality, and documentation guidance (CLAUDE.md).

Changes

Cohort / File(s) Summary
Documentation & Configuration
CLAUDE.md, dependencies.yaml, pyproject.toml
Adds Claude Code guidance documentation, introduces pytest-cov dependency, refactors pytest config to [tool.pytest.ini_options] with updated testpaths, and adds coverage configuration sections with source/omit/exclude patterns and 95% coverage threshold.
CLI Test Module
rapids_cli/tests/test_cli.py
New test suite validating CLI help output, subcommand invocation (doctor/debug), exit codes, verbose/dry-run/filter behavior, and run_debug output formats using CliRunner with patched implementations.
Configuration Test Module
rapids_cli/tests/test_config.py
New test suite verifying config object structure, presence of required sections (min_supported_versions, valid_subcommands, os_requirements), and nested configuration keys.
Debug Test Module
rapids_cli/tests/test_debug.py
New test suite covering CUDA driver version gathering, installed package enumeration, command output fallback logic, tool detection, and run_debug output in both console and JSON formats with mocked NVML/filesystem interactions.
Doctor Test Module
rapids_cli/tests/test_doctor.py
New test suite validating doctor_check across pass/fail scenarios, verbose discovery output, dry-run behavior, check filtering, warning handling, and ImportError suppression using mocked entry points.
GPU Test Module
rapids_cli/tests/test_gpu.py
New test suite for GPU detection and compute capability validation, covering success paths, no-GPU scenarios, NVML errors, and compute capability sufficiency checks.
Memory Test Module
rapids_cli/tests/test_memory.py
New test suite validating system/GPU memory detection for single and multiple GPU configurations, and memory-to-GPU ratio validation with warning conditions, using mocked psutil and pynvml calls.
NVLink Test Module
rapids_cli/tests/test_nvlink.py
New test suite verifying NVLink status checks for multi-GPU configurations, single-GPU unsupported scenarios, and error handling for missing GPUs and unsupported NVML features.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~18 minutes

Poem

🐰 A testing feast of grandeur built with care,
Seven modules dancing through the digital air,
With mocks and patches woven tight and true,
Coverage climbs to ninety-five or new! 🎯
The doctor checks, the GPUs aligned—
A robust CLI, so finely designed! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 70.37% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly and clearly describes the main changes: adding a comprehensive unit test suite with code coverage enforcement (95% threshold).

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@CLAUDE.md`:
- Around line 29-30: The README example references a non-existent test file
`rapids_cli/tests/test_cuda.py`; update the example command to point to an
actual test file (e.g., `rapids_cli/tests/test_gpu.py` or one of `test_cli.py`,
`test_config.py`, `test_debug.py`, `test_doctor.py`, `test_memory.py`,
`test_nvlink.py`) so the documented pytest invocation works; replace
`test_cuda.py` with a real test filename in the CLAUDE.md example line.
🧹 Nitpick comments (5)
CLAUDE.md (1)

100-100: Nit: "CLI interface" is redundant.

"CLI" already stands for "Command-Line Interface", so "CLI interface" expands to "Command-Line Interface interface."

Proposed fix
-- `rich` and `rich-click` for terminal output and CLI interface
+- `rich` and `rich-click` for terminal output and CLI
rapids_cli/tests/test_nvlink.py (1)

34-38: Unescaped regex metacharacters in match pattern.

The dots in "GPU not found. Please ensure GPUs are installed." are regex metacharacters that match any character. While unlikely to cause false matches here, use re.escape() or escape the dots for correctness.

Proposed fix
+import re
+
 def test_check_nvlink_status_no_gpu():
     import pynvml
 
     with patch("pynvml.nvmlInit", side_effect=pynvml.NVMLError(1)):
         with pytest.raises(
-            ValueError, match="GPU not found. Please ensure GPUs are installed."
+            ValueError, match=re.escape("GPU not found. Please ensure GPUs are installed.")
         ):
             check_nvlink_status(verbose=False)
rapids_cli/tests/test_debug.py (1)

84-88: Broadly patching pathlib.Path.glob and pathlib.Path.read_text is fragile.

These patches affect all Path instances process-wide, not just the ones in debug.py. If pytest internals, coverage tooling, or rich use Path.glob or Path.read_text during the test, it could cause subtle failures. Consider patching at the module level (e.g., rapids_cli.debug.debug.Path) or using more targeted mocking if issues arise.

rapids_cli/tests/test_doctor.py (1)

101-117: Unused capsys parameter and weak assertion — consider verifying only the filtered check ran.

capsys is declared but never used (Ruff ARG001). More importantly, the test only asserts result is True but doesn't verify that only the cudf entry point was loaded/executed (and cuml was skipped). Without that, the test doesn't actually confirm filtering behavior — it only confirms no crash.

Consider calling mock_ep2.load.assert_not_called() and removing the unused capsys, or using it to verify relevant output.

Proposed improvement
-def test_doctor_check_with_filters(capsys):
+def test_doctor_check_with_filters():
     """Test doctor_check with filters."""
     mock_ep1 = MagicMock()
     mock_ep1.name = "cudf_check"
     mock_ep1.value = "cudf.module:check"
     mock_ep1.load.return_value = mock_passing_check

     mock_ep2 = MagicMock()
     mock_ep2.name = "cuml_check"
     mock_ep2.value = "cuml.module:check"
     mock_ep2.load.return_value = mock_passing_check

     with patch(
         "rapids_cli.doctor.doctor.entry_points", return_value=[mock_ep1, mock_ep2]
     ):
         result = doctor_check(verbose=False, dry_run=False, filters=["cudf"])
         assert result is True
+        mock_ep1.load.assert_called_once()
+        mock_ep2.load.assert_not_called()
rapids_cli/tests/test_memory.py (1)

75-82: Use a raw string for the match pattern to avoid unescaped regex metacharacters.

The match argument is a regex pattern, and the dots in "GPU not found. Please ensure GPUs are installed." are metacharacters that match any character. While unlikely to cause a false positive here, using a raw string with escaped dots is more correct. Flagged by Ruff (RUF043).

Proposed fix
         with pytest.raises(
-            ValueError, match="GPU not found. Please ensure GPUs are installed."
+            ValueError, match=r"GPU not found\. Please ensure GPUs are installed\."
         ):

Copy link
Member

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Packaging changes look good to me. I guess pyyaml must previously have been getting into the environment some other way, but the dependency is needed because rapids-cli has direct usage:

I didn't review the other changes, will leave that to Jacob and Naty.

@mmccarty mmccarty merged commit 2137644 into rapidsai:main Feb 12, 2026
9 checks passed
mmccarty added a commit that referenced this pull request Feb 13, 2026
## Summary

- Overhaul Sphinx docs: add user guide, troubleshooting guide, plugin
development guide, and full API reference (cli, doctor, debug, checks)
- Switch Sphinx theme to pydata-sphinx-theme with RAPIDS branding,
enable autodoc/napoleon/intersphinx/viewcode extensions
- Remove stale doc pages (old `doctor.rst`, `cuda_driver.rst`) and
restructure `index.rst` with proper toctree sections
- Fix `doctor_check()` docstring from NumPy-style to Google-style
(matches ruff pydocstyle convention)
- Add SPDX license headers to docs and `dependencies.yaml`
- Fix Sphinx cross-references for `run_debug` and `CheckResult` to use
fully qualified module paths
- Add `sphinx-llm` extension to generate `llms.txt` for LLM consumption
- Add `sphinx-autobuild` with `make serve` target for live-reload docs
development
- Rewrite doctor.rst with user-facing content explaining cross-layer
compatibility checks and library-bundled health checks
- Move check execution flow details to plugin_development.rst
- Fix incorrect `Raises` section in `doctor_check` docstring (replace
with `Returns`)

Note: This branch also includes the commits from #135 (unit tests and
coverage). The docs-specific changes are in commits after `2137644`.

## Test plan

- [ ] `make -C docs clean html` builds with 0 warnings
- [ ] `pre-commit run --all-files` passes
- [ ] `llms.txt` and `llms-full.txt` generated in build output
- [ ] `make serve` starts live-reload server

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Mike McCarty <mmccarty@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: James Lamb <jaylamb20@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants