feat(build): Cleanup vendor script, add new nemo-platform[all] group#90
Merged
matthewgrossman merged 3 commits intoMay 28, 2026
Conversation
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Contributor
Documentation preview is readyPreview: https://nvidia-nemo.github.io/nemo-platform/pr-preview/pr-90/pr-90/ Built from This preview is deployed from this PR branch, updates when docs changes are pushed, and will be removed when the PR closes. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdds an ChangesInstall [all] extra and vendoring infrastructure
Documentation and CLI updates to [all] extra
Suggested reviewers:
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@tools/nemo-platform-sdk-tools/src/nemo_platform_sdk_tools/sdk/vendor/vendor_package.py`:
- Around line 1192-1201: The current code unconditionally clears
project["optional-dependencies"], removing user-defined extras; instead, only
remove generated extras: when bundle_config is truthy and
"optional-dependencies" in project, iterate the keys of
project["optional-dependencies"] and delete only those whose names match the
generated groups described by the bundle config (e.g., derive the list of
generated group names from bundle_config or a known prefix), set changed = True
if any key was removed, and leave other manual extras intact rather than
replacing the whole table; use the variables project, bundle_config,
member_config, member_content and cleaned_member_content to locate and implement
this selective cleanup.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: a32b6ea0-c663-4deb-8060-85b4cf9793a2
⛔ Files ignored due to path filters (4)
sdk/python/nemo-platform/src/nemo_platform/cli/commands/services/cli.pyis excluded by!sdk/**sdk/python/nemo-platform/tests/vendored/nemo_platform_ext/cli/commands/test_services.pyis excluded by!sdk/**sdk/python/overrides/nemo-platform/README/02_installation.mdis excluded by!sdk/**uv.lockis excluded by!**/*.lock
📒 Files selected for processing (23)
docs/cli/index.mddocs/contributing/skills-spec.mddocs/customizer/tutorials/_snippets/customizer-prereqs.mddocs/customizer/tutorials/distillation-customization-job.ipynbdocs/customizer/tutorials/dpo-customization-job.ipynbdocs/customizer/tutorials/embedding-customization-job.ipynbdocs/customizer/tutorials/lora-customization-job.ipynbdocs/customizer/tutorials/optimize-throughput.ipynbdocs/customizer/tutorials/sft-customization-job.ipynbdocs/evaluator/tutorials/run-llm-judge-evaluation.mddocs/example-applications/tool-calling.ipynbdocs/pysdk/index.mddocs/safe-synthesizer/tutorials/differential-privacy.mddocs/safe-synthesizer/tutorials/index.mddocs/safe-synthesizer/tutorials/safe-synthesizer-101.mddocs/support-matrix.mdpackages/nemo_platform/README.mdpackages/nemo_platform/pyproject.tomlpackages/nemo_platform_ext/src/nemo_platform_ext/cli/commands/services/cli.pypackages/nemo_platform_ext/tests/cli/commands/test_services.pypackages/nemo_platform_plugin/pyproject.tomlplugins/nemo-data-designer/pyproject.tomltools/nemo-platform-sdk-tools/src/nemo_platform_sdk_tools/sdk/vendor/vendor_package.py
💤 Files with no reviewable changes (1)
- packages/nemo_platform_plugin/pyproject.toml
Contributor
|
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
|
Actionable comments posted: 0 |
matthewgrossman
commented
May 28, 2026
CI was failing because three tests referenced helpers removed by this PR (_remove_marked_optional_dependency_groups, _annotate_optional_dependency_groups), and a fourth test asserted per-key markers on mixed scripts/entry-points tables — a mode that no longer exists. * Replace the three removed-helper tests with coverage for the new _refresh_bundle_owned_optional_dependencies (preserves hand-written extras, drops stale generated extras, emits the marker for unmarked bundle-owned keys, alphabetizes vendor-owned). * Tighten _annotate_generated_project_table so it only emits the table-level header when the table is wholly bundle-generated. Mixed tables (containing any hand-written entry) are left unannotated, which matches the rebuild model and what the wrapper actually emits today. * Update test_annotate_generated_project_entries_marks_generated_entries accordingly: wholly-generated entry-point tables get the header, mixed scripts/entry-point tables stay unannotated. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
|
Actionable comments posted: 0 |
mckornfield
approved these changes
May 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes AIRCORE-654.
Summary
Adds a new
nemo-platform[all]extra as the recommended user-facing install path, and uses the opportunity to clean upvendor_package.py.What changed
nemo-platform[all]extrapip install nemo-platform[all]now resolves to the same dependency set as[services]. The existing[services]extra is preserved for backwards compatibility. User-facing docs, tutorials, and the CLI's "missing pyleak" hint message have all been updated to recommend[all].allis declared as a hand-written extra directly inpackages/nemo_platform/pyproject.toml, sitting alongside the auto-generated extras in the same table.Vendor flow simplification
The vendor flow already supported a "comment-marked = generated, no-comment = hand-written" convention for
[project.optional-dependencies], but the implementation had grown to several passes (annotate, reset-marked, copy-from-bundle, sort) with a lot of incidental complexity around carrying comment trivia through tomlkit table rewrites.This PR consolidates that into a single function (
_refresh_bundle_owned_optional_dependencies) that:# Generated from [tool.bundle-package]; do not edit by hand.marker comment immediately above it.[tool.bundle-package]).The marker comment is the load-bearing signal that distinguishes vendor-owned from hand-written. This is the same convention the vendor flow was already using, just implemented more directly.
Dead code removal
Removed ~10 helpers in
vendor_package.pythat had no callers (_merge_project_scripts,_merge_project_entrypoints, four_remove_*script/entry-point helpers,_merge_client_source_imports,_build_module_rewrites,_remove_empty_generated_project_tables,_merge_project_optional_dependencies). Some were dead before this PR; some became dead as a result of the refresh-function consolidation.Net:
vendor_package.pyis ~110 lines smaller.Verification
make vendoris idempotent on the wrapper,nemo-data-designer, andnemo-platform-plugin.# Generated...-marked extra in both wrapper and plugin pyprojects; both are removed on the next vendor run.allon the wrapper,testonnemo-data-designer) survive across vendor runs.packages/nemo_platform_ext/tests/cli/commands/test_services.py).uv run pre-commit run -apasses (ruff, format, ty, uv lock).uv lock --checkconfirmsnemo-platform[all]resolves to the same dependency set as[services].Acceptance criteria (from AIRCORE-654)
uv tool install 'nemo-platform[all]'installs the same service-capable package dependencies currently provided bynemo-platform[services].nemo-platform[services]installs continue to work.nemo-platform[all]for the full packaged install path.Signed-off-by: Matthew Grossman mgrossman@nvidia.com
Summary by CodeRabbit
Documentation
nemo-platform[all]instead ofnemo-platform[services]across CLI docs, tutorials, and SDK guidance.Tests
Chores