docs(kubernetes): add GPU sharing with HAMi guide#517
Conversation
Document the HAMi GPU virtualization addon for tenant Kubernetes clusters, covering architecture, setup, fractional resource requests, custom configuration, and known glibc limitations. Signed-off-by: Arsolitt <arsolitt@gmail.com>
✅ Deploy Preview for cozystack ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughIntroduces a new documentation page explaining GPU sharing with HAMi (Heterogeneous Application Module interface) in tenant Kubernetes clusters. The page covers HAMi's integration components, configuration setup, fractional GPU resource requests, customization options, and known limitations. Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~5 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Arsolitt (Arsolitt)
requested review from
Andrei Kvapil (kvaps) and
Timofei Larkin (lllamnyp)
as code owners
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
content/en/docs/next/kubernetes/gpu-sharing.md (1)
141-144:⚠️ Potential issue | 🔴 CriticalFix unmatched trailing Markdown code fence (docs build risk).
There’s a stray closing ``` at the end of the file (after the musl/libc paragraph). This will break Markdown/Hugo rendering and can fail the doc build.
✅ Proposed fix
### Alpine / musl libc HAMi-core is incompatible with musl libc. Only glibc-based container images (Debian, Ubuntu, RHEL) are supported. -```🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@content/en/docs/next/kubernetes/gpu-sharing.md` around lines 141 - 144, Remove the stray closing Markdown code fence found immediately after the "### Alpine / musl libc" paragraph; locate the unmatched ``` at the end of the file and delete it (or, if an intended block was opened earlier, ensure the fence pair is properly balanced by adding a matching opening fence instead). This fixes the dangling code fence that breaks Markdown/Hugo rendering.
🧹 Nitpick comments (2)
content/en/docs/next/kubernetes/gpu-sharing.md (2)
61-79: Align the Pod example with the “prefer gpumem-percentage” guidance (minor clarity).The text recommends preferring
nvidia.com/gpumem-percentage, but the Pod manifest example usesnvidia.com/gpumem. Consider updating the example to usegpumem-percentage(or add a brief note: “Example uses gpumem for concreteness; use gpumem-percentage for portability”).🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@content/en/docs/next/kubernetes/gpu-sharing.md` around lines 61 - 79, The Pod manifest example uses the resource key nvidia.com/gpumem while the text advises preferring nvidia.com/gpumem-percentage; update the example to use nvidia.com/gpumem-percentage instead of nvidia.com/gpumem (or alternatively add a one-line clarifying note after the YAML saying “This example uses gpumem for concreteness; prefer gpumem-percentage for portability”). Locate the manifest block with the container named cuda-app and change the resource key or add the note accordingly to keep example and guidance consistent.
122-140: Add a “how to confirm HAMi-core loaded” step for silent failure modes.You describe several “fails silently / no isolation” cases (Ubuntu 22.04 and 24.04), but the guide doesn’t give a concrete verification/debug step. Since workloads can run without GPU limits when HAMi-core can’t load, it’d be safer UX to add a short section near this table like: “How to verify HAMi-core is active” (e.g., check component logs on startup, confirm
LD_PRELOADinjection, or validate that GPU limits actually constrain memory/compute in a quick test). This directly mitigates the operational risk you’re warning about.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@content/en/docs/next/kubernetes/gpu-sharing.md` around lines 122 - 140, Add a short "How to verify HAMi-core is active" section immediately after the glibc compatibility table that gives concrete, actionable checks: instruct readers to inspect HAMi-core startup logs for successful load messages (referencing "HAMi-core"), verify LD_PRELOAD is injected into the workload container environment (check process env for LD_PRELOAD), and run a quick validation that GPU limits apply (e.g., launch a test workload that would exceed expected memory/compute limits and confirm it is constrained). Mention these three checks explicitly so operators can detect the silent-failure modes on Ubuntu 22.04/24.04 images.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@content/en/docs/next/kubernetes/gpu-sharing.md`:
- Around line 141-144: Remove the stray closing Markdown code fence found
immediately after the "### Alpine / musl libc" paragraph; locate the unmatched
``` at the end of the file and delete it (or, if an intended block was opened
earlier, ensure the fence pair is properly balanced by adding a matching opening
fence instead). This fixes the dangling code fence that breaks Markdown/Hugo
rendering.
---
Nitpick comments:
In `@content/en/docs/next/kubernetes/gpu-sharing.md`:
- Around line 61-79: The Pod manifest example uses the resource key
nvidia.com/gpumem while the text advises preferring
nvidia.com/gpumem-percentage; update the example to use
nvidia.com/gpumem-percentage instead of nvidia.com/gpumem (or alternatively add
a one-line clarifying note after the YAML saying “This example uses gpumem for
concreteness; prefer gpumem-percentage for portability”). Locate the manifest
block with the container named cuda-app and change the resource key or add the
note accordingly to keep example and guidance consistent.
- Around line 122-140: Add a short "How to verify HAMi-core is active" section
immediately after the glibc compatibility table that gives concrete, actionable
checks: instruct readers to inspect HAMi-core startup logs for successful load
messages (referencing "HAMi-core"), verify LD_PRELOAD is injected into the
workload container environment (check process env for LD_PRELOAD), and run a
quick validation that GPU limits apply (e.g., launch a test workload that would
exceed expected memory/compute limits and confirm it is constrained). Mention
these three checks explicitly so operators can detect the silent-failure modes
on Ubuntu 22.04/24.04 images.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 367c9002-4bbe-4528-b07e-5f245a8a8b18
📒 Files selected for processing (1)
content/en/docs/next/kubernetes/gpu-sharing.md
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new documentation page for GPU sharing using HAMi in tenant Kubernetes clusters, detailing its architecture, configuration, and known limitations. Feedback includes a suggestion to use relative links for better documentation portability and a correction regarding the nvidia.com/gpu resource description to accurately reflect that multiple virtual GPUs can be requested.
| [HAMi](https://github.com/Project-HAMi/HAMi) (Heterogeneous AI Computing Virtualization Middleware) is a CNCF Sandbox project that enables fractional GPU sharing in Kubernetes. Instead of dedicating an entire GPU to a single workload, HAMi lets containers request specific amounts of GPU memory and compute cores. | ||
|
|
||
| {{% alert color="info" %}} | ||
| This guide covers GPU sharing for **containers in tenant Kubernetes clusters**. For GPU passthrough to virtual machines on the management cluster, see [GPU Passthrough](/docs/next/virtualization/gpu/). |
There was a problem hiding this comment.
Using an absolute path like /docs/next/ makes the link fragile if the documentation version changes or if the site is viewed in a different context (e.g., a preview environment). It is generally better to use a relative link to maintain portability across different versions of the documentation.
| This guide covers GPU sharing for **containers in tenant Kubernetes clusters**. For GPU passthrough to virtual machines on the management cluster, see [GPU Passthrough](/docs/next/virtualization/gpu/). | |
| This guide covers GPU sharing for **containers in tenant Kubernetes clusters**. For GPU passthrough to virtual machines on the management cluster, see [GPU Passthrough](../virtualization/gpu/). |
There was a problem hiding this comment.
Absolute /docs/next/ paths are the documented convention for this repo (see CLAUDE.md line 70). They are intentionally rewritten to versioned paths by hack/release_next.sh during the release process. Using relative paths would bypass this rewrite mechanism and produce broken links after version promotion.
|
|
||
| | Resource | Description | | ||
| | --- | --- | | ||
| | `nvidia.com/gpu` | Number of virtual GPUs (always `1` per container) | |
There was a problem hiding this comment.
The description for nvidia.com/gpu states it is "always 1 per container". While this is the standard for fractional sharing, HAMi does support requesting multiple virtual GPUs (e.g., nvidia.com/gpu: 2) if the pod needs to span multiple physical GPUs or multiple partitions. Unless this is a specific restriction of the Cozystack integration, it would be more accurate to describe it as the number of virtual GPUs requested by the container.
| | `nvidia.com/gpu` | Number of virtual GPUs (always `1` per container) | | |
| | `nvidia.com/gpu` | Number of virtual GPUs requested | |
There was a problem hiding this comment.
Good catch — HAMi does support requesting multiple virtual GPUs per container. Updated the description to "Number of virtual GPUs requested" in 985f68a.
Add a note after the Pod YAML example explaining when to prefer gpumem-percentage over absolute gpumem values. Signed-off-by: Arsolitt <arsolitt@gmail.com>
HAMi supports requesting multiple virtual GPUs per container, so the "always 1" constraint was inaccurate. Signed-off-by: Arsolitt <arsolitt@gmail.com>
Aleksei Sviridkin (lexfrei)
left a comment
Aleksei Sviridkin (lexfrei)
left a comment
There was a problem hiding this comment.
Cross-checked the technical claims against the feature PR (cozystack/cozystack#2484) and upstream sources:
- glibc compatibility table is consistent with the system-package README and matches HAMi#1190 maintainer thread (Ubuntu 22.04 / 2.35: memory works, compute fails; Ubuntu 24.04 / 2.39: both fail).
- The deliberate choice of
nvcr.io/nvidia/cuda:11.8.0-base-ubuntu20.04in the example pod is a thoughtful detail — the demo gets full isolation because Ubuntu 20.04 ships glibc 2.31 (< 2.34). - The warning alert about silent failure ("workloads still run but without any GPU resource limits") is exactly what an operator needs to know.
nvidia.com/GA102GL_A10matches the example inpackages/apps/vm-instance/values.yaml, the GPU Passthrough cross-link resolves tocontent/en/docs/next/virtualization/gpu.md, theinstanceType: u1.xlargeis a kubevirt-instancetype name (orthogonal to the cozy-libresourcesPresetrename in cozystack/cozystack#1959, which uses the same naming pattern but is a different concept — no collision).- Citation chain is cleaner here than in the system README (no HAMi#173, which is the weak link there).
LGTM.
## What this PR does Documentation: cozystack/website#517 Integrates [HAMi](https://github.com/Project-HAMi/HAMi) v2.8.1 (CNCF Sandbox) into Cozystack as a system-level package for fractional GPU sharing in tenant Kubernetes clusters. The integration covers three layers: - **System chart** (`packages/system/hami/`): Vendors upstream HAMi Helm chart with device plugin, scheduler extender, mutating webhook, and RuntimeClass configuration. The broken DRA subchart was removed — it renders resources even when disabled and references unpublished images. - **Kubernetes app addon** (`packages/apps/kubernetes/`): HAMi exposed as an optional toggle (`hami.enabled`). Automatically disables GPU Operator's native device plugin when active to avoid conflicts. Enforces hard dependency on GPU Operator. - **Platform registration** (`packages/core/platform/`): HAMi declared as PackageSource with gpu-operator dependency, included in the iaas bundle. **Known limitation**: HAMi-core relies on a private glibc symbol (`_dl_sym`) removed in glibc 2.34, which breaks compute isolation on modern container images (Ubuntu 22.04+, PyTorch/TensorFlow official images). Alpine/musl is entirely incompatible. See the package README for details and upstream issue references. ### Screenshots N/A — no UI changes. ### Release note ```release-note feat(hami): add HAMi GPU virtualization as an optional system package for fractional GPU sharing. Enables memory and compute isolation for NVIDIA GPUs across tenant workloads. Integrates with GPU Operator and can be enabled per-cluster via the hami.enabled toggle. Note: compute isolation requires glibc < 2.34 in workload containers. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added HAMi GPU virtualization middleware as an optional cluster addon to enable fractional GPU sharing. * HAMi addon includes an enable toggle (default: disabled) and a customizable Helm values override; when enabled it enforces GPU Operator presence and adjusts GPU Operator values accordingly. * New HelmRelease integration for deploying HAMi with dependency ordering and conditional rendering. * **Documentation** * Added HAMi setup guide, compatibility notes, and updated cluster addon configuration reference. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 participants
Summary
Summary by CodeRabbit