Add Lab 7: GPU isolation on k3s without the GPU Operator#560
Merged
hami-robot[bot] merged 2 commits intoJul 8, 2026
Conversation
✅ Deploy Preview for project-hami ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThis PR adds a new Lab 7 tutorial for GPU isolation on k3s without the GPU Operator, plus supporting sidebar, overview, manifest, and Chinese translation updates. ChangesLab 7 Tutorial Addition
Estimated code review effort: 2 (Simple) | ~12 minutes Suggested labels: Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED Approval requirements bypassed by manually added approval. This pull-request has been approved by: saiyam1814 The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
tutorials/labs/examples/07-hami-isolation-k3s/share-two-pods.yaml (1)
13-20: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low valueConsider basic pod hardening for copy-paste safety.
Static analysis flags missing
allowPrivilegeEscalation: falseand non-rootsecurityContexton both containers. Since readers may copy this manifest into shared or production-adjacent clusters, adding minimal hardening avoids propagating an insecure default pattern.🔒️ Optional hardening
- name: cuda image: nvidia/cuda:12.4.1-devel-ubuntu22.04 command: ["bash", "-c", "nvidia-smi; sleep infinity"] + securityContext: + allowPrivilegeEscalation: false resources: limits: nvidia.com/gpu: 1 nvidia.com/gpumem: 8000Also applies to: 28-35
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tutorials/labs/examples/07-hami-isolation-k3s/share-two-pods.yaml` around lines 13 - 20, The pod spec in the shared examples is missing basic hardening defaults, so update both container definitions (including the cuda container and the other container in this manifest) to run as non-root and prevent privilege escalation. Add a minimal securityContext for each container with allowPrivilegeEscalation set to false and a non-root runAsUser/runAsNonRoot setting so the copied manifest is safer by default.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@i18n/zh/docusaurus-plugin-content-docs-tutorials/current/labs/hami-isolation-k3s.md`:
- Around line 1-385: The document has formatting drift and needs to be
reformatted consistently with the repository’s Prettier settings. Run Prettier
on the Markdown tutorial content in this file and keep the existing wording
unchanged; focus only on whitespace, wrapping, and markdown formatting so the
doc matches CI expectations.
In `@i18n/zh/docusaurus-plugin-content-docs-tutorials/current/overview.md`:
- Line 22: The lab list and the following explanatory text are currently in the
same paragraph, which is fragile for the block-level LabCardGridAuto rendering.
Update the overview content so the experiment list remains one paragraph and
move the trailing explanation about experiments 3, 4, 2, and 7 into its own
separate paragraph in the same MDX section.
In `@tutorials/labs/hami-isolation-k3s.md`:
- Around line 19-21: Reflow the opening prose in this lab so it matches the
documentation line-length/style rules and passes prettier --check. Update the
introductory paragraphs in the tutorial content itself, keeping the same meaning
while wrapping the long sentences more cleanly in the opening section of
hami-isolation-k3s.md.
In `@tutorials/overview.md`:
- Line 20: Separate the LabCardGridAuto block from the explanatory prose by
putting the text in its own paragraph in the overview content. Update the MDX
around LabCardGridAuto so it stands alone as a block, then keep the labs
description as a separate paragraph immediately after it, preserving the
existing wording and references to the lab prerequisites.
---
Nitpick comments:
In `@tutorials/labs/examples/07-hami-isolation-k3s/share-two-pods.yaml`:
- Around line 13-20: The pod spec in the shared examples is missing basic
hardening defaults, so update both container definitions (including the cuda
container and the other container in this manifest) to run as non-root and
prevent privilege escalation. Add a minimal securityContext for each container
with allowPrivilegeEscalation set to false and a non-root runAsUser/runAsNonRoot
setting so the copied manifest is safer by default.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 7c8e85ac-e55c-414c-a08f-a2364b590a4c
📒 Files selected for processing (7)
i18n/zh/docusaurus-plugin-content-docs-tutorials/current/labs/hami-isolation-k3s.mdi18n/zh/docusaurus-plugin-content-docs-tutorials/current/overview.mdsidebars-tutorials.jstutorials/labs/examples/07-hami-isolation-k3s/oversubscribe-pending.yamltutorials/labs/examples/07-hami-isolation-k3s/share-two-pods.yamltutorials/labs/hami-isolation-k3s.mdtutorials/overview.md
| <LabCardGridAuto /> | ||
|
|
||
| 每个实验都列出了各自的前提条件。实验 3 和 4 直接复用实验 1 搭建的集群，一次开机即可完成全部三个实验；实验 2 可在任意笔记本上运行，无需 GPU。 | ||
| 每个实验都列出了各自的前提条件。实验 3 和 4 直接复用实验 1 搭建的集群，一次开机即可完成全部三个实验；实验 2 可在任意笔记本上运行，无需 GPU。实验 7 在租用的 GPU 虚拟机上自行搭建单节点 k3s 集群，不使用 GPU Operator。 |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
将实验列表和后面的说明拆成独立段落。
LabCardGridAuto 会渲染成块级元素，和说明文字写在同一行会导致 MDX 结构不够稳妥。请把后面的说明单独放到一个段落里。
♻️ Proposed fix
-<LabCardGridAuto />
-每个实验都列出了各自的前提条件。实验 3 和 4 直接复用实验 1 搭建的集群，一次开机即可完成全部三个实验；实验 2 可在任意笔记本上运行，无需 GPU。实验 7 在租用的 GPU 虚拟机上自行搭建单节点 k3s 集群，不使用 GPU Operator。
+<LabCardGridAuto />
+
+每个实验都列出了各自的前提条件。实验 3 和 4 直接复用实验 1 搭建的集群，一次开机即可完成全部三个实验；实验 2 可在任意笔记本上运行，无需 GPU。实验 7 在租用的 GPU 虚拟机上自行搭建单节点 k3s 集群，不使用 GPU Operator。📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| 每个实验都列出了各自的前提条件。实验 3 和 4 直接复用实验 1 搭建的集群，一次开机即可完成全部三个实验；实验 2 可在任意笔记本上运行，无需 GPU。实验 7 在租用的 GPU 虚拟机上自行搭建单节点 k3s 集群，不使用 GPU Operator。 | |
| 每个实验都列出了各自的前提条件。实验 3 和 4 直接复用实验 1 搭建的集群，一次开机即可完成全部三个实验；实验 2 可在任意笔记本上运行，无需 GPU。实验 7 在租用的 GPU 虚拟机上自行搭建单节点 k3s 集群，不使用 GPU Operator。 |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@i18n/zh/docusaurus-plugin-content-docs-tutorials/current/overview.md` at line
22, The lab list and the following explanatory text are currently in the same
paragraph, which is fragile for the block-level LabCardGridAuto rendering.
Update the overview content so the experiment list remains one paragraph and
move the trailing explanation about experiments 3, 4, 2, and 7 into its own
separate paragraph in the same MDX section.
Source: Linked repositories
| ## Labs | ||
|
|
||
| <LabCardGridAuto /> Each lab lists its own prerequisites. Labs 3 and 4 continue from the cluster Lab 1 builds, so a single session covers all three; Lab 2 runs on any laptop with no GPU required. | ||
| <LabCardGridAuto /> Each lab lists its own prerequisites. Labs 3 and 4 continue from the cluster Lab 1 builds, so a single session covers all three; Lab 2 runs on any laptop with no GPU required. Lab 7 brings up its own single-node k3s cluster on a rented GPU VM, without the GPU Operator. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Separate the labs grid from the following paragraph.
LabCardGridAuto renders a block element, so keeping prose on the same line can produce awkward MDX output. Put the explanatory text in its own paragraph.
♻️ Proposed fix
-<LabCardGridAuto /> Each lab lists its own prerequisites. Labs 3 and 4 continue from the cluster Lab 1 builds, so a single session covers all three; Lab 2 runs on any laptop with no GPU required. Lab 7 brings up its own single-node k3s cluster on a rented GPU VM, without the GPU Operator.
+<LabCardGridAuto />
+
+Each lab lists its own prerequisites. Labs 3 and 4 continue from the cluster Lab 1 builds, so a single session covers all three; Lab 2 runs on any laptop with no GPU required. Lab 7 brings up its own single-node k3s cluster on a rented GPU VM, without the GPU Operator.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| <LabCardGridAuto /> Each lab lists its own prerequisites. Labs 3 and 4 continue from the cluster Lab 1 builds, so a single session covers all three; Lab 2 runs on any laptop with no GPU required. Lab 7 brings up its own single-node k3s cluster on a rented GPU VM, without the GPU Operator. | |
| <LabCardGridAuto /> | |
| Each lab lists its own prerequisites. Labs 3 and 4 continue from the cluster Lab 1 builds, so a single session covers all three; Lab 2 runs on any laptop with no GPU required. Lab 7 brings up its own single-node k3s cluster on a rented GPU VM, without the GPU Operator. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tutorials/overview.md` at line 20, Separate the LabCardGridAuto block from
the explanatory prose by putting the text in its own paragraph in the overview
content. Update the MDX around LabCardGridAuto so it stands alone as a block,
then keep the labs description as a separate paragraph immediately after it,
preserving the existing wording and references to the lab prerequisites.
Source: Linked repositories
Adds a new hands-on lab (English + Chinese) that builds a single-node k3s cluster on a cloud GPU VM, installs HAMi without the NVIDIA GPU Operator with nvidia as the default containerd runtime, and proves memory isolation end to end: virtualized nvidia-smi inside each pod, a CUDA allocation refused at the slice by HAMi-core, an oversubscribing pod kept Pending with CardInsufficientMemory, and the libvgpu.so /etc/ld.so.preload injection mechanism. Every output was captured from a live run on a GCP g4-standard-48 Spot VM with one 96 GB NVIDIA RTX PRO 6000 Blackwell (k3s v1.36.2+k3s1, HAMi v2.9.0, driver 610.43.02). The lab design is adapted from Lovedeep Singh's (@ld-singh) AI Factory Operations Lab with his permission, credited in the lab. Closes Project-HAMi#559 Signed-off-by: Saiyam Pathak <saiyam911@gmail.com>
saiyam1814
force-pushed
the
add-hami-isolation-lab
branch
from
98e67bf to
f313736
Compare
Contributor
|
thx, pls replace the em dashes (—) with regular hyphens or commas. our repo's style rule: everything else here looks good, good numbering, full zh translation, correct sidebar entry, thx for this lab. |
Signed-off-by: Saiyam Pathak <saiyam911@gmail.com>
Contributor
Author
|
@mesutoezdil em dashes fixed |
Contributor
thx mate! |
Contributor
|
/lgtm |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR adds
A new hands-on tutorial, Lab 7: GPU Isolation on k3s Without the GPU Operator (English + Chinese), plus example manifests and sidebar wiring.
The lab builds a single-node k3s cluster on a cloud GPU VM, installs HAMi without the NVIDIA GPU Operator (HAMi's own device plugin,
nvidiaas the default containerd runtime), and proves memory isolation end to end:gpumemslicesnvidia-smiinside each Pod (8000 MiB shown on a 97887 MiB card)Device 0 OOM 8629780480 / 8388608000) while ~82 GB was physically freePendingwithCardInsufficientMemorylibvgpu.sovia/etc/ld.so.preload+CUDA_DEVICE_MEMORY_LIMIT_0How it differs from Lab 3
Lab 3 proves isolation on the GPU Operator path; this lab covers the other supported path (no Operator, k3s, default runtime) used on edge nodes, bare metal, and rented GPU VMs — and it exposes the injection mechanism that path hides.
Provenance and verification
g4-standard-48Spot VM, one 96 GB NVIDIA RTX PRO 6000 Blackwell, k3s v1.36.2+k3s1, HAMi v2.9.0, driver 610.43.02 (open kernel modules).npm run buildpasses for both locales.Closes #559
Summary by CodeRabbit
Pendingwhen insufficient.