Skip to content

26.07

Latest
Latest

Choose a tag to compare

View all tags
@dholt dholt released this 15 Jul 00:08
26.07
160e340
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

DeepOps 26.07 is the second release of the modern CalVer train, following 26.05 on a six-week cadence.

General

  • New DGX software stack documentation and refreshed nvidia-dgx role covering DGX OS 6 (Ubuntu 22.04), DGX OS 7 (Ubuntu 24.04), DGX Software for RHEL 8, and Enterprise Linux 9, while preserving legacy Ubuntu 18.04/20.04 and RHEL 7 paths (#1346).
  • Clarified current release validation targets versus legacy/community-supported OS paths across the README and docs (#1354).
  • Script debug output now reports a meaningful DeepOps version derived from the checkout's git tag description, with a config/env.sh override (#1357).

Container runtime and OS support

  • New nvidia_container_toolkit role: Ubuntu 24.04+ hosts install the NVIDIA Container Toolkit from the current libnvidia-container repository and configure Docker with nvidia-ctk (#1347).
  • Red Hat family 8+ hosts now use the same role via the official NVIDIA Container Toolkit RPM repository; RPM airgap docs refreshed for EL8/EL9 (#1350).
  • Container Toolkit airgap docs moved from the legacy nvidia-docker wrapper to mirrored package repositories plus nvidia-ctk configuration (#1352).

Provisioning

  • Retired DeepOps-owned DGXIE, minimal PXE/Pixiecore, and Ubuntu 16.04/18.04 preseed assets. MAAS owns machine lifecycle, DHCP/PXE, images, and deployment; DeepOps consumes MAAS tags and dynamic inventory (#1351).

Kubernetes

  • GPU Operator chart default updated to v26.3.3; Kubernetes GPU device plugin and GPU feature discovery charts updated to 0.19.3; MIG manager packages updated to mig-parted v0.14.2; Network Operator default updated to 26.4.0 (#1357).
  • Kubernetes storage helper support boundaries clarified: the NFS subdir provisioner remains the simple default path (chart 4.0.18), NetApp Trident defaults/docs refreshed to Trident 26.02.1 with opt-in backend creation, and the deprecated Rook/Ceph helper now requires explicit opt-in (#1358).

Slurm

  • Slurm default version updated to 26.05.1, the current upstream stable line (#1357).
  • DCGM exporter refreshed to 4.5.3-4.8.2 with current upstream metrics CSVs and a distroless-image-compatible DaemonSet (#1348).

Monitoring

  • Prometheus v3.13.0, Alertmanager v0.33.0, Grafana 13.1.0, node exporter v1.11.1, and kube-prometheus-stack chart 87.5.1 (#1348, #1357).

Examples and documentation

  • CUDA/NGC example images refreshed to current tags across airgap docs, Kubernetes workload examples, and smoke scripts (#1349).
  • RAPIDS/Dask, rootless Docker, RoCE/OFED, NGC-Ready workload, Slurm validation, and registry cache examples refreshed to current NGC and Kubernetes references; new Kubernetes RDMA deployments are pointed at the NVIDIA Network Operator with the DeepOps RoCE role retained as a legacy direct-host path (#1356).
  • Jenkins-era CI harness and Vagrant virtual lab documented as legacy/community-supported reference material (#1353).

Agent operability

  • New AGENTS.md operating guide: repository map, first-time setup, Slurm and Kubernetes golden paths, operating rules, and known gotchas that read as failures but are not (#1360).
  • New scripts/validation/ machine-readable validation tools for automation and AI agents: deepops_doctor.py provisioning preflight, validate_slurm.py, and validate_k8s.py, each with --json flat output, explicit failure lists, and strict exit codes (#1359).
  • New skills/ directory with four reusable procedures (deploy-slurm-cluster, deploy-k8s-gpu-cluster, validate-gpu-cluster, diagnose-driver-install) in the cross-tool agent-skills format (#1360).

Fixes

  • parse_manifest.py firmware version parsing no longer aborts on a non-dict parseable manifest line; adds a regression test (#1355).

Known issues and validation notes

  • Component support boundaries recorded for this release: registry/cache and the refreshed workload examples are supported paths; the RoCE backend role is a documented legacy direct-host path; Open OnDemand, the OpenShift helper role, and the Singularity wrapper are community-supported, with ownership calls planned next release.
  • Release validation ran on single-node GPU hardware: fresh Slurm deployment with a GPU job, fresh Kubernetes deployment with GPU Operator and a CUDA pod, and a MAAS provisioning smoke. Multi-node scheduling and Network Operator install behavior on RDMA-capable networking hardware were not covered in this cycle.
  • Direct in-place Kubernetes upgrades from 23.08 remain blocked by Kubespray's Calico minimum-version prechecks; use the documented staged upgrade or redeploy path for much older clusters.
Assets 2
Loading