Repository navigation
Release v0.8.0
NVCRE v0.8.0
The NVIDIA Cluster Readiness Engine (NVCRE) is a Kubernetes controller that certifies GPU clusters before production use. It runs real training and communication workloads across topology-aware node groups, measures goodput and bandwidth, detects hardware failures, and reports every failed node with a reason. See the README for the full feature list.
Install nvcrectl
The installer is signed. Download it with its bundle, verify, then run — piping it into a shell executes it before anything has checked it.
REL=https://github.com/NVIDIA/cluster-readiness-engine/releases/download/v0.8.0
curl -fsSLO "${REL}/installer"
curl -fsSLO "${REL}/installer.sigstore.json"
cosign verify-blob-attestation \
--bundle installer.sigstore.json \
--type https://slsa.dev/provenance/v1 \
--certificate-identity "https://github.com/NVIDIA/cluster-readiness-engine/.github/workflows/attest.yml@refs/tags/v0.8.0" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
installer
bash installer -v v0.8.0Without cosign
curl -fsSL https://github.com/NVIDIA/cluster-readiness-engine/releases/download/v0.8.0/installer | bash -s -- -v v0.8.0This runs the installer before anything verifies it. TLS proves you reached GitHub; it proves nothing about the asset if the release itself was tampered with. Use it only where installing cosign first is genuinely impractical.
# newest stable release; `releases/latest` never resolves to a pre-release
curl -fsSL https://github.com/NVIDIA/cluster-readiness-engine/releases/latest/download/installer | bashHelm Chart
This release's chart is published at digest sha256:889342a75de8599728625479b3337fc67f4f99e83f8fab2df5c3a40585d48d1a. The signature covers that digest; the version tag is mutable, so verify and install the same digest rather than verifying a digest and installing a tag.
CHART=ghcr.io/nvidia/cluster-readiness-engine
DIGEST=sha256:889342a75de8599728625479b3337fc67f4f99e83f8fab2df5c3a40585d48d1a
# 1. Verify the chart. Requires cosign; the package is public, so no login is needed.
cosign verify \
--certificate-identity "https://github.com/NVIDIA/cluster-readiness-engine/.github/workflows/attest.yml@refs/tags/v0.8.0" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
"${CHART}@${DIGEST}"
# 2. Pull that exact digest, then install the file you just verified.
helm pull "oci://${CHART}@${DIGEST}"
helm install nvcre ./cluster-readiness-engine@${DIGEST/:/-}.tgz \
--namespace nvcre --create-namespaceInstalling with --version v0.8.0 instead resolves the tag at install time, which is not provably the digest you verified.
helm install --verify does not check this signature. That flag looks for a PGP .prov file, a mechanism unrelated to Sigstore, and we publish no .prov files.
Controller image
A different artifact from the chart, verified separately. The release built and signed these exact digests:
| Subject | Digest |
|---|---|
| index (multi-platform) | sha256:0ae58498427d1238dc15b6f4c92479a90104a55670b125042ec9dca60e6b5a3b |
linux/amd64 |
sha256:92a556640f39ff48f175f2bc3f94e14bab180904cd6bb7be9cc48b004b3f2406 |
linux/arm64 |
sha256:b8fd3d57ddd2dcfa399655020f1833f4773801f2df8a689ae406fa25d4c1b60c |
The index carries the signature and the SLSA provenance; each platform digest carries a signature and its own CycloneDX SBOM. Pin the index digest rather than the tag:
IMAGE=ghcr.io/nvidia/cluster-readiness-engine/manager
INDEX=sha256:0ae58498427d1238dc15b6f4c92479a90104a55670b125042ec9dca60e6b5a3b
cosign verify \
--certificate-identity "https://github.com/NVIDIA/cluster-readiness-engine/.github/workflows/attest.yml@refs/tags/v0.8.0" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
"${IMAGE}@${INDEX}"nvcrectl CLI
Download the binary for your platform from the assets below and place it on your $PATH.
| Platform | Binary |
|---|---|
| Linux amd64 | nvcrectl-linux-amd64 |
| Linux arm64 | nvcrectl-linux-arm64 |
| macOS amd64 | nvcrectl-darwin-amd64 |
| macOS arm64 (Apple Silicon) | nvcrectl-darwin-arm64 |
Verify your download
Every asset ships with a Sigstore bundle, <asset>.sigstore.json, carrying a SLSA Build Provenance v1 statement. Verify the artifact itself, not a checksum served from the same place as the artifact:
REL=https://github.com/NVIDIA/cluster-readiness-engine/releases/download/v0.8.0
ASSET=nvcrectl-linux-amd64 # or installer, or any published asset
# The bundle is a separate asset; fetch both.
curl -fsSLO "${REL}/${ASSET}"
curl -fsSLO "${REL}/${ASSET}.sigstore.json"
cosign verify-blob-attestation \
--bundle "${ASSET}.sigstore.json" \
--type https://slsa.dev/provenance/v1 \
--certificate-identity "https://github.com/NVIDIA/cluster-readiness-engine/.github/workflows/attest.yml@refs/tags/v0.8.0" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
"${ASSET}"Each nvcrectl-* binary also ships a CycloneDX SBOM, <binary>.cyclonedx.json. There are two bundles involving it, one segment apart in name, and they prove different things:
| Bundle | Proves |
|---|---|
<binary>.cyclonedx.json.sigstore.json |
the SBOM file is the one we published and was not altered |
<binary>.cyclonedx.sigstore.json |
the SBOM describes that binary — it is bound to the binary as its subject |
Before feeding an SBOM to a scanner, check both. The binding proves an SBOM describes the binary; it says nothing about the bytes of the file on your disk, which is a separate download.
cosign verify-blob-attestation \
--bundle "${ASSET}.cyclonedx.sigstore.json" \
--type cyclonedx \
--certificate-identity "https://github.com/NVIDIA/cluster-readiness-engine/.github/workflows/attest.yml@refs/tags/v0.8.0" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
"${ASSET}"
# 2. the SBOM file you downloaded is the one that was published
cosign verify-blob-attestation \
--bundle "${ASSET}.cyclonedx.json.sigstore.json" \
--type https://slsa.dev/provenance/v1 \
--certificate-identity "https://github.com/NVIDIA/cluster-readiness-engine/.github/workflows/attest.yml@refs/tags/v0.8.0" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
"${ASSET}.cyclonedx.json"Only the first would let a tampered local .cyclonedx.json reach your scanner: the binding attestation carries its own copy of the SBOM and never looks at your file.
checksums.txt is still published and still works, but it is not the verification story: it is served from the same origin as the assets and is itself unsigned, so it detects corruption rather than tampering.
sha256sum --check --ignore-missing checksums.txtThird-Party Notices
Third-party Go module licenses are listed in THIRD_PARTY_NOTICES.md and attached as a release asset.
Support
- Ask questions in GitHub Discussions.
- Report bugs with the issue templates.
- Report security issues per SECURITY.md, never through GitHub issues.
What's Changed
Dependency Updates
- build(deps): bump the docker-images group with 2 updates by @dependabot[bot] in #517
- build(deps): bump the go-dependencies group with 2 updates by @dependabot[bot] in #518
- build(deps-dev): bump fern-api from 5.144.3 to 5.152.2 in /fern in the fern group by @dependabot[bot] in #519
- build(deps): bump the actions group with 4 updates by @dependabot[bot] in #520
Other Changes
- chore(fern): register v0.7.0 by @github-actions[bot] in #484
- fix(catalog): select the nemotron5 attention kernel through --attention-backend by @kaynetu in #487
- fix(platform): mount WorkloadRun volumes, config and init containers on MPI workers by @ndipebot in #483
- feat(rbac): grant the controller write access per run namespace (ADR-092) by @kaynetu in #489
- docs(designs): add ADR-096 for Mokka local development and a UAT runtime lane by @kaynetu in #493
- fix(controller): fail Jobs whose workload create stays rejected by @asivanadi0 in #490
- fix(controller): judge each Job attempt on its own GoodputMeasurement by @asivanadi0 in #492
- feat(catalog): make the dcgm-level4 DCGM host configurable by @Rico00121 in #495
- fix: key report group bandwidth by group name and mark provisional results by @asivanadi0 in #465
- fix: preserve duplicate category identity in watch output by @efegokdemir in #473
- fix(controller): preserve workflow label on bandwidth metrics by @efegokdemir in #476
- fix(controller): record a group retry before retiring the failed attempt by @kaynetu in #496
- feat(api): let a Certification set workload environment variables by @ndipebot in #503
- docs(designs): add ADR-098 for setup init chart values and carry-forward by @kaynetu in #506
- fix(platform): stop force-enabling NVLS in the base NCCL environment by @ndipebot in #502
- feat(uat): add the Mokka local environment and UAT runtime lane (ADR-096) by @kaynetu in #497
- fix(catalog): stop forcing NVLS where no measurement backs it by @ndipebot in #515
- feat(setup): take chart values in setup init and keep them across runs by @kaynetu in #512
- fix(controller): record the failure attribution NVCRE already computes (ADR-099) by @ndipebot in #508
- fix(catalog): give each NCCL variable one owning fragment on OCI by @ndipebot in #516
- docs: prepare the v0.8.0 release by @ndipebot in #523
New Contributors
- @Rico00121 made their first contribution in #495
Full Changelog: v0.7.0...v0.8.0