fix: rewrite firmware collection to use veritas container approach#94
Merged
butler54 merged 7 commits intoJun 2, 2026
Merged
Conversation
Add exclude block to inject-coco-initdata Kyverno policy to skip pods with label coco.io/skip-initdata: "true". Update firmware collection script to add this label to the pod, preventing Kyverno from trying to inject init_data (which would fail since the pod doesn't have coco.io/initdata-configmap annotation). The firmware collection pod doesn't need init_data injection because it only collects measurements from the TEE device - it doesn't attest to KBS or request secrets. Fixes error: mutation policy inject-coco-initdata error: failed to evaluate preconditions: failed to substitute variables in condition key Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Complete rewrite based on Red Hat documentation and veritas usage: - Runs veritas via podman container (quay.io/openshift_sandboxed_containers/coco-tools:1.12) - No cluster pods needed - computes firmware values locally from OCP release artifacts - Auto-detects OCP version from cluster or accepts --ocp-version flag - Extracts reference-values.json from veritas ConfigMap output - Saves to ~/.coco-pattern/firmware-reference-values.json - Uses --hw-xfam-allow x87,sse,avx to prevent attestation failures Previous approach was fundamentally wrong: - Tried to run veritas inside a kata pod on the cluster - Tried to "collect" from /dev/tdx_guest (doesn't work that way) - Veritas doesn't collect from running hardware - it computes expected values from OCP release artifacts (kata RPMs, edk2 firmware, etc.) Now follows the documented approach: https://docs.redhat.com/en/documentation/openshift_sandboxed_containers/1.12 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Veritas outputs firmware reference values as a JSON array:
[{"name": "mr_td", "value": [...]}, ...]
But the trustee-chart RVPS ConfigurationPolicy template expects
an object format:
{"mr_td": [...], "rtmr_1": [...], ...}
Transform the veritas output using jq:
[.[] | {(.name): .value}] | add
This fixes the RVPS policy error:
can't evaluate field mr_td in type []interface {}
Also added jq to prerequisites check.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
Do we need to add the "-r" option to the initial yq command? I'm running this on a Fedora 43 machine with yq 3.4.3 and I get But if I add the "-r" I proper output. |
yq v3 (kislyuk/yq) outputs JSON strings with quotes by default, so the embedded JSON in the YAML ConfigMap stays quoted and jq receives a string instead of an array. Adding -r outputs the raw string, which both yq v3 and v4 handle correctly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
Good catch — this is a yq v3 vs v4 compatibility issue. yq v3 (kislyuk/yq) outputs JSON strings with quotes by default, so the embedded JSON stays quoted and jq receives a string instead of an array. yq v4 (mikefarah/yq) outputs raw scalars by default so it worked on macOS. Fixed in ea4e977 — added |
Generalize collect-firmware-refvals.sh to support both platforms via --platform flag (baremetal default, azure optional). This replaces get-pcr.sh for Azure deployments — veritas pulls the same dm-verity image, verifies its signature via cosign, and extracts PCR values. Azure: outputs to ~/.coco-pattern/measurements.json (pcrStash secret) Baremetal: outputs to ~/.coco-pattern/firmware-reference-values.json Also adds yq -r fix for v3/v4 cross-compatibility and a new 'make collect-azure-refvals' Makefile target. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace get-pcr.sh call with collect-firmware-refvals.sh --platform azure in wrapper.sh. Add missing reference value collection step to wrapper-multicluster.sh (was never collecting PCR values for Vault). Update RHDP README with prerequisites, env vars, and all deployment modes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cherry-picks the veritas container rewrite from the
baremetal-tests-20260529test branch. These changes were validated on a live bare metal cluster but missed being PR'd to main.Changes
scripts/collect-firmware-refvals.sh— replaces the in-cluster kata pod approach with a local podman container approach usingquay.io/openshift_sandboxed_containers/coco-tools:1.12Why
The old script required a running confidential container (kata-cc pod) to collect firmware measurements from inside the TEE. The new approach computes reference values offline from OCP release artifacts via the veritas tool in a container — no cluster pods needed.
pip install veritas-collectdinside podveritasCLI in coco-tools containeroc,jq, cluster accesspodman,yq,jq, pull secretTesting
Validated on bare metal cluster (OCP 4.20.18, Intel TDX) — firmware values collected via this script were loaded to Vault and used for successful attestation (configuration:2, hardware:2, executables:4).
Co-Authored-By: Claude Sonnet 4.5 noreply@anthropic.com