diff --git a/README.md b/README.md index 08c98e82..944454db 100644 --- a/README.md +++ b/README.md @@ -2,27 +2,29 @@ Validated pattern for deploying confidential containers on OpenShift using the [Validated Patterns](https://validatedpatterns.io/) framework. -Confidential containers use hardware-backed Trusted Execution Environments (TEEs) to isolate workloads from cluster and hypervisor administrators. This pattern deploys and configures the Red Hat CoCo stack — including the sandboxed containers operator, Trustee (Key Broker Service), and peer-pod infrastructure — on Azure. +Confidential containers use hardware-backed Trusted Execution Environments (TEEs) to isolate workloads from cluster and hypervisor administrators. This pattern deploys and configures the Red Hat CoCo stack — including the sandboxed containers operator, Trustee (Key Broker Service) operator, and Kata infrastructure — on Azure cloud instances and bare metal. ## Topologies -The pattern provides two deployment topologies: +The pattern provides three deployment topologies: -1. **Single cluster** (`simple` clusterGroup) — deploys all components (Trustee, Vault, ACM, sandboxed containers, workloads) in one cluster. This breaks the RACI separation expected in a remote attestation architecture but simplifies testing and demonstrations. +1. **Single cluster** (`simple` clusterGroup) — deploys all components (Trustee, Vault, ACM, sandboxed containers, workloads) in one cluster on Azure. This breaks the RACI separation expected in a remote attestation architecture but simplifies testing and demonstrations. 2. **Multi-cluster** (`trusted-hub` + `spoke` clusterGroups) — separates the trusted zone from the untrusted workload zone: - **Hub** (`trusted-hub`): Runs Trustee (KBS + attestation service), HashiCorp Vault, ACM, and cert-manager. This cluster is the trust anchor. - **Spoke** (`spoke`): Runs the sandboxed containers operator and confidential workloads. The spoke is imported into ACM and managed from the hub. +3. **Bare metal** (`baremetal` clusterGroup) — deploys all components on bare metal hardware with Intel TDX or AMD SEV-SNP support. NFD (Node Feature Discovery) auto-detects the CPU architecture and configures the appropriate runtime. Supports SNO (Single Node OpenShift) and multi-node clusters. + The topology is controlled by the `main.clusterGroupName` field in `values-global.yaml`. -Currently supports Azure via peer-pods. Peer-pods provision confidential VMs (`Standard_DCas_v5` family) directly on the Azure hypervisor rather than nesting VMs inside worker nodes. +Azure deployments use peer-pods, which provision confidential VMs (`Standard_DCas_v5` family) directly on the Azure hypervisor. Bare metal deployments use layered images and hardware TEE features directly. ## Current version (4.*) Breaking change from v3. This is the first version using GA (Generally Available) releases of the CoCo stack: -- **OpenShift Sandboxed Containers 1.12+** (requires OCP 4.17+) +- **OpenShift Sandboxed Containers 1.12+** (requires OCP 4.19.28+) - **Red Hat Build of Trustee 1.1** (GA release; all versions prior to 1.0 were Technology Preview) - External chart repositories for [Trustee](https://github.com/validatedpatterns/trustee-chart), [sandboxed-containers](https://github.com/validatedpatterns/sandboxed-containers-chart), and [sandboxed-policies](https://github.com/validatedpatterns/sandboxed-policies-chart) - Self-signed certificates via cert-manager (Let's Encrypt no longer required) @@ -42,9 +44,21 @@ All previous versions used pre-GA (Technology Preview) releases of Trustee: ### Prerequisites -- OpenShift 4.17+ cluster on Azure (self-managed via `openshift-install` or ARO) +**Azure deployments:** + +- OpenShift 4.19.28+ cluster on Azure (self-managed via `openshift-install` or ARO) - Azure `Standard_DCas_v5` VM quota in your target region (these are confidential computing VMs and are not available in all regions). See the note below for more details. - Azure DNS hosting the cluster's DNS zone + +**Bare metal deployments:** + +- OpenShift 4.19.28+ cluster on bare metal with Intel TDX or AMD SEV-SNP hardware +- BIOS/firmware configured to enable TDX or SEV-SNP +- Available block devices for LVMS storage (auto-discovered) +- For Intel TDX: an Intel PCS API key from [api.portal.trustedservices.intel.com](https://api.portal.trustedservices.intel.com/) + +**Common:** + - Tools on your workstation: `podman`, `yq`, `jq`, `skopeo` - OpenShift pull secret saved at `~/pull-secret.json` (download from [console.redhat.com](https://console.redhat.com/openshift/downloads)) - Fork the repository — ArgoCD reconciles cluster state against your fork, so changes must be pushed to your remote @@ -53,29 +67,48 @@ All previous versions used pre-GA (Technology Preview) releases of Trustee: These scripts generate the cryptographic material and attestation measurements needed by Trustee and the peer-pod VMs. Run them once before your first deployment. -1. `bash scripts/gen-secrets.sh` — generates KBS key pairs, attestation policy seeds, and copies `values-secret.yaml.template` to `~/values-secret-coco-pattern.yaml` -2. `bash scripts/get-pcr.sh` — retrieves PCR measurements from the peer-pod VM image and stores them at `~/.coco-pattern/measurements.json` (requires `podman`, `skopeo`, and `~/pull-secret.json`) -3. Review and customise `~/values-secret-coco-pattern.yaml` — this file is loaded into Vault and provides secrets to the pattern +1. `bash scripts/gen-secrets.sh` — generates KBS key pairs, PCCS certificates/tokens (for bare metal), and copies `values-secret.yaml.template` to `~/values-secret-coco-pattern.yaml` +2. `bash scripts/get-pcr.sh` — retrieves PCR measurements from the peer-pod VM image and stores them at `~/.coco-pattern/measurements.json` (requires `podman`, `skopeo`, and `~/pull-secret.json`). **Azure only.** Bare metal uses manual PCR collection — see [docs/pcr-reference-values-bare-metal.md](docs/pcr-reference-values-bare-metal.md) for the procedure. Store the measurements at `~/.coco-pattern/measurements.json`. +3. Review and customise `~/values-secret-coco-pattern.yaml` — this file is loaded into Vault and provides secrets to the pattern. For bare metal, uncomment the PCCS secrets section and provide your Intel PCS API key. > **Note:** `gen-secrets.sh` will not overwrite existing secrets. Delete `~/.coco-pattern/` if you need to regenerate. -### Single cluster deployment +### Single cluster deployment (Azure) 1. Set `main.clusterGroupName: simple` in `values-global.yaml` 2. Ensure your Azure configuration is populated in `values-global.yaml` (see `global.azure.*` fields) 3. `./pattern.sh make install` 4. Wait for the cluster to reboot all nodes (the sandboxed containers operator triggers a MachineConfig update). Monitor progress in the ArgoCD UI. -### Multi-cluster deployment +### Multi-cluster deployment (Azure) 1. Set `main.clusterGroupName: trusted-hub` in `values-global.yaml` 2. Deploy the hub cluster: `./pattern.sh make install` 3. Wait for ACM (`MultiClusterHub`) to reach `Running` state on the hub -4. Provision a second OpenShift 4.17+ cluster on Azure for the spoke +4. Provision a second OpenShift 4.19.28+ cluster on Azure for the spoke 5. Import the spoke into ACM with label `clusterGroup=spoke` (see [importing a cluster](https://validatedpatterns.io/learn/importing-a-cluster/)) 6. ACM will automatically deploy the `spoke` clusterGroup applications (sandboxed containers, workloads) to the imported cluster +### Bare metal deployment + +1. Set `main.clusterGroupName: baremetal` in `values-global.yaml` +2. Run `bash scripts/gen-secrets.sh` to generate KBS keys and PCCS secrets +3. For Intel TDX: uncomment the PCCS secrets in `~/values-secret-coco-pattern.yaml` and provide your Intel PCS API key +4. `./pattern.sh make install` +5. Wait for the cluster to reboot nodes (MachineConfig updates for TDX kernel parameters and vsock) + +The system auto-detects your hardware: + +- **NFD** discovers Intel TDX or AMD SEV-SNP capabilities and labels nodes +- **LVMS** auto-discovers available block devices for storage +- **RuntimeClass** `kata-cc` is created automatically pointing to the correct handler (`kata-tdx` or `kata-snp`) +- Both `kata-tdx` and `kata-snp` RuntimeClasses are deployed; only the one matching your hardware has schedulable nodes +- MachineConfigs are deployed for both `master` and `worker` roles (safe on SNO where only master exists) +- PCCS and QGS services deploy unconditionally; DaemonSets only schedule on Intel nodes via NFD labels + +Optional: pin PCCS to a specific node with `bash scripts/get-pccs-node.sh` and set `baremetal.pccs.nodeSelector` in the baremetal chart values. + ## Sample applications Two sample applications are deployed on the cluster running confidential workloads (the single cluster in `simple` mode, or the spoke in multi-cluster mode): @@ -85,7 +118,7 @@ Two sample applications are deployed on the cluster running confidential workloa - `secure` — a confidential container with a strict policy; `oc exec` is denied even for `kubeadmin` - `insecure-policy` — a confidential container with a relaxed policy allowing `oc exec` (useful for testing the Confidential Data Hub) - Each confidential pod runs on its own `Standard_DC2as_v5` Azure VM (visible in the Azure portal). Pods use `runtimeClassName: kata-remote`. + On Azure, each confidential pod runs on its own `Standard_DC2as_v5` Azure VM (visible in the Azure portal) using `runtimeClassName: kata-remote`. On bare metal, pods use `runtimeClassName: kata-cc` and run directly on the underlying TDX or SEV-SNP hardware. - **kbs-access**: A web service that retrieves and presents secrets obtained from the Trustee Key Broker Service (KBS) via the Confidential Data Hub (CDH). Useful for verifying end-to-end attestation and secret delivery in locked-down environments. diff --git a/ansible/init-data-gzipper.yaml b/ansible/init-data-gzipper.yaml index f0e10cf6..c9de0a5f 100644 --- a/ansible/init-data-gzipper.yaml +++ b/ansible/init-data-gzipper.yaml @@ -114,21 +114,33 @@ register: debug_initdata_encoded changed_when: false + - name: Compute raw SHA-256 hash of default initdata + ansible.builtin.shell: | + set -o pipefail + sha256sum "{{ rendered_path }}" | cut -d' ' -f1 + register: raw_hash + changed_when: false + + - name: Compute raw SHA-256 hash of debug initdata + ansible.builtin.shell: | + set -o pipefail + sha256sum "{{ debug_rendered_path }}" | cut -d' ' -f1 + register: debug_raw_hash + changed_when: false + - name: Register init data pcr into a var ansible.builtin.shell: | set -o pipefail - hash=$(sha256sum "{{ rendered_path }}" | cut -d' ' -f1) initial_pcr=0000000000000000000000000000000000000000000000000000000000000000 - PCR8_HASH=$(echo -n "$initial_pcr$hash" | xxd -r -p | sha256sum | cut -d' ' -f1) && echo $PCR8_HASH + PCR8_HASH=$(echo -n "${initial_pcr}{{ raw_hash.stdout }}" | xxd -r -p | sha256sum | cut -d' ' -f1) && echo $PCR8_HASH register: pcr8_hash changed_when: false - name: Register debug init data pcr into a var ansible.builtin.shell: | set -o pipefail - hash=$(sha256sum "{{ debug_rendered_path }}" | cut -d' ' -f1) initial_pcr=0000000000000000000000000000000000000000000000000000000000000000 - PCR8_HASH=$(echo -n "$initial_pcr$hash" | xxd -r -p | sha256sum | cut -d' ' -f1) && echo $PCR8_HASH + PCR8_HASH=$(echo -n "${initial_pcr}{{ debug_raw_hash.stdout }}" | xxd -r -p | sha256sum | cut -d' ' -f1) && echo $PCR8_HASH register: debug_pcr8_hash changed_when: false @@ -147,6 +159,7 @@ data: INITDATA: "{{ initdata_encoded.stdout }}" PCR8_HASH: "{{ pcr8_hash.stdout }}" + RAW_HASH: "{{ raw_hash.stdout }}" version: "0.1.0" algorithm: "sha256" aa.toml: "{{ raw_aa_toml.stdout }}" @@ -168,6 +181,7 @@ data: INITDATA: "{{ debug_initdata_encoded.stdout }}" PCR8_HASH: "{{ debug_pcr8_hash.stdout }}" + RAW_HASH: "{{ debug_raw_hash.stdout }}" version: "0.1.0" algorithm: "sha256" aa.toml: "{{ raw_aa_toml.stdout }}" diff --git a/charts/all/baremetal/Chart.yaml b/charts/all/baremetal/Chart.yaml new file mode 100644 index 00000000..33940799 --- /dev/null +++ b/charts/all/baremetal/Chart.yaml @@ -0,0 +1,9 @@ +apiVersion: v2 +description: Bare metal platform configuration (NFD rules, MachineConfigs, RuntimeClasses, Intel device plugin). +keywords: +- pattern +- upstream +- sandbox +- baremetal +name: baremetal +version: 0.0.1 diff --git a/charts/all/baremetal/templates/kata-nfd.yaml b/charts/all/baremetal/templates/kata-nfd.yaml new file mode 100644 index 00000000..6196e02c --- /dev/null +++ b/charts/all/baremetal/templates/kata-nfd.yaml @@ -0,0 +1,80 @@ +apiVersion: nfd.openshift.io/v1alpha1 +kind: NodeFeatureRule +metadata: + name: consolidated-hardware-features + namespace: openshift-nfd +spec: + rules: + - name: "runtime.kata" + labels: + feature.node.kubernetes.io/runtime.kata: "true" + matchAny: + - matchFeatures: + - feature: cpu.cpuid + matchExpressions: + SSE42: { op: Exists } + VMX: { op: Exists } + - feature: kernel.loadedmodule + matchExpressions: + kvm: { op: Exists } + kvm_intel: { op: Exists } + - matchFeatures: + - feature: cpu.cpuid + matchExpressions: + SSE42: { op: Exists } + SVM: { op: Exists } + - feature: kernel.loadedmodule + matchExpressions: + kvm: { op: Exists } + kvm_amd: { op: Exists } + + - name: "amd.sev-snp" + labels: + amd.feature.node.kubernetes.io/snp: "true" + extendedResources: + sev-snp.amd.com/esids: "@cpu.security.sev.encrypted_state_ids" + matchFeatures: + - feature: cpu.cpuid + matchExpressions: + SVM: { op: Exists } + - feature: cpu.security + matchExpressions: + sev.snp.enabled: { op: Exists } + + - name: "intel.sgx" + labels: + intel.feature.node.kubernetes.io/sgx: "true" + extendedResources: + sgx.intel.com/epc: "@cpu.security.sgx.epc" + matchFeatures: + - feature: cpu.cpuid + matchExpressions: + SGX: { op: Exists } + SGXLC: { op: Exists } + - feature: cpu.security + matchExpressions: + sgx.enabled: { op: IsTrue } + - feature: kernel.config + matchExpressions: + X86_SGX: { op: Exists } + + - name: "intel.tdx" + labels: + intel.feature.node.kubernetes.io/tdx: "true" + extendedResources: + tdx.intel.com/keys: "@cpu.security.tdx.total_keys" + matchFeatures: + - feature: cpu.cpuid + matchExpressions: + VMX: { op: Exists } + - feature: cpu.security + matchExpressions: + tdx.enabled: { op: Exists } + + - name: "ibm.se.enabled" + labels: + ibm.feature.node.kubernetes.io/se: "true" + matchFeatures: + - feature: cpu.security + matchExpressions: + se.enabled: { op: IsTrue } diff --git a/charts/all/baremetal/templates/nfd-instance.yaml b/charts/all/baremetal/templates/nfd-instance.yaml new file mode 100644 index 00000000..97ce9ee1 --- /dev/null +++ b/charts/all/baremetal/templates/nfd-instance.yaml @@ -0,0 +1,12 @@ +apiVersion: nfd.openshift.io/v1 +kind: NodeFeatureDiscovery +metadata: + name: nfd-instance + namespace: openshift-nfd +spec: + operand: + image: registry.redhat.io/openshift4/ose-node-feature-discovery-rhel9:v4.20 + imagePullPolicy: Always + servicePort: 12000 + workerConfig: + configData: | diff --git a/charts/all/baremetal/templates/vsock-mco.yaml b/charts/all/baremetal/templates/vsock-mco.yaml new file mode 100644 index 00000000..f8938f62 --- /dev/null +++ b/charts/all/baremetal/templates/vsock-mco.yaml @@ -0,0 +1,24 @@ +{{- range list "master" "worker" }} +--- +apiVersion: machineconfiguration.openshift.io/v1 +kind: MachineConfig +metadata: + labels: + machineconfiguration.openshift.io/role: {{ . }} + name: 99-enable-coco-{{ . }} +spec: + kernelArguments: + - nohibernate +{{- if $.Values.tdx.enabled }} + - kvm_intel.tdx=1 +{{- end }} + config: + ignition: + version: 3.2.0 + storage: + files: + - path: /etc/modules-load.d/vsock.conf + mode: 0644 + contents: + source: data:text/plain;charset=utf-8;base64,dnNvY2stbG9vcGJhY2sK +{{- end }} diff --git a/charts/all/baremetal/values.yaml b/charts/all/baremetal/values.yaml new file mode 100644 index 00000000..3942cb97 --- /dev/null +++ b/charts/all/baremetal/values.yaml @@ -0,0 +1,2 @@ +tdx: + enabled: true diff --git a/charts/all/coco-kyverno-policies/templates/inject-coco-initdata.yaml b/charts/all/coco-kyverno-policies/templates/inject-coco-initdata.yaml index 397ba39b..56483fe9 100644 --- a/charts/all/coco-kyverno-policies/templates/inject-coco-initdata.yaml +++ b/charts/all/coco-kyverno-policies/templates/inject-coco-initdata.yaml @@ -6,14 +6,14 @@ metadata: policies.kyverno.io/title: Inject CoCo InitData policies.kyverno.io/category: Confidential Computing policies.kyverno.io/severity: medium - policies.kyverno.io/subject: Pod,Deployment + policies.kyverno.io/subject: Pod policies.kyverno.io/description: >- Injects cc_init_data annotation into pods with a kata runtime class by reading from a ConfigMap specified via the coco.io/initdata-configmap - annotation. Kyverno autogen extends this to Deployments, StatefulSets, - DaemonSets, and Jobs automatically. + annotation. Targets Pods only (not Deployments) so that Deployments + remain stable and a rollout restart resolves the latest initdata. argocd.argoproj.io/sync-wave: "1" - pod-policies.kyverno.io/autogen-controllers: Deployment,StatefulSet,DaemonSet,Job + pod-policies.kyverno.io/autogen-controllers: none spec: rules: - name: inject-initdata diff --git a/charts/all/intel-dcap/Chart.yaml b/charts/all/intel-dcap/Chart.yaml new file mode 100644 index 00000000..06095d7f --- /dev/null +++ b/charts/all/intel-dcap/Chart.yaml @@ -0,0 +1,10 @@ +apiVersion: v2 +description: Intel DCAP services (PCCS and QGS) for TDX remote attestation. +keywords: +- pattern +- intel +- tdx +- pccs +- qgs +name: intel-dcap +version: 0.0.1 diff --git a/charts/all/intel-dcap/templates/intel-dpo-sgx.yaml b/charts/all/intel-dcap/templates/intel-dpo-sgx.yaml new file mode 100644 index 00000000..9aaa40c0 --- /dev/null +++ b/charts/all/intel-dcap/templates/intel-dpo-sgx.yaml @@ -0,0 +1,11 @@ +apiVersion: deviceplugin.intel.com/v1 +kind: SgxDevicePlugin +metadata: + name: sgxdeviceplugin-sample +spec: + image: registry.connect.redhat.com/intel/intel-sgx-plugin@sha256:4ac8769c4f0a82b3ea04cf1532f15e9935c71fe390ff5a9dc3ee57f970a65f0b + enclaveLimit: 110 + provisionLimit: 110 + logLevel: 4 + nodeSelector: + intel.feature.node.kubernetes.io/sgx: "true" diff --git a/charts/all/intel-dcap/templates/pccs-deployment.yaml b/charts/all/intel-dcap/templates/pccs-deployment.yaml new file mode 100644 index 00000000..51ec1d35 --- /dev/null +++ b/charts/all/intel-dcap/templates/pccs-deployment.yaml @@ -0,0 +1,69 @@ +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: pccs + namespace: intel-dcap +spec: + replicas: 1 + selector: + matchLabels: + app: pccs + template: + metadata: + labels: + app: pccs + trustedservices.intel.com/cache: pccs + spec: + tolerations: + - effect: NoSchedule + key: node-role.kubernetes.io/master + operator: Exists + serviceAccountName: pccs-service-account + {{- if .Values.baremetal.pccs.nodeSelector }} + nodeSelector: + kubernetes.io/hostname: {{ .Values.baremetal.pccs.nodeSelector }} + {{- end }} + initContainers: + - name: init-seclabel + image: registry.access.redhat.com/ubi9/ubi:9.7-1764578509 + command: [ "sh", "-c", "chcon -Rt container_file_t /var/cache/pccs" ] + volumeMounts: + - name: host-database + mountPath: /var/cache/pccs + securityContext: + runAsUser: 0 + runAsGroup: 0 + privileged: true # Required for chcon to work on host files + containers: + - name: pccs + image: registry.redhat.io/openshift-sandboxed-containers/osc-pccs@sha256:edf57087220516115512dc6687b96045cbc69472a3d6d381619f66beae032ad6 + envFrom: + - secretRef: + name: pccs-secrets + env: + - name: "PCCS_LOG_LEVEL" + value: "info" + - name: "CLUSTER_HTTPS_PROXY" + value: "" + - name: "PCCS_FILL_MODE" + value: "LAZY" + ports: + - containerPort: 8042 + name: pccs-port + volumeMounts: + - name: pccs-tls + mountPath: /opt/intel/pccs/ssl_key + readOnly: true + - name: host-database + mountPath: /var/cache/pccs/ + securityContext: + runAsUser: 0 + volumes: + - name: pccs-tls + secret: + secretName: pccs-tls + - name: host-database + hostPath: + path: /var/cache/pccs/ + type: DirectoryOrCreate diff --git a/charts/all/intel-dcap/templates/pccs-rbac.yaml b/charts/all/intel-dcap/templates/pccs-rbac.yaml new file mode 100644 index 00000000..2122caf0 --- /dev/null +++ b/charts/all/intel-dcap/templates/pccs-rbac.yaml @@ -0,0 +1,49 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: pccs-service-account + namespace: intel-dcap +--- +apiVersion: security.openshift.io/v1 +kind: SecurityContextConstraints +metadata: + name: pccs-scc + annotations: + kubernetes.io/description: "SCC for Intel DCAP PCCS service requiring privileged access and hostPath volumes" +allowHostDirVolumePlugin: true +allowHostIPC: false +allowHostNetwork: false +allowHostPID: false +allowHostPorts: false +allowPrivilegedContainer: true +allowedCapabilities: +- DAC_OVERRIDE +- SETGID +- SETUID +defaultAddCapabilities: null +fsGroup: + type: RunAsAny +priority: null +readOnlyRootFilesystem: false +requiredDropCapabilities: +- KILL +- MKNOD +- SETPCAP +- SYS_CHROOT +runAsUser: + type: RunAsAny +seLinuxContext: + type: MustRunAs +supplementalGroups: + type: RunAsAny +users: +- system:serviceaccount:intel-dcap:pccs-service-account +volumes: +- configMap +- downwardAPI +- emptyDir +- hostPath +- persistentVolumeClaim +- projected +- secret diff --git a/charts/all/intel-dcap/templates/pccs-secrets-eso.yaml b/charts/all/intel-dcap/templates/pccs-secrets-eso.yaml new file mode 100644 index 00000000..5ee91ab9 --- /dev/null +++ b/charts/all/intel-dcap/templates/pccs-secrets-eso.yaml @@ -0,0 +1,37 @@ +--- +apiVersion: "external-secrets.io/v1beta1" +kind: ExternalSecret +metadata: + name: pccs-secrets-eso + namespace: intel-dcap +spec: + refreshInterval: 15s + secretStoreRef: + name: {{ .Values.secretStore.name }} + kind: {{ .Values.secretStore.kind }} + target: + name: pccs-secrets + template: + type: Opaque + data: + PCCS_API_KEY: "{{ "{{ .api_key }}" }}" + PCCS_USER_TOKEN_HASH: "{{ "{{ .user_token_hash }}" }}" + USER_TOKEN: "{{ "{{ .user_token }}" }}" + PCCS_ADMIN_TOKEN_HASH: "{{ "{{ .admin_token_hash }}" }}" + data: + - secretKey: api_key + remoteRef: + key: 'secret/data/hub/pccs' + property: api_key + - secretKey: user_token_hash + remoteRef: + key: 'secret/data/hub/pccs' + property: user_token_hash + - secretKey: user_token + remoteRef: + key: 'secret/data/hub/pccs' + property: user_token + - secretKey: admin_token_hash + remoteRef: + key: 'secret/data/hub/pccs' + property: admin_token_hash diff --git a/charts/all/intel-dcap/templates/pccs-service.yaml b/charts/all/intel-dcap/templates/pccs-service.yaml new file mode 100644 index 00000000..bc83684c --- /dev/null +++ b/charts/all/intel-dcap/templates/pccs-service.yaml @@ -0,0 +1,13 @@ +apiVersion: v1 +kind: Service +metadata: + name: pccs-service + namespace: intel-dcap +spec: + selector: + trustedservices.intel.com/cache: pccs + ports: + - name: pccs + protocol: TCP + port: 8042 + targetPort: pccs-port diff --git a/charts/all/intel-dcap/templates/pccs-tls-eso.yaml b/charts/all/intel-dcap/templates/pccs-tls-eso.yaml new file mode 100644 index 00000000..a7212ae1 --- /dev/null +++ b/charts/all/intel-dcap/templates/pccs-tls-eso.yaml @@ -0,0 +1,24 @@ +--- +apiVersion: "external-secrets.io/v1beta1" +kind: ExternalSecret +metadata: + name: pccs-tls-eso + namespace: intel-dcap +spec: + refreshInterval: 15s + secretStoreRef: + name: {{ .Values.secretStore.name }} + kind: {{ .Values.secretStore.kind }} + target: + name: pccs-tls + template: + type: Opaque + data: + - secretKey: private.pem + remoteRef: + key: 'secret/data/hub/pccs-tls' + property: private_key + - secretKey: file.crt + remoteRef: + key: 'secret/data/hub/pccs-tls' + property: certificate diff --git a/charts/all/intel-dcap/templates/qgs-ds.yaml b/charts/all/intel-dcap/templates/qgs-ds.yaml new file mode 100644 index 00000000..6f10feeb --- /dev/null +++ b/charts/all/intel-dcap/templates/qgs-ds.yaml @@ -0,0 +1,88 @@ +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: tdx-qgs + namespace: intel-dcap +spec: + selector: + matchLabels: + app: tdx-qgs + template: + metadata: + labels: + app: tdx-qgs + annotations: + sgx.intel.com/quote-provider: tdx-qgs + qcnl-conf: '{"pccs_url": "https://pccs-service:8042/sgx/certification/v4/", "use_secure_cert": false, "pck_cache_expire_hours": 168}' + spec: + serviceAccountName: tdx-qgs-service-account + nodeSelector: + intel.feature.node.kubernetes.io/tdx: 'true' + hostNetwork: true + dnsPolicy: ClusterFirstWithHostNet + initContainers: + - name: platform-registration + image: registry.redhat.io/openshift-sandboxed-containers/osc-tdx-qgs@sha256:308d66da94d3116cc16530a4a2cd60a995458f331e1cb3487c6ea3fdae60545b + restartPolicy: Always + command: [ '/usr/bin/dcap-registration-flow' ] + env: + - name: PCCS_URL + value: "https://pccs-service:8042" + - name: SECURE_CERT + value: 'false' + envFrom: + - secretRef: + name: pccs-secrets + securityContext: + readOnlyRootFilesystem: true + allowPrivilegeEscalation: true + privileged: true + capabilities: + drop: + - ALL + add: + - LINUX_IMMUTABLE + volumeMounts: + - name: efivars + mountPath: /sys/firmware/efi/efivars + containers: + - name: tdx-qgs + image: registry.redhat.io/openshift-sandboxed-containers/osc-tdx-qgs@sha256:308d66da94d3116cc16530a4a2cd60a995458f331e1cb3487c6ea3fdae60545b + args: + - -p=4050 + - -n=4 + securityContext: + readOnlyRootFilesystem: true + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + resources: + limits: + sgx.intel.com/epc: "512Ki" + sgx.intel.com/enclave: 1 + sgx.intel.com/provision: 1 + env: + - name: QCNL_CONF_PATH + value: "/run/dcap/qcnl_conf" + - name: XDG_CACHE_HOME + value: "/run/dcap/cache" + volumeMounts: + - name: dcap-qcnl-cache + mountPath: /run/dcap/cache + - name: qcnl-config + mountPath: /run/dcap/ + readOnly: true + volumes: + - name: dcap-qcnl-cache + emptyDir: + sizeLimit: 50Mi + - name: qcnl-config + downwardAPI: + items: + - path: "qcnl_conf" + fieldRef: + fieldPath: metadata.annotations['qcnl-conf'] + - name: efivars + hostPath: + path: /sys/firmware/efi/efivars/ diff --git a/charts/all/intel-dcap/templates/qgs-rbac.yaml b/charts/all/intel-dcap/templates/qgs-rbac.yaml new file mode 100644 index 00000000..2dcb591e --- /dev/null +++ b/charts/all/intel-dcap/templates/qgs-rbac.yaml @@ -0,0 +1,47 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: tdx-qgs-service-account + namespace: intel-dcap +--- +apiVersion: security.openshift.io/v1 +kind: SecurityContextConstraints +metadata: + name: tdx-qgs-scc + annotations: + kubernetes.io/description: "SCC for Intel TDX Quote Generation Service requiring host network access and SGX devices" +allowHostDirVolumePlugin: true +allowHostIPC: false +allowHostNetwork: true +allowHostPID: false +allowHostPorts: false +allowPrivilegedContainer: true +allowedCapabilities: +- LINUX_IMMUTABLE +defaultAddCapabilities: null +fsGroup: + type: RunAsAny +priority: null +readOnlyRootFilesystem: false +requiredDropCapabilities: +- KILL +- MKNOD +- SETPCAP +- SYS_CHROOT +runAsUser: + type: RunAsAny +seLinuxContext: + type: MustRunAs +supplementalGroups: + type: RunAsAny +users: +- system:serviceaccount:intel-dcap:tdx-qgs-service-account +volumes: +- configMap +- downwardAPI +- emptyDir +- hostPath +- persistentVolumeClaim +- projected +- secret diff --git a/charts/all/intel-dcap/values.yaml b/charts/all/intel-dcap/values.yaml new file mode 100644 index 00000000..5910caed --- /dev/null +++ b/charts/all/intel-dcap/values.yaml @@ -0,0 +1,7 @@ +baremetal: + pccs: + nodeSelector: "" # optional: hostname for PCCS pinning. Empty = schedule anywhere with tolerations. + +secretStore: + name: "" + kind: "" diff --git a/charts/coco-supported/hello-openshift/templates/_helpers.tpl b/charts/coco-supported/hello-openshift/templates/_helpers.tpl index cc79680e..ab49aab2 100644 --- a/charts/coco-supported/hello-openshift/templates/_helpers.tpl +++ b/charts/coco-supported/hello-openshift/templates/_helpers.tpl @@ -51,11 +51,14 @@ app.kubernetes.io/instance: {{ .Release.Name }} {{- end }} {{/* -Determine runtime class name based on cluster platform. -Cloud (Azure/AWS) uses "kata-remote" for peer-pods; baremetal uses "kata-cc" for confidential containers. +Determine runtime class name. +If runtimeClassName is explicitly set, use it. +Otherwise, detect from cluster platform: "kata-remote" for Azure/AWS, "kata-cc" for other platforms. */}} {{- define "hello-openshift.runtimeClassName" -}} -{{- if or (eq .Values.global.clusterPlatform "Azure") (eq .Values.global.clusterPlatform "AWS") -}} +{{- if .Values.runtimeClassName -}} +{{- .Values.runtimeClassName -}} +{{- else if or (eq .Values.global.clusterPlatform "Azure") (eq .Values.global.clusterPlatform "AWS") -}} kata-remote {{- else -}} kata-cc diff --git a/charts/coco-supported/hello-openshift/values.yaml b/charts/coco-supported/hello-openshift/values.yaml index c9dbbe0a..39f2b9ff 100644 --- a/charts/coco-supported/hello-openshift/values.yaml +++ b/charts/coco-supported/hello-openshift/values.yaml @@ -1,6 +1,11 @@ # Chart-specific values # Common values are inherited from values-global.yaml +# Runtime class for confidential containers. +# When empty, auto-detected from global.clusterPlatform (kata-remote for Azure/AWS, kata-cc otherwise). +# Bare metal: set to "kata-cc" via values-baremetal.yaml overrides. +runtimeClassName: "" + # Global values used by this chart (overridden by values-global.yaml) global: clusterPlatform: "" # Cluster platform: "Azure" or "AWS" - determines runtime class diff --git a/charts/coco-supported/kbs-access/templates/secure-deployment.yaml b/charts/coco-supported/kbs-access/templates/secure-deployment.yaml index 4a63170c..e53e874f 100644 --- a/charts/coco-supported/kbs-access/templates/secure-deployment.yaml +++ b/charts/coco-supported/kbs-access/templates/secure-deployment.yaml @@ -20,7 +20,7 @@ spec: io.katacontainers.config.hypervisor.default_memory: {{ .Values.defaultMemory | quote }} {{- end }} spec: - runtimeClassName: {{ if or (eq .Values.global.clusterPlatform "Azure") (eq .Values.global.clusterPlatform "AWS") }}kata-remote{{ else }}kata-cc{{ end }} + runtimeClassName: {{ if .Values.runtimeClassName }}{{ .Values.runtimeClassName }}{{ else if or (eq .Values.global.clusterPlatform "Azure") (eq .Values.global.clusterPlatform "AWS") }}kata-remote{{ else }}kata-cc{{ end }} containers: - name: python-access image: ghcr.io/butler54/kbs-access-app:latest diff --git a/charts/coco-supported/kbs-access/values.yaml b/charts/coco-supported/kbs-access/values.yaml index 6d3496ac..a0da355f 100644 --- a/charts/coco-supported/kbs-access/values.yaml +++ b/charts/coco-supported/kbs-access/values.yaml @@ -3,5 +3,6 @@ defaultMemory: "" +runtimeClassName: "" global: clusterPlatform: "" # Cluster platform: "Azure", "AWS", etc. - determines runtime class diff --git a/charts/hub/storage/Chart.yaml b/charts/hub/storage/Chart.yaml new file mode 100644 index 00000000..55383ab4 --- /dev/null +++ b/charts/hub/storage/Chart.yaml @@ -0,0 +1,9 @@ +apiVersion: v2 +description: Deploy and configure storage providers (HPP/LVM) for baremetal clusters +keywords: +- pattern +- storage +- hpp +- lvm +name: storage +version: 0.0.2 diff --git a/charts/hub/storage/templates/hostpathprovisioner.yaml b/charts/hub/storage/templates/hostpathprovisioner.yaml new file mode 100644 index 00000000..909e24e5 --- /dev/null +++ b/charts/hub/storage/templates/hostpathprovisioner.yaml @@ -0,0 +1,11 @@ +{{- if eq .Values.global.storageProvider "hpp" }} +apiVersion: hostpathprovisioner.kubevirt.io/v1beta1 +kind: HostPathProvisioner +metadata: + name: hostpath-provisioner +spec: + imagePullPolicy: IfNotPresent + storagePools: + - name: local + path: {{ .Values.hpp.storagePools.path | default "/var/hpvolumes" }} +{{- end }} diff --git a/charts/hub/storage/templates/hpp-storageclass.yaml b/charts/hub/storage/templates/hpp-storageclass.yaml new file mode 100644 index 00000000..7c86c889 --- /dev/null +++ b/charts/hub/storage/templates/hpp-storageclass.yaml @@ -0,0 +1,13 @@ +{{- if eq .Values.global.storageProvider "hpp" }} +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: {{ .Values.hpp.storageClass.name | default "hostpath-csi" }} + annotations: + storageclass.kubernetes.io/is-default-class: "true" +provisioner: kubevirt.io.hostpath-provisioner +reclaimPolicy: {{ .Values.hpp.storageClass.reclaimPolicy | default "Delete" }} +volumeBindingMode: {{ .Values.hpp.storageClass.volumeBindingMode | default "WaitForFirstConsumer" }} +parameters: + storagePool: local +{{- end }} diff --git a/charts/hub/storage/templates/lvmcluster.yaml b/charts/hub/storage/templates/lvmcluster.yaml new file mode 100644 index 00000000..3a7724bb --- /dev/null +++ b/charts/hub/storage/templates/lvmcluster.yaml @@ -0,0 +1,15 @@ +{{- if eq .Values.global.storageProvider "lvm" }} +apiVersion: lvm.topolvm.io/v1alpha1 +kind: LVMCluster +metadata: + name: {{ .Values.lvmCluster.name | default "lvmcluster" }} + namespace: openshift-storage +spec: + storage: + deviceClasses: + - name: {{ .Values.lvmCluster.deviceClass | default "vg1" }} + thinPoolConfig: + name: thin-pool-1 + sizePercent: 90 + overprovisionRatio: 10 +{{- end }} diff --git a/charts/hub/storage/values.yaml b/charts/hub/storage/values.yaml new file mode 100644 index 00000000..705f5233 --- /dev/null +++ b/charts/hub/storage/values.yaml @@ -0,0 +1,14 @@ +global: + storageProvider: hpp + +lvmCluster: + name: "lvmcluster" + deviceClass: "vg1" + +hpp: + storagePools: + path: /var/hpvolumes + storageClass: + name: hostpath-csi + reclaimPolicy: Delete + volumeBindingMode: WaitForFirstConsumer diff --git a/docs/dell-tdx-configuration.md b/docs/dell-tdx-configuration.md new file mode 100644 index 00000000..ecf078fd --- /dev/null +++ b/docs/dell-tdx-configuration.md @@ -0,0 +1,149 @@ +# Enable Intel TDX on Dell PowerEdge via iDRAC + +This guide provides step-by-step instructions for enabling Intel Trust Domain Extensions (TDX) on Dell PowerEdge servers using the iDRAC console. + +## Prerequisites + +- Dell 16th Generation PowerEdge server: + - PowerEdge R660, R660xs + - PowerEdge R760, R760xs, R760xd2, R760XA + - PowerEdge R860, R960 + - PowerEdge XE8640, XE9640, XE9680 + - PowerEdge C6620, MX760c + - PowerEdge XR5610, XR7620, XR8610t, XR8620t + - PowerEdge T360, T560 +- 5th Gen Intel Xeon Scalable processor with TDX support +- **8 or 16 DIMMs per socket** (required memory configuration) +- Latest BIOS firmware installed + +## Step-by-Step Instructions (Order Matters) + +> **IMPORTANT:** Settings must be configured in this exact order. Some options (like "Multiple Keys") will be greyed out until prerequisite settings are applied. You may need to **save and reboot between steps** for dependent options to become available. + +### 1. Access BIOS Setup via iDRAC + +1. Log into the iDRAC web console +2. Navigate to **Configuration → BIOS Settings** +3. Alternatively, launch **Virtual Console** and press **F2** during POST to enter System Setup + +### 2. Configure Memory Settings (FIRST) + +Navigate to: **System BIOS → Memory Settings** + +| Setting | Value | +| ---------------------- | ----------- | +| **Node Interleaving** | Disabled | + +**Save and reboot** before proceeding. + +### 3. Configure Processor Prerequisites (SECOND) + +Navigate to: **System BIOS → Processor Settings** + +| Setting | Value | +| -------------------------------- | -------- | +| **Logical Processor (x2APIC)** | Enabled | +| **CPU Physical Address Limit** | Disabled | + +**Save and reboot** before proceeding. + +### 4. Enable Memory Encryption - Multiple Keys (THIRD) + +Navigate to: **System BIOS → System Security** + +| Setting | Value | +| --------------------- | -------------- | +| **Memory Encryption** | Multiple Keys | + +> If "Multiple Keys" is still greyed out, verify steps 2 and 3 were applied and the system was rebooted. + +**Save and reboot** before proceeding. + +### 5. Configure TDX Settings (FOURTH) + +Navigate to: **System BIOS → System Security** (or **Processor Settings** depending on BIOS version) + +| Setting | Value | +| ------------------------------------------------- | ------- | +| **Global Memory Integrity** | Disabled | +| **Intel TDX (Trust Domain Extension)** | Enabled | +| **TME-MT/TDX Key Split** | 1 | +| **TDX Secure Arbitration Mode Loader (SEAM)** | Enabled | + +### 6. Configure SGX Settings (FIFTH) + +Navigate to: **System BIOS → Processor Settings → Software Guard Extensions (SGX)** + +| Setting | Value | +| -------------------- | ------------------------- | +| **Intel SGX** | Enabled | +| **SGX Factory Reset** | Off | +| **SGX PRMRR Size** | As needed (e.g., 64GB) | + +### 7. Final Save and Reboot + +1. Press **Escape** to exit menus +2. Select **Save Changes and Exit** +3. System will reboot with TDX enabled + +## Configuration Summary (Order of Operations) + +```text +1. Disable Node Interleaving → Save & Reboot +2. Enable x2APIC Mode → Save & Reboot +3. Disable CPU Physical Address Limit → Save & Reboot +4. Set Memory Encryption = Multiple Keys → Save & Reboot +5. Disable Global Memory Integrity +6. Enable Intel TDX +7. Set TME-MT/TDX Key Split = 1 +8. Enable SEAM Loader +9. Enable Intel SGX → Final Save & Reboot +``` + +## Verification + +After the OS boots, verify TDX is enabled: + +```bash +# Check kernel messages for TDX +dmesg | grep -i tdx +# Should show: "virt/tdx: BIOS enabled: private KeyID range: [X, Y)" + +# Check for TDX module +ls /sys/firmware/tdx_seam/ +``` + +## Troubleshooting + +### "Multiple Keys" Option is Greyed Out + +This is typically caused by: + +1. **Node Interleaving is Enabled** - Must be disabled first +2. **x2APIC Mode is Disabled** - Must be enabled first +3. **CPU Physical Address Limit is Enabled** - Must be disabled first +4. **System not rebooted** - Some changes require reboot before dependent options appear +5. **Insufficient DIMMs** - Requires 8 or 16 DIMMs per socket + +### Settings Not Available + +If TDX-related settings are not visible: + +1. Ensure BIOS firmware is updated to the latest version +2. Verify your processor supports TDX (5th Gen Xeon Scalable required) +3. Contact Dell support for BIOS with TDX support + +### TDX Not Detected by OS + +If the OS doesn't detect TDX after configuration: + +1. Verify all settings are correctly applied in the order specified +2. Ensure the OS/kernel supports TDX (Linux 6.2+ recommended) +3. Check that Memory Encryption is set to "Multiple Keys" (not "Single Key") + +## References + +- [Dell: Enable Intel TDX on Dell 16G Intel Servers](https://www.dell.com/support/kbdoc/en-us/000226452/enableinteltdxondell16g) +- [Intel TDX Enabling Guide - Hardware Setup](https://cc-enabling.trustedservices.intel.com/intel-tdx-enabling-guide/04/hardware_setup/) +- [Dell Info Hub: Enable Intel TDX in BIOS](https://infohub.delltechnologies.com/en-us/l/securing-ai-workloads-on-dell-poweredge-with-intel-xeon-processors-using-intel-trust-domain-extensions/appendix-b-enable-intel-r-tdx-in-bios/) +- [Linux Kernel TDX Documentation](https://docs.kernel.org/arch/x86/tdx.html) diff --git a/docs/nfd-matchall-bug.md b/docs/nfd-matchall-bug.md new file mode 100644 index 00000000..7dbb479f --- /dev/null +++ b/docs/nfd-matchall-bug.md @@ -0,0 +1,122 @@ +# Bug Report: NFD `matchAll` Field Silently Dropped, Causing False TEE Labels + +## Summary + +`matchAll` is not a valid field in the NFD `NodeFeatureRule.spec.rules[]` schema. The correct field for AND-logic matching is `matchFeatures` (top-level on each rule). When `matchAll` is used, the OpenShift NFD operator silently drops it during the `nfd.openshift.io/v1alpha1` to `nfd.k8s-sigs.io/v1alpha1` conversion, leaving rules with no match predicates. These empty rules match every node unconditionally, applying all TEE labels regardless of hardware. + +## Impact + +- Every node receives ALL TEE labels: `intel.feature.node.kubernetes.io/tdx`, `amd.feature.node.kubernetes.io/snp`, `ibm.feature.node.kubernetes.io/se`, and `intel.feature.node.kubernetes.io/sgx` +- The OpenShift sandboxed containers operator fails with: **`"multiple TEE platforms detected; only one per cluster supported"`** +- KataConfig cannot reconcile, so no kata runtime handler is installed +- All confidential container pods fail with: `failed to find runtime handler kata-snp from runtime list` + +## Root Cause + +The NFD `Rule` schema supports two match fields: + +| Field | Behavior | Valid? | +|-------|----------|--------| +| `matchFeatures` | Top-level list of feature matchers; ALL must match (AND) | Yes | +| `matchAny` | List of match groups; ANY must match (OR) | Yes | +| `matchAll` | **Does not exist in the NFD API** | No | + +When the chart template uses `matchAll`: + +```yaml +# BROKEN - matchAll is not a valid field +- name: "amd.sev-snp" + labels: + amd.feature.node.kubernetes.io/snp: "true" + matchAll: + - matchFeatures: + - feature: cpu.security + matchExpressions: + sev.snp.enabled: { op: Exists } +``` + +The OpenShift NFD operator creates a shadow resource under `nfd.k8s-sigs.io/v1alpha1`. During this conversion, `matchAll` is an unrecognized field and is silently stripped. The resulting live resource has: + +```yaml +# RESULT - no match conditions, matches every node +- name: "amd.sev-snp" + labels: + amd.feature.node.kubernetes.io/snp: "true" + labelsTemplate: "" + varsTemplate: "" +``` + +## Evidence + +**Node:** `master-03` (Intel Xeon, model family 6, ID 207, vendor: Intel) + +**NFD-reported hardware features (`cpu.security`):** + +```bash +sgx.enabled: "true" +sgx.epc: "4257210368" +``` + +Note: `sev.snp.enabled`, `tdx.enabled`, and `se.enabled` are **not present** in the node's feature data. + +**Labels applied to the node (all false positives except sgx):** + +```bash +amd.feature.node.kubernetes.io/snp=true # FALSE - Intel CPU, no SEV-SNP +intel.feature.node.kubernetes.io/tdx=true # FALSE - no tdx.enabled in cpu.security +ibm.feature.node.kubernetes.io/se=true # FALSE - Intel CPU, no SE +intel.feature.node.kubernetes.io/sgx=true # CORRECT - sgx.enabled is true +feature.node.kubernetes.io/runtime.kata=true # CORRECT - matchAny works (valid field) +``` + +**Sandbox operator log:** + +```text +INFO controllers.KataConfig failed to detect TEE platform + {"err": "multiple TEE platforms detected; only one per cluster supported"} +``` + +## Fix + +Replace `matchAll` with `matchFeatures` in each rule. The `matchFeatures` list at the rule level uses AND logic (all entries must match), which is the intended behavior. + +Additionally, add vendor-discriminating CPUID checks to prevent cross-platform false positives: + +```yaml +# FIXED - uses matchFeatures (valid field) with vendor guard +- name: "amd.sev-snp" + labels: + amd.feature.node.kubernetes.io/snp: "true" + matchFeatures: + - feature: cpu.cpuid + matchExpressions: + SVM: { op: Exists } # AMD-only CPUID flag + - feature: cpu.security + matchExpressions: + sev.snp.enabled: { op: Exists } +``` + +| Rule | `matchAll` (broken) | `matchFeatures` (fixed) | Vendor guard added | +|------|---------------------|-------------------------|--------------------| +| `amd.sev-snp` | Dropped silently | AND: SVM + sev.snp.enabled | `SVM` (AMD) | +| `intel.sgx` | Dropped silently | AND: SGX + SGXLC + sgx.enabled + X86_SGX | `SGX`, `SGXLC` (Intel) | +| `intel.tdx` | Dropped silently | AND: VMX + tdx.enabled | `VMX` (Intel) | +| `ibm.se.enabled` | Dropped silently | AND: se.enabled | None (s390x only) | + +## Affected Versions + +Any deployment using the consolidated `NodeFeatureRule` (`consolidated-hardware-features`) introduced in commit `57ec5f4`. The original separate NFD rule files (`amd-nfd-rules.yaml`, `intel-nfd-rules.yaml`) used `matchFeatures` correctly but the consolidation mistakenly introduced `matchAll`. + +## Remediation + +After deploying the corrected chart: + +```bash +# Remove false labels so NFD can re-evaluate +oc label node amd.feature.node.kubernetes.io/snp- \ + intel.feature.node.kubernetes.io/tdx- \ + ibm.feature.node.kubernetes.io/se- + +# Restart sandbox operator to re-evaluate KataConfig +oc delete pod -n openshift-sandboxed-containers-operator -l app=controller-manager +``` diff --git a/docs/pcr-reference-values-bare-metal.md b/docs/pcr-reference-values-bare-metal.md new file mode 100644 index 00000000..ad2e4d2c --- /dev/null +++ b/docs/pcr-reference-values-bare-metal.md @@ -0,0 +1,198 @@ +# Collecting PCR Reference Values on Bare Metal for RVPS Attestation + +## Overview + +The Trustee attestation service uses the Reference Value Provider Service (RVPS) to verify that a confidential VM is running the expected software stack. RVPS compares measurements from the TEE attestation quote against pre-registered reference values. On Azure, these values are populated automatically. On bare metal, they must be collected manually and pushed to Vault. + +This document covers the collection of vTPM PCR values for bare-metal TDX deployments, **excluding PCR8** (init data), which is computed separately by the `init-data-gzipper.yaml` imperative playbook. + +## What Gets Measured + +### PCR Registers Used by RVPS + +The trustee chart's RVPS policy (`rvps-values-policies.yaml`) consumes these PCR values: + +| PCR | Contents | Source | +|-----|----------|--------| +| PCR03 | Boot loader configuration | `pcr-stash` secret | +| PCR08 | Init data (CoCo-specific) | `initdata` ConfigMap (computed by `init-data-gzipper.yaml`) | +| PCR09 | Boot loader code | `pcr-stash` secret | +| PCR11 | BitLocker access control / TPM event log | `pcr-stash` secret | +| PCR12 | Boot events | `pcr-stash` secret | + +PCR08 is excluded from this procedure because it represents the hash of the CoCo init data TOML, which changes with each deployment configuration. It is handled by the `ansible/init-data-gzipper.yaml` playbook, which renders the init data template, computes `sha256(zero_pcr || sha256(initdata.toml))`, and stores the result in the `imperative/initdata` ConfigMap as `PCR8_HASH`. + +### TDX RTMR to PCR Mapping + +For TDX on bare metal, the vTPM inside the confidential VM maps measurements to PCR registers. The underlying TDX hardware uses Runtime Measurement Registers (RTMRs): + +| RTMR | Contents | Corresponding PCRs | +|------|----------|---------------------| +| MRTD | TD build-time measurement (firmware) | N/A (separate field in quote) | +| RTMR[0] | TDVF configuration, ACPI tables | PCR 0-1 | +| RTMR[1] | OS kernel, boot parameters, initrd | PCR 4-7 | +| RTMR[2] | OS applications, IMA measurements | PCR 8-15 | +| RTMR[3] | Reserved | N/A | + +## Collection Methods + +### Method 1: From a Running Confidential VM (Recommended) + +Launch a confidential container pod, exec into it, and read the vTPM PCR values. This gives you the actual measurements for your specific firmware + kernel + configuration. + +**Prerequisites:** + +- A working CoCo deployment with kata runtime installed +- `tpm2-tools` available in the guest (or use a debug pod) + +**Steps:** + +1. Launch a confidential pod: + + ```bash + oc run pcr-collector --image=registry.access.redhat.com/ubi9/ubi:latest \ + --restart=Never --overrides='{"spec":{"runtimeClassName":"kata-cc"}}' \ + -- sleep 3600 + ``` + +1. Install tpm2-tools inside the pod: + + ```bash + oc exec pcr-collector -- dnf install -y tpm2-tools + ``` + +1. Read PCR values (SHA-256 bank): + + ```bash + oc exec pcr-collector -- tpm2_pcrread sha256:3,9,11,12 + ``` + + Example output: + + ```text + sha256: + 3 : 0x3D458CFE55CC03EA1F443F1562BEE8DF30100AB2E1C4B6E5FE4568E7B0E6745A + 9 : 0x96A18E5C5E3E9AEC7FE5B8A1C6A02E8D6A4E8C6B3E9A7F5B2D4C8E1A3F6B9D2 + 11: 0x0000000000000000000000000000000000000000000000000000000000000000 + 12: 0x0000000000000000000000000000000000000000000000000000000000000000 + ``` + +1. Clean up: + + ```bash + oc delete pod pcr-collector + ``` + +### Method 2: Pre-Calculation with tdx-measure + +The [virtee/tdx-measure](https://github.com/virtee/tdx-measure) tool computes expected TDX measurement registers offline from firmware and kernel binaries, without requiring a running TD. + +**Install:** + +```bash +cargo install tdx-measure +``` + +**Usage:** + +Create a metadata JSON file describing your boot components: + +```json +{ + "firmware": "/path/to/OVMF.fd", + "kernel": "/path/to/vmlinuz", + "initrd": "/path/to/initrd.img", + "cmdline": "console=ttyS0", + "memory_mb": 4096, + "vcpus": 2 +} +``` + +Compute all measurements: + +```bash +tdx-measure metadata.json --direct-boot true --json +``` + +Compute platform-only (MRTD + RTMR0, excludes kernel/initrd): + +```bash +tdx-measure metadata.json --platform-only --json +``` + +Compute runtime-only (RTMR1 + RTMR2, kernel + initrd): + +```bash +tdx-measure metadata.json --runtime-only --json +``` + +**Note:** You need to extract the firmware (OVMF/TDVF), kernel, and initrd images from your OpenShift node. These can be found in the kata containers payload. + +### Method 3: From the Attestation Quote + +If you have a running deployment with attestation disabled (`global.coco.secured: false`), you can capture the raw attestation quote and extract measurements: + +1. Deploy with `bypassAttestation: true` in the trustee configuration +2. Launch a confidential pod +3. The attestation agent logs the quote contents — extract PCR values from the quote structure +4. Use these values as your reference baseline + +## Pushing Values to Vault + +Once you have collected PCR values, push them to Vault in the format expected by the `pcrs-eso` ExternalSecret: + +```bash +# Format: JSON object with measurements.sha256.pcrNN keys +oc exec -n vault vault-0 -- vault kv put secret/hub/pcrStash \ + json='{ + "measurements": { + "sha256": { + "pcr03": "", + "pcr09": "", + "pcr11": "", + "pcr12": "" + } + } + }' +``` + +The `pcrs-eso` ExternalSecret will then sync this into a `pcr-stash` Kubernetes secret in the `trustee-operator-system` namespace. The RVPS policy (`rvps-values-policies.yaml`) reads from this secret to populate the `rvps-reference-values` ConfigMap. + +## Pipeline Summary + +```text +tpm2_pcrread (inside CoCo pod) + | + v +Vault: secret/data/hub/pcrStash (pcr03, pcr09, pcr11, pcr12) + | + v (ExternalSecret: pcrs-eso) +K8s Secret: pcr-stash (trustee-operator-system) + | + v (ACM Policy: rvps-policy) +ConfigMap: rvps-reference-values (trustee-operator-system) + | + v +Attestation Service RVPS (compares against TD quote) +``` + +PCR8 follows a separate path: + +```text +ansible/init-data-gzipper.yaml + | + v +ConfigMap: initdata (imperative namespace) + | contains: INITDATA, PCR8_HASH + v (ACM Policy: rvps-policy) +ConfigMap: rvps-reference-values (merged with pcr-stash values) +``` + +## References + +- [Red Hat: How to deploy confidential containers on bare metal](https://developers.redhat.com/articles/2025/02/19/how-deploy-confidential-containers-bare-metal) +- [Red Hat: Introducing Confidential Containers Trustee](https://www.redhat.com/en/blog/introducing-confidential-containers-trustee-attestation-services-solution-overview-and-use-cases) +- [Intel: Runtime Integrity Measurement and Attestation in a Trust Domain](https://www.intel.com/content/www/us/en/developer/articles/community/runtime-integrity-measure-and-attest-trust-domain.html) +- [virtee/tdx-measure](https://github.com/virtee/tdx-measure) - Pre-calculate TDX measurements offline +- [CoCo Attestation Service RVPS docs](https://github.com/confidential-containers/attestation-service/blob/main/docs/rvps.md) +- [CoCo Attestation Policies](https://confidentialcontainers.org/docs/attestation/policies/) diff --git a/overrides/values-BareMetal.yaml b/overrides/values-BareMetal.yaml new file mode 100644 index 00000000..59cb0ab8 --- /dev/null +++ b/overrides/values-BareMetal.yaml @@ -0,0 +1,2 @@ +# Bare metal platform overrides. +# Storage-specific values moved to overrides/values-storage-*.yaml. diff --git a/overrides/values-None.yaml b/overrides/values-None.yaml new file mode 100644 index 00000000..172b55e3 --- /dev/null +++ b/overrides/values-None.yaml @@ -0,0 +1,2 @@ +# None platform overrides. +# Storage-specific values moved to overrides/values-storage-*.yaml. diff --git a/overrides/values-storage-external.yaml b/overrides/values-storage-external.yaml new file mode 100644 index 00000000..72b69f73 --- /dev/null +++ b/overrides/values-storage-external.yaml @@ -0,0 +1,2 @@ +# External storage: uses cluster default StorageClass. +# No vault storageClass override — vault uses whatever default exists. diff --git a/overrides/values-storage-hpp.yaml b/overrides/values-storage-hpp.yaml new file mode 100644 index 00000000..1bd94b8b --- /dev/null +++ b/overrides/values-storage-hpp.yaml @@ -0,0 +1,8 @@ +vault: + server: + dataStorage: + storageClass: hostpath-csi + +global: + objectStorage: + backingStorageClass: "hostpath-csi" diff --git a/overrides/values-storage-lvm.yaml b/overrides/values-storage-lvm.yaml new file mode 100644 index 00000000..3c49baed --- /dev/null +++ b/overrides/values-storage-lvm.yaml @@ -0,0 +1,8 @@ +vault: + server: + dataStorage: + storageClass: lvms-vg1 + +global: + objectStorage: + backingStorageClass: "lvms-vg1" diff --git a/scripts/gen-secrets.sh b/scripts/gen-secrets.sh index c487bcac..24fe3407 100755 --- a/scripts/gen-secrets.sh +++ b/scripts/gen-secrets.sh @@ -28,9 +28,58 @@ if [ ! -f "${KBS_PRIVATE_KEY}" ]; then openssl pkey -in "${KBS_PRIVATE_KEY}" -pubout -out "${KBS_PUBLIC_KEY}" fi +## PCCS secrets for bare metal Intel TDX deployments +PCCS_PRIVATE_KEY="${COCO_SECRETS_DIR}/pccs_private.pem" +PCCS_CERTIFICATE="${COCO_SECRETS_DIR}/pccs_certificate.pem" +PCCS_USER_TOKEN_FILE="${COCO_SECRETS_DIR}/pccs_user_token" +PCCS_ADMIN_TOKEN_FILE="${COCO_SECRETS_DIR}/pccs_admin_token" + +if [ ! -f "${PCCS_PRIVATE_KEY}" ]; then + echo "Creating PCCS TLS certificate" + openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \ + -keyout "${PCCS_PRIVATE_KEY}" \ + -out "${PCCS_CERTIFICATE}" \ + -subj "/C=US/ST=Denial/L=Springfield/O=Dis/CN=pccs-service.intel-dcap.svc.cluster.local" +fi + +if [ ! -f "${PCCS_USER_TOKEN_FILE}" ]; then + echo "Creating PCCS user token" + echo "usertoken" > "${PCCS_USER_TOKEN_FILE}" +fi +echo -n "usertoken" | sha512sum | tr -d '[:space:]-' > "${COCO_SECRETS_DIR}/pccs_user_token_hash" + +if [ ! -f "${PCCS_ADMIN_TOKEN_FILE}" ]; then + echo "Creating PCCS admin token" + echo "admintoken" > "${PCCS_ADMIN_TOKEN_FILE}" +fi +echo -n "admintoken" | sha512sum | tr -d '[:space:]-' > "${COCO_SECRETS_DIR}/pccs_admin_token_hash" + ## Copy a sample values file if this stuff doesn't exist if [ ! -f "${VALUES_FILE}" ]; then - echo "No values file was found copying template.. please review before deploying" + echo + echo "========================================================================" + echo "IMPORTANT: Created values-secret file at ${VALUES_FILE}" + echo "========================================================================" + echo cp "${SCRIPT_DIR}/../values-secret.yaml.template" "${VALUES_FILE}" + echo "ACTION REQUIRED: Review and customize this file before deploying:" + echo + echo " For Azure deployments:" + echo " - SSH debug is optional (uncomment sshKey if needed)" + echo " - PCCS secrets can remain commented out" + echo + echo " For Bare Metal deployments:" + echo " - Uncomment the PCCS secrets section" + echo " - Add your Intel PCS API key (get from https://api.portal.trustedservices.intel.com/)" + echo " - SSH debug is optional (uncomment sshKey if needed)" + echo " - See docs/pcr-reference-values-bare-metal.md for PCR collection" + echo + echo " Security policies:" + echo " - Default is 'insecure' (accepts all images)" + echo " - For production, configure 'signed' policy with cosign keys" + echo + echo "File location: ${VALUES_FILE}" + echo "========================================================================" + echo fi diff --git a/scripts/get-pccs-node.sh b/scripts/get-pccs-node.sh new file mode 100755 index 00000000..32b4313b --- /dev/null +++ b/scripts/get-pccs-node.sh @@ -0,0 +1,10 @@ +#!/usr/bin/env bash +# Detects a node with Intel TDX support for PCCS deployment. +# Usage: bash scripts/get-pccs-node.sh +NODE=$(oc get nodes -l intel.feature.node.kubernetes.io/tdx=true \ + -o jsonpath='{.items[0].metadata.name}' 2>/dev/null) +if [ -z "$NODE" ]; then + echo "ERROR: No TDX-capable nodes found" >&2 + exit 1 +fi +echo "$NODE" diff --git a/values-baremetal.yaml b/values-baremetal.yaml new file mode 100644 index 00000000..7548917d --- /dev/null +++ b/values-baremetal.yaml @@ -0,0 +1,261 @@ +# Bare metal deployment for confidential containers. +# Supports Intel TDX and AMD SEV-SNP via auto-detection (NFD). +# Set main.clusterGroupName: baremetal in values-global.yaml to use. + +clusterGroup: + name: baremetal + isHubCluster: true + namespaces: + - open-cluster-management + - vault + - golang-external-secrets + - openshift-sandboxed-containers-operator + - trustee-operator-system + - cert-manager-operator + - cert-manager + - hello-openshift + - kbs-access + - openshift-cnv + - openshift-storage + - openshift-nfd + - baremetal + - intel-dcap + - kyverno + + subscriptions: + acm: + name: advanced-cluster-management + namespace: open-cluster-management + sandbox: + name: sandboxed-containers-operator + namespace: openshift-sandboxed-containers-operator + source: redhat-operators + channel: stable + installPlanApproval: Manual + csv: sandboxed-containers-operator.v1.12.0 + trustee: + name: trustee-operator + namespace: trustee-operator-system + source: redhat-operators + channel: stable + installPlanApproval: Manual + csv: trustee-operator.v1.1.0 + cert-manager: + name: openshift-cert-manager-operator + namespace: cert-manager-operator + channel: stable-v1 + lvm-operator: + name: lvms-operator + namespace: openshift-storage + source: redhat-operators + channel: stable-4.20 + installPlanApproval: Automatic + cnv: + name: kubevirt-hyperconverged + namespace: openshift-cnv + source: redhat-operators + channel: stable + installPlanApproval: Automatic + nfd: + name: nfd + namespace: openshift-nfd + channel: stable + intel-device-plugins: + name: intel-device-plugins-operator + namespace: openshift-operators + source: certified-operators + channel: stable + projects: + - hub + - vault + - trustee + - golang-external-secrets + - sandbox + - workloads + - default + + # Explicitly mention the cluster-state based overrides we plan to use for this pattern. + # We can use self-referential variables because the chart calls the tpl function with these variables defined + sharedValueFiles: + - '/overrides/values-{{ $.Values.global.clusterPlatform }}.yaml' + - '/overrides/values-storage-{{ $.Values.global.storageProvider }}.yaml' + + applications: + acm: + name: acm + namespace: open-cluster-management + project: hub + chart: acm + chartVersion: 0.1.* + + vault: + name: vault + namespace: vault + project: vault + chart: hashicorp-vault + chartVersion: 0.1.* + + secrets-operator: + name: golang-external-secrets + namespace: golang-external-secrets + project: golang-external-secrets + chart: golang-external-secrets + chartVersion: 0.1.* + + trustee: + name: trustee + namespace: trustee-operator-system + project: trustee + repoURL: https://github.com/butler54/trustee-chart.git + targetRevision: feature/baremetal-attestation + path: . + overrides: + - name: global.coco.secured + value: "true" + - name: kbs.admin.format + value: "v1.1" + - name: kbs.https.enabled + value: "false" + - name: kbs.secretResources[0].name + value: kbsres1 + - name: kbs.secretResources[0].key + value: secret/data/hub/kbsres1 + - name: kbs.tdx.enabled + value: "true" + - name: kbs.tdx.collateralService + value: "https://pccs-service.intel-dcap.svc.cluster.local:8042/sgx/certification/v4/" + - name: kbs.secretResources[1].name + value: passphrase + - name: kbs.secretResources[1].key + value: secret/data/hub/passphrase + + storage: + name: storage + namespace: openshift-storage + project: hub + path: charts/hub/storage + + baremetal: + name: baremetal + namespace: baremetal + project: hub + path: charts/all/baremetal + + sandbox: + name: sandbox + namespace: openshift-sandboxed-containers-operator + project: sandbox + chart: sandboxed-containers + chartVersion: 0.2.* + overrides: + - name: global.secretStore.backend + value: vault + - name: secretStore.name + value: vault-backend + - name: secretStore.kind + value: ClusterSecretStore + - name: enablePeerPods + value: "false" + + + intel-dcap: + name: intel-dcap + namespace: intel-dcap + project: hub + path: charts/all/intel-dcap + overrides: + - name: secretStore.name + value: vault-backend + - name: secretStore.kind + value: ClusterSecretStore + + sandbox-policies: + name: sandbox-policies + namespace: openshift-sandboxed-containers-operator + chart: sandboxed-policies + chartVersion: 0.1.* + + kbs-access: + name: kbs-access + namespace: kbs-access + project: workloads + path: charts/coco-supported/kbs-access + syncPolicy: + automated: + prune: true + overrides: + - name: defaultMemory + value: "8192" + + hello-openshift: + name: hello-openshift + namespace: hello-openshift + project: workloads + path: charts/coco-supported/hello-openshift + syncPolicy: + automated: + prune: true + + kyverno: + name: kyverno + namespace: kyverno + project: hub + repoURL: https://kyverno.github.io/kyverno/ + chart: kyverno + chartVersion: 3.7.* + syncPolicy: + automated: {} + retry: + limit: 20 + syncOptions: + - ServerSideApply=true + overrides: + - name: admissionController.container.securityContext + value: "null" + - name: admissionController.initContainer.securityContext + value: "null" + - name: backgroundController.securityContext + value: "null" + - name: cleanupController.securityContext + value: "null" + - name: reportsController.securityContext + value: "null" + - name: crds.migration.securityContext + value: "null" + - name: webhooksCleanup.securityContext + value: "null" + - name: test.securityContext + value: "null" + - name: crds.groups.wgpolicyk8s.policyreports + value: "false" + - name: crds.groups.wgpolicyk8s.clusterpolicyreports + value: "false" + - name: reportsController.enabled + value: "false" + + coco-kyverno-policies: + name: coco-kyverno-policies + namespace: openshift-sandboxed-containers-operator + project: sandbox + path: charts/all/coco-kyverno-policies + + imperative: + # NOTE: We *must* use lists and not hashes. As hashes lose ordering once parsed by helm + # The default schedule is every 10 minutes: imperative.schedule + # Total timeout of all jobs is 1h: imperative.activeDeadlineSeconds + # imagePullPolicy is set to always: imperative.imagePullPolicy + # For additional overrides that apply to the jobs, please refer to + # https://validatedpatterns.io/imperative-actions/#additional-job-customizations + image: ghcr.io/butler54/imperative-container:latest + serviceAccountCreate: true + adminServiceAccountCreate: true + serviceAccountName: imperative-admin-sa + jobs: + - name: install-deps + playbook: ansible/install-deps.yaml + verbosity: -vvv + timeout: 3600 + - name: init-data-gzipper + playbook: ansible/init-data-gzipper.yaml + verbosity: -vvv + timeout: 3600 diff --git a/values-global.yaml b/values-global.yaml index d7c202a3..7cfbda9f 100644 --- a/values-global.yaml +++ b/values-global.yaml @@ -1,5 +1,6 @@ global: pattern: coco-pattern + storageProvider: hpp # Options: hpp, lvm, external secretStore: # Warning: This must be present even if it is set to none. backend: vault # none, vault, kubernetes @@ -13,6 +14,7 @@ global: coco: securityPolicyFlavour: "insecure" # insecure, signed or reject is expected. secured: true # true or false. If true, the cluster will be secured. If false, the cluster will be insecure. + bypassAttestation: false # Enable SSH key injection into podvm for debugging. Do not enable in production. # Also requires: COCO_ENABLE_SSH_DEBUG=true ./scripts/gen-secrets.sh # and uncommenting the sshKey block in values-secret.yaml.template. @@ -24,7 +26,7 @@ main: # WARNING # This default configuration uses a single cluster on azure. # It fundamentally violates the separation of duties. - clusterGroupName: simple + clusterGroupName: baremetal multiSourceConfig: enabled: true clusterGroupChartVersion: 0.9.* diff --git a/values-secret.yaml.template b/values-secret.yaml.template index 4ed9d158..f2631f56 100644 --- a/values-secret.yaml.template +++ b/values-secret.yaml.template @@ -7,20 +7,73 @@ version: "2.0" secrets: + # =================================================================== + # Core KBS Secrets (Always Required) + # =================================================================== - # SSH keys for podvm debug access (optional). - # To enable: set global.coco.enableSSHDebug=true in values-global.yaml, - # run COCO_ENABLE_SSH_DEBUG=true ./scripts/gen-secrets.sh, - # then uncomment the block below. - #- name: sshKey - # vaultPrefixes: - # - global - # fields: - # - name: id_rsa.pub - # path: ~/.coco-pattern/id_rsa.pub - # - name: id_rsa - # path: ~/.coco-pattern/id_rsa + # KBS public key - used by workloads to verify KBS identity + - name: kbsPublicKey + vaultPrefixes: + - hub + fields: + - name: publicKey + path: ~/.coco-pattern/kbsPublicKey + + # KBS private key - used by KBS to sign responses + # Generated by gen-secrets.sh alongside kbsPublicKey + - name: kbsPrivateKey + vaultPrefixes: + - global + fields: + - name: privateKey + path: ~/.coco-pattern/kbsPrivateKey + + # Auto-generated test secrets provided by KBS + - name: kbsres1 + vaultPrefixes: + - hub + fields: + - name: key1 + value: '' + onMissingValue: generate + vaultPolicy: validatedPatternDefaultPolicy + - name: key2 + value: '' + onMissingValue: generate + vaultPolicy: validatedPatternDefaultPolicy + - name: key3 + value: '' + onMissingValue: generate + vaultPolicy: validatedPatternDefaultPolicy + + # Auto-generated passphrase (example secret) + - name: passphrase + vaultPrefixes: + - hub + fields: + - name: passphrase + value: '' + onMissingValue: generate + vaultPolicy: validatedPatternDefaultPolicy + + # Attestation status tracking + - name: attestationStatus + vaultPrefixes: + - hub + fields: + - name: status + value: 'attested' + - name: random + value: '' + onMissingValue: generate + vaultPolicy: validatedPatternDefaultPolicy + + # =================================================================== + # Security Policies & Attestation + # =================================================================== + # Image security policies (insecure, reject, signed) + # Controls which container images are allowed to run in confidential pods - name: securityPolicyConfig vaultPrefixes: - hub @@ -72,7 +125,9 @@ secrets: # PCR measurements for attestation. - # Required: run ./scripts/get-pcr.sh before deploying. + # Azure: run ./scripts/get-pcr.sh to extract from peer-pod image + # Bare metal: manually collect from running confidential VM + # See docs/pcr-reference-values-bare-metal.md for collection procedure - name: pcrStash vaultPrefixes: - hub @@ -81,48 +136,60 @@ secrets: path: ~/.coco-pattern/measurements.json - - name: attestationStatus - vaultPrefixes: - - hub - fields: - - name: status - value: 'attested' - - name: random - value: '' - onMissingValue: generate - vaultPolicy: validatedPatternDefaultPolicy - - - name: kbsPublicKey - vaultPrefixes: - - hub - fields: - - name: publicKey - path: ~/.coco-pattern/kbsPublicKey + # =================================================================== + # Optional - SSH Debug Access + # =================================================================== + # SSH keys for podvm debug access (optional). + # To enable: set global.coco.enableSSHDebug=true in values-global.yaml, + # run COCO_ENABLE_SSH_DEBUG=true ./scripts/gen-secrets.sh, + # then uncomment the block below. + #- name: sshKey + # vaultPrefixes: + # - global + # fields: + # - name: id_rsa.pub + # path: ~/.coco-pattern/id_rsa.pub + # - name: id_rsa + # path: ~/.coco-pattern/id_rsa - - name: kbsres1 - vaultPrefixes: - - hub - fields: - - name: key1 - value: '' - onMissingValue: generate - vaultPolicy: validatedPatternDefaultPolicy - - name: key2 - value: '' - onMissingValue: generate - vaultPolicy: validatedPatternDefaultPolicy - - name: key3 - value: '' - onMissingValue: generate - vaultPolicy: validatedPatternDefaultPolicy + # =================================================================== + # Optional - Bare Metal Intel TDX + # =================================================================== - - name: passphrase - vaultPrefixes: - - hub - fields: - - name: passphrase - value: '' - onMissingValue: generate - vaultPolicy: validatedPatternDefaultPolicy + # PCCS secrets for bare metal Intel TDX deployments. + # Uncomment these sections for bare metal deployments. + # Run ./scripts/gen-secrets.sh first to generate tokens and certificates. + # You must provide your Intel PCS API key in the api_key field. + # Get an API key from: https://api.portal.trustedservices.intel.com/ + #- name: pccs + # vaultPrefixes: + # - hub + # fields: + # - name: api_key + # value: '' + # - name: user_token_hash + # path: ~/.coco-pattern/pccs_user_token_hash + # - name: user_token + # path: ~/.coco-pattern/pccs_user_token + # - name: admin_token_hash + # path: ~/.coco-pattern/pccs_admin_token_hash + # - name: admin_token + # path: ~/.coco-pattern/pccs_admin_token + # - name: db_username + # value: '' + # onMissingValue: generate + # vaultPolicy: validatedPatternDefaultPolicy + # - name: db_password + # value: '' + # onMissingValue: generate + # vaultPolicy: validatedPatternDefaultPolicy + #- name: pccs-tls + # vaultPrefixes: + # - hub + # fields: + # - name: private_key + # path: ~/.coco-pattern/pccs_private.pem + # - name: certificate + # path: ~/.coco-pattern/pccs_certificate.pem