diff --git a/content/patterns/coco-pattern/_index.adoc b/content/patterns/coco-pattern/_index.adoc new file mode 100644 index 000000000..c593ee0b4 --- /dev/null +++ b/content/patterns/coco-pattern/_index.adoc @@ -0,0 +1,70 @@ +--- +title: Confidential Containers pattern +date: 2024-10-03 +tier: sandbox +summary: This pattern helps you get started with deploying confidential containers in OpenShift Container Platform. +rh_products: +- Red Hat OpenShift Container Platform +- Red Hat Advanced Cluster Management +- Red Hat OpenShift Sandbox Containers +industries: +- General +aliases: /coco-pattern/ +pattern_logo: coco-logo.png +links: + install: coco-getting-started + help: https://groups.google.com/g/validatedpatterns + bugs: https://github.com/validatedpatterns/coco-pattern/issues +--- +:toc: +:imagesdir: /images +:_content-type: ASSEMBLY +include::modules/comm-attributes.adoc[] + += About coco-pattern + +Confidential computing is a technology for securing data in use. It uses a https://en.wikipedia.org/wiki/Trusted_execution_environment[Trusted Execution Environment] provided within the hardware of the processor to prevent access from others who have access to the system. +https://confidentialcontainers.org/[Confidential containers] is a project to standardize the consumption of confidential computing by making the security boundary for confidential computing to be a Kubernetes pod. [Kata containers](https://katacontainers.io/) is used to establish the boundary via a shim VM. + +A core goal of confidential computing is to use this technology to isolate the workload from both Kubernetes and hypervisor administrators. + +image::coco-pattern/isolation.png[Schematic describing the isolation of confidential contains from the hosting system] + + +This pattern uses https://docs.redhat.com/en/documentation/openshift_sandboxed_containers/1.7/html/user_guide/deploying-on-azure#deploying-cc_azure-cc[Red Hat OpenShift sandbox containers] to deploy and configure confidential containers on Microsoft Azure. + +It deploys three copies of 'Hello OpenShift' to demonstrate some of the security boundaries that enforced with confidential containers. + +== Requirements + +- An an azure account with the link:./coco-pattern-azure-requirements/[required access rights] +- An OpenShift cluster, within the Azure environment updated beyond 4.16.10 + + +== Security considerations + +**This pattern is a demonstration only and contains configuration that is not best practice** + +- The default configuration deploys everything in a single cluster for testing purposes. The https://www.ietf.org/archive/id/draft-ietf-rats-architecture-22.html[RATS] architecture mandates that the Key Broker Service (e.g. https://github.com/confidential-containers/trustee[Trustee]) is in a trusted security zone. +- The https://github.com/confidential-containers/trustee/tree/main/attestation-service[Attestation Service] has wide open security policies. + +== Future work + +* Deploying the environment the 'Trusted' environment including the KBS on a separate cluster to the secured workloads +* Deploying to alternative environments supporting confidential computing including bare metal x86 clusters; IBM Cloud; IBM Z +* Finishing the sample AI application + +== Architecture + +Confidential Containers typically has two environments. A trusted zone, and an untrusted zone. In these zones, Trustee, and the sandbox container operator are deployed, respectively. + +** For demonstration purposes the pattern currently is converged on one cluster** + +image::coco-pattern/overview-schematic.png[Schematic describing the high level architecture of confidential containers] + + + + +== References +- https://docs.redhat.com/en/documentation/openshift_sandboxed_containers/1.7/html/user_guide/about-osc#about-confidential-containers_about-osc[OpenShift sandboxed containers documentation] +- https://www.redhat.com/en/blog/exploring-openshift-confidential-containers-solution[OpenShift confidential containers solution blog] diff --git a/content/patterns/coco-pattern/coco-pattern-azure-requirements.adoc b/content/patterns/coco-pattern/coco-pattern-azure-requirements.adoc new file mode 100644 index 000000000..4f2813a5c --- /dev/null +++ b/content/patterns/coco-pattern/coco-pattern-azure-requirements.adoc @@ -0,0 +1,70 @@ +--- +title: Azure requirements +weight: 10 +aliases: /coco-pattern/coco-pattern-azure-requirements/ +--- + +:toc: +:_content-type: ASSEMBLY +include::modules/comm-attributes.adoc[] + +:imagesdir: ../../../images += Azure requirements + +This demo currently has been tested only on azure. +The configuration tested used the `openshift-install`. +https://docs.openshift.com/container-platform/4.16/installing/installing_azure/installing-azure-default.html[OpenShift documentation] contains details on how to do this. + +The documentation outlines https://docs.openshift.com/container-platform/4.16/installing/installing_azure/installing-azure-account.html[minimum required configuration] for an azure account. + +== Changes required + +Do not accept default sizes for OpenShift install. It is recommended to up the workers to at least `Standard_D8s_v5`. +This can be done by using `openshift-install create install-config` first and adjusting the workers under platform e.g.: + +[source,yaml] +---- +- architecture: amd64 + hyperthreading: Enabled + name: worker + platform: + azure: + type: Standard_D8s_v5 + replicas: 3 +---- + +On a cloud provider the virtual machines for the kata containers use "peer pods" which are running directly on the cloud provider's hypervisor (see the diagram below). +This means that access is required to the "confidential computing" virtual machine class. On Azure the `Standard_DCas_v5` class of virtual machines are used. +These virtual machines are *NOT* available in all regions. Users will also need to up the specific limits for `Standard_DC2as_v5` virtual machines. + +image::coco-pattern/peer_pods.png[Schematic diagram of peer pods vs standard kata containers] + +DNS for the openshift cluster also *MUST* be provided by azure DNS. + +== Azure configuration required for the validated pattern + +The validated pattern requires access to azure apis to provision peer-pod VMs and to obtain certificates from let's encrypt. + +Azure configuration information must be provided in two places: + +- The a secret must be loaded using a ../../../learn/secrets-management-in-the-validated-patterns-framework/[values-secret] file. + The https://github.com/validatedpatterns/coco-pattern/blob/main/values-secret.yaml.template[`values-secret.yaml.template`] file provides the appropriate structure + +- A broader set of information about the cluster is required in https://github.com/validatedpatterns/coco-pattern/blob/main/values-global.yaml[`values-global.yaml`] (see below). + + +[source,yaml] +---- +global: + azure: + clientID: '' # Service principle ID + subscriptionID: '' + tenantID: '' # Tenant ID + DNSResGroup: '' # Resource group for the azure DNS hosted zone + hostedZoneName: '' # the hosted zone name + clusterResGroup: '' # Resource group of the cluster + clusterSubnet: '' # subnet of the cluster + clusterNSG: '' # network security group of the worker nodes in the cluster + clusterRegion: '' +---- + diff --git a/content/patterns/coco-pattern/coco-pattern-getting-started.adoc b/content/patterns/coco-pattern/coco-pattern-getting-started.adoc new file mode 100644 index 000000000..86fbcd1a2 --- /dev/null +++ b/content/patterns/coco-pattern/coco-pattern-getting-started.adoc @@ -0,0 +1,86 @@ +--- +title: Getting started +weight: 10 +aliases: /coco-pattern/coco-pattern-getting-started/ +--- + +:toc: + +:_content-type: ASSEMBLY +include::modules/comm-attributes.adoc[] + + +== Deploying + +1. Install an link:../coco-pattern-azure-requirements/[OpenShift Cluster on Azure] + +2. Update the link:../coco-pattern-azure-requirements/#_azure_configuration_required_for_the_validated_pattern[required Azure configuration and secrets] + +3. `./pattern.sh make install` + +4. Wait: The cluster needs to reboot all nodes at least once, and reprovision the ingress to use the let's encrypt certificates. + +5. If the services do not come up use the ArgoCD UI to triage potential timeouts. + +== Simple Confidential container tests + +The pattern deploys some simple tests of CoCo with this pattern. +A "Hello Openshift" (e.g. `curl` to return "Hello Openshift!") application has been deployed in three form factor. + +1. A vanilla kubernetes pod at `oc get pods -n hello-openshift standard` +2. A confidential container `oc get pods -n hello-openshift secure` +3. A confidential container with a relaxed policy at `oc get pods -n hello-openshift insecure-policy` + +In this case the insecure policy is designed to allow a user to be able to exec into the confidential container. +Typically this is disabled by an immutable policy established at pod creation time. + + +Doing a `oc get pods` for either of the pods running a confidential container should show the `runtimeClassName: kata-remote` for the pod. + +// Add a azure portal image grab next boot +Logging into azure once the pods have been provisioned will show that each of these two pods has been provisioned with it's own `Standard_DC2as_v5` virtual machine. + +=== `oc exec` testing + +In a OpenShift cluster without confidential containers, Role Based Access Control (RBAC), may be used to prevent users from execing into a container to mutate it. +However: + +1. Cluster admins can always circumvent this capability +2. Anyone logged into the node directly can also circumvent this capability. + +Confidential containers can prevent this. Running: `oc exec -n hello-openshift -it secure -- bash` will result in a denial of access, irrespective of the user undertaking the action, including `kubeadmin`. +For running this with either the standard pod `oc exec -n hello-openshift -it standard -- bash`, or the CoCo pod with the policy disabled `oc exec -n hello-openshift -it insecure-policy -- bash` will allow shell access. + + + +=== Confidential Data Hub testing + +Part of the CoCo VM is a component called the Confidential Data Hub (CDH), which simplifies access to the Trustee Key Broker service for end applications. +Find out more about how the CDH and Trustee work together https://www.redhat.com/en/blog/introducing-confidential-containers-trustee-attestation-services-solution-overview-and-use-cases[here]. + +image::coco-pattern/trustee.png[] + +The CDH presents to containers within the pod (only), via a localhost URL. The CoCo container with an insecure policy can be used for testing the behaviour. + + +- `oc exec -n hello-openshift -it insecure-policy -- bash` to get a shell into a confidential container + +- https://github.com/validatedpatterns/coco-pattern/blob/main/charts/hub/trustee/templates/kbs.yaml[Trustee's configuration] specifies the list of secrets which the KBS can access with the `kbsSecretResources` attribute. + +- Secrets within the CDH can be accessed (by default) at `http://127.0.0.1:8006/cdh/resource/default/$K8S_SECRET/$K8S_SECRET_KEY`. + +- In this case `http://127.0.0.1:8006/cdh/resource/default/passphrase/passphrase` by default will return a string which was randomly generated when the pattern was deployed. + +- This should be the same as result as `oc get secrets -n trustee-operator-system passphrase -o yaml | yq '.data.passphrase'` | base64 -d` + +- Tailing the logs for the kbs container e.g. `oc logs -n trustee-operator-system kbs-deployment-5b574bccd6-twjxh -f` shows the evidence which is flowing to the KBS from the CDH. + + + + + + + + + + diff --git a/static/images/coco-pattern/isolation.png b/static/images/coco-pattern/isolation.png new file mode 100644 index 000000000..9c78c5450 Binary files /dev/null and b/static/images/coco-pattern/isolation.png differ diff --git a/static/images/coco-pattern/overview-schematic.png b/static/images/coco-pattern/overview-schematic.png new file mode 100644 index 000000000..2ba0bae00 Binary files /dev/null and b/static/images/coco-pattern/overview-schematic.png differ diff --git a/static/images/coco-pattern/peer_pods.png b/static/images/coco-pattern/peer_pods.png new file mode 100644 index 000000000..16aa4f4fc Binary files /dev/null and b/static/images/coco-pattern/peer_pods.png differ diff --git a/static/images/coco-pattern/trustee.png b/static/images/coco-pattern/trustee.png new file mode 100644 index 000000000..cca62f395 Binary files /dev/null and b/static/images/coco-pattern/trustee.png differ diff --git a/static/images/logos/coco-logo.png b/static/images/logos/coco-logo.png new file mode 100644 index 000000000..2c147a172 Binary files /dev/null and b/static/images/logos/coco-logo.png differ