Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MGMT-13627: Add ConfidentialVM options to AzureMachineProviderSpec #1403

Conversation

mresvanis
Copy link
Contributor

@mresvanis mresvanis commented Feb 13, 2023

This change adds the Confidential VM configuration options to the AzureMachineProviderSpec, in order to support Confidential Computing on Azure.

Specifically, it introduces the OS disk security profile, which includes the SecurityEncryptionType and DiskEncryptionSetID fields and adds the SecureBoot and VirtualizedTrustedPlatformModule fields in the security profile of the VM.

The OS disk security profile SecurityEncryptionType field defines whether the VM is a Confidential VM. It cannot be defined alongside the EncryptionAtHost. When SecurityEncryptionType is defined the VirtualizedTrustedPlatformModule should be set to Enabled, while SecureBoot can be either Enabled or Disabled.

Possible values for the SecurityEncryptionType are:

  • DiskWithVMGuestState, OS disk encryption before VM deployment that uses platform-managed keys (PMK) or a customer-managed key (CMK)
  • VMGuestStateOnly, no OS disk confidential encryption

When the SecurityEncryptionType is set to DiskWithVMGuestState, the DiskEncryptionSetID can be specified to allow using customer-managed keys to encrypt the OS disk and the VMGuestState blob.

Feature link: https://issues.redhat.com/browse/OCPBU-233

Cluster API provider Azure PR: kubernetes-sigs/cluster-api-provider-azure#3265

Related PRs for Confidential Compute support on GCP:

Signed-off-by: Michail Resvanis mresvani@redhat.com

@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 13, 2023

@mresvanis: This pull request references MGMT-13627 which is a valid jira issue.

In response to this:

This change adds the Confidential VM configuration options to the AzureMachineProviderSpec, in order to support Confidential Computing on Azure.

Specifically, the SecurityEncryptionType field defines whether the VM is a Confidential VM. It cannot be defined alongside the EncryptionAtHost. When SecurityEncryptionType is defined the VirtualizedTrustedPlatformModule should be set to Enabled, while SecureBoot can be either Enabled or Disabled.

Possible values for the SecurityEncryptionType are:

  • DiskWithVMGuestState, OS disk encryption before VM deployment that uses platform-managed keys (PMK) or a customer-managed key (CMK)
  • VMGuestStateOnly, no OS disk confidential encryption

When the SecurityEncryptionType is set to DiskWithVMGuestState, the SecureVMDiskEncryptionSetID can be specified as well.

Feature link: https://issues.redhat.com/browse/OCPBU-233

Signed-off-by: Michail Resvanis mresvani@redhat.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Feb 13, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 13, 2023

Hello @mresvanis! Some important instructions when contributing to openshift/api:
API design plays an important part in the user experience of OpenShift and as such API PRs are subject to a high level of scrutiny to ensure they follow our best practices. If you haven't already done so, please review the OpenShift API Conventions and ensure that your proposed changes are compliant. Following these conventions will help expedite the api review process for your PR.

For merging purposes, this repository follows the no-Feature-Freeze process which means that in addition to the standard lgtm and approved labels this repository requires either:

bugzilla/valid-bug - applied if your PR references a valid bugzilla bug

OR

qe-approved, docs-approved, and px-approved - these labels can be applied by anyone in the openshift org via the /label <labelname> command.

Who should apply these qe/docs/px labels?

  • For a no-Feature-Freeze team who is merging a feature before code freeze, they need to get those labels applied to their api repo PR by the appropriate teams (i.e. qe, docs, px)
  • For a Feature Freeze (traditional) team who is merging a feature before FF, they can self-apply the labels (via /label commands), they are basically irrelevant for those teams
  • For a Feature Freeze team who is merging a feature after FF, the PR should be rejected barring an exception

@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 13, 2023

@mresvanis: This pull request references MGMT-13627 which is a valid jira issue.

In response to this:

This change adds the Confidential VM configuration options to the AzureMachineProviderSpec, in order to support Confidential Computing on Azure.

Specifically, the SecurityEncryptionType field defines whether the VM is a Confidential VM. It cannot be defined alongside the EncryptionAtHost. When SecurityEncryptionType is defined the VirtualizedTrustedPlatformModule should be set to Enabled, while SecureBoot can be either Enabled or Disabled.

Possible values for the SecurityEncryptionType are:

  • DiskWithVMGuestState, OS disk encryption before VM deployment that uses platform-managed keys (PMK) or a customer-managed key (CMK)
  • VMGuestStateOnly, no OS disk confidential encryption

When the SecurityEncryptionType is set to DiskWithVMGuestState, the SecureVMDiskEncryptionSetID can be specified as well.

Feature link: https://issues.redhat.com/browse/OCPBU-233

Related PR for GCP: #1384

Signed-off-by: Michail Resvanis mresvani@redhat.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 13, 2023

@mresvanis: This pull request references MGMT-13627 which is a valid jira issue.

In response to this:

This change adds the Confidential VM configuration options to the AzureMachineProviderSpec, in order to support Confidential Computing on Azure.

Specifically, the SecurityEncryptionType field defines whether the VM is a Confidential VM. It cannot be defined alongside the EncryptionAtHost. When SecurityEncryptionType is defined the VirtualizedTrustedPlatformModule should be set to Enabled, while SecureBoot can be either Enabled or Disabled.

Possible values for the SecurityEncryptionType are:

  • DiskWithVMGuestState, OS disk encryption before VM deployment that uses platform-managed keys (PMK) or a customer-managed key (CMK)
  • VMGuestStateOnly, no OS disk confidential encryption

When the SecurityEncryptionType is set to DiskWithVMGuestState, the SecureVMDiskEncryptionSetID can be specified as well.

Feature link: https://issues.redhat.com/browse/OCPBU-233

Related PRs for GCP:

Signed-off-by: Michail Resvanis mresvani@redhat.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mresvanis mresvanis force-pushed the mgmt-13627-add-azure-machine-confidential-vm-options branch 2 times, most recently from c8a6882 to 9de5dcf Compare February 13, 2023 17:52
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 14, 2023

@mresvanis: This pull request references MGMT-13627 which is a valid jira issue.

In response to this:

This change adds the Confidential VM configuration options to the AzureMachineProviderSpec, in order to support Confidential Computing on Azure.

Specifically, the SecurityEncryptionType field defines whether the VM is a Confidential VM. It cannot be defined alongside the EncryptionAtHost. When SecurityEncryptionType is defined the VirtualizedTrustedPlatformModule should be set to Enabled, while SecureBoot can be either Enabled or Disabled.

Possible values for the SecurityEncryptionType are:

  • DiskWithVMGuestState, OS disk encryption before VM deployment that uses platform-managed keys (PMK) or a customer-managed key (CMK)
  • VMGuestStateOnly, no OS disk confidential encryption

When the SecurityEncryptionType is set to DiskWithVMGuestState, the SecureVMDiskEncryptionSetID can be specified as well.

Feature link: https://issues.redhat.com/browse/OCPBU-233

Related PRs for Confidential Compute support on GCP:

Signed-off-by: Michail Resvanis mresvani@redhat.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mresvanis mresvanis force-pushed the mgmt-13627-add-azure-machine-confidential-vm-options branch 5 times, most recently from e787b3c to e9a3595 Compare February 17, 2023 09:13
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 17, 2023

@mresvanis: This pull request references MGMT-13627 which is a valid jira issue.

In response to this:

This change adds the Confidential VM configuration options to the AzureMachineProviderSpec, in order to support Confidential Computing on Azure.

Specifically, it introduces the OS disk security profile, which includes the SecurityEncryptionType and DiskEncryptionSetID fields and adds the SecureBoot and vTPM fields in the security profile of the VM.

The OS disk security profile SecurityEncryptionType field defines whether the VM is a Confidential VM.
It cannot be defined alongside the EncryptionAtHost. When SecurityEncryptionType is defined the VirtualizedTrustedPlatformModule should be set to Enabled, while SecureBoot can be either Enabled or Disabled.

Possible values for the SecurityEncryptionType are:

  • DiskWithVMGuestState, OS disk encryption before VM deployment that uses platform-managed keys (PMK) or a customer-managed key (CMK)
  • VMGuestStateOnly, no OS disk confidential encryption

When the SecurityEncryptionType is set to DiskWithVMGuestState, the DiskEncryptionSetID can be specified to allow using Customer-managed keys to encrypt the OS disk and the VMGuestState blob.

Feature link: https://issues.redhat.com/browse/OCPBU-233

Related PRs for Confidential Compute support on GCP:

Signed-off-by: Michail Resvanis mresvani@redhat.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mresvanis
Copy link
Contributor Author

mresvanis commented Feb 20, 2023

/hold
We should probably hold this PR until:

  • either the necessary RHCOS / RHEL features are present, in order to have full support for Confidential VMs on Azure (i.e. with support for OS attestation)
  • or we decide that support for SecureBoot, vTPM and OS disk encryption on Confidential VMs, but without OS attestation, is good enough as a first step

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 20, 2023
@mresvanis mresvanis force-pushed the mgmt-13627-add-azure-machine-confidential-vm-options branch from e9a3595 to e0f1ffd Compare March 9, 2023 13:47
@mresvanis
Copy link
Contributor Author

@mresvanis
Copy link
Contributor Author

mresvanis commented Mar 10, 2023

/unhold
I believe the consensus is that we can enable support for Confidential VMs and Trusted launch for VMs already, in order to have a head start for support of Confidential OpenShift Clusters in the future.

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 10, 2023
@mresvanis mresvanis force-pushed the mgmt-13627-add-azure-machine-confidential-vm-options branch from e0f1ffd to ad7cb37 Compare June 14, 2023 15:11
@openshift-ci openshift-ci bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 14, 2023
@mresvanis mresvanis force-pushed the mgmt-13627-add-azure-machine-confidential-vm-options branch 2 times, most recently from e89f31d to 092a0cc Compare June 16, 2023 12:58
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 16, 2023

@mresvanis: This pull request references MGMT-13627 which is a valid jira issue.

In response to this:

This change adds the Confidential VM configuration options to the AzureMachineProviderSpec, in order to support Confidential Computing on Azure.

Specifically, it introduces the OS disk security profile, which includes the SecurityEncryptionType and DiskEncryptionSetID fields and adds the SecureBoot and vTPM fields in the security profile of the VM.

The OS disk security profile SecurityEncryptionType field defines whether the VM is a Confidential VM.
It cannot be defined alongside the EncryptionAtHost. When SecurityEncryptionType is defined the VirtualizedTrustedPlatformModule should be set to Enabled, while SecureBoot can be either Enabled or Disabled.

Possible values for the SecurityEncryptionType are:

  • DiskWithVMGuestState, OS disk encryption before VM deployment that uses platform-managed keys (PMK) or a customer-managed key (CMK)
  • VMGuestStateOnly, no OS disk confidential encryption

When the SecurityEncryptionType is set to DiskWithVMGuestState, the DiskEncryptionSetID can be specified to allow using Customer-managed keys to encrypt the OS disk and the VMGuestState blob.

Feature link: https://issues.redhat.com/browse/OCPBU-233

Cluster API provider Azure PR: kubernetes-sigs/cluster-api-provider-azure#3265

Related PRs for Confidential Compute support on GCP:

Signed-off-by: Michail Resvanis mresvani@redhat.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mresvanis
Copy link
Contributor Author

Hi @JoelSpeed and @soltysh, now that the respective upstream PR is merged I would very much appreciate your review and feedback for this PR. Many thanks.

@JoelSpeed
Copy link
Contributor

I will try to review next week. A quick scan suggests that the names of some of the fields might need to be changed, since they don't fit the conventions (they don't fit upstream conventions either, eg VTpmEnabled should be VTPMEnabled, oh and it's a bool, which we don't allow either).

Could you please review our conventions and make the appropriate changes?

The API need not match upstream, we would prefer it matches conventions downstream where possible, so long as the API can be converted to upstream's at a later point, that's ok

@mresvanis mresvanis force-pushed the mgmt-13627-add-azure-machine-confidential-vm-options branch from 092a0cc to 3f4e0b1 Compare June 19, 2023 13:58
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 19, 2023

@mresvanis: This pull request references MGMT-13627 which is a valid jira issue.

In response to this:

This change adds the Confidential VM configuration options to the AzureMachineProviderSpec, in order to support Confidential Computing on Azure.

Specifically, it introduces the OS disk security profile, which includes the SecurityEncryptionType and DiskEncryptionSetID fields and adds the SecureBoot and vTPM fields in the security profile of the VM.

The OS disk security profile SecurityEncryptionType field defines whether the VM is a Confidential VM.
It cannot be defined alongside the EncryptionAtHost. When SecurityEncryptionType is defined the VirtualizedTrustedPlatformModule should be set to Enabled, while SecureBoot can be either Enabled or Disabled.

Possible values for the SecurityEncryptionType are:

  • DiskWithVMGuestState, OS disk encryption before VM deployment that uses platform-managed keys (PMK) or a customer-managed key (CMK)
  • VMGuestStateOnly, no OS disk confidential encryption

When the SecurityEncryptionType is set to DiskWithVMGuestState, the DiskEncryptionSetID can be specified to allow using customer-managed keys to encrypt the OS disk and the VMGuestState blob.

Feature link: https://issues.redhat.com/browse/OCPBU-233

Cluster API provider Azure PR: kubernetes-sigs/cluster-api-provider-azure#3265

Related PRs for Confidential Compute support on GCP:

Signed-off-by: Michail Resvanis mresvani@redhat.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mresvanis
Copy link
Contributor Author

@JoelSpeed I think I have covered the conformance to our API conventions, just a kind reminder for a review. Thanks in advance.

machine/v1beta1/types_azureprovider.go Show resolved Hide resolved
machine/v1beta1/types_azureprovider.go Outdated Show resolved Hide resolved
machine/v1beta1/types_azureprovider.go Outdated Show resolved Hide resolved
machine/v1beta1/types_azureprovider.go Outdated Show resolved Hide resolved
machine/v1beta1/types_azureprovider.go Show resolved Hide resolved
machine/v1beta1/types_azureprovider.go Outdated Show resolved Hide resolved
machine/v1beta1/types_azureprovider.go Outdated Show resolved Hide resolved
machine/v1beta1/types_azureprovider.go Outdated Show resolved Hide resolved
machine/v1beta1/types_azureprovider.go Show resolved Hide resolved
machine/v1beta1/types_azureprovider.go Show resolved Hide resolved
@mresvanis mresvanis force-pushed the mgmt-13627-add-azure-machine-confidential-vm-options branch 4 times, most recently from 325f265 to 0a96906 Compare July 3, 2023 13:39
@mresvanis mresvanis force-pushed the mgmt-13627-add-azure-machine-confidential-vm-options branch 2 times, most recently from 29db5be to dd9b827 Compare July 6, 2023 12:43
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 6, 2023

@mresvanis: This pull request references MGMT-13627 which is a valid jira issue.

In response to this:

This change adds the Confidential VM configuration options to the AzureMachineProviderSpec, in order to support Confidential Computing on Azure.

Specifically, it introduces the OS disk security profile, which includes the SecurityEncryptionType and DiskEncryptionSetID fields and adds the SecureBoot and VirtualizedTrustedPlatformModule fields in the security profile of the VM.

The OS disk security profile SecurityEncryptionType field defines whether the VM is a Confidential VM.
It cannot be defined alongside the EncryptionAtHost. When SecurityEncryptionType is defined the VirtualizedTrustedPlatformModule should be set to Enabled, while SecureBoot can be either Enabled or Disabled.

Possible values for the SecurityEncryptionType are:

  • DiskWithVMGuestState, OS disk encryption before VM deployment that uses platform-managed keys (PMK) or a customer-managed key (CMK)
  • VMGuestStateOnly, no OS disk confidential encryption

When the SecurityEncryptionType is set to DiskWithVMGuestState, the DiskEncryptionSetID can be specified to allow using customer-managed keys to encrypt the OS disk and the VMGuestState blob.

Feature link: https://issues.redhat.com/browse/OCPBU-233

Cluster API provider Azure PR: kubernetes-sigs/cluster-api-provider-azure#3265

Related PRs for Confidential Compute support on GCP:

Signed-off-by: Michail Resvanis mresvani@redhat.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 6, 2023

@mresvanis: This pull request references MGMT-13627 which is a valid jira issue.

In response to this:

This change adds the Confidential VM configuration options to the AzureMachineProviderSpec, in order to support Confidential Computing on Azure.

Specifically, it introduces the OS disk security profile, which includes the SecurityEncryptionType and DiskEncryptionSetID fields and adds the SecureBoot and VirtualizedTrustedPlatformModule fields in the security profile of the VM.

The OS disk security profile SecurityEncryptionType field defines whether the VM is a Confidential VM. It cannot be defined alongside the EncryptionAtHost. When SecurityEncryptionType is defined the VirtualizedTrustedPlatformModule should be set to Enabled, while SecureBoot can be either Enabled or Disabled.

Possible values for the SecurityEncryptionType are:

  • DiskWithVMGuestState, OS disk encryption before VM deployment that uses platform-managed keys (PMK) or a customer-managed key (CMK)
  • VMGuestStateOnly, no OS disk confidential encryption

When the SecurityEncryptionType is set to DiskWithVMGuestState, the DiskEncryptionSetID can be specified to allow using customer-managed keys to encrypt the OS disk and the VMGuestState blob.

Feature link: https://issues.redhat.com/browse/OCPBU-233

Cluster API provider Azure PR: kubernetes-sigs/cluster-api-provider-azure#3265

Related PRs for Confidential Compute support on GCP:

Signed-off-by: Michail Resvanis mresvani@redhat.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mresvanis
Copy link
Contributor Author

/test verify

Copy link
Contributor

@JoelSpeed JoelSpeed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you can update the required tags to match the correct format, there's one I highlighted and one a few lines below, after that, this LGTM

machine/v1beta1/types_azureprovider.go Outdated Show resolved Hide resolved
@mresvanis mresvanis force-pushed the mgmt-13627-add-azure-machine-confidential-vm-options branch from dd9b827 to bf194f4 Compare July 7, 2023 11:15
…ineProviderSpec

This change introduces the security profile to the OS disk parameters
and adds the SecurityType and UEFISettings sections to the VM's security
profile.

The SecurityType defines whether the VM is a Confidential or a Trusted Launch
VM. This field should be set to one of the two options, in order to enable
the UEFISettings section.

The UEFISettings fields include the VirtualizedTrustedPlatformModule and
SecureBoot fields, which can be set to either Enabled or Disabled.

The OS disk security profile includes the SecurityEncryptionType and the
DiskEncryptionSet fields. The SecurityEncryptionType can only be used when
the VM is a Confidential one. When SecurityEncryptionType is set, vTPM
should be enabled and the VM security profile's SecurityType should also
be set to ConfidentialVM.

Possible values for the SecurityEncryptionType are:
- DiskWithVMGuestState, OS disk encryption before VM deployment that
  uses platform-managed keys (PMK) or a customer-managed key (CMK)
- VMGuestStateOnly, no OS disk confidential encryption

When the SecurityEncryptionType is set to DiskWithVMGuestState, the
DiskEncryptionSet can be specified to allow for customer-managed keys
to be used to encrypt the OS disk and the VMGuestState blob.

Signed-off-by: Michail Resvanis <mresvani@redhat.com>
@mresvanis mresvanis force-pushed the mgmt-13627-add-azure-machine-confidential-vm-options branch from bf194f4 to 173c2ae Compare July 7, 2023 11:21
@mresvanis
Copy link
Contributor Author

/retest-required

@JoelSpeed
Copy link
Contributor

/test verify

@JoelSpeed
Copy link
Contributor

/lgtm

The verify failure is in code that's already in master, looks like maybe there's an ordering issue coming out somewhere, will need to be investigated

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 7, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 7, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JoelSpeed, mresvanis

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 7, 2023
@JoelSpeed
Copy link
Contributor

/test verify

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 7, 2023

@mresvanis: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-merge-robot openshift-merge-robot merged commit 21c0ce7 into openshift:master Jul 7, 2023
7 checks passed
@mresvanis mresvanis deleted the mgmt-13627-add-azure-machine-confidential-vm-options branch July 7, 2023 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants