Introduce API endpoints for SEV attestation #7197

vasiliy-ul · 2022-02-08T13:32:55Z

What this PR does / why we need it:

The PR introduces new API endpoints for SEV attestation. It is based on the initial design proposal. It allows to perform the necessary pre-launch attestation steps:

retrieving platform certificates for verification sev/fetchcertchain: the API call is routed to the node where the VM is scheduled and it is processed by virt-handler. The certificates are fetched from libvirt by node-labeller in the init-container context (thus ATM the support of certificates rotation is limited); UPD: the call is routed to the virt-launcher pod where the corresponding libvirt API is invoked;
setting session parameters sev/setupsession: the call is routed to the virt-launcher pod where the corresponding libvirt API is invoked;
querying launch measurement sev/querylaunchmeasurement: the call is routed to the virt-launcher pod where the corresponding libvirt API is invoked. The measurement is returned to the caller;
injecting launch secret sev/injectsecret: the call is routed to the virt-launcher pod where the corresponding libvirt API is invoked.

The attestation process can be run either manually by calling the corresponding endpoints (e.g. using virtctl) or in an automated way by leveraging an attestation server and a controller that will monitor the VMI state and perform the required actions. The E2E functional test in tests/launchsecurity/sev.go demonstrates the general flow.

Attestation can be requested via VMI spec yaml:

spec:
  startStrategy: "Paused"
  domain:
    firmware:
      bootloader:
        efi:
          secureBoot: false
    launchSecurity:
      sev:
        attestation: {}

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

The implementation of the new virtctl commands mapped to the SEV endpoints is not in the scope of this PR and will be added later in a follow-up (same applies to the attestation controller: not in the scope ATM). UPD: Extend virtctl to Support Interactive SEV Attestation #7595
The PR requires libvirt >= 8.0.0: rpm: Update virtualization packages #7277
There is a proposal to generalize the attestation flow in libvirt: https://listman.redhat.com/archives/libvir-list/2022-February/msg00857.html. Any thoughts on that are welcome.

Release note:

Experimantal support of SEV attestation via the new API endpoints

Introduce sev/querylaunchmeasurement API endpoint for VMI. Apart from the measurement itself it returns the data needed to calculate the expected value as specified in AMD SEV specification: HMAC(0x04 || API_MAJOR || API_MINOR || BUILD || GCTX.POLICY || GCTX.LD || MNONCE; GCTX.TIK) Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Introduce sev/setupsession API endpoint for VMI. It can be used to provide the session launch blob and the Diffie-Hellman key for a guest that has been scheduled to run on a particular node. Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Introduce sev/injectlaunchsecret API endpoint for VMI to inject an encrypted secret into a paused guest. Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

This reverts fetching of the certs from the node labeller. If fetched directly in virt-launcher context that will automatically handle certificates rotation since the API call will return the most recent state. Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Ensure the request fails if VMI is not running. Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Do not hardcode the expected parameters since they are bound to a specific hardware instance. Instead try to detect those in runtime using virsh cli. Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Use sevctl tool to prepare the parameters in runtime. Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

vasiliy-ul · 2023-06-23T08:23:01Z

/test pull-kubevirt-e2e-k8s-1.26-sev

vasiliy-ul · 2023-06-23T10:57:21Z

/test pull-kubevirt-e2e-k8s-1.26-sig-storage

vasiliy-ul · 2023-06-27T05:47:41Z

Hi @alicefr, @iholder101. Previously, you reviewed the code here. Could you please re-lgtm/approve the PR (as you see fit)? I now rebased it and fixed all the conflicts (mostly with the generated code). There are no functional changes. The only thing is that I've moved the addition of sevctl tool to the test image into a separate PR (which is already merged #9892). Thanks!

@alicefr, @iholder101, a gentle ping :) Would appreciate it if you could re-lgtm or approve the PR. Thanks!

alicefr · 2023-06-27T06:23:16Z

pkg/virt-launcher/virtwrap/manager_test.go

+
+			manager, _ := NewLibvirtDomainManager(mockConn, testVirtShareDir, testEphemeralDiskDir, nil, ovmfDir, ephemeralDiskCreatorMock, metadataCache)
+			sevMeasurementInfo, err := manager.GetLaunchMeasurement(vmi)
+			if runtime.GOARCH == "amd64" {


Shouldn't we skip all the SEV suite if this isn't amd64?

I am not sure about skipping, since it may hide potential issues. If the code is expected to error-out on non-amd64, IMHO, better to have a test in place to ensure it. In this particular case, manager.GetLaunchMeasurement fails due to missing ovmf code binary with sev support.

Ok, fine to me!

alicefr · 2023-06-27T06:28:22Z

tests/launchsecurity/sev.go

@@ -197,6 +187,31 @@ var _ = Describe("[sig-compute]AMD Secure Encrypted Virtualization (SEV)", decor
 		}
 	}

+	parseVirshInfo := func(info string, expectedKeys []string) map[string]string {


This might be a good function to move to a shared file for utils. It can be done in a following PR

Agree, this is a good candidate for common utils. Though, on the other hand, I think it makes sense to move it when there are several users of that function. Otherwise, the utils tend to grow.

alicefr · 2023-06-27T06:29:03Z

tests/launchsecurity/sev.go

+		return entries
+	}
+
+	toUint := func(s string) uint {


Same as above we could move it to a common shared file

alicefr · 2023-06-27T06:36:40Z

/approve
/hold
Let's give some time to @iholder101 if he wants to have another look, but to me, it looks good
@vasiliy-ul great work and thanks for the perseverance!

kubevirt-bot · 2023-06-27T06:36:51Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alicefr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [alicefr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

vasiliy-ul · 2023-06-27T06:47:07Z

Thanks a lot @alicefr for the review and valuable input!

iholder101 · 2023-06-29T14:33:48Z

Sorry for the delay @vasiliy-ul!

Thanks very much for this PR! Appreciate your contributions a lot, I know it wasn't easy :)

/lgtm
/unhold

kubevirt-commenter-bot · 2023-06-30T03:19:21Z

/retest-required
This bot automatically retries required jobs that failed/flaked on approved PRs.
Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

kubevirt-bot · 2023-06-30T03:38:39Z

@vasiliy-ul: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-kubevirt-e2e-k8s-1.23-sig-compute	`460fdec`	link	true	`/test pull-kubevirt-e2e-k8s-1.23-sig-compute`
pull-kubevirt-fossa	`3528f6a`	link	false	`/test pull-kubevirt-fossa`

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

vasiliy-ul · 2023-06-30T05:34:41Z

Thank you for the review, @iholder101!

/test pull-kubevirt-unit-test

dbasunag · 2023-07-19T17:55:04Z

Looks like this PR broke https://github.com/openshift/openshift-restclient-python/blob/master/openshift/dynamic/discovery.py#L133, which expects VMInstancesSEVFetchCertChain/VMInstancesSEVQueryLaunchMeasurement/VMInstancesSEVSetupSession/VMInstancesSEVInjectLaunchSecret to be of the format 'a/b' not a/b/c

cc: @myakove

vasiliy-ul marked this pull request as draft February 8, 2022 13:33

kubevirt-bot requested review from kbidarkar and rmohr February 8, 2022 13:33

kubevirt-bot added the size/XL label Feb 8, 2022

vasiliy-ul mentioned this pull request Feb 8, 2022

AMD SEV technology enablement #6991

Open

15 tasks

kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 10, 2022

vasiliy-ul force-pushed the sev-attestation branch from 5fb5971 to 3355916 Compare February 10, 2022 11:35

kubevirt-bot added size/XXL and removed size/XL needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Feb 10, 2022

vasiliy-ul force-pushed the sev-attestation branch from 3355916 to 68657c2 Compare February 11, 2022 13:23

vasiliy-ul force-pushed the sev-attestation branch 2 times, most recently from e16ee9f to 7bb8eb2 Compare March 9, 2022 13:44

vasiliy-ul force-pushed the sev-attestation branch 2 times, most recently from 775b7e0 to bb14be1 Compare March 18, 2022 09:08

kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 22, 2022

vasiliy-ul force-pushed the sev-attestation branch from bb14be1 to b9e1b16 Compare April 4, 2022 09:03

kubevirt-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 4, 2022

vasiliy-ul force-pushed the sev-attestation branch from b9e1b16 to 229f33d Compare April 5, 2022 16:41

kubevirt-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Apr 5, 2022

vasiliy-ul changed the title ~~WIP Introduce API endpoints for SEV attestation~~ Introduce API endpoints for SEV attestation Apr 5, 2022

vasiliy-ul marked this pull request as ready for review April 5, 2022 16:44

kubevirt-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 5, 2022

kubevirt-bot requested review from rnetser and yuhaohaoyu April 5, 2022 16:44

vasiliy-ul force-pushed the sev-attestation branch from 229f33d to d0266e0 Compare April 6, 2022 07:32

vasiliy-ul added 9 commits June 23, 2023 09:49

Allow to inject SEV launch secret into a guest

23f24a9

Introduce sev/injectlaunchsecret API endpoint for VMI to inject an encrypted secret into a paused guest. Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Add functional test for SEV attestation API

38424c8

Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Extend vnc and console subresource unit tests

41e3df4

Ensure the request fails if VMI is not running. Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Query SEV attestation settings in the tests

0d64f69

Do not hardcode the expected parameters since they are bound to a specific hardware instance. Instead try to detect those in runtime using virsh cli. Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Do not hardcode SEV session in the tests

f981a39

Use sevctl tool to prepare the parameters in runtime. Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

Expect SEV test to fail on non-amd64 architectures

f8464bd

Signed-off-by: Vasiliy Ulyanov <vulyanov@suse.de>

vasiliy-ul force-pushed the sev-attestation branch from 3a830dd to f8464bd Compare June 23, 2023 08:22

kubevirt-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 23, 2023

alicefr reviewed Jun 27, 2023

View reviewed changes

kubevirt-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 27, 2023

kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 27, 2023

kubevirt-bot added lgtm Indicates that a PR is ready to be merged. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Jun 29, 2023

kubevirt-bot merged commit 34daa9c into kubevirt:main Jun 30, 2023
37 checks passed

vasiliy-ul deleted the sev-attestation branch June 30, 2023 08:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce API endpoints for SEV attestation #7197

Introduce API endpoints for SEV attestation #7197

vasiliy-ul commented Feb 8, 2022 •

edited

vasiliy-ul commented Jun 23, 2023

vasiliy-ul commented Jun 23, 2023

vasiliy-ul commented Jun 27, 2023

alicefr Jun 27, 2023 •

edited

vasiliy-ul Jun 27, 2023

alicefr Jun 27, 2023

alicefr Jun 27, 2023 •

edited

vasiliy-ul Jun 27, 2023

alicefr Jun 27, 2023

alicefr commented Jun 27, 2023

kubevirt-bot commented Jun 27, 2023

vasiliy-ul commented Jun 27, 2023

iholder101 commented Jun 29, 2023

kubevirt-commenter-bot commented Jun 30, 2023

kubevirt-bot commented Jun 30, 2023 •

edited

vasiliy-ul commented Jun 30, 2023

dbasunag commented Jul 19, 2023 •

edited

Introduce API endpoints for SEV attestation #7197

Introduce API endpoints for SEV attestation #7197

Conversation

vasiliy-ul commented Feb 8, 2022 • edited

vasiliy-ul commented Jun 23, 2023

vasiliy-ul commented Jun 23, 2023

vasiliy-ul commented Jun 27, 2023

alicefr Jun 27, 2023 • edited

Choose a reason for hiding this comment

vasiliy-ul Jun 27, 2023

Choose a reason for hiding this comment

alicefr Jun 27, 2023

Choose a reason for hiding this comment

alicefr Jun 27, 2023 • edited

Choose a reason for hiding this comment

vasiliy-ul Jun 27, 2023

Choose a reason for hiding this comment

alicefr Jun 27, 2023

Choose a reason for hiding this comment

alicefr commented Jun 27, 2023

kubevirt-bot commented Jun 27, 2023

vasiliy-ul commented Jun 27, 2023

iholder101 commented Jun 29, 2023

kubevirt-commenter-bot commented Jun 30, 2023

kubevirt-bot commented Jun 30, 2023 • edited

vasiliy-ul commented Jun 30, 2023

dbasunag commented Jul 19, 2023 • edited

vasiliy-ul commented Feb 8, 2022 •

edited

alicefr Jun 27, 2023 •

edited

alicefr Jun 27, 2023 •

edited

kubevirt-bot commented Jun 30, 2023 •

edited

dbasunag commented Jul 19, 2023 •

edited