libvirt: deploy kbs for libvirt provider in e2e test

[huoqifeng]

Fixes: #1777

[huoqifeng]

• Install k8s cluster and enable libvirt provider in this cluster
• Create a file /root/libvirt.properties like

# cat /root/libvirt.properties 
CLUSTER_NAME = "peer-pods"
KBS_IMAGE = "ghcr.io/confidential-containers/key-broker-service"
KBS_IMAGE_TAG = "v0.8.2"

• Clone trustee repo

cd cloud-api-adaptor/src/cloud-api-adaptor/test/e2e
git clone https://github.com/confidential-containers/trustee.git 

• Run e2e test

cd cloud-api-adaptor/src/cloud-api-adaptor
export TEST_PROVISION_FILE="/root/libvirt.properties"
export CLOUD_PROVIDER=libvirt
export DEPLOY_KBS=true
export TEST_INSTALL_CAA=no
export TEST_TEARDOWN=no
make test-e2e

• Check KBS pod

# kubectl get po -A |grep kbs
coco-tenant                      kbs-6f58d6dc94-xj9z8                               1/1     Running      0               12m

• e2e test logs

# make test-e2e
go test -v -tags=libvirt -timeout 60m -count=1 ./test/e2e
time="2024-04-08T01:22:37Z" level=info msg="Do setup"
time="2024-04-08T01:22:37Z" level=info msg="Deploying kbs"
time="2024-04-08T01:22:37Z" level=info msg="creating key.bin"
/root/src/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e
time="2024-04-08T01:22:37Z" level=info msg="Creating kbs install overlay"
time="2024-04-08T01:22:37Z" level=info msg="Customize the overlay yaml file"
time="2024-04-08T01:22:37Z" level=info msg="Updating kbs image with \"ghcr.io/confidential-containers/key-broker-service\""
time="2024-04-08T01:22:37Z" level=info msg="Updating kbs image tag with \"v0.8.2\""
time="2024-04-08T01:22:37Z" level=info msg="Creating kbs install overlay"
time="2024-04-08T01:22:37Z" level=info msg="Install Kbs"
Pod IP: 
time="2024-04-08T01:22:37Z" level=info msg="KBS PARAMScc_kbc::http://:8080:"
time="2024-04-08T01:22:37Z" level=info msg="Creating namespace 'coco-pp-e2e-test-3e99118f'..."
time="2024-04-08T01:22:37Z" level=info msg="Wait for namespace 'coco-pp-e2e-test-3e99118f' be ready..."
time="2024-04-08T01:22:44Z" level=info msg="Wait for default serviceaccount in namespace 'coco-pp-e2e-test-3e99118f'..."
time="2024-04-08T01:22:44Z" level=info msg="default serviceAccount exists, namespace 'coco-pp-e2e-test-3e99118f' is ready for use"
=== RUN   TestLibvirtCreateSimplePod
=== RUN   TestLibvirtCreateSimplePod/SimplePeerPod_test
    assessment_runner.go:259: Waiting for containers in pod: simple-test are ready
=== RUN   TestLibvirtCreateSimplePod/SimplePeerPod_test/PodVM_is_created
time="2024-04-08T01:23:24Z" level=info msg="Deleting pod simple-test..."
time="2024-04-08T01:23:29Z" level=info msg="Pod simple-test has been successfully deleted within 60s"
--- PASS: TestLibvirtCreateSimplePod (45.70s)
    --- PASS: TestLibvirtCreateSimplePod/SimplePeerPod_test (45.70s)
        --- PASS: TestLibvirtCreateSimplePod/SimplePeerPod_test/PodVM_is_created (0.00s)
=== RUN   TestLibvirtCreateSimplePodWithNydusAnnotation
=== RUN   TestLibvirtCreateSimplePodWithNydusAnnotation/SimplePeerPod_test
...

[stevenhorsman]

The code looks reasonable and the instructions and output you've provided seem to work. I do wonder if we need to document them (noteable where to check out trustee to and the DEPLOY_KBS flag and options) in the e2e README, but I appreciate that you didn't add these, so I'm not sure if should block this PR.

[mkulke]

lgtm

[huoqifeng]

The code looks reasonable and the instructions and output you've provided seem to work. I do wonder if we need to document them (noteable where to check out trustee to and the DEPLOY_KBS flag and options) in the e2e README, but I appreciate that you didn't add these, so I'm not sure if should block this PR.

Thanks @stevenhorsman , We still have some gaps to provide a e2e guide besides deploying KBS, I'll add the guide when have the TEE enabled pod running.