Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kola: add kubeadm tests #171

Merged
merged 3 commits into from May 5, 2021
Merged

Conversation

tormath1
Copy link
Contributor

@tormath1 tormath1 commented Apr 26, 2021

In this PR, we add kubernetes tests using kubeadm and ignition to setup the cluster.

A user-data script will be uploaded through ignition on both worker and controller in order to run the required commands. ignition will take care of pulling required binaries / uploading various files (user-data, manifest).

Test will assert that there is two running nodes and it will deploy a nginx pod.

How to use

./build kola

Testing done

It has been tested on qemu platform with alpha-2823.0.0:

$ sudo ./bin/kola run -b cl -p qemu --qemu-image flatcar_production_qemu_image.img kubeadm.base
=== RUN   kubeadm.base
=== RUN   kubeadm.base/node_readiness
=== RUN   kubeadm.base/nginx_deployment
--- PASS: kubeadm.base (317.05s)
        cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /etc/systemd/system/kubelet.service.
        cluster.go:117: 	[WARNING Service-Docker]: docker service is not enabled, please run 'systemctl enable docker.service'
        cluster.go:117: 	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    --- PASS: kubeadm.base/node_readiness (52.85s)
    --- PASS: kubeadm.base/nginx_deployment (22.12s)
PASS, output in _kola_temp/qemu-2021-04-26-1847-14719

and on GCP with latest alpha (alpha-2857.0.0) (note --gce-machinetype=n1-standard-2):

./bin/kola run --basename=${BASENAME} \
    --gce-image=${IMAGE_NAME} \
    --gce-json-key=${GCE_JSON_KEY} \
    --parallel=4 --gce-machinetype=n1-standard-2 --platform=gce \
    ${TEST_NAME}
=== RUN   kubeadm.base
=== RUN   kubeadm.base/node_readiness
=== RUN   kubeadm.base/nginx_deployment
--- PASS: kubeadm.base (294.59s)
    --- PASS: kubeadm.base/node_readiness (25.84s)
    --- PASS: kubeadm.base/nginx_deployment (15.90s)
PASS, output in _kola_temp/gce-2021-04-30-1131-290038

What's missing

  • try to use ignition instead of script to provision the nodes (worker and master)
  • test on other platforms
  • add some unit-tests maybe to test the logic
  • decide what we should do: should we merge kubeadm and kubernetes tests ? should we drop the current kubernetes tests in favor of this one ? let's keep them in a separate test, then we will remove slowly the kubernetes ones

@tormath1
Copy link
Contributor Author

@pothos comments have been addressed. Ignition is also now used to download binaries file and also to upload "user-data" and nginx manifest.

@tormath1 tormath1 marked this pull request as ready for review April 30, 2021 09:42
@tormath1 tormath1 requested a review from a team April 30, 2021 09:42
@tormath1
Copy link
Contributor Author

tormath1 commented May 5, 2021

@pothos thanks for the review - comments have been addressed !

Copy link
Member

@pothos pothos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to go, thanks!
In a follow up PR we can exclude problematic platforms if we see in the CI that some platforms do not work for whatever reason.

@tormath1 tormath1 force-pushed the tormath1/kubernetes-kubeadm branch from 0d5b883 to b6ffb7e Compare May 5, 2021 15:09
@tormath1 tormath1 merged commit 5ee38bd into flatcar-master May 5, 2021
@tormath1 tormath1 deleted the tormath1/kubernetes-kubeadm branch May 5, 2021 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants