[CI] Add install-k8s role

[gman0]

What this PR does / why we need it:

This PR adds install-k8s role for the CI Ansible playbooks.

Which issue this PR fixes(if applicable):
fixes #

Special notes for reviewers:

install-k3s brings in many OpenStack dependencies. manila-csi e2e test (#1656) doesn't install these, so it fails to run.

Release note:

NONE

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign adisky after the PR has been reviewed.
You can assign the PR to them by writing /assign @adisky in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[gman0]

PTAL @lingxiankong @ramineni @jichenjc

[jichenjc]

just curious , I think k3s is a k8s compatible suite that we used for our test
if k3s role has dependency, why not split it and we introduce a k8s role so we will have
k8s and k3s both in our test?

[gman0]

@jichenjc this is a copy-paste from what we had in OpenLab, where we used k3s for multi-node setup, and then regular k8s built from source and run with local-cluster-up.sh for the rest.

[gman0]

The setup in this PR is known to work well, but I do agree that cleaner solution would be to do as you suggested.

[jichenjc]

thanks for the info, maybe other guys have comments .. I personally think if we have one (k3s), unless really needed, I prefer to not introduce other test backend to make consistent and easy maintain

[gouthampacha]

thanks for the info, maybe other guys have comments .. I personally think if we have one (k3s), unless really needed, I prefer to not introduce other test backend to make consistent and easy maintain

my 0.02 cents: it feels like the install-k3s role seems to do a lot more heavy-lifting to get k3s working - i.e., creating master, worker VMs, setting up networking, FIPs etc. I see this as a simpler alternative for testing components like manila that can be made to work without compute/image/volume/networking dependencies.

[lingxiankong]

install-k3s role complies with the practical environment for most of CPO components, that are running in a k8s cluster on top of openstack, that's why we create VMs and inside which we install k3s.

Maybe manila-csi-plugin can work with a k8s cluster that is running not only in openstack environment but also baremetal or others, if yes, then install-k3s may be a little overkill, we could add another role to simplify this.

I have 2 questions about this PR:

1. Why do we have to build k8s binaries from master each time for just testing manila-csi-plugin rather than download the pre-built artifacts from k8s release repo?
2. In this PR, I only see this ansible role is building binaries but not running a local cluster?

[gman0]

@lingxiankong thanks for the input! If it's ok to run with stable k8s releases (I'm not sure why this wasn't the case already in OpenLab), there are easier ways to get a cluster going. Would kubeadm be acceptable? A role for this already exists in https://github.com/theopenlab/openlab-zuul-jobs/tree/master/roles/create-single-k8s-cluster-with-kubeadm , maybe we can borrow it and use it here too. As far as manila-csi-plugin goes, this is completely sufficient as it doesn't need any of the cloud provider stuff.

[lingxiankong]

@lingxiankong thanks for the input! If it's ok to run with stable k8s releases (I'm not sure why this wasn't the case already in OpenLab), there are easier ways to get a cluster going.

Some historical reasons :-)

Would kubeadm be acceptable? A role for this already exists in https://github.com/theopenlab/openlab-zuul-jobs/tree/master/roles/create-single-k8s-cluster-with-kubeadm , maybe we can borrow it and use it here too. As far as manila-csi-plugin goes, this is completely sufficient as it doesn't need any of the cloud provider stuff.

Yes, kubeadm could definitely do the job. However, compared with k3s or other lightweight alternatives, kubeadm is apparent not the best option :-) Because with the others, you can also choose to not create VMs inside devstack, but just run their binaries directly on the CI host. I believe the CI best practice is to get rid of as more things that are not relevant as possible.

Again, it's totally your choice for manila-csi-plugin, above is just my 2 cents. Feel free to use any option you think is easy to implement and maintain.

[ramineni]

@gman0 I dont quite understand this PR.

Are you planning to use local-up-cluster.sh for bring cluster up or any other? I don't see a reason to build the k8s binaries every time the job is run.

If you want to use local-up-cluster.sh , that's not recommended way forward actually . That script changes a lot and not reliable. So we planned to move everything to kubeadm , now k3s is lightweight option, so using that.

[gman0]

@ramineni

I dont quite understand this PR.

The goal is to have a simple single-node k8s cluster deployment. I don't particularly care about the way we achieve this, so the initial idea was to replicate the install-k8s role we used in OpenLab jobs, but I'm open for other options of course, as long as they are easy enough to implement and maintain.

I don't see a reason to build the k8s binaries every time the job is run.

@lingxiankong and I already agreed to get rid of building the binaries in comments above :)

If you want to use local-up-cluster.sh , that's not recommended way forward actually . That script changes a lot and not reliable. So we planned to move everything to kubeadm , now k3s is lightweight option, so using that.

Thanks for pointing that out. If it's considered volatile, I prefer to stay away from it. I'm trying out k3s right now and seeing how the existing install-k3s role could be modified to accommodate single-node deployments. I'll open a separate PR for that.

[gman0]

Hi all, please take a look at #1669 . If you think the new one is a step in the right direction, I can close this PR.

[gman0]

Closing. #1669 got merged instead.