-
Notifications
You must be signed in to change notification settings - Fork 157
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bump Kubernetes version to 1.24.9 in CI #1106
Conversation
Need to ensure we also test on the K8S version EKS has. |
44b0ad2
to
5a39a41
Compare
Unless this is critical, let's defer this to future milestones. |
Our CI uses 1.23.x which will be considered End of Life in 3 months. |
16747a4
to
639c3db
Compare
d5de8d7
to
8e76325
Compare
.github/actions/setup-local-volume-provisioner/manifests/50_daemonset.yaml
Outdated
Show resolved
Hide resolved
.github/actions/setup-local-volume-provisioner/manifests/50_daemonset.yaml
Outdated
Show resolved
Hide resolved
8e76325
to
1f74b85
Compare
@tnozicka Thanks for detailed review, all your comments are addressed. PTAL |
bf064e4
to
0707324
Compare
var ( | ||
registry = initRegistry() | ||
|
||
imageConfigs = initImageConfigs(registry) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fwiw, kube has it so the downstreams can customize universal images (like for bash, so they play well with their security, like user ids). I don't think we have the need but I don't feel strong one way or the other
c0c3847
to
5e83100
Compare
5e83100
to
57b442b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think thet e2e could avoid waiting for deletion if you'd use UUID and put the name into annotation but it's not worth redoing it.
Got last 3 simple nits, rest lgtm
57b442b
to
d4c4785
Compare
Currently used minikube with `none` driver requires dockershim coming from 3rd party provider when Kubernetes version above 1.24 is requested. The main reason why we used minikube it's storage provisioner which comes with it. Even though it's not perfect and we have issues with it (scylladb#974). Adding new dependency doesn't seem right for our usecase. This change introduces spawning Kube cluster using dedicated tool `kubeadm`. It gives us more customizable setup, then the previous one, some of the existing hacks are now a configuration option. It also allows us to choose different CRI, in this case I used cri-o. Missing storage provisioner was replaced with battle tested upstream kubernetes-sigs/sig-storage-local-static-provisioner. Fixes scylladb#1137 Fixes scylladb#975
Test logic relied on fact that minikube storage provisioner did not set any NodeAffinity for provisioned PVs. This allowed logic to trigger cleanup logic by adding a NodeAffinity pointing to not existing Node. When proper local volume proviosiner is used, and NodeAffinity is configured for new PVs, test no longer have option to trigger cleanup logic because NodeAffinity field is immutable. To overcome this, the ScyllaCluster is using custom StorageClass not backed by any provisioner running in the cluster. ScyllaCluster is going to request PVC from that StorageClass, and the test is going to request a clone of the original PVC from the default StorageClass to get any storage. Then the bound PV is rebounded to the original PVC but with empty NodeAffinity. This allows the test to trigger the orphaned PV cleanup logic.
d4c4785
to
83c37ed
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/approve
/lgtm
Great job @zimnx, bringing us to new Kube is important and this also makes our storage setup much closer to production! Which is also why you hit the test issues and had to make a complex fix there. Thanks!
Currently used minikube with
none
driver requires dockershim comingfrom 3rd party provider when Kubernetes version above 1.24 is requested.
The main reason why we used minikube it's storage provisioner which
comes with it. Even though it's not perfect and we have issues with it (#974).
Adding new dependency doesn't seem right for our usecase.
This change introduces spawning Kube cluster using dedicated tool
kubeadm
.It gives us more customizable setup, then the previous one, some of the
existing hacks are now a configuration option.
It also allows us to choose different CRI, in this case I used cri-o.
Missing storage provisioner was replaced with battle tested upstream
kubernetes-sigs/sig-storage-local-static-provisioner.
Fixes #1137
Fixes #975
Requires: