-
Notifications
You must be signed in to change notification settings - Fork 789
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
resourceLock = endpoints, endpointsleases or leases ? #3161
Comments
Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗 |
Argh this is related to the semverCompare and eks reported version i figure. Could you render a dummy resource that renders what the helm value {{ .Capabilities.KubeVersion.Version }} renders to and play around with what the if / else clause renders to if you adjust that from ita dynamic value to a hardcoded string you experiment with to understand what is wrong with the semverCompare statement. To do this, do "helm create dummy-chart", update a configmap to include an if/else statement and let it render a comment or another comment, and use "helm template ." to render it against your k8s cluster with the version 1.27 on eks. / From a mobile, on vacation, i'll review any PRs if you open one about this, note that we need to ensure all semverCompare is updated in the chart |
Hello 👋🏻 Here is an example output with an The apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-configmap
data:
Capabilities.KubeVersion.Version: {{ .Capabilities.KubeVersion.Version }}
Capabilities.KubeVersion.Major: {{ .Capabilities.KubeVersion.Major }}
Capabilities.KubeVersion.Minor: {{ .Capabilities.KubeVersion.Minor }}
{{- if semverCompare ">=1.28.0-0" .Capabilities.KubeVersion.Version }}
Capabilities.KubeVersion.Version: if__{{ .Capabilities.KubeVersion.Version }}
{{- else }}
Capabilities.KubeVersion.Version: else__{{ .Capabilities.KubeVersion.Version }}
{{- end }} The output : (with the command ---
# Source: dummy-chart/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: testchart-configmap
data:
Capabilities.KubeVersion.Version: v1.27.3-eks-a5565ad
Capabilities.KubeVersion.Major: 1
Capabilities.KubeVersion.Minor: 27+
Capabilities.KubeVersion.Version: else__v1.27.3-eks-a5565ad With that condition, the file diff --git a/jupyterhub/templates/scheduling/user-scheduler/configmap.yaml b/jupyterhub/templates/scheduling/user-scheduler/configmap.yaml
index 0f142b01..d4d959d2 100644
--- a/jupyterhub/templates/scheduling/user-scheduler/configmap.yaml
+++ b/jupyterhub/templates/scheduling/user-scheduler/configmap.yaml
@@ -29,7 +29,11 @@ data:
{{- end }}
kind: KubeSchedulerConfiguration
leaderElection:
- resourceLock: endpointsleases
+ {{- if semverCompare ">=1.27.0-0" .Capabilities.KubeVersion.Version }}
+ resourceLock: leases
+ {{- else }}
+ resourceLock: endpoints
+ {{- end }}
resourceName: {{ include "jupyterhub.user-scheduler-lock.fullname" . }}
resourceNamespace: "{{ .Release.Namespace }}"
profiles: |
I think the issue is that you ended up with v1beta3 in the first place, using k8s 1.27, semverCompare should have rendered you to get v1 of that kube-scheduler config file. You should also have got "leases" in your PR instead of "endpoints". So, why doesnt semverCompare work with the EKS provided version? Why does semverCompare used by you result in the wrong version? What version of helm are you using? |
Also, are you using helm directly, or using it indirectly via argocd or similar? If templates arent rendered against a k8s api-server, what value does the .Version field take? |
I don't think there is a bug because this is tested to work I think, but I may be wrong. I'm happy to look into this further given information about:
|
Bug description
That issue is close to the issue #3128.
The value of
resourceLock
used is not correct. Indeed, I have that value in the ConfigMapuser-scheduler
:but the log from the pod
user-scheduler
returns :Expected behaviour
That value shoud be
leases
as returned by the log.The value is hardcoded. Maybe a comparison with the k8s version should be done like this ⬇️ but I do not know from which version of k8s the value
leases
should be used.Actual behaviour
Change the value of
resourceLock
toleases
.Your personal set up
Here is the version of k8s (eks) I'm using :
The text was updated successfully, but these errors were encountered: