-
Notifications
You must be signed in to change notification settings - Fork 864
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kubeflow manifests and kustomize 5 #2388
Comments
The situation with Kustomize 5 is that:
I'll expose more technical details in the following messages, but I propose that we do the switch and just go with Kustomize 5 |
Specifically, if we try to build the
We could try to do |
Next up, if we'd try to apply these manifests we'll see that the generated manifests look like this:
The problem here is that the line To get around this we can just parse the output and remove such lines with something like
|
Lastly, in order to solve the initial issue we had with ordering we'll need to use the sortOptions:
order: legacy
legacySortOptions:
orderFirst:
- Namespace
- ResourceQuota
- StorageClass
- CustomResourceDefinition
- MutatingWebhookConfiguration
- ServiceAccount
- PodSecurityPolicy
- Role
- ClusterRole
- RoleBinding
- ClusterRoleBinding
- ConfigMap
- Secret
- Endpoints
- Service
- LimitRange
- PriorityClass
- PersistentVolume
- PersistentVolumeClaim
- Deployment
- StatefulSet
- CronJob
- PodDisruptionBudget
orderLast:
- ValidatingWebhookConfiguration Pay close attention to the In later versions of kustomize this order was changed, and is what produced the ordering issue described in #1797 The only issue with the above though is that users can't use it with earlier versions of Kustomize, since only Kustomize 5 can recognize the |
I propose that we just go to Kustomize 5 altogether. We'll have to user the So KF 1.7 will support the latest and greatest Kustomize 5. Then from our side we'll need to do the work to update the KF manifests to remove the warnings. Any takers? /help |
@kimwnasptd: Please ensure the request meets the requirements listed here. If this request no longer meets these requirements, the label can be removed In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
yes please kustomize 5.0 only. @akgraner anyone from the community outreach who could do that? |
Kustomize 5.0 seems like the right move, we have been on 3.2 for a while, I can put this issue on the Community Meeting Agenda. I appreciate the "help wanted" request. I suspect the qualified number of candidates will include those who have infra, which might mean that likely contributors would be at the distributions or perhaps large users. Still time is tight if we want this in 1.7. |
Thanks Kimonas for the detailed explanation of the issues and solutions. I support the upgrade of 5.0 for the manifest repo, however, am concerned with the timing to be included in the 1.7. This would mean that any distributions / end users would need to modify their existing kustomize version and test their custom manifest works as expected. With the issues that you have mentioned above + knowing that it is a breaking change, is there a strong reason to include it in the 1.7 when we're already in the middle of distribution testing and very close to the release date? Would it be better to include the update in the main branch but not cherry pick for the release and give manifest wg time to fix the warning and other issues that might come up and everyone else more time to make changes needed to prepare for the breaking changes introduced by kustomize5? |
Small update that Kustomize has released a We can update the the instructions to propose that version to be used instead. But considering that we are very close to release I'm not sure on whether we'd like to do this change now. Reminder that we have the https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv5.0.1 |
I can confirm with Kustomize 5.0.1, the following one liner works for KF 1.7.0 manifests with my on-prem k8s
|
I was trying to install 1.8.0-rc1 with Kustomize 5.1.1 and I still see the following warnings
Is the effort still on to remove the warning? Any helping hand needed? ✋🏻 |
@midhun1998 we just fixed the istio and knative warnings in the last weeks. You can create a similar PR for any other components directly maintained by the manifests WG. If they are synchronized as for example KFP, you might have to fix it at the source repository. Feel free to join our manifest WG meetings to discuss such things. |
We support Kustomize 5.2.1+ only now so lets close. If there are warnings lets create follow up PRs. |
Kustomize 5 has just been released, with the fix we wanted for ordering kubernetes-sigs/kustomize#3794 #1797.
Will use this issue to expose the current status of installing the Kubeflow manifests with Kustomize 5.
/cc @DomFleischmann @annajung @juliusvonkohout
The text was updated successfully, but these errors were encountered: