WIP: To provide a gist of moving the deployment to kustomize.

[rverma-jm]

Related issue

#107

Proposed changes

Start of migrating from helm to kustomize.

Checklist

• I have read the contributing guidelines
• I have read the security policy
• I confirm that this pull request does not address a security vulnerability. If this pull request addresses a security
  vulnerability, I confirm that I got green light (please contact security@ory.sh) from the maintainers to push the changes.
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation within the code base (if appropriate)

[aeneasr]

Thank you! However, I don't think this is taking a good direction, because:

1. A lot of people (including us) depend on helm
2. Helm v3 builds on top of feedback and criticism of helm v2 and is using kubectl internally (afaik)
3. Kustomize has no lifecycle management (to my knowledge) built in and is therefore missing critical things like rollback

Additionally, this PR appears to be an output of helm render with a fixed configuration. That's not how we want people to consume these templates, because they offer more than the one configuration chosen during export.

What would make sense (in my head) instead would be a guide of how to use these helm templates together with kustomize, so for example (again please be aware that I have limited knowledge of kustomize):

1. Pick a base config
2. Run helm render
3. Add additional kustomize config
4. Run kubectl

[rverma-jm]

Thank you! However, I don't think this is taking a good direction, because:

1. A lot of people (including us) depend on helm
2. Helm v3 builds on top of feedback and criticism of helm v2 and is using kubectl internally (afaik)

I am from helm background and used it for close to 4 years for managing k8 workloads. Things are changing now, the base of helm is a templating engine, i.e. for any structural changes you want to introduce, you are introducing changes in your helm chart. An init-container, you need to rollout new template etc...
Support for clientid-clientsercret as secrets, another chart release. Working with kustomize for last 1 year I see that the main difference is the ideology of taking vs giving control to the consumer of package. Its more of also complex vs simple, but I would let you to decide on that.

3. Kustomize has no lifecycle management (to my knowledge) built in and is therefore missing critical things like rollback
   Regarding rollback, I don't see helm as a good engine for rollback. First helm rollout is mostly based on functionality of kubernetes rollback, It basically a combination of applying an annotation as deployment.kubernetes.io/revision: "4" and running command kubectl rollout undo deployment/abc --to-revision=3. To maintain the rollback functionalities on other resources helm create a configmap/secrets in your kube-system namespace with every release.

Rollback isn't something which we can skip, but as per our observation there are tools which are better suited for this purpose. Spinnaker, flux etc... . The important part is more on the release with confidence, with simple strategies of canary/Red black release, we are more focus of rolling out the correct release.
That's my perception but yeah I am totally acknowledging your concerns here.

By the way kustomize support a versioning of base on top of git tag/commit/branch. So a rollback with kustomize is to provide the version with base and revert it manually in your own ovveride kustomization.
The issue/benefit is that, since it use configmap/secrets generator it will use your current config. To rollback on those resources, you have to do a git revert on them.

Additionally, this PR appears to be an output of helm render with a fixed configuration. That's not how we want people to consume these templates, because they offer more than the one configuration chosen during export.
I agreed to the part that its kind of an output of helm render. I combined some resources logically, and removed helm related annotations only. But the main purpose was to give you a gist of what it would look like, made it implicit in the title as well.

What would make sense (in my head) instead would be a guide of how to use these helm templates together with kustomize, so for example (again please be aware that I have limited knowledge of kustomize):

1. Pick a base config
2. Run helm render
3. Add additional kustomize config
4. Run kubectl

This can work, but it would be something like maintaining my own chart. For every new release you do, I have to kind of update from upstream and validate my fork.

[aeneasr]

for any structural changes you want to introduce, you are introducing changes in your helm chart. An init-container, you need to rollout new template etc...

This is exactly the point in using helm for a package like this! The scope of this chart is very limited - it uses and introduces only features supported by each project. We do not use helm as a way to orchestrate our istio deployment - we use helm to distribute our open source stack in the most consumable way. That is what helm is for!

If we need an init container, we will add one (as we did with auto-migrate). There wont be other init containers because they are not in the scope of this package!

This can work, but it would be something like maintaining my own chart. For every new release you do, I have to kind of update from upstream and validate my fork.

Not if this process is automated, which could also be part of the documentation around this process. I'd also be happy to offer a versioned, base kustomize template in this repo if its generation is fully automated and its scope is more than just rendered output of helm with its base config.

By the way kustomize support a versioning of base on top of git tag/commit/branch. So a rollback with kustomize is to provide the version with base and revert it manually in your own ovveride kustomization.

sounds to me like you need to have a git repo anyways :)

I think I can say that we wont switch to Kustomize because even though your arguments come from experience in dealing with helm / kustomize, I fear that helm is the closest "all-in-one" solution that most people use at the moment for packaged upstream depemdencies. Deprecating our helm charts is something where the community would run haywire as several (probably more) prod deployments depend on this.

Again, I think adding support for more toolchains is good, but in this case I want to keep the system as is and add another tool in the mix, preferably fully automated so we don't have a lot of maintenance work.

[CLAassistant]

All committers have signed the CLA.