Resource Lifecycle Management v2

[ilya-lesikov]

Before proceeding

• I didn't find a similar issue

Problem

Helm have a few annotations that can change the resource behavior during deployment, i.e. helm.sh/hook-delete-policy and helm.sh/resource-policy, but that's not enough.

Solution (if you have one)

We need to provide mechanisms to easily configure werf behavior during resource deploys and deletions: enable resource recreation, configure conditions for resource cleanups/deletions, configure applying/updating release and service metadata.

Additional information

Refs: #82, #368

[ilya-lesikov]

@kuzaxak Actually the cornerstone of all these lifecycle annotations will be the helm.sh/hook-delete-policy: recreate-on-immutable-error annotation. The question is can it even be implemented.

The kubectl apply --server-side --force-conflicts --field-manager=helm -f myresource.yaml pretty much emulates what we do when we update the resource, and kubectl apply --dry-run=server --server-side --force-conflicts --field-manager=helm is what we do to get the projected "resource-manifest-to-be", which we then use to compute the diffs for release plan and for internals of release install.

If with these "kubectl apply" commands the "recreate-on-immutable-error" behavior can be implemented, then we are good to go for "recreate-on-any-release-changes" or anything else.

[kuzaxak]

If with these "kubectl apply" commands the "recreate-on-immutable-error" behavior can be implemented

Could you expand more on "the "recreate-on-immutable-error" behavior can be implemented", especially what do you mean by immutable-error?

As I understand you, it would mean a strategy for handling updates to resources that contain fields that cannot be changed after the resource is created. For example, we will recreate resources like Job, as we cannot update them.

Do I get you correctly?

[ilya-lesikov]

Correct, this is mostly for Jobs, and somewhat for controllers (they have some immutable fields too). Probably what we want to do is this:

1. dry-run apply the resource
2. if got "cannot update immutable fields" error, then mark the resource as to be recreated (delete + apply), like we do with resources with helm.sh/hook-delete-policy: before-hook-creation. Otherwise, if no immutable error, mark the resource as to be updated (apply)

Thinking on it, the requirements are:

1. dry-run apply must be able to return this immutable error, as non-dry-run apply does
2. dry-run apply shouldn't produce this error if we apply the resource as it is in the cluster, unchanged

If so, then we can implement it. And then "recreate-on-immutable-error" + "recreate-on-any-release-changes" will probably be the combination the user needs in 95%+ cases. "recreate-on-immutable-error" we can even enable by default for Jobs and maybe stateless controllers.

[ilya-lesikov]

Implemented: werf/werf#7139