Skip to content

Dependent Resource Event Filtration with Previous version as Annotation #2249

@csviri

Description

@csviri
Collaborator

When a KubernetesDependentResource makes an update, it adds an annotation with the actual version of the resource:

} else if (usePreviousAnnotation(context)) { // set a new one
eventSource().orElseThrow().addPreviousAnnotation(
Optional.ofNullable(actualResource).map(r -> r.getMetadata().getResourceVersion())
.orElse(null),
target);
}

This is then used to if an event comes it could be skipped (won't trigger the reconciliation) since it is a result of the controller. This halves the reconciliation execution with the same result.
The actual setup that uses matchers works well. No functional issue.

The matchers (SSA based matcher by default) are however an optimization, that other controller might or might do, also there are some special cases that are hard to handle. So in case the matcher simply returns false, and the reconciliation is triggered, while the controller of the dependent resource (not our controller) made changes, the update done by the dependent resource (with the changed resources version in the annotation) will trigger an update (therefore an update event in the informer). While this will be filtered out, it would not create an event (thus there would not be a change (also no resourceVersion change) if the annotation with resource version is not changed.

This causes a real problem just in the case that the controller of the resource is reacting on every change, a typical (and not found more yet) is the Deployment, where there is somehow an exceptional functionality if the annotations are changed the generation is changed in the metadata. What triggers a reconciliation of the Deployment, that results in a change of observedGeneration. Which again triggers the reconciliation on our side. (If the matchers are in place this not causes an infinite loop).

So there are multiple things here:

  • in some cases, we might produce additional changes to the resource using the annotation
  • if the matchers are turned off in some very rare cases this might cause a loop in the reconciliation (in combination with the resources controller, like Deployment)

Note that there is a feature flag there to not use this annotation.

Should we do anything regarding this? Well would be nicer to have a simple algorithm without the annotation. The previous algorithm was more complex but did not use the annotation. So we might revisit this and see if we can do better.

Activity

shawkins

shawkins commented on Feb 26, 2024

@shawkins
Collaborator

if the matchers are turned off in some very rare cases this might cause a loop in the reconciliation (in combination with the resources controller, like Deployment)

Would you just consider adding java docs to the ResourceUpdaterMatcher.matches and KubernetesDependentResource.match methods warning about this scenario?

The only simple alternative I can think of would be make the previous value based upon a hash rather than the resourceVersion. That will make it stable (as long as the serialized form of the resource is stable), but will be more intensive to compute / compare.

added this to the 5.0 milestone on Mar 19, 2024
added
needs-discussionIssue needs to be discussed more before working on it
on Mar 19, 2024
removed this from the 5.0 milestone on Mar 27, 2024
added this to the 5.2 milestone on Nov 1, 2024
added 3 commits that reference this issue on Apr 1, 2025
599002a
2264e7d
bf95051
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Participants

      @shawkins@csviri

      Issue actions

        Dependent Resource Event Filtration with Previous version as Annotation · Issue #2249 · operator-framework/java-operator-sdk