[FLINK-33803] Set observedGeneration at end of reconciliation

[justin-chen]

What is the purpose of the change

https://issues.apache.org/jira/browse/FLINK-33803

This pull requests adds the observedGeneration field to the FlinkDeployment CRD status. This field is used to persist the generation that was last acted on by the operator. At the end of a successful reconciliation (reconcile, RecoveryFailureException, DeploymentFailedException ), the operator sets the observedGeneration field to the current generation number (from the CRD metadata).

Brief change log

• Operator sets observedGeneration field within FlinkDeploymentStatus after a completed reconciliation

Verifying this change

This change added tests and can be verified as follows:

• Test setting observedGeneration based on generation metadata taken from lastReconciledSpec

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changes to the CustomResourceDescriptors: yes
• Core observer or reconciler logic that is regularly executed: yes

Documentation

• Does this pull request introduce a new feature? No
• If yes, how is the feature documented? Docs

[gyfora]

Please see the comment about adopting the already existing observed generation logic from lastReconciledSpec and simply moving it into a top level field.

Also this change should be general to both FlinkDeployments and FlinkSessionJobs.

[gyfora]

The change looks good overall and thanks for updating the existing usage 👍

I added a small comment to where we should set the generation so that it's a bit more intuitive when reading the code but that's an easy change.

[gyfora]

Please generate the docs via 'mvn clean install -DskipTests -Pgenerate-docs'

[justin-chen]

Updated docs with mvn clean install -DskipTests -Pgenerate-docs

[mxm]

This looks good. Any further comments @gyfora?

[gyfora]

🚢