🛠️ Refactor metadata flattener

[arybolovlev]

Description

This PR has the following changes:

• Split the flattenMetadata function into 2 independent ones:
  • flattenMetadataFields -- to flatten all metadata fields, this one is used to flatten metadata of the datasources and template specs
  • flattenMetadata -- to cut some labels and annotations and then flatten metadata
• Update all data sources read functions to be independent from corresponding resources, in this case, we can use different metadata flattenters
• Update documentation to mention that data sources and templates will keep all annotations and labels, in spite of ignore_annotations and ignore_labels settings
• Add a new function removeGeneratedLabels to remove job controller server-generated labels.

Acceptance tests

• Have you added an acceptance test for the functionality being added?
• Have you run the acceptance tests on this branch?

Output from acceptance testing:

• AKS: https://github.com/hashicorp/terraform-provider-kubernetes/actions/runs/6968742155
• EKS: https://github.com/hashicorp/terraform-provider-kubernetes/actions/runs/6968743067
• GKE: https://github.com/hashicorp/terraform-provider-kubernetes/actions/runs/6968744026

Release Note

Release note for CHANGELOG:

We have updated the logic of data sources and now the provider will return all annotations and labels attached to the object, in spite of the `ignore_annotations` and `ignore_labels` provider settings. In addition to that, a list of ignored labels when they are attached to `kubernetes_job(_v1)` and `kubernetes_cron_job(_v1)` resources were extended with labels `batch.kubernetes.io/controller-uid` and `batch.kubernetes.io/job-name` since they aim to replace `controller-uid` and `job-name` in the future Kubernetes releases.

References

Fix: #2336
Fix: #2340

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[SarahFrench]

I had a quick look and obviously don't have full context, but one thing I looked for but couldn't find was unit tests for the flattenMetadataFields and flattenMetadata functions. I did see that the acceptance tests check that data sources have one or more annotations/labels present, though. Is there any behaviour of those functions that could benefit from more explicit testing?

[arybolovlev]

I had a quick look and obviously don't have full context, but one thing I looked for but couldn't find was unit tests for the flattenMetadataFields and flattenMetadata functions. I did see that the acceptance tests check that data sources have one or more annotations/labels present, though. Is there any behaviour of those functions that could benefit from more explicit testing?

I have added a unit test for the flattenMetadataFields function and working on the flattenMetadata one as we have discussed. Thanks!

[jastr945]

👍

[arybolovlev]

@SarahFrench I have added a unit test for the flattenMetadata function. Big thank you for the idea and inspiration!

[SarahFrench]

Thanks for adding those unit tests - they help me to understand the difference between the two functions. I'm glad that the two behaviours have been split apart!

Approving because the acceptance tests have passed and the unit tests look good and pass 🚀

[alexsomesan]

Looks great!

[alexsomesan]

These first two are new. Could this be a breaking change if any existing configuration is depending on them?

[arybolovlev]

I actually discovered these two labels when switched Job and CronJob from flattenMetadata to flattenMetadataFields. Once I did it, tests were failing due to batch.kubernetes.io labels.

I think that should not cause a breaking change. In the original flattenMetadata function, we only made a conditional exception on annotations when a prefix was passed and kept cutting labels. These two labels were cut anyway.