fix: Remove optional variable attribute experiment from project

[bryantbiggs]

What does this PR do?

• Remove optional variable attribute experiment from project. This removes the version pin of < 1.3.0 from the project itself, but there is additional work still before users can use 1.3.0+ on this project. Namely, the two external addons needed to be updated
  • Vault
  • Ondat

Motivation

• Relates to [Bug]: There is no function named "defaults". in terraform 1.3 alpha #659
• Relates to Remove warnings from Terraform output #754
• Relates to Update to be compatible with terraform 1.3.0+ #988

More

• Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)
• Yes, I have added a new example under examples to support my PR
• Yes, I have created another PR for add-ons under add-ons repo (if applicable)
• Yes, I have updated the docs for this feature
• Yes, I ran pre-commit run -a with this PR

Note: Not all the PRs require a new example and/or doc page. In general:

• Use an existing example when possible to demonstrate a new addons usage
• A new docs page under docs/add-ons/* is required for new a new addon

For Moderators

• E2E Test successfully complete before merge?

Additional Notes

[askulkarni2]

LGTM!

[Hokwang]

@bryantbiggs tetrate_istio has also problem.

│ Error: Experiment has concluded
│
│   on .terraform/modules/eks_kubernetes_addons.tetrate_istio.cni/modules/kubernetes-addons/helm-addon/locals.tf line 2, in terraform:
│    2:   experiments = [module_variable_optional_attrs]
│
│ Experiment "module_variable_optional_attrs" is no longer available. The final feature corresponding to this experiment differs from the experimental
│ form and is available in the Terraform language from Terraform v1.3.0 onwards.
╵
╷
│ Error: Experiment has concluded
│
│   on .terraform/modules/eks_kubernetes_addons.tetrate_istio.gateway/modules/kubernetes-addons/helm-addon/locals.tf line 2, in terraform:
│    2:   experiments = [module_variable_optional_attrs]
│
│ Experiment "module_variable_optional_attrs" is no longer available. The final feature corresponding to this experiment differs from the experimental
│ form and is available in the Terraform language from Terraform v1.3.0 onwards.
╵
╷
│ Error: Experiment has concluded
│
│   on .terraform/modules/eks_kubernetes_addons.tetrate_istio.istiod/modules/kubernetes-addons/helm-addon/locals.tf line 2, in terraform:
│    2:   experiments = [module_variable_optional_attrs]
│
│ Experiment "module_variable_optional_attrs" is no longer available. The final feature corresponding to this experiment differs from the experimental
│ form and is available in the Terraform language from Terraform v1.3.0 onwards.

[Hokwang]

I think you can remove ondat, tetrate_istio, valut modules in main.tf. It is easy and fast way.
As you know, addon module calls other addons which is managed other repo is not good way, if we can use helm chart in this repo.

And also user can use it in their code directly.