Release of optional attributes in TF 1.3 breaks modules using the experimental feature even if compatible

[BzSpi]

Hello,

Since Terraform 1.3, and the release of the optional attributes as a mainstream feature, using module that declare the experiments block breaks the Terraform stack because it exits with error.
IMHO, Terraform should output a warning and ignore the experimental block.
Currently, with Terraform 1.3, all module that used to declare the experimental block are unusable. Also, it is not possible to have module compatible with Terrarorm <=1.2 & 1.3 if optioanl attributes are used.
Also, I understand that the default value behavior for this feature has changed (in better) with 1.3 version but it breaks even if default values are not used.

Terraform Version

1.3.0-alpha20220622

Terraform Configuration Files

In a module:

terraform {
  required_version = ">= 1.1.0"
  experiments      = [module_variable_optional_attrs]
  required_providers {
   ...
  }
}

Expected Behavior

Terraform should behave fine and warns the user that the experimental feature declaration is ignored.

Actual Behavior

Terraform exits with follofing error

Error: Module uses experimental features
 
  on versions.tf line 3, in terraform:
   3:   experiments      = [module_variable_optional_attrs]

Experimental features are intended only for gathering early feedback on new
language designs, and so are available only in alpha releases of Terraform.

Steps to Reproduce

1. terraform init

Additional Context

We discovered the issue by having a CI that tests our module on all compatible Terraform versions (and also the future ones)

References

[jbardin]

Hi @BzSpi,

The experiments feature is intentionally designed such that users adding an experimental feature to a configuration must remove it at the conclusion of the experiment. This discourages the use of experiments in production situations where they may be unstable or even unsuitable for production, and ensures some acceptance of any changes encompassed by either the removal or conclusion of the feature.

Since experiments have now been limited only to alpha releases going forward, there really isn't any case where experimental features should end up in production configuration, limiting the possible impact further. Since the feature is working as intended, we can re-label this as an enhancement request, and see if there is any further feedback from other users.

Thanks!

[apparentlymart]

Indeed, experimental features are not a part of the Terraform language until they are stabilized, and so any module which uses optional attributes as a stable feature should declare that it requires Terraform 1.3 or later, since that is the first version of Terraform whose language truly includes that feature:

terraform {
  required_version = ">= 1.3.0"
}

That the experimental feature happened to be a subset of the final feature is a happy coincidence, but we do not encourage using experimental features in any non-experimental module and so would expect any non-experimental module to either choose between using this new feature and therefore requiring the new version of Terraform, or not using this feature at all and therefore remaining compatible with older versions. This is a typical trade-off to make for any new language feature, and the fact that there happened to be an experimental new version of the language also built into older versions does not change that trade-off.

The common use of the experimental feature in non-experimental modules is a regrettable outcome and so as @jbardin noted in passing we do not intend to ever ship experimental features in stable Terraform releases again. All future experiments will exist only in alpha releases, taking inspiration from Rust's approach where experiments are available only in nightly builds. (Alpha releases are Terraform's closest equivalent to "nightly builds" at this time, despite being published less often than nightly.)

[BzSpi]

Hi @jbardin & @apparentlymart,

Thanks for your replies, we will update our modules accordingly.

[apparentlymart]

Thanks @BzSpi!

In a separate PR I've tweaked how Terraform reports this situation to be explicit about the fact that the experiment has concluded, since we did also accidentally mask that helpful message with the generic one about experiments being available only in alpha releases. That should hopefully help others in a similar situation see that there is a path forward to the stable feature for those using Terraform v1.3.0 or later.

Thanks again!