[19.03] terraform: 0.12.3 -> 0.12.6

[knl]

Motivation for this change

This brings up a lot of bug fixes, most notably for various crashes and terraform fmt which was a regression in the ways how code is formatted.

NOTE Still not ready for merging, as everything Go related fails to build on Darwin after 3590ff2. Working on fixing that.

Fails with:

$ nix-build . -A pkgs.terraform_0_12 --argstr system "x86_64-darwin"
...
building '/nix/store/8i66967wln7z6nal6i4npkl4ks901nhv-swift-corefoundation.drv'...
building '/nix/store/gkgyss9s4hkxm3z2aly7mr4cg3jwrxg0-swift-corefoundation-private.drv'...
unpacking sources
unpacking sources
unpacking source archive /nix/store/q19wpy2aylv4a6z5c6p52ih8pi2kcbpq-source
unpacking source archive /nix/store/q19wpy2aylv4a6z5c6p52ih8pi2kcbpq-source
source root is source/CoreFoundation
source root is source/CoreFoundation
patching sources
patching sources
substituteStream(): WARNING: pattern '#if defined(__GNU__) ||' doesn't match anything in file 'Base.subproj/CFAsmMacros.h'
substituteStream(): WARNING: pattern '#if defined(__GNU__) ||' doesn't match anything in file 'Base.subproj/CFAsmMacros.h'
configuring
configuring
/nix/store/3iqn6s3xabswcqirr7302c3kn4dd7q4k-bootstrap-stage1-stdenv-darwin/setup: ../configure: /usr/bin/env: bad interpreter: Operation not permitted
/nix/store/3iqn6s3xabswcqirr7302c3kn4dd7q4k-bootstrap-stage1-stdenv-darwin/setup: ../configure: /usr/bin/env: bad interpreter: Operation not permitted
builder for '/nix/store/gkgyss9s4hkxm3z2aly7mr4cg3jwrxg0-swift-corefoundation-private.drv' failed with exit code 126
builder for '/nix/store/8i66967wln7z6nal6i4npkl4ks901nhv-swift-corefoundation.drv' failed with exit code 126
cannot build derivation '/nix/store/8x0ymrcaiamka8r3gk9l0wlzyx3lycn8-bootstrap-stage2-stdenv-darwin.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/qww9pydqjlp47g32bl8hrpgvzphn24p1-bootstrap-stage2-stdenv-darwin.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/vr6il6rllr2xg3vws55rnlc0idcrd12h-bootstrap-stage3-stdenv-darwin.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/ycx6ynnf1j14vgkac62skb22pd3y0jws-bootstrap-stage3-stdenv-darwin.drv': 1 dependencies couldn't be built
...

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

Notify maintainers

cc @

[mmahut]

@knl what is the reason for this backport?

[knl]

Would love to have the most recent Terraform on 19.03 that we're using. However, it doesn't build on darwin (yet), since it has been broken since June 20. Working on it.

[mmahut]

@knl we usually really prefer updates to stable branch only in case of regressions and security features to ensure the release stays as stable as possible.

[knl]

@mmahut Good approach! Would it be possible to add that to the PR template for release branches, as one of the questions? That way, people trying to submit upgrades on stable branch can decide early if it's worth creating a PR or not?

This upgrade fixes quite a lot of issues, btw (from: https://github.com/hashicorp/terraform/blob/v0.12.6/CHANGELOG.md):

0.12.6 (July 31, 2019)

NOTES:

backend/s3: After this update, the AWS Go SDK will prefer credentials found via the AWS_PROFILE environment variable when both the AWS_PROFILE environment variable and the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables are statically defined. Previously the SDK would ignore the AWS_PROFILE environment variable, if static environment credentials were also specified. This is listed as a bug fix in the AWS Go SDK release notes. (#22253)

NEW FEATURES:

backend/oss: added support for assume role config (#22186)
config: Resources can now use a for_each meta-argument (#17179)

ENHANCEMENTS:

backend/s3: Add support for assuming role via web identity token via the AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN environment variables (#22253)
backend/s3: Support automatic region validation for me-south-1. For AWS operations to work in the new region, the region must be explicitly enabled as outlined in the AWS Documentation (#22253)
connection/ssh: Improve connection debug messages (#22097)

BUG FIXES:

backend/remote: remove misleading contents from error message (#22148)
backend/s3: Load credentials via the AWS_PROFILE environment variable (if available) when AWS_PROFILE is defined along with AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (#22253)
config: Improve conditionals to returns the correct type when dynamic values are present but unevaluated (#22137)
config: Fix panic when mistakingly using dynamic on an attribute (#22169)
cli: Fix crash with reset connection during init (#22146)
cli: show all deposed instances and prevent crash in show command (#22149)
configs/configupgrade: Fix crash with nil hilNode (#22181)
command/fmt: now formats correctly in presence of here-docs (#21434)
helper/schema: don't skip deprecation check during validation when attribute value is unknown (#22262)
plugin/sdk: allow MinItems > 1 when dynamic blocks (#22221)
plugin/sdk: fix reflect panics in helper/schema validation (#22236)

0.12.5 (July 18, 2019)

ENHANCEMENTS:

command/format: No longer show no-ops in terraform show, since nothing will change (#21907)
backend/s3: Support for assuming role using credential process from the shared AWS configuration file (support profile containing both credential_process and role_arn configurations) (#21908)
connection/ssh: Abort ssh connections when the server is no longer responding (#22037)
connection/ssh: Support ssh diffie-hellman-group-exchange-sha256 key exchange (#22037)

BUG FIXES:

backend/remote: fix conflict with normalized config dir and vcs root working directory (#22096)
backend/remote: be transparent about what filesystem prefix Terraform is uploading to the remote system, and why it's doing that (#22121)
configs: Ensure diagnostics are properly recorded from nested modules (#22098)
core: Prevent inconsistent final plan error when using dynamic in a set-type block (#22057)
lang/funcs: Allow null values in compact function (#22044)
lang/funcs: Pass through empty list in chunklist (#22119)

0.12.4 (July 11, 2019)

NEW FEATURES:

lang/funcs: new abspath function returns the absolute path to a given file (#21409)
backend/swift: support for user configured state object names in swift containers (#17465)

BUG FIXES:

core: Prevent crash when a resource has no current valid instance (#21979)
plugin/sdk: Prevent empty strings from being replaced with default values (#21806)
plugin/sdk: Ensure resource timeouts are not lost when there is an empty plan (#21814)
plugin/sdk: Don't add null elements to diagnostic paths when validating config (#21884)
lang/funcs: Add missing map of bool support for lookup (#21863)
config: Fix issue with downloading BitBucket modules from deprecated V1 API by updating go-getter dependency (#21948)
config: Fix conditionals to evaluate to the correct type when using null (#21957)

[basvandijk]

It's tempting to backport this because it includes lots of bug fixes and looking at the minor version upgrade appears to be a backwards compatible release.

0.12.6 does however include the following change in existing behavior:

backend/s3: After this update, the AWS Go SDK will prefer credentials found via the AWS_PROFILE environment variable when both the AWS_PROFILE environment variable and the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables are statically defined. Previously the SDK would ignore the AWS_PROFILE environment variable, if static environment credentials were also specified. This is listed as a bug fix in the AWS Go SDK release notes. (hashicorp/terraform#22253)

Note that this behavior is going to be reverted back in a new release (hashicorp/terraform#22363).

Terraform seems to cut a release every two/three weeks. So what we could do is wait for the release which reverts this behavior back and backport that. I expect this to happen soon since the last release is already two weeks ago. Then we have an upgrade including lots of bug fixes which doesn't break backward compatibility. I think that warrants a backport.

[mmahut]

@mmahut Good approach! Would it be possible to add that to the PR template for release branches, as one of the questions? That way, people trying to submit upgrades on stable branch can decide early if it's worth creating a PR or not?

Is it possible to define a template for one branch? I thought it is not.

[knl]

Let's wait for 0.12.8, because 0.12.7 is out and they still have that backwards incompatible change for backend/s3

[marsam]

Closing since 19.03 has reached end of support

End of support is planned for end of October 2019, handing over to 19.09.
https://nixos.org/nixos/manual/release-notes.html#sec-release-19.03-highlights