feat: Added support for tfupdate to update version constraints in Terraform configurations

[jrottenberg]

Put an x into the box if that apply:

• This PR introduces breaking change.
• This PR fixes a bug.
• This PR adds new functionality.
• This PR enhances existing functionality.

Description of your changes

Fixes #328

How can we test changes

Working on an existing module, for example : https://github.com/cloudposse/terraform-aws-sso.git

Prepare the pre-commit config :

repos: # pre-commit autoupdate
  - repo: https://github.com/jrottenberg/pre-commit-terraform
    rev: tfupdate
    hooks:
      - id: tfupdate
        name: tfupdate terraform
      - id: tfupdate
        name: tfupdate provider aws
        args:
          - provider
          - aws   

❯ pre-commit run -a
[...]
tfupdate terraform.......................................................Failed
- hook id: tfupdate
- files were modified by this hook
tfupdate provider aws....................................................Failed
- hook id: tfupdate
- files were modified by this hook

And for the diff :

diff --git a/examples/complete/versions.tf b/examples/complete/versions.tf
index 42f8195..764880f 100644
--- a/examples/complete/versions.tf
+++ b/examples/complete/versions.tf
@@ -1,5 +1,5 @@
 terraform {
-  required_version = ">= 0.13.0"
+  required_version = "1.1.5"
 
   required_providers {
     local = "~> 1.2"
diff --git a/modules/account-assignments/versions.tf b/modules/account-assignments/versions.tf
index c9a3bb8..f470932 100644
--- a/modules/account-assignments/versions.tf
+++ b/modules/account-assignments/versions.tf
@@ -1,10 +1,10 @@
 terraform {
-  required_version = ">= 0.13.0"
+  required_version = "1.1.5"
 
   required_providers {
     aws = {
       source  = "hashicorp/aws"
-      version = ">= 3.26.0"
+      version = "4.0.0"
     }
   }

teraform version and the provider are pinned.

On subsequent run :

❯ pre-commit run -a
[...]
tfupdate terraform.......................................................Passed
tfupdate provider aws....................................................Passed

[jrottenberg]

Beside the new feature I had to path the __init__.py file since the warning for terraform_docs_replace was leaking on any python hook.

tfupdate uses subcommands no options it was easier to write the hook in python.

[jrottenberg]

@yermulnik @MaxymVlasov , ok updated the PR as recommended, let me know if it needs anything else.

[MaxymVlasov]

Let's use common functions to parse args and then use function per_dir_hook_unique_part and function run_hook_on_whole_repo as replace for function tfupdate_

First should work only for dirs with changes, second - recursive for whole repo

Check https://github.com/antonbabenko/pre-commit-terraform/blob/master/hooks/terrascan.sh as an example

[jrottenberg]

Let's use common functions to parse args and then use function per_dir_hook_unique_part and function run_hook_on_whole_repo as replace for function tfupdate_

First should work only for dirs with changes, second - recursive for whole repo

Check https://github.com/antonbabenko/pre-commit-terraform/blob/master/hooks/terrascan.sh as an example

tfupdate behaves differently than terrascan, it really runs only against the module as a whole, not individual files. It makes sure the version of terraform or the various providers is pinned to the latest when tfupdate ran.

The version lookup is a bit expansive and I'd go against running it multiple times, as it will give the same value, but could be considered as abuse from github (it uses the api to extract the latest published version).

I'll make the change to use common::parse_cmdline

[jrottenberg]

Ok it is consistent with other hooks in term of passing options but seems a bit verbose args of args, when the command itself (tfupdate) takes only a subcommands and parameters. Either way, let me know if you prefer this version, I'll squash and push.

[MaxymVlasov]

1. Please add installation instructions to README (for all OSes and param in Docker) and Dockerfile.

tfupdate behaves differently than terrascan, it really runs only against the module as a whole, not individual files. It makes sure the version of terraform or the various providers is pinned to the latest when tfupdate ran.

The version lookup is a bit expansive and I'd go against running it multiple times, as it will give the same value, but could be considered as abuse from github (it uses the api to extract the latest published version).

It runs on per-dir basic, that same as for terrascan and many other hooks.

pre-commit-terraform/hooks/_common.sh

Lines 121 to 130 in 458fb28

	# consume modified files passed from pre-commit so that
	# hook runs against only those relevant directories
	local index=0
	for file_with_path in "${files[@]}"; do
	file_with_path="${file_with_path// /__REPLACED__SPACE__}"
	dir_paths[index]=$(dirname "$file_with_path")
	((index += 1))
	done

pre-commit-terraform/hooks/_common.sh

Lines 138 to 143 in 458fb28

	# run hook for each path
	for dir_path in $(echo "${dir_paths[*]}" | tr ' ' '\n' | sort -u); do
	dir_path="${dir_path//__REPLACED__SPACE__/ }"
	pushd "$dir_path" > /dev/null || continue
	per_dir_hook_unique_part "$args" "$dir_path"

So, please use function per_dir_hook_unique_part and function run_hook_on_whole_repo

Just do not use --recurcive in function per_dir_hook_unique_part :)

[jrottenberg]

ok @MaxymVlasov I'm going with the flow, let me know if I'm missing anything else.

Thanks for the feedback.

[MaxymVlasov]

LGTM. Just need to check that all works fine. I'll do that little bit later

[jrottenberg]

Argh, I'll fix the pre-commit.

[yermulnik]

I might be doing something different, but this doesn't work for me on Ubuntu as it returns more than just browser_download_url.
What works for me is e.g. grep -o -E "https://.+?_.+_linux_amd64.tar.gz"

[jrottenberg]

Thank you very much @yermulnik , I was missing the targe on the "tar x".

Let me know what you think @MaxymVlasov

[jrottenberg]

@MaxymVlasov very strange, the pre-commit are passing locally, hadolint included.

[github-actions]

This PR has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this PR will be closed in 10 days

[jrottenberg]

rebased and ready for review

[MaxymVlasov]

Revert some changes

[MaxymVlasov]

Great job!
I fixed all things where I had some questions, so now LGTM

Let's wait to merge #362, rebase here and if all pipelines will be happy and @yermulnik will approve PR, we can go to merge it

[yermulnik]

LGTM

[antonbabenko]

This PR is included in version 1.66.0 🎉

[jrottenberg]

Thank you !