Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

terragrunt: removing terraform dependency #103670

Merged
merged 1 commit into from Nov 18, 2020
Merged

Conversation

@ivankovnatsky
Copy link
Contributor

@ivankovnatsky ivankovnatsky commented Nov 13, 2020

Since terragrunt would trigger downloading all the plugins and basing on terraform 0.12 branch whereas people could use it against 0.11 or 0.13 terraform.

fixes #103631

Motivation for this change

it is very frustrating when you download a statically linked binary and a vast amount of software is downloaded with it. i understand that from a user perspective it might be convenient and make sense to have terraform (not sure though about ALL the providers) along side it.

but terragrunt itself is very niche software, and people would want to have different terraform versions with it.

PS: i'm new to NixOS, and to that packaging as well. i could miss some things, please correct me where applicable, thanks.

i've used to use tgenv/tfenv myself, but downloading providers every time in .terraform directory within your code is a nightmare. NixOS packaging did it very cool i must say with linking provider binaries to .terraform/plugins

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
sevenfourk
Since terragrunt would trigger downloading all the plugins and basing on terraform 0.12 branch whereas people could use it against 0.11 or 0.13 terraform.
@ivankovnatsky
Copy link
Contributor Author

@ivankovnatsky ivankovnatsky commented Nov 13, 2020

nixpkgs-review pr 103670
[nix-shell:~/Sources/*/s3]$ terragrunt init
...
Initializing the backend...

Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/aws versions matching "~> 2.25"...
- Installing hashicorp/aws v2.65.0...
- Installed hashicorp/aws v2.65.0 (unauthenticated)

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

[nix-shell:~/Sources/*/s3]$ cat terragrunt.hcl
include {
  path = find_in_parent_folders()
}

dependencies {
  paths = ["../iam"]
}
@06kellyjac
Copy link
Contributor

@06kellyjac 06kellyjac commented Nov 18, 2020

I agree with your frustrations, I patch my terragrunt to use normal terraform and I've seen several posts on here and the Discourse.

For a while I've been trying to set up a terragrunt.full option like the terraform.full but now that I think about it this is much easier. It makes it a bit less nix-ie but if people want maximum nix we can either put in a terragrunt.full or let people patch this wrapper back in.
Thoughts @zimbatm


As a note for future packages or changes you should be targeting master rather than nixos-unstable.

Although since this is a breaking change for some users this will need to be done carefully and might need to be be merged into staging-next instead 🤔

@ivankovnatsky ivankovnatsky changed the base branch from nixos-unstable to staging-next Nov 18, 2020
@ivankovnatsky
Copy link
Contributor Author

@ivankovnatsky ivankovnatsky commented Nov 18, 2020

@06kellyjac thanks for your input. i've changed target branch to staging-next.

ideally what we would need: to have terraform packaged (whatever version along side terragrunt), as terragrunt obviously useless without terraform, but me being still novice in nixos pushed this change to at least make things simpler, before making them more complex.

and we would probably need to control providers download only in a terraform wrapper.

@zimbatm
Copy link
Member

@zimbatm zimbatm commented Nov 18, 2020

Makes sense. Terraform can be sourced from the path and it gives more flexibility to the user.
Since each user has their own set of providers that they want to use, overriding terragrunt would also work but cause unnecessary rebuild.

@zimbatm zimbatm merged commit 45ed0b7 into NixOS:staging-next Nov 18, 2020
19 checks passed
19 checks passed
tests
Details
action
Details
Evaluation Performance Report Evaluator Performance Report
Details
Wait for ofborg
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="dc8de78"; rev="dc8de78420e1b7bbeb5b47d45d2508c27de88d6d"; } ./pkgs/t
Details
grahamcofborg-eval-lib-tests nix-build --arg pkgs import ./. {} ./lib/tests/release.nix
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="dc8de78"; rev="dc8de78420e1b7bbeb5b47d45d2508c27de88d6d"; } ./nixos/
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="dc8de78"; rev="dc8de78420e1b7bbeb5b47d45d2508c27de88d6d"; } ./nixos/
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="dc8de78"; rev="dc8de78420e1b7bbeb5b47d45d2508c27de88d6d"; } ./nixos/
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="dc8de78"; rev="dc8de78420e1b7bbeb5b47d45d2508c27de88d6d"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="dc8de78"; rev="dc8de78420e1b7bbeb5b47d45d2508c27de88d6d"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="dc8de78"; rev="dc8de78420e1b7bbeb5b47d45d2508c27de88d6d"; } ./pkgs/t
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
terragrunt, terragrunt.passthru.tests on aarch64-linux Success
Details
terragrunt, terragrunt.passthru.tests on x86_64-linux Success
Details
@ivankovnatsky
Copy link
Contributor Author

@ivankovnatsky ivankovnatsky commented Nov 18, 2020

@zimbatm thanks.

kalbasit added a commit to kalbasit/nixpkgs that referenced this pull request Dec 10, 2020
Since terragrunt would trigger downloading all the plugins and basing on terraform 0.12 branch whereas people could use it against 0.11 or 0.13 terraform.

(cherry picked from commit 45ed0b7)
zimbatm pushed a commit that referenced this pull request Dec 10, 2020
Since terragrunt would trigger downloading all the plugins and basing on terraform 0.12 branch whereas people could use it against 0.11 or 0.13 terraform.

(cherry picked from commit 45ed0b7)

Co-authored-by: sevenfourk <sevenfourk@protonmail.ch>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants