Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use OpenTofu to have single .tfvars files for each environment, rather than nested modules #1247

Open
wants to merge 16 commits into
base: dev
Choose a base branch
from

Conversation

GaryGSC
Copy link
Member

@GaryGSC GaryGSC commented Jun 25, 2024

Intro

OpenTofu is a drop-in replacement for Terraform; just swap the terraform binary with tofu. There was a fork. Their feature sets are slowly diverging.

One of the latest features in OpenTofu is the ability to use variables in backend configurations. This allows us to significantly DRY up our infrastructure code. I don't expect Terraform to implement this feature any time soon: making it easier to use alternative backends conflicts with Hashicorp's business model.


Why bother switching?

Before After
image image

Usage

Install OpenTofu using your favorite version manager, navigate to the terraform/app directory, and then run:

  • aws sso login
  • tofu init -var-file=dev.tfvars
  • tofu plan -var-file=dev.tfvars

Discussion

This has fewer footguns than before. Now that backends are configured via .tfvars, if you run tofu init -var-file=dev.tfvars and then accidentally use production variables later (tofu plan -var-file=prd.tfvars), Tofu will warn you that your backend configuration has changed and refuse to proceed. 🙂

Now that we don't have .tfvars and .tfbackend files competing for autocompletion, typing commands is more ergonomic. tofu plan -var-file=d + Tab autocompletes to tofu plan -var-file=dev.tfvars. 🙂

When running tofu init with a different environment from what's configured on your local machine, Tofu still prompts users to choose between -migrate-state and -reconfigure. 😕

I haven't bothered replacing the word "Terraform" with "Tofu" in all the places, yet.

This module restructuring clobbers the existing state, but I don't really want to fill up the template with a bunch of moved blocks.


Supersedes #980

… local backend

This makes it harder to do the wrong thing. With this change, we avoid
a potential footgun when developers use `terraform init` instead of
`terraform init -backend-config=dev.s3.tfbackend`.

This isn't a functional change because we were already using encryption
on everything in our state buckets.
Docs needed to be updated either way.

I know we've done some bikeshedding on the directory name before. I
didn't previously care whether it was called terraform/, iac/,
terraform-iac/ or anything else. Nowadays, I have a tiny reason to
prefer the name terraform/: it plays nicely with automatic folder
icons. The other options don't.
Swap setup-terraform for setup-opentofu and start using the tofu
binary instead of terraform, which should be a drop-in replacement.

I didn't want to muck with all the directory names, etc., yet.
This is possible in the newest OpenTofu version, v1.8.0-alpha1, and in
fact it's the motivation for migrating.

While I was at it, I updated some provider versions that I apparently
downgraded by accident while rebasing this branch back onto `dev`.

Refs: https://opentofu.org/blog/help-us-test-opentofu-1-8-0-alpha1/#early-variablelocals-evaluation
.github/workflows/deploy.yml Outdated Show resolved Hide resolved
.github/workflows/ci-iac.yml Outdated Show resolved Hide resolved
@GaryGSC GaryGSC changed the base branch from prd to dev June 25, 2024 16:07
@GaryGSC

This comment was marked as resolved.

Copy link

github-actions bot commented Jun 25, 2024

Terraform Plan:

will create 60 resources:

  • aws_dynamodb_table - my_dynamo_table
  • aws_iam_policy - my_dynamo_policy
  • aws_iam_policy - my_s3_policy
  • aws_s3_bucket - my_s3_bucket
  • aws_s3_bucket - my_s3_bucket_logs
  • aws_s3_bucket_lifecycle_configuration - my_s3_bucket
  • aws_s3_bucket_lifecycle_configuration - my_s3_bucket_logs
  • aws_s3_bucket_logging - my_s3_bucket
  • aws_s3_bucket_public_access_block - default
  • aws_s3_bucket_public_access_block - default_logs
  • aws_s3_bucket_server_side_encryption_configuration - my_s3_bucket
  • aws_s3_bucket_server_side_encryption_configuration - my_s3_bucket_logs
  • aws_s3_bucket_versioning - my_s3_bucket
  • aws_alb - alb
  • aws_alb_listener - http_to_https
  • aws_alb_listener - https
  • aws_alb_listener - test_listener
  • aws_alb_target_group - blue
  • aws_alb_target_group - green
  • aws_appautoscaling_policy - down
  • aws_appautoscaling_policy - up
  • aws_appautoscaling_target - default
  • aws_cloudwatch_log_group - container_log_group
  • aws_cloudwatch_metric_alarm - down
  • aws_cloudwatch_metric_alarm - up
  • aws_codedeploy_app - app
  • aws_codedeploy_deployment_group - deploymentgroup
  • aws_ecs_cluster - new_cluster
  • aws_ecs_service - service
  • aws_ecs_task_definition - task_def
  • aws_iam_policy - secrets_access
  • aws_iam_role - task_execution_role
  • aws_iam_role - task_role
  • aws_iam_role_policy_attachment - secret_task_policy_attach
  • aws_iam_role_policy_attachment - secrets_policy_attach
  • aws_iam_role_policy_attachment - task_execution_policy_attach
  • aws_iam_role_policy_attachment - task_policy_attach
  • aws_iam_role_policy_attachment - task_policy_attach
  • aws_route53_record - a_record
  • aws_route53_record - aaaa_record
  • aws_security_group - alb-sg
  • aws_security_group - fargate_service_sg
  • local_file - appspec_json
  • aws_cloudwatch_log_group - lambda_logs
  • aws_iam_policy - s3_access
  • aws_iam_role - test_lambda
  • aws_iam_role_policy - test_lambda
  • aws_iam_role_policy_attachment - s3_access
  • aws_lambda_function - test_lambda
  • aws_s3_bucket - postman_bucket
  • aws_s3_bucket - postman_bucket_logs
  • aws_s3_bucket_lifecycle_configuration - postman_bucket
  • aws_s3_bucket_lifecycle_configuration - postman_bucket_logs
  • aws_s3_bucket_logging - postman_bucket
  • aws_s3_bucket_public_access_block - default
  • aws_s3_bucket_public_access_block - default_logs
  • aws_s3_bucket_server_side_encryption_configuration - postman_bucket
  • aws_s3_bucket_server_side_encryption_configuration - postman_bucket_logs
  • aws_s3_object - collections
  • aws_s3_object - environments

will delete 59 resources:

  • aws_dynamodb_table - my_dynamo_table
  • aws_iam_policy - my_dynamo_policy
  • aws_iam_policy - my_s3_policy
  • aws_s3_bucket - my_s3_bucket
  • aws_s3_bucket - my_s3_bucket_logs
  • aws_s3_bucket_lifecycle_configuration - my_s3_bucket
  • aws_s3_bucket_lifecycle_configuration - my_s3_bucket_logs
  • aws_s3_bucket_logging - my_s3_bucket
  • aws_s3_bucket_public_access_block - default
  • aws_s3_bucket_public_access_block - default_logs
  • aws_s3_bucket_server_side_encryption_configuration - my_s3_bucket
  • aws_s3_bucket_server_side_encryption_configuration - my_s3_bucket_logs
  • aws_s3_bucket_versioning - my_s3_bucket
  • aws_alb - alb
  • aws_alb_listener - http_to_https
  • aws_alb_listener - https
  • aws_alb_listener - test_listener
  • aws_alb_target_group - blue
  • aws_alb_target_group - green
  • aws_appautoscaling_policy - down
  • aws_appautoscaling_policy - up
  • aws_appautoscaling_target - default
  • aws_cloudwatch_log_group - container_log_group
  • aws_cloudwatch_metric_alarm - down
  • aws_cloudwatch_metric_alarm - up
  • aws_codedeploy_app - app
  • aws_codedeploy_deployment_group - deploymentgroup
  • aws_ecs_cluster - new_cluster
  • aws_ecs_service - service
  • aws_ecs_task_definition - task_def
  • aws_iam_policy - secrets_access
  • aws_iam_role - task_execution_role
  • aws_iam_role - task_role
  • aws_iam_role_policy_attachment - secret_task_policy_attach
  • aws_iam_role_policy_attachment - secrets_policy_attach
  • aws_iam_role_policy_attachment - task_execution_policy_attach
  • aws_iam_role_policy_attachment - task_policy_attach
  • aws_iam_role_policy_attachment - task_policy_attach
  • aws_route53_record - a_record
  • aws_route53_record - aaaa_record
  • aws_security_group - alb-sg
  • aws_security_group - fargate_service_sg
  • aws_cloudwatch_log_group - lambda_logs
  • aws_iam_policy - s3_access
  • aws_iam_role - test_lambda
  • aws_iam_role_policy - test_lambda
  • aws_iam_role_policy_attachment - s3_access
  • aws_lambda_function - test_lambda
  • aws_s3_bucket - postman_bucket
  • aws_s3_bucket - postman_bucket_logs
  • aws_s3_bucket_lifecycle_configuration - postman_bucket
  • aws_s3_bucket_lifecycle_configuration - postman_bucket_logs
  • aws_s3_bucket_logging - postman_bucket
  • aws_s3_bucket_public_access_block - default
  • aws_s3_bucket_public_access_block - default_logs
  • aws_s3_bucket_server_side_encryption_configuration - postman_bucket
  • aws_s3_bucket_server_side_encryption_configuration - postman_bucket_logs
  • aws_s3_object - collections
  • aws_s3_object - environments

see details

@GaryGSC

This comment was marked as resolved.

@GaryGSC

This comment was marked as resolved.

.github/workflows/ci-iac.yml Outdated Show resolved Hide resolved
@GaryGSC GaryGSC requested review from a team June 25, 2024 20:42
@GaryGSC
Copy link
Member Author

GaryGSC commented Jul 8, 2024

We're waiting on a mainline release of OpenTofu 1.8.0 before merging this, BTW.

dependabot bot and others added 2 commits July 29, 2024 23:23
Bumps [terraform-aws-modules/iam/aws](https://github.com/terraform-aws-modules/terraform-aws-iam) from 5.41.0 to 5.42.0.
- [Release notes](https://github.com/terraform-aws-modules/terraform-aws-iam/releases)
- [Changelog](https://github.com/terraform-aws-modules/terraform-aws-iam/blob/master/CHANGELOG.md)
- [Commits](terraform-aws-modules/terraform-aws-iam@v5.41.0...v5.42.0)

---
updated-dependencies:
- dependency-name: terraform-aws-modules/iam/aws
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…ac/dev/setup/dev/terraform-aws-modules/iam/aws-5.42.0' into opentofu

# Conflicts:
#	terraform-iac/dev/setup/.terraform.lock.hcl
#	terraform-iac/modules/setup/setup.tf
@GaryGSC
Copy link
Member Author

GaryGSC commented Jul 30, 2024

It's been released! 🎉

jvisker pushed a commit to byu-oit/tfvm that referenced this pull request Aug 19, 2024
Add support for OpenTofu with the useOpenTofu flag in the tfvm config.
This is because OIT will be adopting OpenTofu over Terraform
byu-oit/hw-fargate-api#1247 and our internal
tool seems to be better than alternatives, in my opinion.

---------

Co-authored-by: tab518 <tylerablackham@gmail.com>
# Conflicts:
#	terraform-iac/modules/setup/setup.tf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants