Enhance README to take advantage of `terraform.tf` #46

jmcgeheeiv · 2020-05-18T02:43:09Z

what

Add terraform.tf to the usage instructions

why

The current usage does not account for terraform.tf, which is actually quite brilliant functionality

references

Personal experience using this over the last few months

jmcgeheeiv · 2020-05-18T18:40:58Z

I added a destroy procedure. Using this module this way works very smoothly. Very nice architecture!

schollii · 2020-05-21T11:53:28Z

I was just thinking the other day of doing something like this, I didn't know such functionality was already in place just not documented!

However I'm wondering since it is so far undocumented can we make the following changes before this goes "public":

rename those 3 input variables, they are unnecessarily verbose and clash with rest. Given the context of this module, it would be sufficient to call them just backend_file_path etc.
the path to the generated file should probably default to root module path, ie almost always the module that ultimately needs a backend. So the count 0/1 test should be whether the filename has been defined, not whether the path has been defined.
I think you have to empty bucket after state moved locally, before terraform destroy, otherwise you get an error.

I would be happy to submit some PR for same changes in other repos if that helps, I don't remember seeing any of them advertise this approach either.

osterman · 2020-05-21T21:34:27Z

This is an interesting approach!

schollii · 2020-05-28T19:33:31Z

pretty sure you will have to set force_destroy to true before doing the final apply since the bucket is versioned

jmcgeheeiv · 2020-05-29T01:36:48Z

Yes, I encountered that last time I did terraform destroy. I'm also reconsidering whether the backend file should be added to .gitignore, or whether it should be committed to Git. I want to try the workflow a few more times, but right now I'm away from this part of the project now. Still, the basic workflow is quite good. It is CloudPosse that deserves credit for that, @osterman.

osterman · 2020-05-29T01:50:52Z

Sorry we haven’t been able to review this yet. We’ve been slammed with new projects.

Has anyone else had a chance to test this out?

jmcgeheeiv · 2020-05-29T17:56:42Z

No rush, @osterman. In fact, I will change this to WIP because I still want to check into the issues I mentioned in my last comment.

schollii · 2020-05-30T12:30:17Z

@jmcgeheeiv even though the file is auto-generated I think it should be committed: if there is a change in the tf state module, or you make a change in input values, that you didn't realize would affect it, you will be able to see the change with git diff. Whereas without having committed it, you will only see what the plan tells you has changed, there will be no source to check for diff to help understand cause of change.

Also is there any disadvantage? If a timestamp or other attribute in the generated file changed at every apply, that would be a good reason to gitignore it, but that's not the case.

jmcgeheeiv · 2020-05-30T16:39:13Z

@schollii, in my most recent pass through the workflow I have made these changes:

Specify file name ./backend.tf. terraform.tf is not descriptive.
Commit ./backend.tf to git
Do something more forceful to remove the state bucket. @schollii, could you help by describing the exact command you recommend?

Shall I implement these changes in the documentation?

schollii · 2020-05-30T23:07:44Z

I gather this will work on v0.16.0? Pretty sure I had issues with 0.17.0 (#47). It seems like it's only the docs that are changing. I believe the workflow is this:

create backend-s3-bootstrap.tf
terraform init
terraform apply: creates backend.tf and local state
terraform init: moves local state to s3
terraform apply: creates other state if added, so this is optional if no additional resources created
...
remove backend.tf
terraform init: copy state from s3 to local
edit bucket to have force_destroy=true
terraform apply: updates only bucket; I'm not 100% this is necessary
terraform destroy

However I cannot verify that right now, but if you don't do that, then during terraform destroy you will get an error that the bucket has contents. After I deleted manually from AWS console a couple weeks ago, the destroy worked.

schollii · 2020-06-09T00:19:12Z

@jmcgeheeiv I have verified the workflow and documented at #44 (comment) since PR's are harder to find. HTH.

Gowiem · 2020-06-18T19:36:40Z

@jmcgeheeiv Definitely like the idea of getting this merged. I personally didn't know about this functionality and it sounds much preferred! Weighing in on a couple things:

backend.tf > terraform.tf 👍
Committing backend.tf to git -- I only see negatives in not doing this. I think this is the right call.

Happy to get this PR approved, merged, and cut for you as soon as you and @schollii suss out the final changes you want to make to the README. Thanks gents!

schollii

Looks good to me. FWIW I use this in my backend-aws.tf (the file that uses the tfstate_backend module):

  terraform_backend_config_file_name = "backend.tf"
  terraform_backend_config_file_path = var.tf_state_is_remote ? "." : ""
  force_destroy                      = ! var.tf_state_is_remote

then in my settings.auto.tfvars I just define tf_state_is_remote to be true or false. You just have to remember when you toggle between true and false to do a terraform init. Then the apply will actually remove or create the backend.tf, pretty cool. But I think this is too complicated to explain in the readme.

To be honest Hashicorp should do this automatically, ie it should be possible to tell the s3 backend block what bucket etc to create and it should create and use it, so we wouldn't have to go through this rigmarole.

in #44 (comment)

jmcgeheeiv · 2020-06-19T04:50:38Z

Modified to match @schollii's procedure in #44 without question--because he seems like such a trustworthy chap.

@schollii, @Gowiem, please scrutinize.

Gowiem

@jmcgeheeiv I think that looks great! 👍

Since this is a collab with @schollii, I'll let him weigh in too before merging.

Thanks gents!

@schollii

Today I had an opportunity to destroy a deployment, and I found an error in my destroy proceedure. I also added @schollii as a contributor.

@schollii

Today I had an opportunity to destroy a deployment, and I found an error in my destroy proceedure. I also added @schollii as a contributor.

jmcgeheeiv · 2020-06-19T21:08:34Z

Today I had an opportunity to destroy a deployment, and I found an error in my destroy procedure (wrong value for force_destroy).

I also added @schollii as a contributor. I hope you do not mind. If this bothers you, I can remove you from my fork and nobody will be the wiser.

…backend

osterman · 2020-06-23T17:39:18Z

/rebuild-readme

Gowiem · 2020-06-23T22:25:55Z

@jmcgeheeiv Sorry to do it to ya, but it looks like there are some merge conflicts. Mind fixing and we'll get this merged?

aknysh · 2020-06-24T03:46:59Z

README.md

+   definition file. Note that when `terraform_backend_config_file_path` is
+   empty (the default), no file is created.
+
+1. `terraform init`. This downloads Terraform modules and providers.


why all steps are numbered 1? :)
I see it was like that before, but maybe now is a good time to fix that.

aknysh · 2020-06-24T03:47:21Z

README.md

+      force_destroy                      = true
+    }
+    ```
+1. `terraform apply -target module.terraform_state_backend -auto-approve`.


all steps are numbered as 1

aknysh

thanks @jmcgeheeiv
please see comments

schollii · 2020-06-24T12:15:10Z

@aknysh markdown doesn't require sequence of numbers just first one must be '1'. See https://www.markdownguide.org/basic-syntax/#ordered-lists.

However the readme is understandable without rendering if we use sequence of numbers. For small lists it's manageable, so I'm in favour of fixing.

RothAndrew · 2020-06-24T12:17:31Z

IMO 1s are preferable, since you dont need to reorder them if you add an item in the middle of the list

aknysh · 2020-06-24T14:24:23Z

IMO 1s are preferable, since you dont need to reorder them if you add an item in the middle of the list

ah yes, the user will see it as an unordered list, it's just in the code itself we have 1.

RothAndrew · 2020-06-24T14:34:33Z

They will see it as an ordered list

aknysh · 2020-06-24T15:26:00Z

They will see it as an ordered list

Gowiem · 2020-06-24T16:01:28Z

@aknysh Did you mean to close this? I would think we would still want to get this merged. Mind if I reopen?

aknysh · 2020-06-24T16:02:42Z

@aknysh Did you mean to close this? I would think we would still want to get this merged. Mind if I reopen?

oh sorry, clicked the wrong button, reopening

jmcgeheeiv · 2020-06-24T20:05:14Z

The GitHub style guide specifies that 1. be used

Gowiem · 2020-06-24T21:44:38Z

/test all

Gowiem

I think this looks good and we've hashed it out plently. Let's ship it!

Gowiem · 2020-06-24T21:51:10Z

@jmcgeheeiv @schollii Thanks for the contributions and wading through the updates! This is just a docs update, but still... Released as 0.18.1.

Enhance README to take advantage of terraform.tf

db047de

jmcgeheeiv requested a review from a team as a code owner May 18, 2020 02:43

Add terraform destroy procedure.

e9e06ef

jmcgeheeiv mentioned this pull request May 18, 2020

terraform destroy needs explanation #44

Closed

osterman requested a review from aknysh May 21, 2020 05:30

osterman requested a review from 3h4x May 21, 2020 21:34

jmcgeheeiv changed the title ~~Enhance README to take advantage of terraform.tf~~ WIP: Enhance README to take advantage of terraform.tf May 29, 2020

Gowiem requested review from Gowiem and removed request for a team June 18, 2020 19:36

jmcgeheeiv and others added 2 commits June 18, 2020 20:54

Implement @schollii's changes

8cfdbec

Append to backend file path definition

5efe3f9

schollii previously approved these changes Jun 19, 2020

View reviewed changes

Document usage model proposed by @schollii

a70b7d3

in #44 (comment)

jmcgeheeiv dismissed schollii’s stale review via a70b7d3 June 19, 2020 04:47

Gowiem previously approved these changes Jun 19, 2020

View reviewed changes

jmcgeheeiv changed the title ~~WIP: Enhance README to take advantage of terraform.tf~~ Enhance README to take advantage of terraform.tf Jun 19, 2020

Correct the destroy proceedure

6489668

Today I had an opportunity to destroy a deployment, and I found an error in my destroy proceedure. I also added @schollii as a contributor.

jmcgeheeiv dismissed Gowiem’s stale review via 6489668 June 19, 2020 21:05

Correct the destroy proceedure

b5e5d3a

Today I had an opportunity to destroy a deployment, and I found an error in my destroy proceedure. I also added @schollii as a contributor.

Merge branch 'master' of github.com:jmcgeheeiv/terraform-aws-tfstate-…

ced77bf

…backend

Merge from master and fix conflicts

df5b26a

jmcgeheeiv requested a review from a team as a code owner June 24, 2020 03:22

aknysh reviewed Jun 24, 2020

View reviewed changes

aknysh closed this Jun 24, 2020

aknysh reopened this Jun 24, 2020

Gowiem approved these changes Jun 24, 2020

View reviewed changes

Gowiem merged commit a367b5b into cloudposse:master Jun 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhance README to take advantage of `terraform.tf` #46

Enhance README to take advantage of `terraform.tf` #46

jmcgeheeiv commented May 18, 2020

jmcgeheeiv commented May 18, 2020

schollii commented May 21, 2020

osterman commented May 21, 2020

schollii commented May 28, 2020

jmcgeheeiv commented May 29, 2020

osterman commented May 29, 2020

jmcgeheeiv commented May 29, 2020

schollii commented May 30, 2020 •

edited

jmcgeheeiv commented May 30, 2020

schollii commented May 30, 2020 •

edited

schollii commented Jun 9, 2020

Gowiem commented Jun 18, 2020

schollii left a comment •

edited

jmcgeheeiv commented Jun 19, 2020 •

edited

Gowiem left a comment

jmcgeheeiv commented Jun 19, 2020 •

edited

osterman commented Jun 23, 2020

Gowiem commented Jun 23, 2020

aknysh Jun 24, 2020

aknysh Jun 24, 2020

aknysh left a comment

schollii commented Jun 24, 2020 •

edited

RothAndrew commented Jun 24, 2020 •

edited

aknysh commented Jun 24, 2020

RothAndrew commented Jun 24, 2020 via email •

edited

aknysh commented Jun 24, 2020

Gowiem commented Jun 24, 2020

aknysh commented Jun 24, 2020

jmcgeheeiv commented Jun 24, 2020

Gowiem commented Jun 24, 2020

Gowiem left a comment

Gowiem commented Jun 24, 2020

Enhance README to take advantage of terraform.tf #46

Enhance README to take advantage of terraform.tf #46

Conversation

jmcgeheeiv commented May 18, 2020

what

why

references

jmcgeheeiv commented May 18, 2020

schollii commented May 21, 2020

osterman commented May 21, 2020

schollii commented May 28, 2020

jmcgeheeiv commented May 29, 2020

osterman commented May 29, 2020

jmcgeheeiv commented May 29, 2020

schollii commented May 30, 2020 • edited

jmcgeheeiv commented May 30, 2020

schollii commented May 30, 2020 • edited

schollii commented Jun 9, 2020

Gowiem commented Jun 18, 2020

schollii left a comment • edited

Choose a reason for hiding this comment

jmcgeheeiv commented Jun 19, 2020 • edited

Gowiem left a comment

Choose a reason for hiding this comment

jmcgeheeiv commented Jun 19, 2020 • edited

osterman commented Jun 23, 2020

Gowiem commented Jun 23, 2020

aknysh Jun 24, 2020

Choose a reason for hiding this comment

aknysh Jun 24, 2020

Choose a reason for hiding this comment

aknysh left a comment

Choose a reason for hiding this comment

schollii commented Jun 24, 2020 • edited

RothAndrew commented Jun 24, 2020 • edited

aknysh commented Jun 24, 2020

RothAndrew commented Jun 24, 2020 via email • edited

aknysh commented Jun 24, 2020

Gowiem commented Jun 24, 2020

aknysh commented Jun 24, 2020

jmcgeheeiv commented Jun 24, 2020

Gowiem commented Jun 24, 2020

Gowiem left a comment

Choose a reason for hiding this comment

Gowiem commented Jun 24, 2020

Enhance README to take advantage of `terraform.tf` #46

Enhance README to take advantage of `terraform.tf` #46

schollii commented May 30, 2020 •

edited

schollii commented May 30, 2020 •

edited

schollii left a comment •

edited

jmcgeheeiv commented Jun 19, 2020 •

edited

jmcgeheeiv commented Jun 19, 2020 •

edited

schollii commented Jun 24, 2020 •

edited

RothAndrew commented Jun 24, 2020 •

edited

RothAndrew commented Jun 24, 2020 via email •

edited