terraspace init complains about backends (http)

[gwvandesteeg]

Checklist

• Upgrade Terraspace: Are you using the latest version of Terraspace? This allows Terraspace to fix issues fast. There's an Upgrading Guide: https://terraspace.cloud/docs/misc/upgrading/
• Reproducibility: Are you reporting a bug others will be able to reproduce and not asking a question. If you're unsure or want to ask a question, do so on https://community.boltops.com
• Code sample: Have you put together a code sample to reproduce the issue and make it available? Code samples help speed up fixes dramatically. If it's an easily reproducible issue, then code samples are not needed. If you're unsure, please include a code sample.

My Environment

Software	Version
Operating System	Ubuntu 20.04 LTS
Terraform	1.0.0
Terraspace	0.6.13
Ruby

---

Expected Behaviour

terraspace all init works correctly without reporting issues

Current Behavior

running

terraspace clean all -y
terraspace all init -y

when using the http backend to store state (Gitlab cloud).
Terrastate complains about the backed initialization occuring. This seems to happen generally after a new version of terrspace has come out.

Step-by-step reproduction instructions

$ terraspace all init -y
Building one stack to build all stacks
Building .terraspace-cache/ap-southeast-2/dev/stacks/kubernetes_vpc
Downloading tfstate files for dependencies defined in tfvars...
╷
│ Error: Backend initialization required, please run "terraform init"
│ 
│ Reason: Initial configuration of the requested backend "http"
│ 
│ The "backend" is the interface that Terraform uses to store state,
│ perform operations, etc. If this message is showing up, it means that the
│ Terraform configuration you're using is using a custom configuration for
│ the Terraform backend.
│ 
│ Changes to backend configurations require reinitialization. This allows
│ Terraform to set up the new configuration, copy existing state, etc. Please run
│ "terraform init" with either the "-reconfigure" or "-migrate-state" flags to
│ use the current configuration.
│ 
│ If the change reason above is incorrect, please verify your configuration
│ hasn't changed and try again. At this point, no changes to your existing
│ configuration or state have been made.

<.. snip for brevity .. happens to all stacks involved repeatedly ..>

Code Sample

None available that I can share

Solution Suggestion

not sure where to start

[gwvandesteeg]

None of the environment variables have changed
None of the backends have changed in the project, its still the same
The only change is the version of terraspace, which was update to the new version (0.6.13) from (0.6.11)
The Gemfile was updated to reference the new version of terraspace
The Gemfile.lock was updated by removing and letting ruby re-generate it

Terraspace is installed using the APT package

Using 0.6.11-20210726100211 - works fine
Using 0.6.12-20210809100412 - breaks
Using 0.6.13-20210811100246 - breaks

Only difference is the terraspace version.

The generated *.auto.tfvars, backend.tf, provider.tf.json in the .terraspace-cache path for each stack are perfectly identical (sha256sums match, no diffs) between versions.

config/args/terraform.rb

command("apply",
  args: ["-lock-timeout=22m", "-no-color"],
  env: {TF_IN_AUTOMATION: "1"},
)

command("plan",
  args: ["-lock-timeout=22m", "-no-color"],
  env: {TF_IN_AUTOMATION: "1"},
)

command("destroy",
  args: ["-lock-timeout=22m", "-no-color"],
  env: {TF_IN_AUTOMATION: "1"},
)

config/terraform/backend.tf

terraform {
  backend "http" {
    address        = "<%= http_address %>"
    lock_address   = "<%= http_lock_address %>"
    lock_method    = "<%= http_lock_method %>"
    unlock_address = "<%= http_unlock_address %>"
    unlock_method  = "<%= http_unlock_method %>"
    username       = "<%= http_username %>"
    password       = "<%= http_password %>"
    retry_wait_min = "<%= http_retry_wait_min %>"
  }
}

Those are simple helper functions pulling information from the environment

[gwvandesteeg]

The issue here is because something is calling a terraform state pull prior to the actual init being called during the init call.

$ terraspace all init --yes --exit-on-fail
Building one stack to build all stacks
Building .terraspace-cache/ap-southeast-2/test/stacks/defaults
Downloading tfstate files for dependencies defined in tfvars...
╷
│ Error: Backend initialization required, please run "terraform init"
│ 
│ Reason: Initial configuration of the requested backend "http"
│ 
│ The "backend" is the interface that Terraform uses to store state,
│ perform operations, etc. If this message is showing up, it means that the
│ Terraform configuration you're using is using a custom configuration for
│ the Terraform backend.
│ 
│ Changes to backend configurations require reinitialization. This allows
│ Terraform to set up the new configuration, copy existing state, etc. Please
│ run
│ "terraform init" with either the "-reconfigure" or "-migrate-state" flags
│ to
│ use the current configuration.
│ 
│ If the change reason above is incorrect, please verify your configuration
│ hasn't changed and try again. At this point, no changes to your existing
│ configuration or state have been made.
|
Error running: cd /builds/<Redacted>/<Redacted>/iac/<Redacted>-infrastructure/.terraspace-cache/ap-southeast-2/test/stacks/kubernetes_vpc && terraform state pull > /tmp/terraspace/remote_state/stacks/kubernetes_vpc/state.json

Digging further it looks like the remote_state fetcher is trying to pull the remote state without running init first.

[gwvandesteeg]

Upgraded to 0.6.17, problem still present.

[gwvandesteeg]

Pretty sure this is related to the following section (but not familiar enough with Ruby to be 100%)

terraspace/lib/terraspace/terraform/remote_state/fetcher.rb

Lines 14 to 18 in e816ed5

	def run
	validate! # check child stack exists
	pull
	load
	end

This is causing things to be validated and pulled prior to running init on it.

[Gladskih]

I faced similar problem updating boltops/terraspace:alpine from 3564b8f30ad6 to 6ec6343781fe
I use up not init

[ghost]

We're having similar issues. We're using templated dependencies between stacks (i.e. <%= output'...') %> in tfvars files). When you try to run any stack-specific command after clearing the Terraspace cache directory, you get errors that look like this:

phil@philhowe-infogrid:~/git/tf-infra$ TS_ENV=live AWS_REGION=eu-west-1 bundle exec terraspace build security
Building .terraspace-cache/live/stacks/security
Downloading tfstate files for dependencies defined in tfvars...
╷
│ Error: Backend initialization required, please run "terraform init"
│ 
│ Reason: Initial configuration of the requested backend "s3"
│ 
│ The "backend" is the interface that Terraform uses to store state,
│ perform operations, etc. If this message is showing up, it means that the
│ Terraform configuration you're using is using a custom configuration for
│ the Terraform backend.
│ 
│ Changes to backend configurations require reinitialization. This allows
│ Terraform to set up the new configuration, copy existing state, etc. Please run
│ "terraform init" with either the "-reconfigure" or "-migrate-state" flags to
│ use the current configuration.
│ 
│ If the change reason above is incorrect, please verify your configuration
│ hasn't changed and try again. At this point, no changes to your existing
│ configuration or state have been made.
╵

Error running: cd /home/phil/git/tf-infra/.terraspace-cache/live/stacks/logs && terraform state pull > /tmp/terraspace/remote_state/stacks/logs/state.json
Please fix the error before continuing
╷
│ Error: Backend initialization required, please run "terraform init"
│ 
│ Reason: Initial configuration of the requested backend "s3"
│ 
│ The "backend" is the interface that Terraform uses to store state,
│ perform operations, etc. If this message is showing up, it means that the
│ Terraform configuration you're using is using a custom configuration for
│ the Terraform backend.
│ 
│ Changes to backend configurations require reinitialization. This allows
│ Terraform to set up the new configuration, copy existing state, etc. Please run
│ "terraform init" with either the "-reconfigure" or "-migrate-state" flags to
│ use the current configuration.
│ 
│ If the change reason above is incorrect, please verify your configuration
│ hasn't changed and try again. At this point, no changes to your existing
│ configuration or state have been made.
╵

Error running: cd /home/phil/git/tf-infra/.terraspace-cache/live/stacks/security && terraform state pull > /tmp/terraspace/remote_state/stacks/security/state.json

...and so on, for all the stacks

The workaround is to run terraspace all init before running any other commands. It complains loudly while running but it does run to completion. Once you've run it, subsequent commands work properly as long as you don't delete the cache.

There being a workaround means it's not insurmountable, but it'd be good to get a fix at some point.

---

I've also attached a ZIP of a test project that has this problem:

ts_test.zip.

All the relevant versions (for TS/Ruby/TF) should be defined in the Gemfile/.tool-versions/config.

To reproduce:

1. Delete the Terraspace cache folder (.terraspace-cache)
2. Run terraspace build a or terraspace build b or terraspace build c

The errors shown above should appear.

[ORuessel]

I can reproduce this against version ~>1.1.0. I switch back to version 1.0.6 i can use the output helper without any " terraform init" error message. I get the same error message like you.

I have test every version of my code with terraspace version from 1.0.6 to version 1.1.7. Only with version 1.0.6 it works.

Please fix the error before continuing
╷
│ Error: Backend initialization required, please run "terraform init"
│
│ Reason: Unsetting the previously set backend "s3"
│
│ The "backend" is the interface that Terraform uses to store state,
│ perform operations, etc. If this message is showing up, it means that the
│ Terraform configuration you're using is using a custom configuration for
│ the Terraform backend.

I hope this helps.
Best regards

[ORuessel]

Hello @tongueroo,
do you need more information or can we support in this case ?
Best regards