No way to initialize PG backend in automation without non-zero exit code

[aeneasr]

It is not possible to initialize a PG backend when in automation without encountering one non-zero exit code

Terraform Version

0.12.11

Terraform Configuration Files

terraform {
  backend "pg" {
  }
}

Debug Output

Trying to create a workspace before terraform init:

$ TF_AUTOMATION=yes TF_LOG=debug terraform workspace new foo
2019/10/18 18:27:18 [INFO] Terraform version: 0.12.11  
2019/10/18 18:27:18 [INFO] Go runtime version: go1.13.1
2019/10/18 18:27:18 [INFO] CLI args: []string{"/usr/local/bin/terraform", "workspace", "new", "foo"}
2019/10/18 18:27:18 [DEBUG] Attempting to open CLI config file: /Users/foobar/.terraformrc
2019/10/18 18:27:18 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/10/18 18:27:18 [DEBUG] checking for credentials in "/Users/foobar/.terraform.d/plugins"
2019/10/18 18:27:18 [DEBUG] checking for credentials in "/Users/foobar/.terraform.d/plugins/darwin_amd64"
2019/10/18 18:27:18 [INFO] CLI command args: []string{"workspace", "new", "foo"}
2019/10/18 18:27:18 [DEBUG] New state was assigned lineage "8285c60a-48c6-35ee-c3cf-7b4a71d9782d"
2019/10/18 18:27:18 [TRACE] Meta.Backend: moving from default local state only to "pg" backend
Backend reinitialization required. Please run "terraform init".
Reason: Initial configuration of the requested backend "pg"

The "backend" is the interface that Terraform uses to store state,
perform operations, etc. If this message is showing up, it means that the
Terraform configuration you're using is using a custom configuration for
the Terraform backend.

Changes to backend configurations require reinitialization. This allows
Terraform to setup the new configuration, copy existing state, etc. This is
only done during "terraform init". Please run that command now then try again.

If the change reason above is incorrect, please verify your configuration
hasn't changed and try again. At this point, no changes to your existing
configuration or state have been made.


Error: Initialization required. Please see the error message above.

Trying to run terraform init itself:

$ TF_LOG=debug TF_AUTOMATION=yes terraform init -backend-config="conn_str=postgres://postgres:secret@127.0.0.1:3445/hydra?sslmode=disable" -input=false
2019/10/18 18:28:20 [INFO] Terraform version: 0.12.11  
2019/10/18 18:28:20 [INFO] Go runtime version: go1.13.1
2019/10/18 18:28:20 [INFO] CLI args: []string{"/usr/local/bin/terraform", "init", "-backend-config=conn_str=postgres://postgres:secret@127.0.0.1:3445/hydra?sslmode=disable", "-input=false"}
2019/10/18 18:28:20 [DEBUG] Attempting to open CLI config file: /Users/foobar/.terraformrc
2019/10/18 18:28:20 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/10/18 18:28:20 [DEBUG] checking for credentials in "/Users/foobar/.terraform.d/plugins"
2019/10/18 18:28:20 [DEBUG] checking for credentials in "/Users/foobar/.terraform.d/plugins/darwin_amd64"
2019/10/18 18:28:20 [INFO] CLI command args: []string{"init", "-backend-config=conn_str=postgres://postgres:secret@127.0.0.1:3445/hydra?sslmode=disable", "-input=false"}

Initializing the backend...
2019/10/18 18:28:20 [DEBUG] New state was assigned lineage "b5b6c72d-9632-cc06-c411-7dd40ac93b9c"
2019/10/18 18:28:20 [DEBUG] checking for provider in "."
2019/10/18 18:28:20 [DEBUG] checking for provider in "/usr/local/bin"
2019/10/18 18:28:20 [DEBUG] checking for provider in "/Users/foobar/.terraform.d/plugins"
2019/10/18 18:28:20 [WARN] found legacy provider "terraform-provider-circleci-darwin-amd64"
2019/10/18 18:28:20 [DEBUG] checking for provider in "/Users/foobar/.terraform.d/plugins/darwin_amd64"
2019/10/18 18:28:20 [WARN] found legacy provider "terraform-provider-circleci"
2019/10/18 18:28:20 [DEBUG] found valid plugin: "circleci-darwin-amd64", "0.0.0", "/Users/foobar/.terraform.d/plugins/terraform-provider-circleci-darwin-amd64"
2019/10/18 18:28:20 [DEBUG] found valid plugin: "circleci", "0.0.0", "/Users/foobar/.terraform.d/plugins/darwin_amd64/terraform-provider-circleci"
2019/10/18 18:28:20 [DEBUG] checking for provisioner in "."
2019/10/18 18:28:20 [DEBUG] checking for provisioner in "/usr/local/bin"
2019/10/18 18:28:20 [DEBUG] checking for provisioner in "/Users/foobar/.terraform.d/plugins"
2019/10/18 18:28:20 [DEBUG] checking for provisioner in "/Users/foobar/.terraform.d/plugins/darwin_amd64"
2019/10/18 18:28:20 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory
2019/10/18 18:28:20 [DEBUG] New state was assigned lineage "1495b868-0f84-2a44-6f7d-e6084b40220b"

Successfully configured the backend "pg"! Terraform will automatically
use this backend unless the backend configuration changes.

Error: No existing workspaces.

Use the "terraform workspace" command to create and select a new workspace.
If the backend already contains existing workspaces, you may need to update
the backend configuration.

Expected Behavior

I would expect terraform init to not exit with a non-zero status code or be allowed to use workspace new before init.

Actual Behavior

Running workspace new without init does not work. Running init by itself also fails. What I have to do is run init, expect non-zero exit code, then run workspace new/select.

Steps to Reproduce

Create backend.tf with:

terraform {
  backend "pg" {
  }
}

Run $ TF_AUTOMATION=yes TF_LOG=debug terraform workspace new foo

Run TF_LOG=debug TF_AUTOMATION=yes terraform init -backend-config="conn_str=postgres://postgres:secret@127.0.0.1:3445/hydra?sslmode=disable" -input=false

[aeneasr]

Once a workspace has been created, terraform init actually asks, on consecutive calls in directories without previous state, to select a workspace, despite clearly having TF_AUTOMATION set as well as input=false:

$ TF_AUTOMATION=yes TF_WORKSPACE=asdf terraform init -backend-config="conn_str=postgres://postgres:secret@127.0.0.1:3445/hydra?sslmode=disable" -input=false

Initializing the backend...

Successfully configured the backend "pg"! Terraform will automatically
use this backend unless the backend configuration changes.

The currently selected workspace (asdf) does not exist.
  This is expected behavior when the selected workspace did not have an
  existing non-empty state. Please enter a number to select a workspace:
  
  1. foo

  Enter a value: 

[andrey-dubnik]

Same for TF 0.12.16

terraform init does not create a record in the PG database for the default backend which is expected during the init flow...

[FlorianLaunay]

Same for v0.12.19

@teamterraform any new on this ?

[petr-motejlek]

If at least the fact that there's no workspace existing was reported as a separate exit code, we could catch it, and deal with it in automation.

Right now, terraform init with an empty PG backend will just error out with exit code 1 and a message about no workspace. Why does it even behave differently with the PG backend, compared to the local tfstate files? It does not scream at you when you initialize an empty directory. Instead, it create the workspace; no questions asked :).

[aeneasr]

Yeah it's pretty ridiculous if you want to deal with this - we're catching the output of the commands and check if e.g. terraform init or terraform workspace is included in it to decide what to do next. Needless to say that this will stop working if anyone decides to rewrite the help message.

[amarjothi]

any update on this please? or is there a workaround

[thundertaker]

I also would like this to be fixed the same way as local and other remote backends (for example, consul) do: create default workspace automatically on first terraform init.
Now initialization of a new Terraform environment with pg backend require three steps:

1. terraform init, which fails. Ignore it
2. terraform workspace new default
3. terraform init

[FlorianLaunay]

@teamterraform any update please ?

[FlorianLaunay]

For those who would be interested, personally I deal with this bug via the before_script directive in Gitlab CI/CD :

before_script:
  - terraform --version
  - terraform init -backend-config="conn_str=postgres://${DB_USER}:${DB_PASSWORD}@postgres:5432/${DB_NAME}?sslmode=disable" || echo "Done !"
  - terraform workspace new default || if [ $? -eq 0 ]; then echo "Done !"; else echo "Workspace \"default\" was not created"; fi
  - terraform init

I'm getting around the problem by catching the return code of terraform workspace

[petr-motejlek]

Nice combo, @FlorianLaunay :D

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.