${terraform.workspace} value is default and not actual workspace name (when using the terraform cloud remote backend)

[tomerliberman]

Terraform Version

Terraform v0.12.8

Terraform Configuration Files

resource "aws_instance" "example" {
  count = "${terraform.workspace == "default" ? 5 : 1}"

  # ... other arguments
}

Debug Output

Crash Output

Expected Behavior

Would use actual Cloud Workspace to make the decision

Actual Behavior

terraform.workspace value was "default" and resulted in wrong configuration

Steps to Reproduce

follow this example and use within the cloud offering with a workspace named anything other than default
(https://www.terraform.io/docs/state/workspaces.html#current-workspace-interpolation)

Additional Context

References

[kmoe]

Thanks for the bug report, @tomerliberman. I've reproduced this issue on Terraform Cloud.

Edit: This is not a bug, but expecting a different value for ${terraform.workspace} here is quite understandable! Please see Roger's response below for a full explanation.

[sandangel]

we had to manually change all the terraform.workspace to var.workspace . we just started to have 1 more engineer working on terraform code and we want remote execution to prevent security risk with applying locally. but this bug really a show stopper for us for using terraform cloud.

[tamoanxx]

I am having the same issue and can't figure out why this is occuring.

@kmoe Do you have an update as to when this will be corrected?

[tomazberisa]

We're seeing the same behavior with v0.11.14

Timeline for a fix would be greatly appreciated.

[unfor19]

terraform v0.12.9 - same behavior.

[vliao-semios]

One weird thing is when debugging with terraform console, and type terraform.workspace in the interactive console, it shows the actual remote workspace name.

`
$ tf console

terraform.workspace
wonderful_space

`

[saitotqr]

Same issue here. I must to add static value for workspaces.name to read backend.
Terraform reads default even if I set workspaces.prefix
It's critical blocker for us for using terraform cloud.

data "terraform_remote_state" "tfcloud" {
  backend = "remote"
  config = {
    organization = "org"
    #static will work, but can't use for another workspace
    #workspaces = {
    #  name = "mywrk-prd"
    #}
    # I want to set like this but doesn't work
    workspaces = {
      prefix = "mywrk-"
    }
  }
}

terraform {
  backend "remote" {
    hostname = "app.terraform.io"
    organization = "org"
    workspaces = {
      prefix = "mywrk-"
    }
  }
}

$ terraform --version
Terraform v0.11.13
+ provider.archive v1.3.0
+ provider.aws v2.30.0
+ provider.template v2.1.2

$ terraform workspace list
* prd

$ terraform plan
...
To view this run in a browser, visit:
https://app.terraform.io/app/mycorp/mywrk/runs/run-xxx

Waiting for the plan to start...

Terraform v0.11.13

Configuring remote state backend...
...

Error: Error refreshing state: 1 error(s) occurred:

* data.terraform_remote_state.tfcloud: 1 error(s) occurred:

* data.terraform_remote_state.tfcloud: data.terraform_remote_state.tfcloud: error loading the remote state: default state not supported
You can create a new workspace with the "workspace new" command.

[rberlind]

This is not actually a bug.

It is important to understand that the concept of "workspace" in TFC is different from the one in open source Terraform. In the latter (sometimes called a "local workspace"), the terraform workspace command can be used to set more than one "workspace" for the configuration and the interpolation ${terraform.workspace} can be used to refer to the current workspace set. However, in TFC, each "workspace" (corresponding to one listed in the UI) only ever has the single OSS workspace "default" which is why the interpolation always gives "default".

This does mean that Terraform configurations that use that interpolation must be modified to not use it for TFC if doing remote runs on TFC or TFE.

Please see https://www.terraform.io/docs/cloud/migrate/workspaces.html for instructions on migrating multiple local workspaces to TFC.

[unfor19]

Thank you for your comment reberlind. I've read the guide that you shared, it is still not intuitive.
From how I see it - TFC and TF(local) work exactly the same, the only difference is the 'run locally' and 'run remotely'.

Anyways - I simply recommend people to tick the "run locally" and then everything works as expected.

[dekimsey]

It is important to understand that the concept of "workspace" in TFC is different from the one in open source Terraform. In the latter (sometimes called a "local workspace"), the terraform workspace command can be used to set more than one "workspace" for the configuration and the interpolation ${terraform.workspace} can be used to refer to the current workspace set. However, in TFC, each "workspace" (corresponding to one listed in the UI) only ever has the single OSS workspace "default" which is why the interpolation always gives "default".
This does mean that Terraform configurations that use that interpolation must be modified to not use it for TFC.

Forgive me @rberlind, but I'm having trouble resolving how this with how terraform_remote_state functions in which it needs to know the full name of the state (see #21507). How would I alter the remote state/backend such that it works correctly in TFC? Would I have to define a new variable that just so happens to be the workspace I select with terraform workspace?

To provide examples, iven a main backend of:

terraform {
  backend "remote" {
    organization = "example"
    workspaces {
      prefix = "main-"
    }
  }
}

And a set of workspaces such as:

terraform workspace list
* stg
  demo1

data "terraform_remote_state" "main" {
  backend = "remote"

  config = {
    organization = "example"

    workspaces = {
      name = "main-${terraform.workspace}"
    }
  }
}

[rberlind]

Thanks for your feedback, @dekimsey.

When I described the differences between the "workspace" concepts in Terraform OSS and Terraform Cloud (and Terraform Enterprise), I neglected to mention how the remote backend can help customers map between them. Your comment referring to the workspaces.prefix attribute of the remote backend configuration reminded me of that.

Specifically, if you have a Terraform OSS configuration in a directory with multiple workspaces such as "dev", "qa", and "prod", you can set up the remote backend with the workspaces.prefix option set to something like my-workspaces-. If you run terraform workspace select dev and then run terraform apply, the run on the Terraform Cloud (or Terraform Enterprise) server will be done in a workspace called "my-workspaces-dev". If you run terraform workspace select qa and then run terraform apply, the run on the Terraform Cloud (or Terraform Enterprise) server will be done in a workspace called "my-workspaces-qa". If you run terraform workspace select prod and then run terraform apply, the run on the Terraform Cloud (or Terraform Enterprise) server will be done in a workspace called "my-workspaces-prod".

Note that the ${terraform.workspace} interpolation is NOT used when doing this mapping. Instead, the Terraform OSS workspace name ("dev", "qa", or "prod") will automatically be used in the name of the TFC/TFE workspace when doing runs on the remote server. (The same applies with regard to storing of state in TFC/TFE workspaces if doing local runs.)

The bottom line with regard to the ${terraform.workspace} interpolation is that it should not be used in Terraform configurations that are doing remote runs on TFC or TFE. It should be ok to use it if you are only doing local runs and only using TFC/TFE to store remote state.

Note that the TFC/TFE workspaces will be created for you if they do not already exist and will use the most recent version of Terraform OSS. However, if your workspace needs variables set, especially environment variables such as cloud credentials, you should manually create the workspaces and set any needed variables. If you want to use a version of Terraform OSS for the runs done on the TFC/TFE server in those workspaces, then you should also set the version before your first plan or apply.

Regarding use of remote backend with the terraform_remote_state data source, the workspace containing the state that is specified in that data source's config.workspaces stanza can only specify a name attribute and that using prefix would not be allowed. See the terraform_remote_state documentation: https://www.terraform.io/docs/providers/terraform/d/remote_state.html as well as the actual example under the remote backend doc: https://www.terraform.io/docs/backends/types/remote.html#example-reference

Hopefully, that clarifies how Terraform OSS workspaces can be mapped to TFC/TFE workspaces with the remote backend configuration as well as why the ${terraform.workspace} interpolation should not be used in Terraform configurations doing remote runs on TFC/TFE servers.

[simonklb]

I was also surprised by this when transitioning from local runs to remote runs and saw that my resources (that was tagged with the terraform.workspace variable) became tagged with "default" instead of "dev" etc.

Would love to keep using the terraform.workspace variable in the future instead of having to introduce an extra variable.

[keithrozario]

This is rather confusing, but I think the major issue is simply:

• Terraform defaulting to 'remote' execution mode for new workspaces and deployments.
• Most people still expecting the same behavior as 'local' execution

And since the remote execution mode has a different behavior with regards to the local when it comes to workspaces -- specifically ${terraform.workspace} , this causes a lot of confusion. A simple fix for to simply revert to local deployments via the GUI, and most of the references to workspaces would continue to work.

In the Web GUI, go to the workspace in question, select settings and set the execution mode to local.

[rberlind]

Thanks for sharing your thoughts on this, @keithrozario.

It is true that workspace execution mode does default to remote. That is probably because remote execution used to be the only option before the introduction of "Terraform Cloud" which was a new product that essentially replaced the HashiCorp-hosted implementation of "Terraform Enterprise".

I will file an internal ticket asking whether the Terraform Cloud team thinks that defaulting to local execution mode would be better, at least for customers using the free tier of Terraform Cloud who are most likely to want that.

I'm also adding a ticket asking that execution mode be documented under https://www.terraform.io/docs/cloud/workspaces/settings.html#general

[SeanHood]

For anyone needing a quick workaround, TF_VAR_ATLAS_WORKSPACE_NAME is available in the remote environment. This can be utilised but it will be the full Workspace name you see in Terraform Cloud, not just the short one you see locally (minus any prefixes set).

variable "ATLAS_WORKSPACE_NAME" {
  type = string
}

[keithrozario]

thanks @rberlind , yea, as a free-tier user I struggled to understand why my new workspaces were not deploying. Also, it warned me about a state file as I deployed :(

Would have been helpful if the error message prompted something.

[unfor19]

I've just read through all the replies and ..

TL;DR- I recommend to anyone who's using Terraform to use "run locally" and make sure all the architects have the same version of Terraform, and relevant provider(s) extra software installed.

Since this "run remotely" is not acting as expected by the common user (or to anyone with common sense) it's "simpler" to design a solution that runs locally.
And of course, each team might have its own considerations, I'm talking about teams of up to 5-8 architects who are communicating on a daily basis (Slack channel or something similar).

[rberlind]

@unfor19 : I would note that there are some advantages of doing remote runs with Terraform Cloud including having the runs history for each workspace along with the plan and apply logs and Sentinel policy check logs if Sentinel is used. With local runs, you are only using Terraform Cloud for storing state and cannot use Sentinel policies to restrict the resources being provisioned.

[simonklb]

Also worth mentioning VCS integration which is the reason I transitioned from local to remote runs. It has worked really well (great work!), so no reason to complain other than this small nit.

[unfor19]

@unfor19 : I would note that there are some advantages of doing remote runs with Terraform Cloud including having the runs history for each workspace along with the plan and apply logs and Sentinel policy check logs if Sentinel is used. With local runs, you are only using Terraform Cloud for storing state and cannot use Sentinel policies to restrict the resources being provisioned.

agreed

[Dunedan]

@rberlind Thank you for your explanations. What's the suggested future-proof way of retrieving the name of the current TFC workspace? While the approach with the environment variable might work right now, you mentioned that this is an internal implementation detail, which might change, so I wonder if there is a better way to get the TFC workspace name.

[rberlind]

I cannot share that with you yet, @Dunedan, since the product management and engineering teams are working on improving the use of Terraform OSS workspace names in Terraform Cloud and Terraform Enterprise. At this time, every TFC/TFE workspace will only have the "default" Terraform OSS workspace under the covers.

However, I also want to point out that I misinterpreted what @SeanHood has written in his comment suggesting using the TF_VAR_ATLAS_WORKSPACE_NAME environment variable and the ATLAS_WORKSPACE_NAME Terraform variable. That is something that can always be done, also in the future. I had mistakenly thought he was referring to a pre-defined environment variable that might have existed at some point in the past and might still exist even though I could find no reference to it. (I had neglected to note that it started with "TF_VAR_".)

I'm going to delete that confusing comment.

The one suggestion I would make is that you use a different name such as tfc_workspace_name for the Terraform variable and TF_VAR_tfc_workspace_name for the environment variable. "Atlas" refers to a very old version of Terraform Cloud (and Terraform Enterprise) and should really not be used. In fact, it was the use of "ATLAS" that made me thing @SeanHood was referring to some predefined environment variable. I still should have noticed the TF_VAR_ prefix though.

[Dunedan]

Now I'm more confused than before, because TF_VAR_ATLAS_WORKSPACE_NAME is a pre-defined environment variable in Terraform Cloud which contains the expected TFC workspace name. That's why I was asking if that's future-proof or if there is something more future-proof. As I understand you now that's something we shouldn't rely on, but should set a custom environment variable separately containing the workspace name, correct?

[rberlind]

Apologies @Dunedan and other readers. I'm obviously also confused. I'm reshowing my original comment (which I had hidden, not deleted). I'll ask the Terraform engineering team about this. If you are correct that "TF_VAR_ATLAS_WORKSPACE_NAME is a pre-defined environment variable in Terraform Cloud", I do not believe that fact is documented. Given the use of "ATLAS" in the name, I wonder if this is from the legacy version of Terraform Enterpise. I searched old versions of the docs in GitHub, but did not find it in https://github.com/hashicorp/terraform-website/blob/32b778a42ad3c49498af5ce8a9273ca0d7537bb3/content/source/docs/enterprise/runs/variables-and-configuration.html.md which mentioned various other ATLAS_ env vars.

[rberlind]

@Dunedan: I would note that the use of TF_VAR_ to set Terraform variable values through environment variables IS standard. And to me, the fact that the one in question, TF_VAR_ATLAS_WORKSPACE_NAME, starts with "TF_VAR_" suggests that it is NOT a pre-existing environment variable, but is instead just using the standard mechanism for setting Terraform variables through environment variables if the Terraform variable ATLAS_WORKSPACE_NAME is set in the Terraform code. I think that is why @SeanHood also suggested setting the Terraform variables ATLAS_WORKSPACE_NAME in your Terraform code.

[fprimex]

Hi all, TF Support Engineer here. I have assisted with numerous instances of this confusion, so please allow me to chime in. The OSS engineering team is primarily concerned with the CLI terraform program, and so may not be the best resource for how terraform runs in Terraform Cloud and Enterprise. Since I have responded to a customer before regarding this issue, I had the following write-up ready to supply here.

I'll be happy to address any additional questions you may have.

---

The terraform.workspace interpolation has different values depending on where terraform is run and how the remote backend is configured.

When the remote backend is configured using the name option rather than the prefix option, Terraform CLI workspaces are effectively unsupported and disabled. You can see this locally for yourself by configuring remote with name and then attempting to create a workspace with terraform workspace new foo. Since workspaces are unsupported, the terraform.workspace interpolation is always the value default.

When the remote backend is configured with the prefix option instead, workspaces are supported. With a prefix foo-, a new Terraform CLI workspace bar will be created with terraform workspace new bar. In this case the TFE workspace name will be foo-bar, and locally the terraform.workspace interpolation will produce bar - the workspace name without the prefix value.

In TFE, the backend configuration is always overridden. This ensures that the TFE remote state storage backend for that workspace is always used regardless as to any other backend present in the configuration. The backend override is the remote backend with the name option, therefore the value of terraform.workspace is always default.

It's unfortunately possible to have a configuration locally with a workspace bar, a TFE workspace name of foo-bar, and the terraform.workspace produce the value default.

There are some alternative approaches to keying off of the workspace name. For one, you could create a variable named workspace of your own and set it on the Variables page per workspace. Alternatively, you can use an environment variable that is present in the Terraform worker container, and set this environment variable locally to match should you need to perform a local-only run (not using the remote backend).

There is a variable that should be available in TFE that will work for this purpose. At times it is informative to poke at the container used in TFE to see what is set and what the container layout is like. Here is the output of env run from a null_resource in a local-exec provisioner (and trimmed for readability):

Executing: ["/bin/sh" "-c" "env"]
ATLAS_RUN_ID=run-nsYMgVXgwWDeZazK
TF_VAR_ATLAS_CONFIGURATION_SLUG=brentwoodruff/tfe_demo
MAIL=/var/mail/terraform
SSH_CLIENT=192.168.126.1 54920 22
USER=terraform
LANGUAGE=en_US:
TF_INPUT=0
SHLVL=1
ATLAS_WORKSPACE_NAME=tfe_demo
HOME=/home/terraform
OLDPWD=/home/terraform
TF_X_SHADOW=0
TF_VAR_ATLAS_CONFIGURATION_NAME=tfe_demo
LOGNAME=terraform
TF_VAR_ATLAS_WORKSPACE_SLUG=brentwoodruff/tfe_demo
_=/usr/bin/envdir
ATLAS_CONFIGURATION_SLUG=brentwoodruff/tfe_demo
XDG_SESSION_ID=2
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
TF_VAR_ATLAS_ADDRESS=https://app.terraform.io
ATLAS_TOKEN=<redacted>
XDG_RUNTIME_DIR=/run/user/500
TF_VAR_ATLAS_RUN_ID=run-nsYMgVXgwWDeZazK
TF_APPEND_USER_AGENT=TFE/deac623
LANG=en_US
cccf35992f8f3cd8d1d28f0109dd953e26664531=7c28215aca87789f95b406b8dd91aa5198406750
SHELL=/bin/bash
TF_VAR_ATLAS_WORKSPACE_NAME=tfe_demo
TF_VAR_TFE_RUN_ID=run-nsYMgVXgwWDeZazK
ATLAS_CONFIGURATION_NAME=tfe_demo
PLUGIN_MIN_PORT=10000
ATLAS_WORKSPACE_SLUG=brentwoodruff/tfe_demo
PLUGIN_MAX_PORT=25000
CHECKPOINT_DISABLE=1
TERRAFORM_CONFIG=/tmp/cli.tfrc
PWD=/terraform
SSH_CONNECTION=192.168.126.1 54920 192.168.126.163 22
TF_IN_AUTOMATION=1
TF_FORCE_LOCAL_BACKEND=1
ATLAS_ADDRESS=https://app.terraform.io
TFE_RUN_ID=run-nsYMgVXgwWDeZazK
TF_PLUGIN_MAGIC_COOKIE=<redacted>

In particular there is an environment variable called TF_VAR_ATLAS_WORKSPACE_NAME=tfe_demo. This naming follows the convention needed for setting Terraform variables via environment variables:

https://www.terraform.io/docs/configuration/environment-variables.html#tf_var_name

Therefore, to make this variable available in your Terraform configuration, declare it as:

variable "ATLAS_WORKSPACE_NAME" {}

And then it should be possible to use it in the configuration:

variable "ATLAS_WORKSPACE_NAME" {}
resource "null_resource" "n" {}
output "workspace" { value = "${var.ATLAS_WORKSPACE_NAME}" }
...
Outputs:

workspace = tfe_demo

Please note that this is an implementation detail of TFE and is likely to change, if for no other reason, due to the use of the term ATLAS.

[rberlind]

Thanks @fprimex !

[valery-zhurbenko]

I see new TFC_WORKSPACE_NAME can be used in tfc remote runs :-)

[rberlind]

@valery-zhurbenko : Unfortunately, that is the TFC-level workspace name, not the TF OSS workspace name. It is still the case that the only TF OSS workspace name associated with a TFC/TFE workspace is the "default" OSS workspace.

[dkirrane]

I'm using the prefix and all the environment variables include the prefix.

There doesn't seem to be a way to just get the workspace name as set by terraform workspace new prod

All of these include the prefix:

• TFC_WORKSPACE_NAME
• ATLAS_WORKSPACE_NAME
• TF_VAR_ATLAS_WORKSPACE_NAME
• TF_VAR_TFC_WORKSPACE_NAME

[guspio]

Hi,

Here's what I've set up to retrieve the correct name for the workspace

terraform {
  backend "remote" {
    hostname     = "app.terraform.io"
    organization = "my-organization"

    workspaces {
      prefix = "my-workspace-" # copy your workspace prefix on trimprefix("${var.TFC_WORKSPACE_NAME}", "my-workspace-") function
    }
  }
}

variable "TFC_WORKSPACE_NAME" {
  type = string
  default = "" # An error occurs when you are running TF backend other than Terraform Cloud
}

locals {
  # If your backend is not Terraform Cloud, the value is ${terraform.workspace} 
  # otherwise the value retrieved is that of the TFC_WORKSPACE_NAME with trimprefix
  my_workspace_env = var.TFC_WORKSPACE_NAME != "" ? trimprefix("${var.TFC_WORKSPACE_NAME}", "my-workspace-") : "${terraform.workspace}"
}

output "workspace" {
  value = local.my_workspace_env
}

If you have a better temporary solution...

[jjorissen52]

An alternative below. This allows you to set a workspace name in your remote execution environment via TF_VAR_indicated_workspace

locals {
  # if the workspace name is "default" (such as any run in Terraform Cloud), see 
  # if var.indicated_workspace is different from the active workspace
  workspace_name = terraform.workspace == "default" ? var.indicated_workspace : terraform.workspace
}

variable "indicated_workspace" {
  default = "default"
  description = <<EOF
This is the workspace indicated via the environment or some other mechanism. This is useful in 
an remote execution environment where the workspace name will not necessary be the same as the active
workspace of your local environment, such as when using remote execution in Terraform Cloud.
  EOF
}

So If I am in my workspace dev locally and I want to execute in Terraform Cloud remotely, in my remote workspace environment I can set TF_VAR_indicated_workspace=dev. Once I have done that, workspace_name will contain the desired value during remote execution.

The nice thing here is that I do not need to specify indicated_workspace if it's running locally; the value will always contain the currently active workspace (unless indicated_workspace has been set locally).

[apparentlymart]

Hi all!

Reviewing the discussion here, it seems like this issue was representing a question, and that question has now been answered so I'm going to close the issue. We typically don't use GitHub issues for questions but I understand that at first this seemed like a bug report and only in subsequent discussion did we switch to talking about the details of the current behavior.

If you have any further questions related to this, I'd suggest starting a topic in the community forum.

The Terraform Core and Terraform Cloud teams are continuing to discuss the future of workspaces in Terraform CLI and so the details discussed here might change slightly in future releases, but after reading through the discussion it seems to be correct and current at the time I'm writing this.

If you have bug reports or feedback about Terraform Cloud in particular, you can also contact the Terraform Cloud support team at tf-cloud-support@hashicorp.com. If you contact the support team, be sure to include your Terraform Cloud username and organization name in your request.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.