Authenticate to Terraform providers with user's OIDC token

[bpmct]

Background

Coder currently relies on the provisioner to be authenticated against a cloud API (e.g. AWS or Kubernetes) to create resources on behalf of the user. Terraform has many authentication methods and we describe this in our docs in provider authentication. For standard workflows, this works well:

coderd authenticates to a Kubernetes cluster via a ServiceAccount+RoleBinding which lets it provision pods and PVCs in a specific namespace on behalf of the user.

However, there are some advanced use cases that don't work with this model:

• Coder uses the user's OIDC token to authenticate with the Terraform provider. That way, it can only provision workspaces in regions, resourcegroups, clusters, etc that the user has access to.
• Coder uses the user's OIDC token to fetch secrets via the Azure Key Vault Terraform feature or Hashicorp Vault feature and then mounts them into workspaces via environment variables.
• Coder uses the GitLab Terraform provider to create a new project on behalf of the user matching the workspace name.

Note: All of these are technically possible with a "global/root" token that Coder has access to, but that is a huge security risk for most enterprises.

Current behavior

During a coder templates push (a terraform plan), data.coder_git_auth.access_token and data.coder_workspace.me.owner_oidc_access_token is empty since there is technically workspace owner. The token being empty is fine if the token is just being mounted as an environment variable, but causes a terraform plan to fail if the token is being used to authenticate with a Terraform provider on behalf of the user.

The unfortunate part about this current behavior is, if a push was to succeed, a coder create (terraform apply) would work fine because there is a workspace owner at that point. The provider would authenticate fine. The template isn't broken, Coder just thinks it is during the plan

Proposed behavior

During a coder templates <create/push>, pass the template author's credentials so that Terraform can authenticate using the token and the template version can be pushed. This should pass the credentials for gitauth and oidc.

The template code would look like:

provider "azurerm" {
  features {
    key_vault {
      purge_soft_deleted_keys_on_destroy = true
      recover_soft_deleted_keys          = true
    }
  }

  subscription_id = "xxxxxxxxxxxxxxxx"
  client_id       = "xxxxxxxxxxxxxxxx"
  use_oidc        = true

  # Currently this fails because, during the plan, the token is empty
  oidc_token = data.coder_workspace.me.owner_oidc_access_token
  tenant_id = "xxxxxxxxxxxxxxxx"
}

data "azurerm_client_config" "current" {}

data "azurerm_key_vault" "example" {
  name                = "xxxxxxxxxxxxxxxx"
  resource_group_name = "xxxxxxxxxxxxxxxx"
}

data "azurerm_key_vault_key" "example" {
  name         = "mykey"
  key_vault_id = data.azurerm_key_vault.example.id
}

Benefit to the user

• Users are "magically authenticated" in their workspaces with the disks, secrets, infrastructure they are used to. Coder saved the user a bunch of manual steps to infrastructure they already had access to.

Benefit to the admin

• Admins do not have to give Coder global access to every tool they want to authenticate with in workspaces. Instead, Coder uses the user's existing permissions to fetch the stuff they need.

Risks

I don't have a fantastic understanding of the risks, so I could use some help here! I outlined a couple here:

• Template author can access another template author's token by re-running or fetching output from a past plan: @sreya brought this one up and I'm not sure if this is possible.
• It is more difficult to debug why a build for a given user (or even template author). There is increased variability in builds based on the user's identity. Here, we are limited in the verbosity of Terraform to explain why a user failed to authenticate versus the guarantee that a provisioner key can create pods on behalf of any user.
• Template author token is exposed via workspace metadata: Metadata can be marked as sensitive to hide it from the templates page and endpoints.

Workarounds

I suppose there could be some Terraform logic to see if the token is empty, then use a "fallback" token that the provisioner has that is limited to a dummy user. I haven't tried this in practice.

I mentioned the global/root token above, but that would not cut it for most of our enterprise customers.

Our customers provision team-specific or user-specific infrastructure outside of Coder and configure templates to authenticate with them via managed variables and parameters respectively.

[bpmct]

This contributes to #9116