Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

digitalocean_database_cluster password is always empty #1165

Closed
razum90 opened this issue May 22, 2024 · 5 comments · Fixed by #1166
Closed

digitalocean_database_cluster password is always empty #1165

razum90 opened this issue May 22, 2024 · 5 comments · Fixed by #1166
Labels

Comments

@razum90
Copy link
Contributor

razum90 commented May 22, 2024

Bug Report

Describe the bug

When creating a digitalocean_database_cluster resource, the password attribute for the admin user (doadmin) is always returned as empty.

Affected Resource(s)

  • digitalocean_database_cluster

Expected Behavior

I expect the password attribute to return the actual value.

Actual Behavior

It is not returning the password.

Steps to Reproduce

terraform apply

Terraform Configuration Files

terraform {
  required_providers {
    digitalocean = {
      source  = "digitalocean/digitalocean"
      version = "~> 2.38.0"
    }
    null = {
      source  = "hashicorp/null"
      version = "~> 3.1.0"
    }
  }
}

provider "digitalocean" {
  token = var.do_token
}

variable "do_token" {}
variable "region" {
  description = "DO region"
  type        = string
  default     = "fra1"
}
variable "services_names" {
  description = "Name of the services you want to create"
  type        = object({
    web = string
    api = string
  })
  default = {
    "web" = "web"
    "api" = "api"
  }
}
variable "app_name" {
  description = "Name of the app"
  type        = string
  default     = "your-app-name"
}
variable "environments" {
  description = "Map of environment names and their attributes"
  type        = map(any)
  default = {
    "prod" = {
      "domain" : "your-domain.com",
      "db" : {
        "production" : true,
        "size" : "db-s-1vcpu-1gb",
        "node_count" : 1,
      },
      "api" : {
        "instance_count" : 1,
        "size_slug" : "apps-s-1vcpu-0.5gb",
        "port" : 80
      },
      "web" : {
        "instance_count" : 1,
        "size_slug" : "basic-xxs",
        "port" : 80
      }
    }
  }
}

resource "digitalocean_database_cluster" "db-cluster" {
  for_each = var.environments

  name       = "${each.key}-cluster"
  engine     = "pg"
  version    = "15"
  size       = var.environments[each.key].db.size
  region     = var.region
  node_count = var.environments[each.key].db.node_count
}

resource "digitalocean_database_user" "api-user" {
  for_each = var.environments

  cluster_id = digitalocean_database_cluster.db-cluster[each.key].id
  name       = "${var.services_names.api}-user"
}

resource "digitalocean_database_db" "api-db" {
  for_each = var.environments

  cluster_id = digitalocean_database_cluster.db-cluster[each.key].id
  name       = "${var.services_names.api}-db"
}

resource "null_resource" "grant_permissions" {
  for_each = var.environments

  provisioner "local-exec" {
    command = <<EOT
      docker run --rm -e PGPASSWORD=${digitalocean_database_cluster.db-cluster[each.key].password} postgres:13 psql -h ${digitalocean_database_cluster.db-cluster[each.key].host} -U ${digitalocean_database_cluster.db-cluster[each.key].user} -p ${digitalocean_database_cluster.db-cluster[each.key].port} -d ${digitalocean_database_db.api-db[each.key].name} -c "GRANT ALL PRIVILEGES ON DATABASE ${digitalocean_database_db.api-db[each.key].name} TO ${digitalocean_database_user.api-user[each.key].name};"
    EOT

    environment = {
      PGPASSWORD = digitalocean_database_cluster.db-cluster[each.key].password
    }
  }

  depends_on = [
    digitalocean_database_cluster.db-cluster,
    digitalocean_database_user.api-user,
    digitalocean_database_db.api-db
  ]
}

resource "digitalocean_database_firewall" "db-cluster-fw" {
  for_each = var.environments

  cluster_id = digitalocean_database_cluster.db-cluster[each.key].id

  rule {
    type  = "app"
    value = digitalocean_app.do-app[each.key].id
  }

  depends_on = [null_resource.grant_permissions]
}

resource "digitalocean_app" "do-app" {
  for_each = var.environments

  lifecycle {
    ignore_changes = [
      spec.0.features,
      spec.0.region,
      spec.0.service.0.image,
      spec.0.service.1.image
    ]
  }

  spec {
    name   = "${var.app_name}-${each.key}"
    region = var.region

    domain {
      name = var.environments[each.key].domain
    }

    alert {
      rule = "DEPLOYMENT_FAILED"
    }

    service {
      name               = var.services_names.api
      instance_count     = var.environments[each.key].api.instance_count
      instance_size_slug = var.environments[each.key].api.size_slug

      image {
        registry_type = "DOCKER_HUB"
        repository    = "nginx"
        tag           = "latest"
      }

      http_port = var.environments[each.key].api.port

      env {
        key   = "DB_PASSWORD"
        value = digitalocean_database_user.api-user[each.key].password
      }

      env {
        key   = "DB_HOST"
        value = digitalocean_database_cluster.db-cluster[each.key].private_host
      }

      env {
        key   = "DB_PORT"
        value = digitalocean_database_cluster.db-cluster[each.key].port
      }

      env {
        key   = "DB_NAME"
        value = digitalocean_database_db.api-db[each.key].name
      }

      env {
        key   = "DB_USER"
        value = digitalocean_database_user.api-user[each.key].name
      }
    }

    service {
      name               = var.services_names.web
      instance_count     = var.environments[each.key].web.instance_count
      instance_size_slug = var.environments[each.key].web.size_slug

      image {
        registry_type = "DOCKER_HUB"
        repository    = "nginx"
        tag           = "latest"
      }

      http_port = var.environments[each.key].web.port
    }

    database {
      name         = digitalocean_database_db.api-db[each.key].name
      db_name      = digitalocean_database_db.api-db[each.key].name
      cluster_name = digitalocean_database_cluster.db-cluster[each.key].name
      production   = var.environments[each.key].db.production
    }

    ingress {
      rule {
        component {
          name = var.services_names.api
        }
        match {
          path {
            prefix = "/api"
          }
        }
      }

      rule {
        component {
          name = var.services_names.web
        }

        match {
          path {
            prefix = "/"
          }
        }
      }
    }
  }
}

Terraform version
1.8.4

Debug Output
https://gist.github.com/razum90/780047c0f021b02832b8a67e6be84490

Additional context

I have also tried to save digitalocean_database_cluster.db-cluster[each.key].password as an env variable to my service, and it shows empty there too.

Important Factoids
It seems like the initial call POST /v2/databases returns the password. But the subsequent GET /v2/databases/{id} does not. I guess the GET calls is used for some sort of polling. I suppose it would be an issue in case the response of the GET request is used to build the output of the digitalocean_database_cluster resource.

BR

@andrewsomething
Copy link
Member

Hi @razum90,

Thanks for raising this issue! Though I haven't been able to reproduce it myself. I am able to create a Postrgres cluster PostgreSQL and retrieve the password for the doadmin user. Here's a minimal example:

resource "digitalocean_database_cluster" "postgres-example" {
  name       = "example-postgres-cluster"
  engine     = "pg"
  version    = "15"
  size       = "db-s-1vcpu-1gb"
  region     = "nyc1"
  node_count = 1
}

output "pw" {
  value = digitalocean_database_cluster.postgres-example.password
  sensitive = true
}

And terraform output pw successfully displays the password.

Using your more complex example above failed with an error when the local-exec provisioner runs, but the password for the doadmin user is visible in the output:

│ Error running command '      docker run --rm -e PGPASSWORD=<redacted> postgres:13 psql -h
│ prod-cluster-do-user-612423-0.c.db.ondigitalocean.com -U doadmin -p 25060 -d api-db -c "GRANT ALL PRIVILEGES
│ ON DATABASE api-db TO api-user;"
<snip>
│ ERROR:  syntax error at or near "-"
│ LINE 1: GRANT ALL PRIVILEGES ON DATABASE api-db TO api-user;

There is one case where it is possible a PostgreSQL cluster will not return the password, when you use a read only API token. We should protect against overwriting a password if someone switches to a read-only token post-create.

Are you using a read-only token? Any additional details that might help us get to the bottom of this?

@razum90
Copy link
Contributor Author

razum90 commented May 28, 2024

Hi @andrewsomething!

I removed the local-exec part, (I am not sure if it even works) and tried:

terraform {
  required_providers {
    digitalocean = {
      source  = "digitalocean/digitalocean"
      version = "~> 2.38.0"
    }
    null = {
      source  = "hashicorp/null"
      version = "~> 3.1.0"
    }
  }
}

provider "digitalocean" {
  token = var.do_token
}

variable "do_token" {}
variable "region" {
  description = "DO region"
  type        = string
  default     = "fra1"
}
variable "services_names" {
  description = "Name of the services you want to create"
  type        = object({
    web = string
    api = string
  })
  default = {
    "web" = "web"
    "api" = "api"
  }
}
variable "app_name" {
  description = "Name of the app"
  type        = string
  default     = "your-app-name"
}
variable "environments" {
  description = "Map of environment names and their attributes"
  type        = map(any)
  default = {
    "prod" = {
      "domain" : "your-domain.com",
      "db" : {
        "production" : true,
        "size" : "db-s-1vcpu-1gb",
        "node_count" : 1,
      },
      "api" : {
        "instance_count" : 1,
        "size_slug" : "apps-s-1vcpu-0.5gb",
        "port" : 80
      },
      "web" : {
        "instance_count" : 1,
        "size_slug" : "basic-xxs",
        "port" : 80
      }
    }
  }
}

resource "digitalocean_database_cluster" "db-cluster" {
  for_each = var.environments

  name       = "${each.key}-cluster"
  engine     = "pg"
  version    = "15"
  size       = var.environments[each.key].db.size
  region     = var.region
  node_count = var.environments[each.key].db.node_count
}

resource "digitalocean_database_db" "api-db" {
  for_each = var.environments

  cluster_id = digitalocean_database_cluster.db-cluster[each.key].id
  name       = "${var.services_names.api}-db"
}

resource "digitalocean_database_firewall" "db-cluster-fw" {
  for_each = var.environments

  cluster_id = digitalocean_database_cluster.db-cluster[each.key].id

  rule {
    type  = "app"
    value = digitalocean_app.do-app[each.key].id
  }
}

resource "digitalocean_app" "do-app" {
  for_each = var.environments

  lifecycle {
    ignore_changes = [
      spec.0.features,
      spec.0.region,
      spec.0.service.0.image,
      spec.0.service.1.image
    ]
  }

  spec {
    name   = "${var.app_name}-${each.key}"
    region = var.region

    domain {
      name = var.environments[each.key].domain
    }

    alert {
      rule = "DEPLOYMENT_FAILED"
    }

    service {
      name               = var.services_names.api
      instance_count     = var.environments[each.key].api.instance_count
      instance_size_slug = var.environments[each.key].api.size_slug

      image {
        registry_type = "DOCKER_HUB"
        repository    = "nginx"
        tag           = "latest"
      }

      http_port = var.environments[each.key].api.port

      env {
        key   = "DB_PASSWORD"
        value = digitalocean_database_cluster.db-cluster[each.key].password
      }

      env {
        key   = "DB_HOST"
        value = digitalocean_database_cluster.db-cluster[each.key].private_host
      }

      env {
        key   = "DB_PORT"
        value = digitalocean_database_cluster.db-cluster[each.key].port
      }

      env {
        key   = "DB_NAME"
        value = digitalocean_database_db.api-db[each.key].name
      }

      env {
        key   = "DB_USER"
        value = digitalocean_database_cluster.db-cluster[each.key].user
      }
    }

    service {
      name               = var.services_names.web
      instance_count     = var.environments[each.key].web.instance_count
      instance_size_slug = var.environments[each.key].web.size_slug

      image {
        registry_type = "DOCKER_HUB"
        repository    = "nginx"
        tag           = "latest"
      }

      http_port = var.environments[each.key].web.port
    }

    database {
      name         = digitalocean_database_db.api-db[each.key].name
      db_name      = digitalocean_database_db.api-db[each.key].name
      cluster_name = digitalocean_database_cluster.db-cluster[each.key].name
      production   = var.environments[each.key].db.production
    }

    ingress {
      rule {
        component {
          name = var.services_names.api
        }
        match {
          path {
            prefix = "/api"
          }
        }
      }

      rule {
        component {
          name = var.services_names.web
        }

        match {
          path {
            prefix = "/"
          }
        }
      }
    }
  }
}

And I am getting an empty value as the DB_PASSWORD env variable in the app. I am pretty sure this has worked for me at some point, but for unknown reasons it does not work anymore. I am not sure at all, but I have a vague memory that the creation of the db cluster went quicker right after I created my DO account, could it be that it did not have to perform subsequent GET requests (for the polling) and were just completed after the initial POST request? At this point I have recreated my infrastructure at least 20 times.

Are you sure that in your case the GET /v2/databases/{id} is being performed, and that it's returning the password? As we can see in the GIST I provided in the issue creation (https://gist.github.com/razum90/780047c0f021b02832b8a67e6be84490) the password is not being returned in the GET request, only in the POST one.

I don't switch any tokens, I use the same one for tf. And since that's able to create I guess it's not read-only. Attaching the permissions it has:

Screenshot 2024-05-28 at 22 16 42

BR

@razum90
Copy link
Contributor Author

razum90 commented May 28, 2024

I tried your simple example and I still get this issue. Full gist here: https://gist.github.com/razum90/ff4438ec7e50c7b0c0fe898d30e3e254

@andrewsomething
Copy link
Member

Thanks for the follow up @razum90! I can reproduce the issue now. The difference was using a custom scoped token vs a "full access" one. I've raised this internally as if that is the expected behavior, it needs to be documented.

It also impacts other database engines, not just PostgreSQL. So my suggestion on you PR around removing the special casing for specific engines still stands. Let me know if you can make that change. If not, I can whip up a PR.

@razum90
Copy link
Contributor Author

razum90 commented May 29, 2024

Ah okay interesting. Sure I will do that this evening (I'm in the CEST timezone FYI)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants