Prod rename and clickhouse module refactor #4

hellais · 2024-02-23T10:37:54Z

This PR does the following:

Rename production environment folder to prod (in future we shall have prod and test)
Move the clickhouse related configuration into a separate module (see docs for best-practices for this: https://developer.hashicorp.com/terraform/language/modules/develop/composition).

github-actions · 2024-02-23T10:40:34Z

Terraform Run Output 🤖

Format and Style 🖌`success`

Initialization ⚙️`success`

Validation 🤖`success`

Validation Output


$ terraform validate
Success! The configuration is valid.

Plan 📖`success`

Plan: 2 to add, 3 to change, 3 to destroy.

Show Plan


$ terraform plan
Acquiring state lock. This may take a few moments...
local_file.ansible_inventory: Refreshing state... [id=ab094adabd3ba9227beb2fc69d596fbaa80c9cfa]
null_resource.ansible_update_known_hosts: Refreshing state... [id=4418919173081010018]
aws_cloudwatch_log_group.ecs: Refreshing state... [id=tf-ecs-group/ecs-agent]
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Reading...
aws_iam_role_policy.ecs_task: Refreshing state... [id=ooni_ecs_task_role:ooni_ecs_task_policy]
module.terraform_state_backend.data.aws_region.current: Reading...
aws_vpc.main: Refreshing state... [id=vpc-08b101405472a0b46]
aws_ecs_cluster.main: Refreshing state... [id=arn:aws:ecs:eu-central-1:082866812839:cluster/ooni-ecs-cluster]
aws_iam_role.ecs_task: Refreshing state... [id=ooni_ecs_task_role]
aws_iam_role.app_instance: Refreshing state... [id=tf-ecs-ooni-instance-role]
aws_iam_role.ecs_service: Refreshing state... [id=ooni_ecs_role]
module.terraform_state_backend.aws_dynamodb_table.with_server_side_encryption[0]: Refreshing state... [id=ooni-production-terraform-state-lock]
module.terraform_state_backend.data.aws_region.current: Read complete after 0s [id=eu-central-1]
aws_cloudwatch_log_group.app: Refreshing state... [id=tf-ecs-group/app-dataapi]
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Read complete after 0s [id=2050088263]
aws_acm_certificate.dataapi: Refreshing state... [id=arn:aws:acm:eu-central-1:082866812839:certificate/27b7c5e3-ef70-41e8-a6cf-ac0d58a99613]
data.aws_availability_zones.available: Reading...
data.aws_ssm_parameter.ecs_optimized_ami: Reading...
module.terraform_state_backend.aws_s3_bucket.default[0]: Refreshing state... [id=ooni-production-terraform-state]
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Reading...
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Read complete after 0s [id=2050088263]
aws_iam_instance_profile.app: Refreshing state... [id=tf-ecs-instprofile]
aws_iam_role_policy.ecs_service: Refreshing state... [id=ooni_ecs_role:ooni_ecs_policy]
aws_iam_role_policy.instance: Refreshing state... [id=tf-ecs-ooni-instance-role:TfEcsOONIInstanceRole]
aws_ecs_task_definition.dataapi: Refreshing state... [id=ooni-dataapi-production-td]
aws_route53_record.dataapi_cert_validation["dataapi.prod.ooni.io"]: Refreshing state... [id=Z02418652BOD91LFA5S9X__74a0d0e91f58697b6838bed5a2fb4939.dataapi.prod.ooni.io._CNAME]
data.aws_availability_zones.available: Read complete after 1s [id=eu-central-1]
aws_acm_certificate_validation.dataapi: Refreshing state... [id=2024-02-05 11:11:01.598 +0000 UTC]
data.aws_ssm_parameter.ecs_optimized_ami: Read complete after 1s [id=/aws/service/ecs/optimized-ami/amazon-linux-2/recommended]
aws_internet_gateway.gw: Refreshing state... [id=igw-05319878d828e0a53]
aws_security_group.pg_sg: Refreshing state... [id=sg-06d7cf832e6855a37]
aws_subnet.main[1]: Refreshing state... [id=subnet-02241f60bd951358d]
aws_alb_target_group.dataapi: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:082866812839:targetgroup/ooni-ecs-dataapi/c16001d24f858f20]
aws_subnet.main[0]: Refreshing state... [id=subnet-06d47d18b015109d4]
aws_security_group.lb_sg: Refreshing state... [id=sg-090ff83a71e7ad235]
aws_db_subnet_group.main: Refreshing state... [id=ooni-main]
aws_route_table.r: Refreshing state... [id=rtb-06814accd93aca9ce]
aws_alb.main: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:082866812839:loadbalancer/app/ooni-alb-ecs/4cf8bd00b53f8961]
aws_security_group.instance_sg: Refreshing state... [id=sg-0f39450a61732d69b]
aws_route_table_association.a[1]: Refreshing state... [id=rtbassoc-034c4831c1754e284]
aws_route_table_association.a[0]: Refreshing state... [id=rtbassoc-049f8c5e84cb32e55]
aws_launch_template.app: Refreshing state... [id=lt-01b923563af0432b5]
aws_autoscaling_group.app: Refreshing state... [id=ooni-tier1-production-backend-asg20240119184843559900000002]
aws_db_instance.ooni_pg: Refreshing state... [id=db-LT4NDHTMMWPYB4TLALUJMYTZFU]
module.terraform_state_backend.aws_s3_bucket_versioning.default[0]: Refreshing state... [id=ooni-production-terraform-state]
module.terraform_state_backend.aws_s3_bucket_public_access_block.default[0]: Refreshing state... [id=ooni-production-terraform-state]
module.terraform_state_backend.aws_s3_bucket_server_side_encryption_configuration.default[0]: Refreshing state... [id=ooni-production-terraform-state]
aws_alb_listener.front_end: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:082866812839:listener/app/ooni-alb-ecs/4cf8bd00b53f8961/1cd9240a34a13129]
aws_route53_record.alb_dns: Refreshing state... [id=Z02418652BOD91LFA5S9X_dataapi.prod.ooni.io_A]
aws_alb_listener.front_end_https: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:082866812839:listener/app/ooni-alb-ecs/4cf8bd00b53f8961/b7ec77ceadac94c8]
module.terraform_state_backend.aws_s3_bucket_policy.default[0]: Refreshing state... [id=ooni-production-terraform-state]
aws_route53_record.postgres_dns: Refreshing state... [id=Z035992527R8VEIX2UVO0_postgres.tier0.prod.ooni.nu_CNAME]
module.terraform_state_backend.time_sleep.wait_for_aws_s3_bucket_settings[0]: Refreshing state... [id=2024-02-02T21:32:23Z]
module.terraform_state_backend.aws_s3_bucket_ownership_controls.default[0]: Refreshing state... [id=ooni-production-terraform-state]
aws_ecs_service.dataapi: Refreshing state... [id=arn:aws:ecs:eu-central-1:082866812839:service/ooni-ecs-cluster/ooni-ecs-dataapi-production]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
  - destroy
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # aws_ecs_service.dataapi will be updated in-place
  ~ resource "aws_ecs_service" "dataapi" {
        id                                 = "arn:aws:ecs:eu-central-1:082866812839:service/ooni-ecs-cluster/ooni-ecs-dataapi-production"
        name                               = "ooni-ecs-dataapi-production"
        tags                               = {
            "Name"       = "ooni-tier1-production"
            "Repository" = "https://github.com/ooni/devops"
        }
      ~ task_definition                    = "arn:aws:ecs:eu-central-1:082866812839:task-definition/ooni-dataapi-production-td:10" -> (known after apply)
      ~ triggers                           = {
          ~ "redeployment" = "2024-02-23T17:02:58Z" -> "2024-02-26T09:52:20Z"
        }
        # (14 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

  # aws_ecs_task_definition.dataapi must be replaced
-/+ resource "aws_ecs_task_definition" "dataapi" {
      ~ arn                      = "arn:aws:ecs:eu-central-1:082866812839:task-definition/ooni-dataapi-production-td:10" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:eu-central-1:082866812839:task-definition/ooni-dataapi-production-td" -> (known after apply)
      ~ container_definitions    = jsonencode(
          ~ [
              ~ {
                  - environment      = []
                  - mountPoints      = []
                    name             = "ooni_dataapi"
                  ~ portMappings     = [
                      ~ {
                          - protocol      = "tcp"
                            # (2 unchanged attributes hidden)
                        },
                    ]
                  - secrets          = [
                      - {
                          - name      = "POSTGRES_URL"
                          - valueFrom = "arn:aws:secretsmanager:eu-central-1:082866812839:secret:OONI_PROD_POSTGRES_URL-IQyNqP"
                        },
                      - {
                          - name      = "JWT_ENCRYPTION_KEY"
                          - valueFrom = "arn:aws:secretsmanager:eu-central-1:082866812839:secret:OONI_PROD_JWT_ENCRYPTION_KEY-euqdD9"
                        },
                    ]
                  - volumesFrom      = []
                    # (5 unchanged attributes hidden)
                },
            ] # forces replacement
        )
      - execution_role_arn       = "arn:aws:iam::082866812839:role/ooni_ecs_task_role" -> null # forces replacement
      ~ id                       = "ooni-dataapi-production-td" -> (known after apply)
      + network_mode             = (known after apply)
      - requires_compatibilities = [] -> null
      ~ revision                 = 10 -> (known after apply)
        tags                     = {
            "Name"       = "ooni-tier1-production"
            "Repository" = "https://github.com/ooni/devops"
        }
        # (3 unchanged attributes hidden)
    }

  # aws_iam_role.ecs_task will be destroyed
  # (because aws_iam_role.ecs_task is not in configuration)
  - resource "aws_iam_role" "ecs_task" {
      - arn                   = "arn:aws:iam::082866812839:role/ooni_ecs_task_role" -> null
      - assume_role_policy    = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "sts:AssumeRole"
                      - Effect    = "Allow"
                      - Principal = {
                          - Service = "ecs-tasks.amazonaws.com"
                        }
                      - Sid       = ""
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> null
      - create_date           = "2024-02-23T16:30:06Z" -> null
      - force_detach_policies = false -> null
      - id                    = "ooni_ecs_task_role" -> null
      - managed_policy_arns   = [] -> null
      - max_session_duration  = 3600 -> null
      - name                  = "ooni_ecs_task_role" -> null
      - path                  = "/" -> null
      - tags                  = {
          - "Name"       = "ooni-tier1-production"
          - "Repository" = "https://github.com/ooni/devops"
        } -> null
      - tags_all              = {
          - "Name"       = "ooni-tier1-production"
          - "Repository" = "https://github.com/ooni/devops"
        } -> null
      - unique_id             = "AROARGSZ7G6T6XF6TFK7B" -> null

      - inline_policy {
          - name   = "ooni_ecs_task_policy" -> null
          - policy = jsonencode(
                {
                  - Statement = [
                      - {
                          - Action   = [
                              - "ecs:DeregisterContainerInstance",
                              - "ecs:DiscoverPollEndpoint",
                              - "ecs:Poll",
                              - "ecs:RegisterContainerInstance",
                              - "ecs:Submit*",
                              - "ecs:StartTelemetrySession",
                            ]
                          - Effect   = "Allow"
                          - Resource = [
                              - "*",
                            ]
                          - Sid      = "ecsInstanceRole"
                        },
                      - {
                          - Action   = [
                              - "logs:*",
                              - "cloudwatch:GenerateQuery",
                            ]
                          - Effect   = "Allow"
                          - Resource = "*"
                          - Sid      = "CloudWatchLogsFullAccess"
                        },
                      - {
                          - Action   = [
                              - "secretsmanager:GetResourcePolicy",
                              - "secretsmanager:GetSecretValue",
                              - "secretsmanager:DescribeSecret",
                              - "secretsmanager:ListSecretVersionIds",
                            ]
                          - Effect   = "Allow"
                          - Resource = "*"
                        },
                      - {
                          - Action   = "secretsmanager:ListSecrets"
                          - Effect   = "Allow"
                          - Resource = "*"
                        },
                      - {
                          - Action   = [
                              - "ec2:Describe*",
                              - "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
                              - "elasticloadbalancing:DeregisterTargets",
                              - "elasticloadbalancing:Describe*",
                              - "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
                              - "elasticloadbalancing:RegisterTargets",
                            ]
                          - Effect   = "Allow"
                          - Resource = "*"
                        },
                    ]
                  - Version   = "2012-10-17"
                }
            ) -> null
        }
    }

  # aws_iam_role_policy.ecs_service will be updated in-place
  ~ resource "aws_iam_role_policy" "ecs_service" {
        id     = "ooni_ecs_role:ooni_ecs_policy"
        name   = "ooni_ecs_policy"
      ~ policy = jsonencode(
          ~ {
              ~ Statement = [
                  - {
                      - Action   = [
                          - "ecs:DeregisterContainerInstance",
                          - "ecs:DiscoverPollEndpoint",
                          - "ecs:Poll",
                          - "ecs:RegisterContainerInstance",
                          - "ecs:Submit*",
                          - "ecs:StartTelemetrySession",
                        ]
                      - Effect   = "Allow"
                      - Resource = [
                          - "*",
                        ]
                      - Sid      = "ecsInstanceRole"
                    },
                  - {
                      - Action   = [
                          - "logs:*",
                          - "cloudwatch:GenerateQuery",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                      - Sid      = "CloudWatchLogsFullAccess"
                    },
                  - {
                      - Action   = [
                          - "secretsmanager:GetResourcePolicy",
                          - "secretsmanager:GetSecretValue",
                          - "secretsmanager:DescribeSecret",
                          - "secretsmanager:ListSecretVersionIds",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action   = "secretsmanager:ListSecrets"
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                    {
                        Action   = [
                            "ec2:Describe*",
                            "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
                            "elasticloadbalancing:DeregisterTargets",
                            "elasticloadbalancing:Describe*",
                            "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
                            "elasticloadbalancing:RegisterTargets",
                        ]
                        Effect   = "Allow"
                        Resource = "*"
                    },
                ]
                # (1 unchanged attribute hidden)
            }
        )
        # (1 unchanged attribute hidden)
    }

  # aws_iam_role_policy.ecs_task will be destroyed
  # (because aws_iam_role_policy.ecs_task is not in configuration)
  - resource "aws_iam_role_policy" "ecs_task" {
      - id     = "ooni_ecs_task_role:ooni_ecs_task_policy" -> null
      - name   = "ooni_ecs_task_policy" -> null
      - policy = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = [
                          - "ecs:DeregisterContainerInstance",
                          - "ecs:DiscoverPollEndpoint",
                          - "ecs:Poll",
                          - "ecs:RegisterContainerInstance",
                          - "ecs:Submit*",
                          - "ecs:StartTelemetrySession",
                        ]
                      - Effect   = "Allow"
                      - Resource = [
                          - "*",
                        ]
                      - Sid      = "ecsInstanceRole"
                    },
                  - {
                      - Action   = [
                          - "logs:*",
                          - "cloudwatch:GenerateQuery",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                      - Sid      = "CloudWatchLogsFullAccess"
                    },
                  - {
                      - Action   = [
                          - "secretsmanager:GetResourcePolicy",
                          - "secretsmanager:GetSecretValue",
                          - "secretsmanager:DescribeSecret",
                          - "secretsmanager:ListSecretVersionIds",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action   = "secretsmanager:ListSecrets"
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action   = [
                          - "ec2:Describe*",
                          - "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
                          - "elasticloadbalancing:DeregisterTargets",
                          - "elasticloadbalancing:Describe*",
                          - "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
                          - "elasticloadbalancing:RegisterTargets",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> null
      - role   = "ooni_ecs_task_role" -> null
    }

  # aws_iam_role_policy.instance will be updated in-place
  ~ resource "aws_iam_role_policy" "instance" {
        id     = "tf-ecs-ooni-instance-role:TfEcsOONIInstanceRole"
        name   = "TfEcsOONIInstanceRole"
      ~ policy = jsonencode(
          ~ {
              ~ Statement = [
                    # (1 unchanged element hidden)
                    {
                        Action   = [
                            "logs:*",
                            "cloudwatch:GenerateQuery",
                        ]
                        Effect   = "Allow"
                        Resource = "*"
                        Sid      = "CloudWatchLogsFullAccess"
                    },
                  - {
                      - Action   = [
                          - "secretsmanager:GetResourcePolicy",
                          - "secretsmanager:GetSecretValue",
                          - "secretsmanager:DescribeSecret",
                          - "secretsmanager:ListSecretVersionIds",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action   = "secretsmanager:ListSecrets"
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                  - {
                      - Action   = [
                          - "ec2:Describe*",
                          - "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
                          - "elasticloadbalancing:DeregisterTargets",
                          - "elasticloadbalancing:Describe*",
                          - "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
                          - "elasticloadbalancing:RegisterTargets",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                    },
                ]
                # (1 unchanged attribute hidden)
            }
        )
        # (1 unchanged attribute hidden)
    }

  # local_file.ansible_inventory will be created
  + resource "local_file" "ansible_inventory" {
      + content              = <<-EOT
            # Do not edit!
            # autogenerated by terraform in ooni/devops
            [all]
            
            [clickhouse_servers]
        EOT
      + content_base64sha256 = (known after apply)
      + content_base64sha512 = (known after apply)
      + content_md5          = (known after apply)
      + content_sha1         = (known after apply)
      + content_sha256       = (known after apply)
      + content_sha512       = (known after apply)
      + directory_permission = "0777"
      + file_permission      = "0777"
      + filename             = "./ansible/inventory.ini"
      + id                   = (known after apply)
    }

Plan: 2 to add, 3 to change, 3 to destroy.

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.


Pusher	@hellais
Action	pull_request
Working Directory	./tf/environments/prod
Workflow	.github/workflows/check_deploy.yml
Last updated	Mon, 26 Feb 2024 09:52:27 GMT

github-actions · 2024-02-23T10:44:10Z

Ansible Run Output 🤖

Ansible Playbook Recap 🔍


clickhouse.tier1.prod.ooni.nu : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0

Ansible playbook output 📖`success`

Show Execution


$ ansible-playbook playbook.yml --check --diff -i inventory.ini

PLAY [ClickHouse servers] ******************************************************

TASK [Gathering Facts] *********************************************************
fatal: [clickhouse.tier1.prod.ooni.nu]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname clickhouse.tier1.prod.ooni.nu: Name or service not known", "unreachable": true}

PLAY RECAP *********************************************************************
clickhouse.tier1.prod.ooni.nu : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0


Pusher	@hellais
Action	pull_request
Working Directory	./tf/environments/prod
Workflow	.github/workflows/check_deploy.yml
Last updated	Mon, 26 Feb 2024 09:53:39 GMT

hellais added 4 commits February 23, 2024 11:24

Rename production to prod

d8dae80

move clickhouse production into separate sub-module

9a53093

Comment out reference to clickhouse module

ddbe005

Fix paths to prod

25e0549

Bump dataapi version

4ed5ee5

hellais merged commit 43da7da into main Feb 23, 2024
2 checks passed

hellais deleted the prod-rename branch February 23, 2024 15:21

hellais changed the title ~~Prod rename~~ Prod rename and clickhouse module refactor Feb 26, 2024

hellais mentioned this pull request Feb 26, 2024

Start outlining the test environment #6

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prod rename and clickhouse module refactor #4

Prod rename and clickhouse module refactor #4

hellais commented Feb 23, 2024

github-actions bot commented Feb 23, 2024 •

edited

github-actions bot commented Feb 23, 2024 •

edited

Prod rename and clickhouse module refactor #4

Prod rename and clickhouse module refactor #4

Conversation

hellais commented Feb 23, 2024

github-actions bot commented Feb 23, 2024 • edited

Terraform Run Output 🤖

Format and Style 🖌success

Initialization ⚙️success

Validation 🤖success

Plan 📖success

github-actions bot commented Feb 23, 2024 • edited

Ansible Run Output 🤖

Ansible Playbook Recap 🔍

Ansible playbook output 📖success

github-actions bot commented Feb 23, 2024 •

edited

Format and Style 🖌`success`

Initialization ⚙️`success`

Validation 🤖`success`

Plan 📖`success`

github-actions bot commented Feb 23, 2024 •

edited

Ansible playbook output 📖`success`