Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Provision of resources in a popular cloud providers for PostgreSQL cluster deployment #464

Open
wants to merge 274 commits into
base: master
Choose a base branch
from
Open

Provision of resources in a popular cloud providers for PostgreSQL cluster deployment #464

wants to merge 274 commits into from

Conversation

vitabaks
Copy link
Owner

@vitabaks vitabaks commented Sep 2, 2023 •
edited

Issue: #458

  1. Provision of cloud resources (virtual machine + disk) to a popular cloud providers for PostgreSQL cluster deployment:

    • AWS
    • GCP
    • Azure
    • DigitalOcean
    • Hetzner Cloud

  2. Create a Cloud Load Balancer - entry point for connecting to a database in a cluster (if 'cloud_load_balancer' is 'true', default: true)

    • AWS Elastic Load Balancer (ELB) - external and internal
    • GCP Proxy Network Load Balancer - external only (TODO: internal)
    • Azure
    • DigitalOcean Load Balancer - external and internal
    • Hetzner Cloud Load Balancer - external and internal

  3. Create a Bucket in the cloud and configure backups to this bucket (if 'pgbackrest_install' or 'wal_g_instal' is 'true', default: false)

    • AWS S3 Bucket
    • GCP Bucket
    • Azure Blob Storage
    • DigitalOcean Spaces
    • Hetzner Cloud (?)

Role: cloud-resources

Examples:

AWS

export AWS_ACCESS_KEY_ID=********
export AWS_SECRET_ACCESS_KEY=********

ansible-playbook deploy_pgcluster.yml \
  --user=ubuntu --extra-vars \
    "cloud_provider=aws \
     cloud_load_balancer=true \
     server_count=3 \
     server_location=us-east-2 \
     server_type=m5.xlarge \
     server_image=ami-05fb0b8c1424f266b \
     volume_size=100 \
     ssh_public_keys=\"$(cat $HOME/.ssh/id_rsa.pub)\""

Note: "ami-05fb0b8c1424f266b" is Ubuntu 22.04 LTS (from 2023-12-07)

GCP

export GCP_SERVICE_ACCOUNT_CONTENTS='{
  ********
}'

ansible-playbook deploy_pgcluster.yml \
  --user=root --private-key=$HOME/.ssh/id_rsa --extra-vars \
    "cloud_provider='gcp' \
     server_count=3 \
     server_location='us-central1' \
     server_type='n2-standard-4' \
     server_image='projects/ubuntu-os-cloud/global/images/family/ubuntu-2204-lts' \
     volume_size=100 \
     ssh_key_content=\"$(cat $HOME/.ssh/id_rsa.pub)\""

Azure

export AZURE_SUBSCRIPTION_ID=********
export AZURE_CLIENT_ID=********
export AZURE_SECRET=********
export AZURE_TENANT=********

ansible-playbook deploy_pgcluster.yml \
  --user=azureadmin --private-key=$HOME/.ssh/id_rsa --extra-vars \
    "cloud_provider='azure' \
     server_count=3 \
     server_location='eastus' \
     server_type='Standard_D2s_v3' \
     volume_size=100 \
     ssh_key_content=\"$(cat $HOME/.ssh/id_rsa.pub)\""

Note: instead of 'server_image', use variables: azure_vm_image_offer, azure_vm_image_publisher, azure_vm_image_sku, azure_vm_image_version. Default: '0001-com-ubuntu-server-jammy', 'Canonical', '22_04-lts-gen2', 'latest'.

DigitalOcean

export DO_API_TOKEN=********

ansible-playbook deploy_pgcluster.yml \
  --user=root --extra-vars \
    "cloud_provider='digitalocean' \
     cloud_load_balancer=true \
     server_count=3 \
     server_location='nyc1' \
     server_type='g-4vcpu-16gb' \
     server_image='ubuntu-22-04-x64' \
     volume_size=100 \
     ssh_public_keys=\"$(cat $HOME/.ssh/id_rsa.pub)\""

Hetzner Cloud

export HCLOUD_API_TOKEN=********

ansible-playbook deploy_pgcluster.yml \
  --user=root --extra-vars \
    "cloud_provider='hetzner' \
     cloud_load_balancer=true \
     server_count=3 \
     server_location='ash' \
     server_type='CCX23' \
     server_image='ubuntu-22.04' \
     volume_size=100 \
     ssh_public_keys=\"$(cat $HOME/.ssh/id_rsa.pub)\""

Additionally:

  • Docker image 'vitabaks/postgresql_cluster'
    • includes repository code, ansible, and all necessary dependencies.
  • New roles:
    • authorized-keys
      • These SSH public keys will be added to the server's ~/.ssh/authorized_keys file.
      • variable: ssh_public_keys (optional, default '')
    • mount
      • Configure mount points in /etc/fstab and mount the file system
      • variables:
        • mount (optional, default '')
        • pg_data_mount_path (default '/pgdata')
        • pg_data_mount_fstype (default 'ext4')
      • an empty disk will be automatically detected for cloud providers
      • ZFS pool will be created if 'zfs' specified in the mount.fstype variable.
  • Automatically generate passwords for patroni_superuser_password, patroni_replication_password, pgbouncer_auth_password (if not defined)
    • and print superuser password in “connection info” after deployment
    • if the variable 'mask_password' is 'true' (default 'false'), do not print the superuser password in connection info
  • install perf, FlameGraph and postgres dbgsym/debuginfo packages
    • variable install_perf, default 'false'
  • Extension Auto-Setup
    • This feature simplifies the integration and configuration process for PostgreSQL third-party extensions. This automated approach eliminates the need for manual configuration, allowing users to seamlessly integrate PostgreSQL extensions. To activate the "Extension Auto-Setup", specify the enable_<extension_name>=true variable.
    • List of extensions: timescaledb, citus, pg_repack, pg_cron, pgaudit, pgvector, postgis, pgrouting, pg_stat_kcache, pg_wait_sampling, pg_partman
  • Add the ability to execute pre and post-deploy command
    • Variables: pre_deploy_command, post_deploy_command
    • This can be a direct command, a bash script content, or a path to a script on the host.

Consider sponsoring the project via GitHub or Patreon

@vitabaks vitabaks self-assigned this Sep 2, 2023
@vitabaks vitabaks changed the title Cloud Provision of cloud resources to a popular cloud providers for PostgreSQL cluster deployment Sep 2, 2023
@vitabaks vitabaks marked this pull request as draft September 2, 2023 18:31
@vitabaks
Copy link
Owner Author

vitabaks commented Sep 2, 2023 •
edited

Test: AWS

export AWS_ACCESS_KEY_ID=**********
export AWS_SECRET_ACCESS_KEY=**********

ansible-playbook deploy_pgcluster.yml \
  --user=ubuntu --private-key=$HOME/.ssh/Vitaliy.pem --extra-vars \
    "provision=aws \
     servers_count=3 \
     server_type=m5.large \
     server_image=ami-07d445a5585fce0a1 \
     server_location=us-east-2 \
     volume_size=60 \
     ssh_key_name=Vitaliy"

Result:

PLAY [Provision of cloud resources (virtual machine + disk) for PostgreSQL HA Cluster (on AWS)] ****************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Ensure that 'boto3' dependency is present on controlling host] *************************************************************************************************************************
ok: [localhost -> 127.0.0.1]

TASK [cloud-resources : AWS: Gather information about default VPC] *********************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : AWS: Gather information about VPC subnet for default VPC] ******************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Set variable: vpc_id] ******************************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Set variable: server_network] **********************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : AWS: Create or modify Security Group] **************************************************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : AWS: Create or modify EC2 instance] ****************************************************************************************************************************************************
changed: [localhost] => (item=postgres-cluster-pgnode01)
changed: [localhost] => (item=postgres-cluster-pgnode02)
changed: [localhost] => (item=postgres-cluster-pgnode03)

TASK [cloud-resources : Wait for EC2 instance to be available via SSH] *****************************************************************************************************************************************
ok: [localhost] => (item=18.223.205.113)
ok: [localhost] => (item=3.135.186.47)
ok: [localhost] => (item=3.144.48.18)

TASK [cloud-resources : Show EC2 instance info] ****************************************************************************************************************************************************************
ok: [localhost] => (item=18.223.205.113) => {
    "msg": [
        "instance ID is i-0215e0f507b416b43",
        "instance Image is ami-07d445a5585fce0a1",
        "instance Name is postgres-cluster-pgnode01",
        "instance Type is m5.large",
        "Block Storage Size is 60 gigabytes",
        "Public IP is 18.223.205.113",
        "Private IP is 172.31.0.73"
    ]
}
ok: [localhost] => (item=3.135.186.47) => {
    "msg": [
        "instance ID is i-0733c8a9411959a41",
        "instance Image is ami-07d445a5585fce0a1",
        "instance Name is postgres-cluster-pgnode02",
        "instance Type is m5.large",
        "Block Storage Size is 60 gigabytes",
        "Public IP is 3.135.186.47",
        "Private IP is 172.31.3.250"
    ]
}
ok: [localhost] => (item=3.144.48.18) => {
    "msg": [
        "instance ID is i-0eee7f56136793cf6",
        "instance Image is ami-07d445a5585fce0a1",
        "instance Name is postgres-cluster-pgnode03",
        "instance Type is m5.large",
        "Block Storage Size is 60 gigabytes",
        "Public IP is 3.144.48.18",
        "Private IP is 172.31.7.185"
    ]
}

TASK [cloud-resources : Add host to "postgres_cluster", "master" groups (in-memory inventory)] *****************************************************************************************************************
ok: [localhost] => (item=18.223.205.113)

TASK [cloud-resources : Add host to "postgres_cluster", "replica" groups (in-memory inventory)] ****************************************************************************************************************
ok: [localhost] => (item=3.135.186.47)
ok: [localhost] => (item=3.144.48.18)

TASK [cloud-resources : Add host to "etcd_cluster" group (in-memory inventory)] ********************************************************************************************************************************
ok: [localhost] => (item=18.223.205.113)
ok: [localhost] => (item=3.135.186.47)
ok: [localhost] => (item=3.144.48.18)

PLAY [Deploy PostgreSQL HA Cluster (based on "Patroni" and "etcd")] ********************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.0.73]
ok: [172.31.7.185]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [172.31.0.73]
ok: [172.31.3.250]
ok: [172.31.7.185]

TASK [System information] **************************************************************************************************************************************************************************************
ok: [172.31.0.73] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz, count: 1, cores: 1",
        "Disk space total": "77.75 GB",
        "Kernel": "5.15.0-1030-aws",
        "Memory": "7.56 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "m5.large",
        "Virtualization type": "kvm"
    }
}
ok: [172.31.3.250] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz, count: 1, cores: 1",
        "Disk space total": "77.75 GB",
        "Kernel": "5.15.0-1030-aws",
        "Memory": "7.47 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "m5.large",
        "Virtualization type": "kvm"
    }
}
ok: [172.31.7.185] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz, count: 1, cores: 1",
        "Disk space total": "77.75 GB",
        "Kernel": "5.15.0-1030-aws",
        "Memory": "7.56 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "m5.large",
        "Virtualization type": "kvm"
    }
}

TASK [pre-checks : Set max_connections from vars or use default] ***********************************************************************************************************************************************
ok: [172.31.0.73] => (item={'option': 'max_connections', 'value': '500'})

TASK [pre-checks : PgBouncer | Calculate pool_size] ************************************************************************************************************************************************************
ok: [172.31.0.73] => (item={'name': 'postgres', 'dbname': 'postgres', 'pool_parameters': ''})

TASK [pre-checks : PgBouncer | Calculate total pool_size] ******************************************************************************************************************************************************
ok: [172.31.0.73]

TASK [pre-checks : PgBouncer | Show total pool_size] ***********************************************************************************************************************************************************
ok: [172.31.0.73] => {
    "pgbouncer_total_pool_size": "20"
}

TASK [pre-checks : PostgreSQL | check that data directory "/var/lib/postgresql/15/main" is not initialized] ****************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.7.185]
ok: [172.31.0.73]

TASK [Update apt cache] ****************************************************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [Make sure the gnupg and apt-transport-https packages are present] ****************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [Make sure that the iproute is installed] *****************************************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.7.185]
ok: [172.31.0.73]

PLAY [Configure etcd Cluster and System Settings] **************************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [172.31.0.73]
ok: [172.31.3.250]
ok: [172.31.7.185]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [172.31.0.73]
ok: [172.31.3.250]
ok: [172.31.7.185]

TASK [Update apt cache] ****************************************************************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.7.185]
ok: [172.31.0.73]

TASK [Make sure the gnupg and apt-transport-https packages are present] ****************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.7.185]
ok: [172.31.0.73]

TASK [sysctl : Build a sysctl_conf dynamic variable] ***********************************************************************************************************************************************************
ok: [172.31.0.73] => (item=etcd_cluster)
ok: [172.31.0.73] => (item=master)
ok: [172.31.3.250] => (item=etcd_cluster)
ok: [172.31.0.73] => (item=postgres_cluster)
ok: [172.31.3.250] => (item=postgres_cluster)
ok: [172.31.7.185] => (item=etcd_cluster)
ok: [172.31.3.250] => (item=replica)
ok: [172.31.7.185] => (item=postgres_cluster)
ok: [172.31.7.185] => (item=replica)

TASK [sysctl : Setting kernel parameters] **********************************************************************************************************************************************************************
changed: [172.31.3.250] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [172.31.7.185] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [172.31.0.73] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [172.31.7.185] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [172.31.3.250] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [172.31.0.73] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [172.31.7.185] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [172.31.3.250] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [172.31.0.73] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [172.31.3.250] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [172.31.7.185] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [172.31.0.73] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [172.31.3.250] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [172.31.7.185] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [172.31.0.73] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [172.31.3.250] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [172.31.7.185] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [172.31.0.73] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [172.31.3.250] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [172.31.7.185] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [172.31.0.73] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [172.31.3.250] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [172.31.7.185] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [172.31.0.73] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [172.31.3.250] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [172.31.7.185] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [172.31.0.73] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [172.31.3.250] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [172.31.7.185] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [172.31.0.73] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [172.31.3.250] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [172.31.7.185] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [172.31.0.73] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [172.31.3.250] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [172.31.7.185] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [172.31.0.73] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [172.31.3.250] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [172.31.7.185] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [172.31.0.73] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [172.31.3.250] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [172.31.0.73] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [172.31.7.185] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [172.31.3.250] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [172.31.0.73] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [172.31.7.185] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [172.31.3.250] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
changed: [172.31.0.73] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
changed: [172.31.7.185] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})

TASK [etcd : Make sure the unzip/tar packages are present] *****************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [etcd : Download "etcd" package] **************************************************************************************************************************************************************************
changed: [172.31.0.73] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)
[WARNING]: Module remote_tmp /tmp/root/ansible did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct
permissions manually
changed: [172.31.3.250] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)
changed: [172.31.7.185] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)

TASK [etcd : Extract "etcd" into /tmp] *************************************************************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.7.185]
changed: [172.31.3.250]

TASK [etcd : Copy "etcd" and "etcdctl" binary files to /usr/local/bin/] ****************************************************************************************************************************************
changed: [172.31.3.250] => (item=etcd)
changed: [172.31.7.185] => (item=etcd)
changed: [172.31.0.73] => (item=etcd)
changed: [172.31.0.73] => (item=etcdctl)
changed: [172.31.3.250] => (item=etcdctl)
changed: [172.31.7.185] => (item=etcdctl)

TASK [etcd : Add etcd user] ************************************************************************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.3.250]
changed: [172.31.7.185]

TASK [etcd : Create etcd conf directory] ***********************************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [etcd : Create etcd data directory] ***********************************************************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.3.250]
changed: [172.31.7.185]

TASK [etcd : Generate conf file "/etc/etcd/etcd.conf"] *********************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

TASK [etcd : Copy systemd service file] ************************************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

TASK [etcd : Enable and start etcd service] ********************************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [etcd : Wait for port 2379 to become open on the host] ****************************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.0.73]
ok: [172.31.7.185]

TASK [etcd : Wait until the etcd cluster is healthy] ***********************************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.0.73]
ok: [172.31.7.185]

TASK [etcd : cluster health] ***********************************************************************************************************************************************************************************
ok: [172.31.0.73] => {
    "msg": "http://172.31.0.73:2379 is healthy: successfully committed proposal: took = 3.457788ms"
}
ok: [172.31.3.250] => {
    "msg": "http://172.31.3.250:2379 is healthy: successfully committed proposal: took = 3.650855ms"
}
ok: [172.31.7.185] => {
    "msg": "http://172.31.7.185:2379 is healthy: successfully committed proposal: took = 3.018372ms"
}

PLAY [consul.yml | Consul Playbook] ****************************************************************************************************************************************************************************

PLAY [consul.yml | Configure Consul instances] *****************************************************************************************************************************************************************
skipping: no hosts matched

PLAY [deploy_pgcluster.yml | Postgres Cluster Configuration] ***************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [172.31.0.73]
ok: [172.31.3.250]
ok: [172.31.7.185]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [172.31.0.73]
ok: [172.31.3.250]
ok: [172.31.7.185]

TASK [add-repository : Add repository apt-key] *****************************************************************************************************************************************************************
changed: [172.31.0.73] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})
changed: [172.31.3.250] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})
changed: [172.31.7.185] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})

TASK [add-repository : Add repository] *************************************************************************************************************************************************************************
changed: [172.31.3.250] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})
changed: [172.31.7.185] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})
changed: [172.31.0.73] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})

TASK [packages : Install system packages] **********************************************************************************************************************************************************************
ok: [172.31.3.250] => (item=['python3'])
ok: [172.31.7.185] => (item=['python3'])
ok: [172.31.0.73] => (item=['python3'])
ok: [172.31.3.250] => (item=python3)
ok: [172.31.7.185] => (item=python3)
ok: [172.31.0.73] => (item=python3)
changed: [172.31.7.185] => (item=python3-dev)
changed: [172.31.3.250] => (item=python3-dev)
changed: [172.31.0.73] => (item=python3-dev)
changed: [172.31.7.185] => (item=python3-psycopg2)
changed: [172.31.3.250] => (item=python3-psycopg2)
changed: [172.31.0.73] => (item=python3-psycopg2)
ok: [172.31.7.185] => (item=python3-setuptools)
ok: [172.31.3.250] => (item=python3-setuptools)
ok: [172.31.0.73] => (item=python3-setuptools)
changed: [172.31.3.250] => (item=python3-pip)
ok: [172.31.3.250] => (item=curl)
changed: [172.31.7.185] => (item=python3-pip)
ok: [172.31.3.250] => (item=less)
ok: [172.31.7.185] => (item=curl)
changed: [172.31.0.73] => (item=python3-pip)
ok: [172.31.3.250] => (item=sudo)
ok: [172.31.7.185] => (item=less)
ok: [172.31.0.73] => (item=curl)
ok: [172.31.3.250] => (item=vim)
ok: [172.31.7.185] => (item=sudo)
ok: [172.31.0.73] => (item=less)
ok: [172.31.3.250] => (item=gcc)
ok: [172.31.7.185] => (item=vim)
ok: [172.31.0.73] => (item=sudo)
ok: [172.31.7.185] => (item=gcc)
ok: [172.31.0.73] => (item=vim)
ok: [172.31.0.73] => (item=gcc)
changed: [172.31.3.250] => (item=jq)
ok: [172.31.3.250] => (item=iptables)
changed: [172.31.7.185] => (item=jq)
ok: [172.31.7.185] => (item=iptables)
changed: [172.31.0.73] => (item=jq)
ok: [172.31.0.73] => (item=iptables)
changed: [172.31.3.250] => (item=acl)
changed: [172.31.7.185] => (item=acl)
changed: [172.31.3.250] => (item=dnsutils)
changed: [172.31.0.73] => (item=acl)
changed: [172.31.7.185] => (item=dnsutils)
changed: [172.31.0.73] => (item=dnsutils)

TASK [packages : PostgreSQL | ensure postgresql database-cluster manager package] ******************************************************************************************************************************
changed: [172.31.7.185]
changed: [172.31.3.250]
changed: [172.31.0.73]

TASK [packages : PostgreSQL | disable initializing of a default postgresql cluster] ****************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

TASK [packages : PostgreSQL | disable log rotation with logrotate for postgresql] ******************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.7.185]
changed: [172.31.3.250]

TASK [packages : Install PostgreSQL packages] ******************************************************************************************************************************************************************
changed: [172.31.3.250] => (item=postgresql-15)
ok: [172.31.3.250] => (item=postgresql-client-15)
changed: [172.31.0.73] => (item=postgresql-15)
ok: [172.31.3.250] => (item=postgresql-contrib-15)
ok: [172.31.0.73] => (item=postgresql-client-15)
ok: [172.31.0.73] => (item=postgresql-contrib-15)
changed: [172.31.7.185] => (item=postgresql-15)
ok: [172.31.7.185] => (item=postgresql-client-15)
ok: [172.31.7.185] => (item=postgresql-contrib-15)
changed: [172.31.0.73] => (item=postgresql-server-dev-15)
changed: [172.31.3.250] => (item=postgresql-server-dev-15)
changed: [172.31.7.185] => (item=postgresql-server-dev-15)

TASK [sudo : Add user to /etc/sudoers.d/] **********************************************************************************************************************************************************************
changed: [172.31.3.250] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})
changed: [172.31.0.73] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})
changed: [172.31.7.185] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})

TASK [swap : Ensure swap exists] *******************************************************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.0.73]
ok: [172.31.7.185]

TASK [swap : Create swap file] *********************************************************************************************************************************************************************************
changed: [172.31.7.185]
changed: [172.31.0.73]
changed: [172.31.3.250]

TASK [swap : Set permissions on swap file] *********************************************************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.3.250]
changed: [172.31.7.185]

TASK [swap : Make swap file if necessary] **********************************************************************************************************************************************************************
changed: [172.31.7.185]
changed: [172.31.0.73]
changed: [172.31.3.250]

TASK [swap : Run swapon on the swap file] **********************************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

TASK [swap : Manage swap file entry in fstab] ******************************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

TASK [sysctl : Build a sysctl_conf dynamic variable] ***********************************************************************************************************************************************************
ok: [172.31.0.73] => (item=etcd_cluster)
ok: [172.31.0.73] => (item=master)
ok: [172.31.3.250] => (item=etcd_cluster)
ok: [172.31.0.73] => (item=postgres_cluster)
ok: [172.31.3.250] => (item=postgres_cluster)
ok: [172.31.7.185] => (item=etcd_cluster)
ok: [172.31.3.250] => (item=replica)
ok: [172.31.7.185] => (item=postgres_cluster)
ok: [172.31.7.185] => (item=replica)

TASK [sysctl : Setting kernel parameters] **********************************************************************************************************************************************************************
ok: [172.31.0.73] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [172.31.3.250] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [172.31.7.185] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [172.31.3.250] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [172.31.0.73] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [172.31.7.185] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [172.31.3.250] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [172.31.0.73] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [172.31.7.185] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [172.31.3.250] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [172.31.0.73] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [172.31.7.185] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [172.31.3.250] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [172.31.0.73] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [172.31.7.185] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [172.31.3.250] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [172.31.7.185] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [172.31.0.73] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [172.31.3.250] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [172.31.0.73] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [172.31.7.185] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [172.31.3.250] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [172.31.7.185] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [172.31.0.73] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [172.31.3.250] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [172.31.7.185] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [172.31.0.73] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [172.31.3.250] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [172.31.0.73] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [172.31.7.185] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [172.31.3.250] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [172.31.7.185] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [172.31.0.73] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [172.31.3.250] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [172.31.7.185] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [172.31.0.73] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [172.31.3.250] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [172.31.3.250] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [172.31.7.185] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [172.31.0.73] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [172.31.3.250] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [172.31.7.185] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [172.31.0.73] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [172.31.3.250] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
ok: [172.31.7.185] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [172.31.0.73] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [172.31.7.185] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
ok: [172.31.0.73] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})

TASK [transparent_huge_pages : Create systemd service "disable-transparent-huge-pages.service"] ****************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

RUNNING HANDLER [transparent_huge_pages : Start disable-transparent-huge-pages service] ************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.3.250]
changed: [172.31.7.185]

TASK [pam_limits : Linux PAM limits | add or modify nofile limits] *********************************************************************************************************************************************
changed: [172.31.0.73] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [172.31.3.250] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [172.31.7.185] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [172.31.3.250] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})
changed: [172.31.0.73] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})
changed: [172.31.7.185] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})

TASK [locales : Generate locales] ******************************************************************************************************************************************************************************
ok: [172.31.3.250] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})
ok: [172.31.0.73] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})
ok: [172.31.7.185] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})

TASK [locales : Set locale "en_US.utf-8" into /etc/default/locale] *********************************************************************************************************************************************
changed: [172.31.3.250] => (item=LANG=en_US.utf-8)
changed: [172.31.0.73] => (item=LANG=en_US.utf-8)
changed: [172.31.7.185] => (item=LANG=en_US.utf-8)
changed: [172.31.3.250] => (item=LANGUAGE=en_US.utf-8)
changed: [172.31.0.73] => (item=LANGUAGE=en_US.utf-8)
changed: [172.31.7.185] => (item=LANGUAGE=en_US.utf-8)
changed: [172.31.3.250] => (item=LC_ALL=en_US.utf-8)
changed: [172.31.0.73] => (item=LC_ALL=en_US.utf-8)
changed: [172.31.7.185] => (item=LC_ALL=en_US.utf-8)

PLAY [balancers.yml | Configure load balancers] ****************************************************************************************************************************************************************
skipping: no hosts matched

PLAY [deploy_pgcluster.yml | Install and configure pgBackRest] *************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.0.73]
ok: [172.31.7.185]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [172.31.0.73]
ok: [172.31.3.250]
ok: [172.31.7.185]

PLAY [deploy_pgcluster.yml | PostgreSQL Cluster Deployment] ****************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [172.31.0.73]
ok: [172.31.3.250]
ok: [172.31.7.185]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [172.31.0.73]
ok: [172.31.3.250]
ok: [172.31.7.185]

TASK [cron : Make sure that the cron package is installed] *****************************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.0.73]
ok: [172.31.7.185]

TASK [pgbouncer : Install pgbouncer package] *******************************************************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.7.185]
changed: [172.31.3.250]

TASK [pgbouncer : Ensure config directory "/etc/pgbouncer" exist] **********************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [pgbouncer : Check if pgbouncer systemd service file exists] **********************************************************************************************************************************************
ok: [172.31.7.185]
ok: [172.31.3.250]
ok: [172.31.0.73]

TASK [pgbouncer : Stop and disable standard init script] *******************************************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.7.185]
changed: [172.31.3.250]

TASK [pgbouncer : Configure pgbouncer systemd service file] ****************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [pgbouncer : Ensure pgbouncer service is enabled] *********************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

TASK [pgbouncer : Enable log rotation with logrotate] **********************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [pgbouncer : Configure pgbouncer.ini] *********************************************************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.7.185]
changed: [172.31.3.250]

TASK [pgpass : Configure a password file (/var/lib/postgresql/.pgpass)] ****************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

RUNNING HANDLER [pgbouncer : Restart pgbouncer service] ********************************************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.7.185]
changed: [172.31.3.250]

RUNNING HANDLER [pgbouncer : Wait for port "6432" to become open on the host] **********************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.0.73]
ok: [172.31.7.185]

RUNNING HANDLER [Restart pgbouncer service] ********************************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

RUNNING HANDLER [Wait for port "6432" to become open on the host] **********************************************************************************************************************************************
ok: [172.31.0.73]
ok: [172.31.7.185]
ok: [172.31.3.250]

TASK [patroni : Copy patroni requirements.txt file] ************************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

TASK [patroni : Install setuptools] ****************************************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [patroni : Install requirements] **************************************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [patroni : Install patroni] *******************************************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [patroni : Create conf directory] *************************************************************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.3.250]
changed: [172.31.7.185]

TASK [patroni : Generate conf file "/etc/patroni/patroni.yml"] *************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

TASK [patroni : Copy systemd service file "/etc/systemd/system/patroni.service"] *******************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

TASK [patroni : Prepare PostgreSQL | make sure the postgresql log directory "/var/log/postgresql" exists] ******************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.7.185]
changed: [172.31.3.250]

TASK [patroni : Prepare PostgreSQL | make sure PostgreSQL data directory "/var/lib/postgresql/15/main" exists] *************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.3.250]
changed: [172.31.7.185]

TASK [patroni : Prepare PostgreSQL | make sure the postgresql config files exists] *****************************************************************************************************************************
ok: [172.31.0.73]
ok: [172.31.3.250]
ok: [172.31.7.185]

TASK [patroni : Prepare PostgreSQL | generate default postgresql config files] *********************************************************************************************************************************
changed: [172.31.7.185]
changed: [172.31.0.73]
changed: [172.31.3.250]

TASK [patroni : Prepare PostgreSQL | make sure the data directory "/var/lib/postgresql/15/main" is empty] ******************************************************************************************************
changed: [172.31.0.73] => (item=absent)
changed: [172.31.3.250] => (item=absent)
changed: [172.31.7.185] => (item=absent)
changed: [172.31.3.250] => (item=directory)
changed: [172.31.0.73] => (item=directory)
changed: [172.31.7.185] => (item=directory)

TASK [patroni : Start patroni service on the Master server] ****************************************************************************************************************************************************
changed: [172.31.0.73]

TASK [patroni : Wait for port 8008 to become open on the host] *************************************************************************************************************************************************
ok: [172.31.0.73]

TASK [patroni : Check PostgreSQL is started and accepting connections on Master] *******************************************************************************************************************************
ok: [172.31.0.73]

TASK [patroni : Wait for the cluster to initialize (master is the leader with the lock)] ***********************************************************************************************************************
ok: [172.31.0.73]

TASK [patroni : Prepare PostgreSQL | generate pg_hba.conf] *****************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.0.73]
changed: [172.31.7.185]

TASK [patroni : Prepare PostgreSQL | reload for apply the pg_hba.conf] *****************************************************************************************************************************************
ok: [172.31.3.250]
changed: [172.31.0.73]
ok: [172.31.7.185]

TASK [patroni : Start patroni service on Replica servers] ******************************************************************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]

TASK [patroni : Wait for port 8008 to become open on the host] *************************************************************************************************************************************************
ok: [172.31.7.185]
ok: [172.31.3.250]

TASK [patroni : Check that the patroni is healthy on the replica server] ***************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.7.185]

TASK [patroni : Turning off postgresql autostart from config "start.conf" (will be managed by patroni)] ********************************************************************************************************
changed: [172.31.3.250]
changed: [172.31.7.185]
changed: [172.31.0.73]

TASK [patroni : Disable "postgresql@15-main" service] **********************************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.0.73]
ok: [172.31.7.185]

TASK [patroni : Add PATRONICTL_CONFIG_FILE environment variable into /etc/environment] *************************************************************************************************************************
changed: [172.31.0.73]
changed: [172.31.3.250]
changed: [172.31.7.185]

TASK [postgresql-users : Make sure the PostgreSQL users are present] *******************************************************************************************************************************************
changed: [172.31.0.73] => (item=pgbouncer)

TASK [pgbouncer/config : Ensure config directory "/etc/pgbouncer" exist] ***************************************************************************************************************************************
ok: [172.31.3.250]
ok: [172.31.0.73]
ok: [172.31.7.185]

TASK [pgbouncer/config : Update pgbouncer.ini] *****************************************************************************************************************************************************************
ok: [172.31.0.73]
ok: [172.31.3.250]
ok: [172.31.7.185]

TASK [pgbouncer/config : Create function 'user_search' for pgbouncer 'auth_query' option in all databases] *****************************************************************************************************
changed: [172.31.0.73]

TASK [deploy-finish : Get postgresql users list] ***************************************************************************************************************************************************************
ok: [172.31.0.73]

TASK [deploy-finish : PostgreSQL list of users] ****************************************************************************************************************************************************************
ok: [172.31.0.73] => {
    "users_result.stdout_lines": [
        "                                    List of roles",
        " Role name  |                         Attributes                         | Member of ",
        "------------+------------------------------------------------------------+-----------",
        " pgbouncer  |                                                            | {}",
        " postgres   | Superuser, Create role, Create DB, Replication, Bypass RLS | {}",
        " replicator | Replication                                                | {}"
    ]
}

TASK [deploy-finish : Get postgresql database list] ************************************************************************************************************************************************************
ok: [172.31.0.73]

TASK [deploy-finish : PostgreSQL list of databases] ************************************************************************************************************************************************************
ok: [172.31.0.73] => {
    "dbs_result.stdout_lines": [
        "   name   |  owner   | encoding |   collate   |    ctype    |  size   | tablespace ",
        "----------+----------+----------+-------------+-------------+---------+------------",
        " postgres | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 7453 kB | pg_default",
        "(1 row)"
    ]
}

TASK [deploy-finish : Check postgresql cluster health] *********************************************************************************************************************************************************
ok: [172.31.0.73]

TASK [deploy-finish : PostgreSQL Cluster health] ***************************************************************************************************************************************************************
ok: [172.31.0.73] => {
    "patronictl_result.stdout_lines": [
        "+ Cluster: postgres-cluster -----+---------+-----------+----+-----------+",
        "| Member          | Host         | Role    | State     | TL | Lag in MB |",
        "+-----------------+--------------+---------+-----------+----+-----------+",
        "| ip-172-31-0-73  | 172.31.0.73  | Leader  | running   |  1 |           |",
        "| ip-172-31-3-250 | 172.31.3.250 | Replica | streaming |  1 |         0 |",
        "| ip-172-31-7-185 | 172.31.7.185 | Replica | streaming |  1 |         0 |",
        "+-----------------+--------------+---------+-----------+----+-----------+"
    ]
}

TASK [deploy-finish : Create list of nodes] ********************************************************************************************************************************************************************
ok: [172.31.0.73]

TASK [deploy-finish : PostgreSQL Cluster connection info] ******************************************************************************************************************************************************
ok: [172.31.0.73] => {
    "msg": [
        "+------------------------------------------------+",
        "address 172.31.0.73,172.31.3.250,172.31.7.185",
        "port 6432 (pgbouncer)",
        "+------------------------------------------------+"
    ]
}

PLAY RECAP *****************************************************************************************************************************************************************************************************
172.31.0.73                : ok=104  changed=58   unreachable=0    failed=0    skipped=348  rescued=0    ignored=0   
172.31.3.250               : ok=89   changed=55   unreachable=0    failed=0    skipped=328  rescued=0    ignored=0   
172.31.7.185               : ok=89   changed=55   unreachable=0    failed=0    skipped=328  rescued=0    ignored=0   
localhost                  : ok=13   changed=2    unreachable=0    failed=0    skipped=18   rescued=0    ignored=0

image

passed

@vitabaks
Copy link
Owner Author

vitabaks commented Sep 3, 2023 •
edited

Test: GCP

export GCP_SERVICE_ACCOUNT_CONTENTS='{
  *****
}'

ansible-playbook deploy_pgcluster.yml \
  --user=root --private-key=$HOME/.ssh/id_rsa --extra-vars \
    "provision='gcp' \
     servers_count=5 \
     server_type='n2-standard-4' \
     server_image='projects/ubuntu-os-cloud/global/images/family/ubuntu-2204-lts' \
     server_location='us-central1' \
     volume_size=60 \
     ssh_key_content=\"$(cat $HOME/.ssh/id_rsa.pub)\" \
     patroni_cluster_name=vitaliy-test-pgcluster"

Result:

PLAY [Provision of cloud resources (virtual machine + disk) for PostgreSQL HA Cluster (on GCP)] ****************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Ensure that 'google-auth' dependency is present on controlling host] *******************************************************************************************************************
ok: [localhost -> 127.0.0.1]

TASK [cloud-resources : GCP: Gather information about project] *************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Set variable: gcp_network_name] ********************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : GCP: Create or modify private firewall rule] *******************************************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : GCP: Create or modify instance] ********************************************************************************************************************************************************
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode01)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode02)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode03)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode04)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode05)

TASK [cloud-resources : Wait for host to be available via SSH] *************************************************************************************************************************************************
ok: [localhost] => (item=34.70.223.37)
ok: [localhost] => (item=34.134.213.137)
ok: [localhost] => (item=34.66.129.241)
ok: [localhost] => (item=35.192.39.18)
ok: [localhost] => (item=35.223.176.129)

TASK [cloud-resources : Show GCP instance info] ****************************************************************************************************************************************************************
ok: [localhost] => (item=34.70.223.37) => {
    "msg": [
        "instance ID is 1335351888603344788",
        "instance Name is vitaliy-test-pgcluster-pgnode01",
        "instance Image is ubuntu-2204-lts",
        "instance Type is n2-standard-4",
        "Block Storage Size is 60 gigabytes",
        "Public IP is 34.70.223.37",
        "Private IP is 10.128.15.213"
    ]
}
ok: [localhost] => (item=34.134.213.137) => {
    "msg": [
        "instance ID is 8329019901739626345",
        "instance Name is vitaliy-test-pgcluster-pgnode02",
        "instance Image is ubuntu-2204-lts",
        "instance Type is n2-standard-4",
        "Block Storage Size is 60 gigabytes",
        "Public IP is 34.134.213.137",
        "Private IP is 10.128.15.216"
    ]
}
ok: [localhost] => (item=34.66.129.241) => {
    "msg": [
        "instance ID is 1438317652146220924",
        "instance Name is vitaliy-test-pgcluster-pgnode03",
        "instance Image is ubuntu-2204-lts",
        "instance Type is n2-standard-4",
        "Block Storage Size is 60 gigabytes",
        "Public IP is 34.66.129.241",
        "Private IP is 10.128.15.217"
    ]
}
ok: [localhost] => (item=35.192.39.18) => {
    "msg": [
        "instance ID is 7150671383286607695",
        "instance Name is vitaliy-test-pgcluster-pgnode04",
        "instance Image is ubuntu-2204-lts",
        "instance Type is n2-standard-4",
        "Block Storage Size is 60 gigabytes",
        "Public IP is 35.192.39.18",
        "Private IP is 10.128.15.218"
    ]
}
ok: [localhost] => (item=35.223.176.129) => {
    "msg": [
        "instance ID is 6973353061470793537",
        "instance Name is vitaliy-test-pgcluster-pgnode05",
        "instance Image is ubuntu-2204-lts",
        "instance Type is n2-standard-4",
        "Block Storage Size is 60 gigabytes",
        "Public IP is 35.223.176.129",
        "Private IP is 10.128.15.219"
    ]
}

TASK [cloud-resources : Inventory | Initialize ip_addresses variable] ******************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Inventory | Extract IP addresses] ******************************************************************************************************************************************************
ok: [localhost] => (item=public_ip: 34.70.223.37, private_ip: 10.128.15.213)
ok: [localhost] => (item=public_ip: 34.134.213.137, private_ip: 10.128.15.216)
ok: [localhost] => (item=public_ip: 34.66.129.241, private_ip: 10.128.15.217)
ok: [localhost] => (item=public_ip: 35.192.39.18, private_ip: 10.128.15.218)
ok: [localhost] => (item=public_ip: 35.223.176.129, private_ip: 10.128.15.219)

TASK [cloud-resources : Inventory | Add host to 'postgres_cluster', 'master' groups] ***************************************************************************************************************************
ok: [localhost] => (item=34.70.223.37)

TASK [cloud-resources : Inventory | Add host to 'postgres_cluster', 'replica' groups] **************************************************************************************************************************
ok: [localhost] => (item=34.134.213.137)
ok: [localhost] => (item=34.66.129.241)
ok: [localhost] => (item=35.192.39.18)
ok: [localhost] => (item=35.223.176.129)

TASK [cloud-resources : Inventory | Add host to 'etcd_cluster' group] ******************************************************************************************************************************************
ok: [localhost] => (item=34.70.223.37)
ok: [localhost] => (item=34.134.213.137)
ok: [localhost] => (item=34.66.129.241)
ok: [localhost] => (item=35.192.39.18)
ok: [localhost] => (item=35.223.176.129)

PLAY [Deploy PostgreSQL HA Cluster (based on "Patroni" and "etcd")] ********************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]
ok: [10.128.15.213]
ok: [10.128.15.216]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]

TASK [System information] **************************************************************************************************************************************************************************************
ok: [10.128.15.213] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "Intel(R) Xeon(R) CPU @ 2.80GHz, count: 1, cores: 2",
        "Disk space total": "78.0 GB",
        "Kernel": "6.2.0-1012-gcp",
        "Memory": "15.61 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "Google Compute Engine",
        "Virtualization type": "kvm"
    }
}
ok: [10.128.15.216] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "Intel(R) Xeon(R) CPU @ 2.80GHz, count: 1, cores: 2",
        "Disk space total": "78.0 GB",
        "Kernel": "6.2.0-1012-gcp",
        "Memory": "15.61 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "Google Compute Engine",
        "Virtualization type": "kvm"
    }
}
ok: [10.128.15.217] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "Intel(R) Xeon(R) CPU @ 2.80GHz, count: 1, cores: 2",
        "Disk space total": "78.0 GB",
        "Kernel": "6.2.0-1012-gcp",
        "Memory": "15.61 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "Google Compute Engine",
        "Virtualization type": "kvm"
    }
}
ok: [10.128.15.218] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "Intel(R) Xeon(R) CPU @ 2.80GHz, count: 1, cores: 2",
        "Disk space total": "78.0 GB",
        "Kernel": "6.2.0-1012-gcp",
        "Memory": "15.61 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "Google Compute Engine",
        "Virtualization type": "kvm"
    }
}
ok: [10.128.15.219] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "Intel(R) Xeon(R) CPU @ 2.80GHz, count: 1, cores: 2",
        "Disk space total": "78.0 GB",
        "Kernel": "6.2.0-1012-gcp",
        "Memory": "15.61 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "Google Compute Engine",
        "Virtualization type": "kvm"
    }
}

TASK [pre-checks : Set max_connections from vars or use default] ***********************************************************************************************************************************************
ok: [10.128.15.213] => (item={'option': 'max_connections', 'value': '500'})

TASK [pre-checks : PgBouncer | Calculate pool_size] ************************************************************************************************************************************************************
ok: [10.128.15.213] => (item={'name': 'postgres', 'dbname': 'postgres', 'pool_parameters': ''})

TASK [pre-checks : PgBouncer | Calculate total pool_size] ******************************************************************************************************************************************************
ok: [10.128.15.213]

TASK [pre-checks : PgBouncer | Show total pool_size] ***********************************************************************************************************************************************************
ok: [10.128.15.213] => {
    "pgbouncer_total_pool_size": "20"
}

TASK [pre-checks : PostgreSQL | check that data directory "/var/lib/postgresql/15/main" is not initialized] ****************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.219]
ok: [10.128.15.218]
ok: [10.128.15.216]
ok: [10.128.15.217]

TASK [Update apt cache] ****************************************************************************************************************************************************************************************
changed: [10.128.15.219]
changed: [10.128.15.213]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.216]

TASK [Make sure the gnupg and apt-transport-https packages are present] ****************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.219]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.216]

TASK [Make sure that the iproute is installed] *****************************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.219]
ok: [10.128.15.217]
ok: [10.128.15.218]

PLAY [Configure etcd Cluster and System Settings] **************************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]
ok: [10.128.15.213]
ok: [10.128.15.216]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]

TASK [Update apt cache] ****************************************************************************************************************************************************************************************
ok: [10.128.15.216]
ok: [10.128.15.219]
ok: [10.128.15.213]
ok: [10.128.15.218]
ok: [10.128.15.217]

TASK [Make sure the gnupg and apt-transport-https packages are present] ****************************************************************************************************************************************
ok: [10.128.15.218]
ok: [10.128.15.217]
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.219]

TASK [sysctl : Build a sysctl_conf dynamic variable] ***********************************************************************************************************************************************************
ok: [10.128.15.213] => (item=etcd_cluster)
ok: [10.128.15.213] => (item=master)
ok: [10.128.15.216] => (item=etcd_cluster)
ok: [10.128.15.213] => (item=postgres_cluster)
ok: [10.128.15.216] => (item=postgres_cluster)
ok: [10.128.15.217] => (item=etcd_cluster)
ok: [10.128.15.216] => (item=replica)
ok: [10.128.15.217] => (item=postgres_cluster)
ok: [10.128.15.218] => (item=etcd_cluster)
ok: [10.128.15.217] => (item=replica)
ok: [10.128.15.218] => (item=postgres_cluster)
ok: [10.128.15.219] => (item=etcd_cluster)
ok: [10.128.15.218] => (item=replica)
ok: [10.128.15.219] => (item=postgres_cluster)
ok: [10.128.15.219] => (item=replica)

TASK [sysctl : Setting kernel parameters] **********************************************************************************************************************************************************************
changed: [10.128.15.213] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [10.128.15.219] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [10.128.15.216] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [10.128.15.217] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [10.128.15.218] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [10.128.15.213] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [10.128.15.219] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [10.128.15.213] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [10.128.15.219] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [10.128.15.217] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [10.128.15.216] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [10.128.15.213] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [10.128.15.218] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [10.128.15.219] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [10.128.15.217] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [10.128.15.213] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [10.128.15.219] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [10.128.15.218] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [10.128.15.217] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [10.128.15.216] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [10.128.15.219] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [10.128.15.213] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [10.128.15.218] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [10.128.15.219] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [10.128.15.217] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [10.128.15.213] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [10.128.15.218] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [10.128.15.216] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [10.128.15.213] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [10.128.15.219] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [10.128.15.217] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [10.128.15.218] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [10.128.15.213] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [10.128.15.219] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [10.128.15.216] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [10.128.15.217] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [10.128.15.218] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [10.128.15.213] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [10.128.15.219] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [10.128.15.217] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [10.128.15.213] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [10.128.15.216] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [10.128.15.219] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [10.128.15.218] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [10.128.15.213] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [10.128.15.219] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [10.128.15.218] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [10.128.15.217] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [10.128.15.216] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [10.128.15.213] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [10.128.15.219] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [10.128.15.218] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [10.128.15.217] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [10.128.15.213] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [10.128.15.219] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [10.128.15.218] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [10.128.15.217] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [10.128.15.216] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [10.128.15.213] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [10.128.15.219] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [10.128.15.217] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [10.128.15.218] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [10.128.15.213] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
changed: [10.128.15.219] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
changed: [10.128.15.217] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [10.128.15.216] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [10.128.15.217] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [10.128.15.218] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [10.128.15.216] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [10.128.15.217] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [10.128.15.218] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [10.128.15.216] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [10.128.15.217] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
changed: [10.128.15.218] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [10.128.15.216] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [10.128.15.218] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
changed: [10.128.15.216] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [10.128.15.216] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [10.128.15.216] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [10.128.15.216] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})

TASK [etcd : Make sure the unzip/tar packages are present] *****************************************************************************************************************************************************
changed: [10.128.15.219]
changed: [10.128.15.213]
changed: [10.128.15.217]
changed: [10.128.15.216]
changed: [10.128.15.218]

TASK [etcd : Download "etcd" package] **************************************************************************************************************************************************************************
changed: [10.128.15.219] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)
changed: [10.128.15.217] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)
changed: [10.128.15.213] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)
[WARNING]: Module remote_tmp /tmp/root/ansible did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct
permissions manually
changed: [10.128.15.216] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)
changed: [10.128.15.218] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)

TASK [etcd : Extract "etcd" into /tmp] *************************************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.216]
changed: [10.128.15.219]

TASK [etcd : Copy "etcd" and "etcdctl" binary files to /usr/local/bin/] ****************************************************************************************************************************************
changed: [10.128.15.213] => (item=etcd)
changed: [10.128.15.217] => (item=etcd)
changed: [10.128.15.216] => (item=etcd)
changed: [10.128.15.218] => (item=etcd)
changed: [10.128.15.219] => (item=etcd)
changed: [10.128.15.213] => (item=etcdctl)
changed: [10.128.15.217] => (item=etcdctl)
changed: [10.128.15.216] => (item=etcdctl)
changed: [10.128.15.218] => (item=etcdctl)
changed: [10.128.15.219] => (item=etcdctl)

TASK [etcd : Add etcd user] ************************************************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.217]
changed: [10.128.15.216]
changed: [10.128.15.219]
changed: [10.128.15.218]

TASK [etcd : Create etcd conf directory] ***********************************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.216]

TASK [etcd : Create etcd data directory] ***********************************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.216]

TASK [etcd : Generate conf file "/etc/etcd/etcd.conf"] *********************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.216]

TASK [etcd : Copy systemd service file] ************************************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.217]
changed: [10.128.15.219]
changed: [10.128.15.216]
changed: [10.128.15.218]

TASK [etcd : Enable and start etcd service] ********************************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.213]
changed: [10.128.15.219]

TASK [etcd : Wait for port 2379 to become open on the host] ****************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.217]
ok: [10.128.15.216]
ok: [10.128.15.218]
ok: [10.128.15.219]

TASK [etcd : Wait until the etcd cluster is healthy] ***********************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.218]
ok: [10.128.15.217]
ok: [10.128.15.216]
ok: [10.128.15.219]

TASK [etcd : cluster health] ***********************************************************************************************************************************************************************************
ok: [10.128.15.213] => {
    "msg": "http://10.128.15.213:2379 is healthy: successfully committed proposal: took = 4.548685ms"
}
ok: [10.128.15.216] => {
    "msg": "http://10.128.15.216:2379 is healthy: successfully committed proposal: took = 3.359365ms"
}
ok: [10.128.15.217] => {
    "msg": "http://10.128.15.217:2379 is healthy: successfully committed proposal: took = 4.280164ms"
}
ok: [10.128.15.218] => {
    "msg": "http://10.128.15.218:2379 is healthy: successfully committed proposal: took = 3.928711ms"
}
ok: [10.128.15.219] => {
    "msg": "http://10.128.15.219:2379 is healthy: successfully committed proposal: took = 3.204371ms"
}

PLAY [consul.yml | Consul Playbook] ****************************************************************************************************************************************************************************

PLAY [consul.yml | Configure Consul instances] *****************************************************************************************************************************************************************
skipping: no hosts matched

PLAY [deploy_pgcluster.yml | Postgres Cluster Configuration] ***************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.217]
ok: [10.128.15.216]
ok: [10.128.15.218]
ok: [10.128.15.219]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]

TASK [add-repository : Add repository apt-key] *****************************************************************************************************************************************************************
changed: [10.128.15.213] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})
changed: [10.128.15.217] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})
changed: [10.128.15.219] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})
changed: [10.128.15.218] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})
changed: [10.128.15.216] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})

TASK [add-repository : Add repository] *************************************************************************************************************************************************************************
changed: [10.128.15.213] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})
changed: [10.128.15.217] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})
changed: [10.128.15.218] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})
changed: [10.128.15.216] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})
changed: [10.128.15.219] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})

TASK [packages : Install system packages] **********************************************************************************************************************************************************************
ok: [10.128.15.213] => (item=['python3'])
ok: [10.128.15.217] => (item=['python3'])
ok: [10.128.15.216] => (item=['python3'])
ok: [10.128.15.219] => (item=['python3'])
ok: [10.128.15.218] => (item=['python3'])
ok: [10.128.15.213] => (item=python3)
ok: [10.128.15.217] => (item=python3)
ok: [10.128.15.216] => (item=python3)
ok: [10.128.15.219] => (item=python3)
ok: [10.128.15.218] => (item=python3)
changed: [10.128.15.217] => (item=python3-dev)
changed: [10.128.15.219] => (item=python3-dev)
changed: [10.128.15.213] => (item=python3-dev)
changed: [10.128.15.218] => (item=python3-dev)
changed: [10.128.15.216] => (item=python3-dev)
changed: [10.128.15.217] => (item=python3-psycopg2)
changed: [10.128.15.219] => (item=python3-psycopg2)
ok: [10.128.15.217] => (item=python3-setuptools)
changed: [10.128.15.218] => (item=python3-psycopg2)
changed: [10.128.15.216] => (item=python3-psycopg2)
changed: [10.128.15.213] => (item=python3-psycopg2)
ok: [10.128.15.219] => (item=python3-setuptools)
ok: [10.128.15.218] => (item=python3-setuptools)
ok: [10.128.15.213] => (item=python3-setuptools)
ok: [10.128.15.216] => (item=python3-setuptools)
changed: [10.128.15.217] => (item=python3-pip)
changed: [10.128.15.219] => (item=python3-pip)
changed: [10.128.15.218] => (item=python3-pip)
changed: [10.128.15.213] => (item=python3-pip)
ok: [10.128.15.217] => (item=curl)
ok: [10.128.15.219] => (item=curl)
ok: [10.128.15.218] => (item=curl)
changed: [10.128.15.216] => (item=python3-pip)
ok: [10.128.15.213] => (item=curl)
ok: [10.128.15.217] => (item=less)
ok: [10.128.15.219] => (item=less)
ok: [10.128.15.218] => (item=less)
ok: [10.128.15.213] => (item=less)
ok: [10.128.15.216] => (item=curl)
ok: [10.128.15.217] => (item=sudo)
ok: [10.128.15.219] => (item=sudo)
ok: [10.128.15.218] => (item=sudo)
ok: [10.128.15.213] => (item=sudo)
ok: [10.128.15.216] => (item=less)
ok: [10.128.15.217] => (item=vim)
ok: [10.128.15.219] => (item=vim)
ok: [10.128.15.218] => (item=vim)
ok: [10.128.15.213] => (item=vim)
ok: [10.128.15.216] => (item=sudo)
ok: [10.128.15.217] => (item=gcc)
ok: [10.128.15.219] => (item=gcc)
ok: [10.128.15.218] => (item=gcc)
ok: [10.128.15.213] => (item=gcc)
ok: [10.128.15.216] => (item=vim)
ok: [10.128.15.216] => (item=gcc)
changed: [10.128.15.219] => (item=jq)
changed: [10.128.15.217] => (item=jq)
changed: [10.128.15.218] => (item=jq)
changed: [10.128.15.213] => (item=jq)
ok: [10.128.15.219] => (item=iptables)
ok: [10.128.15.217] => (item=iptables)
ok: [10.128.15.218] => (item=iptables)
ok: [10.128.15.213] => (item=iptables)
changed: [10.128.15.216] => (item=jq)
ok: [10.128.15.216] => (item=iptables)
changed: [10.128.15.219] => (item=acl)
changed: [10.128.15.217] => (item=acl)
changed: [10.128.15.213] => (item=acl)
changed: [10.128.15.218] => (item=acl)
changed: [10.128.15.216] => (item=acl)
changed: [10.128.15.219] => (item=dnsutils)
changed: [10.128.15.217] => (item=dnsutils)
changed: [10.128.15.213] => (item=dnsutils)
changed: [10.128.15.218] => (item=dnsutils)
changed: [10.128.15.216] => (item=dnsutils)

TASK [packages : PostgreSQL | ensure postgresql database-cluster manager package] ******************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.216]
changed: [10.128.15.217]

TASK [packages : PostgreSQL | disable initializing of a default postgresql cluster] ****************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.217]
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.219]

TASK [packages : PostgreSQL | disable log rotation with logrotate for postgresql] ******************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.219]

TASK [packages : Install PostgreSQL packages] ******************************************************************************************************************************************************************
changed: [10.128.15.213] => (item=postgresql-15)
changed: [10.128.15.216] => (item=postgresql-15)
changed: [10.128.15.217] => (item=postgresql-15)
changed: [10.128.15.218] => (item=postgresql-15)
ok: [10.128.15.213] => (item=postgresql-client-15)
ok: [10.128.15.216] => (item=postgresql-client-15)
changed: [10.128.15.219] => (item=postgresql-15)
ok: [10.128.15.217] => (item=postgresql-client-15)
ok: [10.128.15.218] => (item=postgresql-client-15)
ok: [10.128.15.213] => (item=postgresql-contrib-15)
ok: [10.128.15.216] => (item=postgresql-contrib-15)
ok: [10.128.15.219] => (item=postgresql-client-15)
ok: [10.128.15.217] => (item=postgresql-contrib-15)
ok: [10.128.15.218] => (item=postgresql-contrib-15)
ok: [10.128.15.219] => (item=postgresql-contrib-15)
changed: [10.128.15.216] => (item=postgresql-server-dev-15)
changed: [10.128.15.213] => (item=postgresql-server-dev-15)
changed: [10.128.15.218] => (item=postgresql-server-dev-15)
changed: [10.128.15.219] => (item=postgresql-server-dev-15)
changed: [10.128.15.217] => (item=postgresql-server-dev-15)

TASK [sudo : Add user to /etc/sudoers.d/] **********************************************************************************************************************************************************************
changed: [10.128.15.213] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})
changed: [10.128.15.216] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})
changed: [10.128.15.219] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})
changed: [10.128.15.218] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})
changed: [10.128.15.217] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})

TASK [swap : Ensure swap exists] *******************************************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]

TASK [swap : Create swap file] *********************************************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.219]

TASK [swap : Set permissions on swap file] *********************************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.219]

TASK [swap : Make swap file if necessary] **********************************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.219]

TASK [swap : Run swapon on the swap file] **********************************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.219]

TASK [swap : Manage swap file entry in fstab] ******************************************************************************************************************************************************************
changed: [10.128.15.219]
changed: [10.128.15.213]
changed: [10.128.15.218]
changed: [10.128.15.216]
changed: [10.128.15.217]

TASK [sysctl : Build a sysctl_conf dynamic variable] ***********************************************************************************************************************************************************
ok: [10.128.15.213] => (item=etcd_cluster)
ok: [10.128.15.213] => (item=master)
ok: [10.128.15.216] => (item=etcd_cluster)
ok: [10.128.15.213] => (item=postgres_cluster)
ok: [10.128.15.216] => (item=postgres_cluster)
ok: [10.128.15.217] => (item=etcd_cluster)
ok: [10.128.15.216] => (item=replica)
ok: [10.128.15.217] => (item=postgres_cluster)
ok: [10.128.15.218] => (item=etcd_cluster)
ok: [10.128.15.217] => (item=replica)
ok: [10.128.15.218] => (item=postgres_cluster)
ok: [10.128.15.218] => (item=replica)
ok: [10.128.15.219] => (item=etcd_cluster)
ok: [10.128.15.219] => (item=postgres_cluster)
ok: [10.128.15.219] => (item=replica)

TASK [sysctl : Setting kernel parameters] **********************************************************************************************************************************************************************
ok: [10.128.15.213] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [10.128.15.216] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [10.128.15.217] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [10.128.15.218] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [10.128.15.219] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [10.128.15.213] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [10.128.15.216] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [10.128.15.217] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [10.128.15.218] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [10.128.15.219] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [10.128.15.213] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [10.128.15.216] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [10.128.15.217] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [10.128.15.218] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [10.128.15.219] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [10.128.15.213] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [10.128.15.216] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [10.128.15.217] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [10.128.15.219] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [10.128.15.218] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [10.128.15.213] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [10.128.15.216] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [10.128.15.217] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [10.128.15.219] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [10.128.15.213] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [10.128.15.216] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [10.128.15.217] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [10.128.15.213] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [10.128.15.216] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [10.128.15.213] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [10.128.15.216] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [10.128.15.217] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [10.128.15.219] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [10.128.15.213] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [10.128.15.216] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [10.128.15.218] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [10.128.15.217] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [10.128.15.213] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [10.128.15.216] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [10.128.15.219] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [10.128.15.217] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [10.128.15.213] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [10.128.15.216] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [10.128.15.219] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [10.128.15.218] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [10.128.15.217] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [10.128.15.213] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [10.128.15.217] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [10.128.15.213] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [10.128.15.219] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [10.128.15.216] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [10.128.15.217] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [10.128.15.213] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [10.128.15.218] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [10.128.15.219] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [10.128.15.216] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [10.128.15.217] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [10.128.15.213] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [10.128.15.216] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [10.128.15.219] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [10.128.15.217] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [10.128.15.213] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
ok: [10.128.15.218] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [10.128.15.216] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [10.128.15.217] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [10.128.15.219] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [10.128.15.216] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
ok: [10.128.15.218] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [10.128.15.217] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
ok: [10.128.15.219] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [10.128.15.218] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [10.128.15.219] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [10.128.15.219] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [10.128.15.218] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [10.128.15.219] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
ok: [10.128.15.218] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [10.128.15.218] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [10.128.15.218] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [10.128.15.218] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [10.128.15.218] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})

TASK [transparent_huge_pages : Create systemd service "disable-transparent-huge-pages.service"] ****************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.218]
changed: [10.128.15.216]
changed: [10.128.15.219]
changed: [10.128.15.217]

RUNNING HANDLER [transparent_huge_pages : Start disable-transparent-huge-pages service] ************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.219]

TASK [pam_limits : Linux PAM limits | add or modify nofile limits] *********************************************************************************************************************************************
changed: [10.128.15.213] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [10.128.15.218] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [10.128.15.217] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [10.128.15.216] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [10.128.15.213] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})
changed: [10.128.15.218] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})
changed: [10.128.15.217] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})
changed: [10.128.15.216] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})
changed: [10.128.15.219] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [10.128.15.219] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})

TASK [locales : Generate locales] ******************************************************************************************************************************************************************************
ok: [10.128.15.213] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})
ok: [10.128.15.218] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})
ok: [10.128.15.216] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})
ok: [10.128.15.217] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})
ok: [10.128.15.219] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})

TASK [locales : Set locale "en_US.utf-8" into /etc/default/locale] *********************************************************************************************************************************************
changed: [10.128.15.213] => (item=LANG=en_US.utf-8)
changed: [10.128.15.218] => (item=LANG=en_US.utf-8)
changed: [10.128.15.216] => (item=LANG=en_US.utf-8)
changed: [10.128.15.217] => (item=LANG=en_US.utf-8)
changed: [10.128.15.213] => (item=LANGUAGE=en_US.utf-8)
changed: [10.128.15.219] => (item=LANG=en_US.utf-8)
changed: [10.128.15.218] => (item=LANGUAGE=en_US.utf-8)
changed: [10.128.15.216] => (item=LANGUAGE=en_US.utf-8)
changed: [10.128.15.217] => (item=LANGUAGE=en_US.utf-8)
changed: [10.128.15.213] => (item=LC_ALL=en_US.utf-8)
changed: [10.128.15.218] => (item=LC_ALL=en_US.utf-8)
changed: [10.128.15.216] => (item=LC_ALL=en_US.utf-8)
changed: [10.128.15.217] => (item=LC_ALL=en_US.utf-8)
changed: [10.128.15.219] => (item=LANGUAGE=en_US.utf-8)
changed: [10.128.15.219] => (item=LC_ALL=en_US.utf-8)

PLAY [balancers.yml | Configure load balancers] ****************************************************************************************************************************************************************
skipping: no hosts matched

PLAY [deploy_pgcluster.yml | Install and configure pgBackRest] *************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]

PLAY [deploy_pgcluster.yml | PostgreSQL Cluster Deployment] ****************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]

TASK [cron : Make sure that the cron package is installed] *****************************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]

TASK [pgbouncer : Install pgbouncer package] *******************************************************************************************************************************************************************
changed: [10.128.15.219]
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.213]
changed: [10.128.15.218]

TASK [pgbouncer : Ensure config directory "/etc/pgbouncer" exist] **********************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.219]

TASK [pgbouncer : Check if pgbouncer systemd service file exists] **********************************************************************************************************************************************
ok: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]

TASK [pgbouncer : Stop and disable standard init script] *******************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.217]
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.219]

TASK [pgbouncer : Configure pgbouncer systemd service file] ****************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.219]
changed: [10.128.15.213]

TASK [pgbouncer : Ensure pgbouncer service is enabled] *********************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.213]

TASK [pgbouncer : Enable log rotation with logrotate] **********************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.213]
changed: [10.128.15.219]

TASK [pgbouncer : Configure pgbouncer.ini] *********************************************************************************************************************************************************************
changed: [10.128.15.218]
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.219]
changed: [10.128.15.213]

TASK [pgpass : Configure a password file (/var/lib/postgresql/.pgpass)] ****************************************************************************************************************************************
changed: [10.128.15.218]
changed: [10.128.15.216]
changed: [10.128.15.219]
changed: [10.128.15.217]
changed: [10.128.15.213]

RUNNING HANDLER [pgbouncer : Restart pgbouncer service] ********************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.213]

RUNNING HANDLER [pgbouncer : Wait for port "6432" to become open on the host] **********************************************************************************************************************************
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]
ok: [10.128.15.213]

RUNNING HANDLER [Restart pgbouncer service] ********************************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.213]

RUNNING HANDLER [Wait for port "6432" to become open on the host] **********************************************************************************************************************************************
ok: [10.128.15.216]
ok: [10.128.15.217]
ok: [10.128.15.218]
ok: [10.128.15.219]
ok: [10.128.15.213]

TASK [patroni : Copy patroni requirements.txt file] ************************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.219]
changed: [10.128.15.213]

TASK [patroni : Install setuptools] ****************************************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.219]
changed: [10.128.15.213]

TASK [patroni : Install requirements] **************************************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.219]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.213]

TASK [patroni : Install patroni] *******************************************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.219]
changed: [10.128.15.218]
changed: [10.128.15.213]

TASK [patroni : Create conf directory] *************************************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.213]

TASK [patroni : Generate conf file "/etc/patroni/patroni.yml"] *************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.219]
changed: [10.128.15.218]
changed: [10.128.15.217]
changed: [10.128.15.213]

TASK [patroni : Copy systemd service file "/etc/systemd/system/patroni.service"] *******************************************************************************************************************************
changed: [10.128.15.218]
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.219]
changed: [10.128.15.213]

TASK [patroni : Prepare PostgreSQL | make sure the postgresql log directory "/var/log/postgresql" exists] ******************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.213]
changed: [10.128.15.217]
changed: [10.128.15.219]

TASK [patroni : Prepare PostgreSQL | make sure PostgreSQL data directory "/var/lib/postgresql/15/main" exists] *************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.213]
changed: [10.128.15.217]

TASK [patroni : Prepare PostgreSQL | make sure the postgresql config files exists] *****************************************************************************************************************************
ok: [10.128.15.216]
ok: [10.128.15.218]
ok: [10.128.15.219]
ok: [10.128.15.217]
ok: [10.128.15.213]

TASK [patroni : Prepare PostgreSQL | generate default postgresql config files] *********************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.217]
changed: [10.128.15.213]

TASK [patroni : Prepare PostgreSQL | make sure the data directory "/var/lib/postgresql/15/main" is empty] ******************************************************************************************************
changed: [10.128.15.216] => (item=absent)
changed: [10.128.15.218] => (item=absent)
changed: [10.128.15.219] => (item=absent)
changed: [10.128.15.216] => (item=directory)
changed: [10.128.15.218] => (item=directory)
changed: [10.128.15.219] => (item=directory)
changed: [10.128.15.213] => (item=absent)
changed: [10.128.15.217] => (item=absent)
changed: [10.128.15.213] => (item=directory)
changed: [10.128.15.217] => (item=directory)

TASK [patroni : Start patroni service on the Master server] ****************************************************************************************************************************************************
changed: [10.128.15.213]

TASK [patroni : Wait for port 8008 to become open on the host] *************************************************************************************************************************************************
ok: [10.128.15.213]

TASK [patroni : Check PostgreSQL is started and accepting connections on Master] *******************************************************************************************************************************
ok: [10.128.15.213]

TASK [patroni : Wait for the cluster to initialize (master is the leader with the lock)] ***********************************************************************************************************************
ok: [10.128.15.213]

TASK [patroni : Prepare PostgreSQL | generate pg_hba.conf] *****************************************************************************************************************************************************
changed: [10.128.15.213]
changed: [10.128.15.218]
changed: [10.128.15.216]
changed: [10.128.15.217]
changed: [10.128.15.219]

TASK [patroni : Prepare PostgreSQL | reload for apply the pg_hba.conf] *****************************************************************************************************************************************
changed: [10.128.15.213]
ok: [10.128.15.216]
ok: [10.128.15.218]
ok: [10.128.15.217]
ok: [10.128.15.219]

TASK [patroni : Start patroni service on Replica servers] ******************************************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.217]

TASK [patroni : Wait for port 8008 to become open on the host] *************************************************************************************************************************************************
ok: [10.128.15.219]
ok: [10.128.15.218]
ok: [10.128.15.216]
ok: [10.128.15.217]

TASK [patroni : Check that the patroni is healthy on the replica server] ***************************************************************************************************************************************
ok: [10.128.15.216]
ok: [10.128.15.218]
ok: [10.128.15.219]
ok: [10.128.15.217]

TASK [patroni : Turning off postgresql autostart from config "start.conf" (will be managed by patroni)] ********************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.213]
changed: [10.128.15.217]

TASK [patroni : Disable "postgresql@15-main" service] **********************************************************************************************************************************************************
ok: [10.128.15.216]
ok: [10.128.15.219]
ok: [10.128.15.218]
ok: [10.128.15.213]
ok: [10.128.15.217]

TASK [patroni : Add PATRONICTL_CONFIG_FILE environment variable into /etc/environment] *************************************************************************************************************************
changed: [10.128.15.216]
changed: [10.128.15.218]
changed: [10.128.15.219]
changed: [10.128.15.213]
changed: [10.128.15.217]

TASK [postgresql-users : Make sure the PostgreSQL users are present] *******************************************************************************************************************************************
changed: [10.128.15.213] => (item=pgbouncer)

TASK [pgbouncer/config : Ensure config directory "/etc/pgbouncer" exist] ***************************************************************************************************************************************
ok: [10.128.15.216]
ok: [10.128.15.218]
ok: [10.128.15.213]
ok: [10.128.15.217]
ok: [10.128.15.219]

TASK [pgbouncer/config : Update pgbouncer.ini] *****************************************************************************************************************************************************************
ok: [10.128.15.216]
ok: [10.128.15.218]
ok: [10.128.15.219]
ok: [10.128.15.217]
ok: [10.128.15.213]

TASK [pgbouncer/config : Create function 'user_search' for pgbouncer 'auth_query' option in all databases] *****************************************************************************************************
changed: [10.128.15.213]

TASK [deploy-finish : Get postgresql users list] ***************************************************************************************************************************************************************
ok: [10.128.15.213]

TASK [deploy-finish : PostgreSQL list of users] ****************************************************************************************************************************************************************
ok: [10.128.15.213] => {
    "users_result.stdout_lines": [
        "                                    List of roles",
        " Role name  |                         Attributes                         | Member of ",
        "------------+------------------------------------------------------------+-----------",
        " pgbouncer  |                                                            | {}",
        " postgres   | Superuser, Create role, Create DB, Replication, Bypass RLS | {}",
        " replicator | Replication                                                | {}"
    ]
}

TASK [deploy-finish : Get postgresql database list] ************************************************************************************************************************************************************
ok: [10.128.15.213]

TASK [deploy-finish : PostgreSQL list of databases] ************************************************************************************************************************************************************
ok: [10.128.15.213] => {
    "dbs_result.stdout_lines": [
        "   name   |  owner   | encoding |   collate   |    ctype    |  size   | tablespace ",
        "----------+----------+----------+-------------+-------------+---------+------------",
        " postgres | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 7453 kB | pg_default",
        "(1 row)"
    ]
}

TASK [deploy-finish : Check postgresql cluster health] *********************************************************************************************************************************************************
ok: [10.128.15.213]

TASK [deploy-finish : PostgreSQL Cluster health] ***************************************************************************************************************************************************************
ok: [10.128.15.213] => {
    "patronictl_result.stdout_lines": [
        "+ Cluster: vitaliy-test-pgcluster +---------------+---------+-----------+----+-----------+",
        "| Member                          | Host          | Role    | State     | TL | Lag in MB |",
        "+---------------------------------+---------------+---------+-----------+----+-----------+",
        "| vitaliy-test-pgcluster-pgnode01 | 10.128.15.213 | Leader  | running   |  1 |           |",
        "| vitaliy-test-pgcluster-pgnode02 | 10.128.15.216 | Replica | streaming |  1 |         0 |",
        "| vitaliy-test-pgcluster-pgnode03 | 10.128.15.217 | Replica | streaming |  1 |         0 |",
        "| vitaliy-test-pgcluster-pgnode04 | 10.128.15.218 | Replica | streaming |  1 |         0 |",
        "| vitaliy-test-pgcluster-pgnode05 | 10.128.15.219 | Replica | streaming |  1 |         0 |",
        "+---------------------------------+---------------+---------+-----------+----+-----------+"
    ]
}

TASK [deploy-finish : Create list of nodes] ********************************************************************************************************************************************************************
ok: [10.128.15.213]

TASK [deploy-finish : PostgreSQL Cluster connection info] ******************************************************************************************************************************************************
ok: [10.128.15.213] => {
    "msg": [
        "+------------------------------------------------+",
        "address 10.128.15.213,10.128.15.216,10.128.15.217,10.128.15.218,10.128.15.219",
        "port 6432 (pgbouncer)",
        "+------------------------------------------------+"
    ]
}

PLAY RECAP *****************************************************************************************************************************************************************************************************
10.128.15.213              : ok=104  changed=58   unreachable=0    failed=0    skipped=348  rescued=0    ignored=0   
10.128.15.216              : ok=89   changed=55   unreachable=0    failed=0    skipped=328  rescued=0    ignored=0   
10.128.15.217              : ok=89   changed=55   unreachable=0    failed=0    skipped=328  rescued=0    ignored=0   
10.128.15.218              : ok=89   changed=55   unreachable=0    failed=0    skipped=328  rescued=0    ignored=0   
10.128.15.219              : ok=89   changed=55   unreachable=0    failed=0    skipped=328  rescued=0    ignored=0   
localhost                  : ok=13   changed=2    unreachable=0    failed=0    skipped=41   rescued=0    ignored=0

image

passed

@vitabaks
Copy link
Owner Author

vitabaks commented Sep 10, 2023 •
edited

Test: DigitalOcean

export DO_API_TOKEN=********

ansible-playbook deploy_pgcluster.yml \
  --user=root --private-key=$HOME/.ssh/id_rsa --extra-vars \
    "provision='digitalocean' \
     servers_count=3 \
     server_type='g-4vcpu-16gb' \
     server_image='ubuntu-22-04-x64' \
     server_location='nyc1' \
     volume_size=100 \
     ssh_key_name=vitaliy \
     patroni_cluster_name=vitaliy-test-pgcluster"

Result:

PLAY [Provision of cloud resources (virtual machine + disk) for PostgreSQL HA Cluster (on DIGITALOCEAN)] *******************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Ensure that 'dopy' dependency is present on controlling host] **************************************************************************************************************************
ok: [localhost -> 127.0.0.1]

TASK [cloud-resources : DigitalOcean: Gather information about SSH keys] ***************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : DigitalOcean: Get Fingerprint for SSH key 'vitaliy'] ***********************************************************************************************************************************
ok: [localhost] => (item=vitaliy)

TASK [cloud-resources : DigitalOcean: Gather information about VPC] ********************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Extract ip_range from default VPC] *****************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : DigitalOcean: Create a tag 'vitaliy-test-pgcluster'] ***********************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : DigitalOcean: Create or modify SSH firewall rule] **************************************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : DigitalOcean: Create or modify private firewall rule] **********************************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : DigitalOcean: Create or modify Droplet] ************************************************************************************************************************************************
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode01)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode02)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode03)

TASK [cloud-resources : Set variable: droplet_result] **********************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : DigitalOcean: Create or modify Block Storage] ******************************************************************************************************************************************
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode01-storage)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode02-storage)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode03-storage)

TASK [cloud-resources : DigitalOcean: Attach Block Storage to Droplet] *****************************************************************************************************************************************
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode01-storage)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode02-storage)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode03-storage)

TASK [cloud-resources : Wait for host to be available via SSH] *************************************************************************************************************************************************
ok: [localhost] => (item=198.199.90.240)
ok: [localhost] => (item=167.99.228.9)
ok: [localhost] => (item=159.89.84.26)

TASK [cloud-resources : Show Droplet info] *********************************************************************************************************************************************************************
ok: [localhost] => (item=198.199.90.240) => {
    "msg": [
        "ID: 374097982",
        "Name: vitaliy-test-pgcluster-pgnode01",
        "Image: Ubuntu 22.04 (LTS) x64",
        "Type: g-4vcpu-16gb",
        "Volume Size: 100 GB",
        "Public IP: 198.199.90.240",
        "Private IP: 10.116.0.2"
    ]
}
ok: [localhost] => (item=167.99.228.9) => {
    "msg": [
        "ID: 374098039",
        "Name: vitaliy-test-pgcluster-pgnode02",
        "Image: Ubuntu 22.04 (LTS) x64",
        "Type: g-4vcpu-16gb",
        "Volume Size: 100 GB",
        "Public IP: 167.99.228.9",
        "Private IP: 10.116.0.3"
    ]
}
ok: [localhost] => (item=159.89.84.26) => {
    "msg": [
        "ID: 374098074",
        "Name: vitaliy-test-pgcluster-pgnode03",
        "Image: Ubuntu 22.04 (LTS) x64",
        "Type: g-4vcpu-16gb",
        "Volume Size: 100 GB",
        "Public IP: 159.89.84.26",
        "Private IP: 10.116.0.4"
    ]
}

TASK [cloud-resources : Inventory | Initialize ip_addresses variable] ******************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Inventory | Extract IP addresses] ******************************************************************************************************************************************************
ok: [localhost] => (item=public_ip: 198.199.90.240, private_ip: 10.116.0.2)
ok: [localhost] => (item=public_ip: 167.99.228.9, private_ip: 10.116.0.3)
ok: [localhost] => (item=public_ip: 159.89.84.26, private_ip: 10.116.0.4)

TASK [cloud-resources : Inventory | Add host to 'postgres_cluster', 'master' groups] ***************************************************************************************************************************
ok: [localhost] => (item=198.199.90.240)

TASK [cloud-resources : Inventory | Add host to 'postgres_cluster', 'replica' groups] **************************************************************************************************************************
ok: [localhost] => (item=167.99.228.9)
ok: [localhost] => (item=159.89.84.26)

TASK [cloud-resources : Inventory | Add host to 'etcd_cluster' group] ******************************************************************************************************************************************
ok: [localhost] => (item=198.199.90.240)
ok: [localhost] => (item=167.99.228.9)
ok: [localhost] => (item=159.89.84.26)

PLAY [Deploy PostgreSQL HA Cluster (based on "Patroni" and "etcd")] ********************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.3]
ok: [10.116.0.2]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.116.0.2]
ok: [10.116.0.3]
ok: [10.116.0.4]

TASK [System information] **************************************************************************************************************************************************************************************
ok: [10.116.0.2] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz, count: 1, cores: 4",
        "Disk space total": "48.6 GB",
        "Kernel": "5.15.0-67-generic",
        "Memory": "15.63 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "Droplet",
        "Virtualization type": "kvm"
    }
}
ok: [10.116.0.3] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz, count: 1, cores: 4",
        "Disk space total": "48.6 GB",
        "Kernel": "5.15.0-67-generic",
        "Memory": "15.63 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "Droplet",
        "Virtualization type": "kvm"
    }
}
ok: [10.116.0.4] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz, count: 1, cores: 4",
        "Disk space total": "48.6 GB",
        "Kernel": "5.15.0-67-generic",
        "Memory": "15.63 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "Droplet",
        "Virtualization type": "kvm"
    }
}

TASK [pre-checks : Set max_connections from vars or use default] ***********************************************************************************************************************************************
ok: [10.116.0.2] => (item={'option': 'max_connections', 'value': '500'})

TASK [pre-checks : PgBouncer | Calculate pool_size] ************************************************************************************************************************************************************
ok: [10.116.0.2] => (item={'name': 'postgres', 'dbname': 'postgres', 'pool_parameters': ''})

TASK [pre-checks : PgBouncer | Calculate total pool_size] ******************************************************************************************************************************************************
ok: [10.116.0.2]

TASK [pre-checks : PgBouncer | Show total pool_size] ***********************************************************************************************************************************************************
ok: [10.116.0.2] => {
    "pgbouncer_total_pool_size": "20"
}

TASK [pre-checks : PostgreSQL | check that data directory "/var/lib/postgresql/15/main" is not initialized] ****************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.2]
ok: [10.116.0.3]

TASK [Update apt cache] ****************************************************************************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.2]
ok: [10.116.0.3]

TASK [Make sure the gnupg and apt-transport-https packages are present] ****************************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.3]
ok: [10.116.0.2]

TASK [Make sure that the iproute is installed] *****************************************************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.3]
ok: [10.116.0.2]

PLAY [Configure etcd Cluster and System Settings] **************************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.2]
ok: [10.116.0.3]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.116.0.2]
ok: [10.116.0.3]
ok: [10.116.0.4]

TASK [Update apt cache] ****************************************************************************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.2]
ok: [10.116.0.3]

TASK [Make sure the gnupg and apt-transport-https packages are present] ****************************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.3]
ok: [10.116.0.2]

TASK [sysctl : Build a sysctl_conf dynamic variable] ***********************************************************************************************************************************************************
ok: [10.116.0.2] => (item=etcd_cluster)
ok: [10.116.0.2] => (item=master)
ok: [10.116.0.3] => (item=etcd_cluster)
ok: [10.116.0.2] => (item=postgres_cluster)
ok: [10.116.0.3] => (item=postgres_cluster)
ok: [10.116.0.4] => (item=etcd_cluster)
ok: [10.116.0.3] => (item=replica)
ok: [10.116.0.4] => (item=postgres_cluster)
ok: [10.116.0.4] => (item=replica)

TASK [sysctl : Setting kernel parameters] **********************************************************************************************************************************************************************
changed: [10.116.0.4] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [10.116.0.3] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [10.116.0.4] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [10.116.0.3] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [10.116.0.2] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [10.116.0.4] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [10.116.0.3] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [10.116.0.2] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [10.116.0.4] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [10.116.0.3] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [10.116.0.2] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [10.116.0.4] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [10.116.0.3] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [10.116.0.2] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [10.116.0.4] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [10.116.0.3] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [10.116.0.2] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [10.116.0.4] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [10.116.0.3] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [10.116.0.4] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [10.116.0.2] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [10.116.0.3] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [10.116.0.4] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [10.116.0.2] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [10.116.0.3] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [10.116.0.4] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [10.116.0.3] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [10.116.0.2] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [10.116.0.4] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [10.116.0.3] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [10.116.0.2] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [10.116.0.4] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [10.116.0.3] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [10.116.0.2] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [10.116.0.4] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [10.116.0.2] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [10.116.0.3] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [10.116.0.4] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [10.116.0.2] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [10.116.0.3] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [10.116.0.4] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [10.116.0.2] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [10.116.0.3] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [10.116.0.4] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
changed: [10.116.0.2] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [10.116.0.3] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
changed: [10.116.0.2] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [10.116.0.2] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})

TASK [etcd : Make sure the unzip/tar packages are present] *****************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.3]
changed: [10.116.0.2]

TASK [etcd : Download "etcd" package] **************************************************************************************************************************************************************************
changed: [10.116.0.4] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)
[WARNING]: Module remote_tmp /tmp/root/ansible did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct
permissions manually
changed: [10.116.0.3] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)
changed: [10.116.0.2] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)

TASK [etcd : Extract "etcd" into /tmp] *************************************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.3]
changed: [10.116.0.2]

TASK [etcd : Copy "etcd" and "etcdctl" binary files to /usr/local/bin/] ****************************************************************************************************************************************
changed: [10.116.0.4] => (item=etcd)
changed: [10.116.0.3] => (item=etcd)
changed: [10.116.0.2] => (item=etcd)
changed: [10.116.0.4] => (item=etcdctl)
changed: [10.116.0.2] => (item=etcdctl)
changed: [10.116.0.3] => (item=etcdctl)

TASK [etcd : Add etcd user] ************************************************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.4]
changed: [10.116.0.3]

TASK [etcd : Create etcd conf directory] ***********************************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.4]
changed: [10.116.0.3]

TASK [etcd : Create etcd data directory] ***********************************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.3]
changed: [10.116.0.4]

TASK [etcd : Generate conf file "/etc/etcd/etcd.conf"] *********************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.3]
changed: [10.116.0.4]

TASK [etcd : Copy systemd service file] ************************************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.3]
changed: [10.116.0.2]

TASK [etcd : Enable and start etcd service] ********************************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.4]
changed: [10.116.0.3]

TASK [etcd : Wait for port 2379 to become open on the host] ****************************************************************************************************************************************************
ok: [10.116.0.3]
ok: [10.116.0.4]
ok: [10.116.0.2]

TASK [etcd : Wait until the etcd cluster is healthy] ***********************************************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.3]
ok: [10.116.0.2]

TASK [etcd : cluster health] ***********************************************************************************************************************************************************************************
ok: [10.116.0.2] => {
    "msg": "http://10.116.0.2:2379 is healthy: successfully committed proposal: took = 5.598689ms"
}
ok: [10.116.0.3] => {
    "msg": "http://10.116.0.3:2379 is healthy: successfully committed proposal: took = 4.467634ms"
}
ok: [10.116.0.4] => {
    "msg": "http://10.116.0.4:2379 is healthy: successfully committed proposal: took = 6.142228ms"
}

PLAY [consul.yml | Consul Playbook] ****************************************************************************************************************************************************************************

PLAY [consul.yml | Configure Consul instances] *****************************************************************************************************************************************************************
skipping: no hosts matched

PLAY [deploy_pgcluster.yml | Postgres Cluster Configuration] ***************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.116.0.3]
ok: [10.116.0.4]
ok: [10.116.0.2]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.116.0.2]
ok: [10.116.0.3]
ok: [10.116.0.4]

TASK [add-repository : Add repository apt-key] *****************************************************************************************************************************************************************
changed: [10.116.0.3] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})
changed: [10.116.0.4] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})
changed: [10.116.0.2] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})

TASK [add-repository : Add repository] *************************************************************************************************************************************************************************
changed: [10.116.0.4] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})
changed: [10.116.0.3] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})
changed: [10.116.0.2] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})

TASK [packages : Install system packages] **********************************************************************************************************************************************************************
ok: [10.116.0.2] => (item=['python3'])
ok: [10.116.0.4] => (item=['python3'])
ok: [10.116.0.3] => (item=['python3'])
ok: [10.116.0.4] => (item=python3)
ok: [10.116.0.3] => (item=python3)
ok: [10.116.0.2] => (item=python3)
changed: [10.116.0.4] => (item=python3-dev)
changed: [10.116.0.3] => (item=python3-dev)
changed: [10.116.0.4] => (item=python3-psycopg2)
changed: [10.116.0.2] => (item=python3-dev)
ok: [10.116.0.4] => (item=python3-setuptools)
changed: [10.116.0.3] => (item=python3-psycopg2)
ok: [10.116.0.3] => (item=python3-setuptools)
changed: [10.116.0.2] => (item=python3-psycopg2)
changed: [10.116.0.4] => (item=python3-pip)
ok: [10.116.0.2] => (item=python3-setuptools)
ok: [10.116.0.4] => (item=curl)
ok: [10.116.0.4] => (item=less)
changed: [10.116.0.3] => (item=python3-pip)
ok: [10.116.0.4] => (item=sudo)
ok: [10.116.0.3] => (item=curl)
ok: [10.116.0.4] => (item=vim)
ok: [10.116.0.3] => (item=less)
ok: [10.116.0.4] => (item=gcc)
ok: [10.116.0.3] => (item=sudo)
ok: [10.116.0.3] => (item=vim)
ok: [10.116.0.3] => (item=gcc)
changed: [10.116.0.4] => (item=jq)
ok: [10.116.0.4] => (item=iptables)
changed: [10.116.0.2] => (item=python3-pip)
ok: [10.116.0.2] => (item=curl)
changed: [10.116.0.3] => (item=jq)
ok: [10.116.0.2] => (item=less)
ok: [10.116.0.3] => (item=iptables)
changed: [10.116.0.4] => (item=acl)
ok: [10.116.0.2] => (item=sudo)
ok: [10.116.0.2] => (item=vim)
changed: [10.116.0.4] => (item=dnsutils)
changed: [10.116.0.3] => (item=acl)
ok: [10.116.0.2] => (item=gcc)
changed: [10.116.0.3] => (item=dnsutils)
changed: [10.116.0.2] => (item=jq)
ok: [10.116.0.2] => (item=iptables)
changed: [10.116.0.2] => (item=acl)
changed: [10.116.0.2] => (item=dnsutils)

TASK [packages : PostgreSQL | ensure postgresql database-cluster manager package] ******************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.2]
changed: [10.116.0.3]

TASK [packages : PostgreSQL | disable initializing of a default postgresql cluster] ****************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.2]
changed: [10.116.0.3]

TASK [packages : PostgreSQL | disable log rotation with logrotate for postgresql] ******************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.4]
changed: [10.116.0.3]

TASK [packages : Install PostgreSQL packages] ******************************************************************************************************************************************************************
changed: [10.116.0.4] => (item=postgresql-15)
changed: [10.116.0.3] => (item=postgresql-15)
ok: [10.116.0.4] => (item=postgresql-client-15)
changed: [10.116.0.2] => (item=postgresql-15)
ok: [10.116.0.3] => (item=postgresql-client-15)
ok: [10.116.0.4] => (item=postgresql-contrib-15)
ok: [10.116.0.2] => (item=postgresql-client-15)
ok: [10.116.0.3] => (item=postgresql-contrib-15)
ok: [10.116.0.2] => (item=postgresql-contrib-15)
changed: [10.116.0.4] => (item=postgresql-server-dev-15)
changed: [10.116.0.3] => (item=postgresql-server-dev-15)
changed: [10.116.0.2] => (item=postgresql-server-dev-15)

TASK [sudo : Add user to /etc/sudoers.d/] **********************************************************************************************************************************************************************
changed: [10.116.0.4] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})
changed: [10.116.0.2] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})
changed: [10.116.0.3] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})

TASK [swap : Ensure swap exists] *******************************************************************************************************************************************************************************
ok: [10.116.0.2]
ok: [10.116.0.4]
ok: [10.116.0.3]

TASK [swap : Create swap file] *********************************************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.3]
changed: [10.116.0.2]

TASK [swap : Set permissions on swap file] *********************************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.4]
changed: [10.116.0.3]

TASK [swap : Make swap file if necessary] **********************************************************************************************************************************************************************
changed: [10.116.0.3]
changed: [10.116.0.4]
changed: [10.116.0.2]

TASK [swap : Run swapon on the swap file] **********************************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.3]
changed: [10.116.0.4]

TASK [swap : Manage swap file entry in fstab] ******************************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.3]
changed: [10.116.0.4]

TASK [sysctl : Build a sysctl_conf dynamic variable] ***********************************************************************************************************************************************************
ok: [10.116.0.2] => (item=etcd_cluster)
ok: [10.116.0.2] => (item=master)
ok: [10.116.0.3] => (item=etcd_cluster)
ok: [10.116.0.2] => (item=postgres_cluster)
ok: [10.116.0.3] => (item=postgres_cluster)
ok: [10.116.0.4] => (item=etcd_cluster)
ok: [10.116.0.3] => (item=replica)
ok: [10.116.0.4] => (item=postgres_cluster)
ok: [10.116.0.4] => (item=replica)

TASK [sysctl : Setting kernel parameters] **********************************************************************************************************************************************************************
ok: [10.116.0.2] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [10.116.0.3] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [10.116.0.4] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [10.116.0.4] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [10.116.0.2] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [10.116.0.4] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [10.116.0.4] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [10.116.0.3] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [10.116.0.4] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [10.116.0.3] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [10.116.0.4] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [10.116.0.3] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [10.116.0.4] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [10.116.0.3] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [10.116.0.4] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [10.116.0.3] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [10.116.0.4] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [10.116.0.2] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [10.116.0.3] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [10.116.0.4] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [10.116.0.3] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [10.116.0.4] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [10.116.0.2] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [10.116.0.3] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [10.116.0.4] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [10.116.0.2] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [10.116.0.3] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [10.116.0.4] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [10.116.0.2] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [10.116.0.4] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [10.116.0.3] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [10.116.0.4] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [10.116.0.3] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [10.116.0.2] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [10.116.0.4] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
ok: [10.116.0.3] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [10.116.0.2] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [10.116.0.3] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [10.116.0.2] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [10.116.0.3] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [10.116.0.2] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [10.116.0.3] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
ok: [10.116.0.2] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [10.116.0.2] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [10.116.0.2] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [10.116.0.2] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [10.116.0.2] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [10.116.0.2] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})

TASK [transparent_huge_pages : Create systemd service "disable-transparent-huge-pages.service"] ****************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.4]
changed: [10.116.0.3]

RUNNING HANDLER [transparent_huge_pages : Start disable-transparent-huge-pages service] ************************************************************************************************************************
changed: [10.116.0.3]
changed: [10.116.0.2]
changed: [10.116.0.4]

TASK [pam_limits : Linux PAM limits | add or modify nofile limits] *********************************************************************************************************************************************
changed: [10.116.0.2] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [10.116.0.4] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [10.116.0.3] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [10.116.0.2] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})
changed: [10.116.0.4] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})
changed: [10.116.0.3] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})

TASK [locales : Generate locales] ******************************************************************************************************************************************************************************
ok: [10.116.0.4] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})
ok: [10.116.0.3] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})
ok: [10.116.0.2] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})

TASK [locales : Set locale "en_US.utf-8" into /etc/default/locale] *********************************************************************************************************************************************
changed: [10.116.0.4] => (item=LANG=en_US.utf-8)
changed: [10.116.0.2] => (item=LANG=en_US.utf-8)
changed: [10.116.0.3] => (item=LANG=en_US.utf-8)
changed: [10.116.0.4] => (item=LANGUAGE=en_US.utf-8)
changed: [10.116.0.2] => (item=LANGUAGE=en_US.utf-8)
changed: [10.116.0.3] => (item=LANGUAGE=en_US.utf-8)
changed: [10.116.0.4] => (item=LC_ALL=en_US.utf-8)
changed: [10.116.0.2] => (item=LC_ALL=en_US.utf-8)
changed: [10.116.0.3] => (item=LC_ALL=en_US.utf-8)

PLAY [balancers.yml | Configure load balancers] ****************************************************************************************************************************************************************
skipping: no hosts matched

PLAY [deploy_pgcluster.yml | Install and configure pgBackRest] *************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.116.0.2]
ok: [10.116.0.3]
ok: [10.116.0.4]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.116.0.2]
ok: [10.116.0.3]
ok: [10.116.0.4]

PLAY [deploy_pgcluster.yml | PostgreSQL Cluster Deployment] ****************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.116.0.2]
ok: [10.116.0.3]
ok: [10.116.0.4]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.116.0.2]
ok: [10.116.0.3]
ok: [10.116.0.4]

TASK [cron : Make sure that the cron package is installed] *****************************************************************************************************************************************************
ok: [10.116.0.2]
ok: [10.116.0.3]
ok: [10.116.0.4]

TASK [pgbouncer : Install pgbouncer package] *******************************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.3]
changed: [10.116.0.2]

TASK [pgbouncer : Ensure config directory "/etc/pgbouncer" exist] **********************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.2]
changed: [10.116.0.3]

TASK [pgbouncer : Check if pgbouncer systemd service file exists] **********************************************************************************************************************************************
ok: [10.116.0.2]
ok: [10.116.0.4]
ok: [10.116.0.3]

TASK [pgbouncer : Stop and disable standard init script] *******************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.4]
changed: [10.116.0.3]

TASK [pgbouncer : Configure pgbouncer systemd service file] ****************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.4]
changed: [10.116.0.3]

TASK [pgbouncer : Ensure pgbouncer service is enabled] *********************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.4]
changed: [10.116.0.3]

TASK [pgbouncer : Enable log rotation with logrotate] **********************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.2]
changed: [10.116.0.3]

TASK [pgbouncer : Configure pgbouncer.ini] *********************************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.3]
changed: [10.116.0.2]

TASK [pgpass : Configure a password file (/var/lib/postgresql/.pgpass)] ****************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.2]
changed: [10.116.0.3]

RUNNING HANDLER [pgbouncer : Restart pgbouncer service] ********************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.2]
changed: [10.116.0.3]

RUNNING HANDLER [pgbouncer : Wait for port "6432" to become open on the host] **********************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.3]
ok: [10.116.0.2]

RUNNING HANDLER [Restart pgbouncer service] ********************************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.2]
changed: [10.116.0.3]

RUNNING HANDLER [Wait for port "6432" to become open on the host] **********************************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.3]
ok: [10.116.0.2]

TASK [patroni : Copy patroni requirements.txt file] ************************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.3]
changed: [10.116.0.2]

TASK [patroni : Install setuptools] ****************************************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.2]
changed: [10.116.0.3]

TASK [patroni : Install requirements] **************************************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.3]
changed: [10.116.0.2]

TASK [patroni : Install patroni] *******************************************************************************************************************************************************************************
changed: [10.116.0.3]
changed: [10.116.0.2]
changed: [10.116.0.4]

TASK [patroni : Create conf directory] *************************************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.3]
changed: [10.116.0.4]

TASK [patroni : Generate conf file "/etc/patroni/patroni.yml"] *************************************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.3]
changed: [10.116.0.4]

TASK [patroni : Copy systemd service file "/etc/systemd/system/patroni.service"] *******************************************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.3]
changed: [10.116.0.4]

TASK [patroni : Prepare PostgreSQL | make sure the postgresql log directory "/var/log/postgresql" exists] ******************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.3]
changed: [10.116.0.4]

TASK [patroni : Prepare PostgreSQL | make sure PostgreSQL data directory "/var/lib/postgresql/15/main" exists] *************************************************************************************************
changed: [10.116.0.2]
changed: [10.116.0.3]
changed: [10.116.0.4]

TASK [patroni : Prepare PostgreSQL | make sure the postgresql config files exists] *****************************************************************************************************************************
ok: [10.116.0.2]
ok: [10.116.0.4]
ok: [10.116.0.3]

TASK [patroni : Prepare PostgreSQL | generate default postgresql config files] *********************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.2]
changed: [10.116.0.3]

TASK [patroni : Prepare PostgreSQL | make sure the data directory "/var/lib/postgresql/15/main" is empty] ******************************************************************************************************
changed: [10.116.0.4] => (item=absent)
changed: [10.116.0.3] => (item=absent)
changed: [10.116.0.4] => (item=directory)
changed: [10.116.0.3] => (item=directory)
changed: [10.116.0.2] => (item=absent)
changed: [10.116.0.2] => (item=directory)

TASK [patroni : Start patroni service on the Master server] ****************************************************************************************************************************************************
changed: [10.116.0.2]

TASK [patroni : Wait for port 8008 to become open on the host] *************************************************************************************************************************************************
ok: [10.116.0.2]

TASK [patroni : Check PostgreSQL is started and accepting connections on Master] *******************************************************************************************************************************
ok: [10.116.0.2]

TASK [patroni : Wait for the cluster to initialize (master is the leader with the lock)] ***********************************************************************************************************************
ok: [10.116.0.2]

TASK [patroni : Prepare PostgreSQL | generate pg_hba.conf] *****************************************************************************************************************************************************
changed: [10.116.0.4]
changed: [10.116.0.3]
changed: [10.116.0.2]

TASK [patroni : Prepare PostgreSQL | reload for apply the pg_hba.conf] *****************************************************************************************************************************************
ok: [10.116.0.3]
ok: [10.116.0.4]
changed: [10.116.0.2]

TASK [patroni : Start patroni service on Replica servers] ******************************************************************************************************************************************************
changed: [10.116.0.3]
changed: [10.116.0.4]

TASK [patroni : Wait for port 8008 to become open on the host] *************************************************************************************************************************************************
ok: [10.116.0.3]
ok: [10.116.0.4]

TASK [patroni : Check that the patroni is healthy on the replica server] ***************************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.3]

TASK [patroni : Turning off postgresql autostart from config "start.conf" (will be managed by patroni)] ********************************************************************************************************
changed: [10.116.0.3]
changed: [10.116.0.4]
changed: [10.116.0.2]

TASK [patroni : Disable "postgresql@15-main" service] **********************************************************************************************************************************************************
ok: [10.116.0.3]
ok: [10.116.0.4]
ok: [10.116.0.2]

TASK [patroni : Add PATRONICTL_CONFIG_FILE environment variable into /etc/environment] *************************************************************************************************************************
changed: [10.116.0.3]
changed: [10.116.0.4]
changed: [10.116.0.2]

TASK [postgresql-users : Make sure the PostgreSQL users are present] *******************************************************************************************************************************************
changed: [10.116.0.2] => (item=pgbouncer)

TASK [pgbouncer/config : Ensure config directory "/etc/pgbouncer" exist] ***************************************************************************************************************************************
ok: [10.116.0.3]
ok: [10.116.0.4]
ok: [10.116.0.2]

TASK [pgbouncer/config : Update pgbouncer.ini] *****************************************************************************************************************************************************************
ok: [10.116.0.4]
ok: [10.116.0.3]
ok: [10.116.0.2]

TASK [pgbouncer/config : Create function 'user_search' for pgbouncer 'auth_query' option in all databases] *****************************************************************************************************
changed: [10.116.0.2]

TASK [deploy-finish : Get postgresql users list] ***************************************************************************************************************************************************************
ok: [10.116.0.2]

TASK [deploy-finish : PostgreSQL list of users] ****************************************************************************************************************************************************************
ok: [10.116.0.2] => {
    "users_result.stdout_lines": [
        "                                    List of roles",
        " Role name  |                         Attributes                         | Member of ",
        "------------+------------------------------------------------------------+-----------",
        " pgbouncer  |                                                            | {}",
        " postgres   | Superuser, Create role, Create DB, Replication, Bypass RLS | {}",
        " replicator | Replication                                                | {}"
    ]
}

TASK [deploy-finish : Get postgresql database list] ************************************************************************************************************************************************************
ok: [10.116.0.2]

TASK [deploy-finish : PostgreSQL list of databases] ************************************************************************************************************************************************************
ok: [10.116.0.2] => {
    "dbs_result.stdout_lines": [
        "   name   |  owner   | encoding |   collate   |    ctype    |  size   | tablespace ",
        "----------+----------+----------+-------------+-------------+---------+------------",
        " postgres | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 7453 kB | pg_default",
        "(1 row)"
    ]
}

TASK [deploy-finish : Check postgresql cluster health] *********************************************************************************************************************************************************
ok: [10.116.0.2]

TASK [deploy-finish : PostgreSQL Cluster health] ***************************************************************************************************************************************************************
ok: [10.116.0.2] => {
    "patronictl_result.stdout_lines": [
        "+ Cluster: vitaliy-test-pgcluster +------------+---------+-----------+----+-----------+",
        "| Member                          | Host       | Role    | State     | TL | Lag in MB |",
        "+---------------------------------+------------+---------+-----------+----+-----------+",
        "| vitaliy-test-pgcluster-pgnode01 | 10.116.0.2 | Leader  | running   |  1 |           |",
        "| vitaliy-test-pgcluster-pgnode02 | 10.116.0.3 | Replica | streaming |  1 |         0 |",
        "| vitaliy-test-pgcluster-pgnode03 | 10.116.0.4 | Replica | streaming |  1 |         0 |",
        "+---------------------------------+------------+---------+-----------+----+-----------+"
    ]
}

TASK [deploy-finish : Create list of nodes] ********************************************************************************************************************************************************************
ok: [10.116.0.2]

TASK [deploy-finish : PostgreSQL Cluster connection info] ******************************************************************************************************************************************************
ok: [10.116.0.2] => {
    "msg": [
        "+------------------------------------------------+",
        "address 10.116.0.2,10.116.0.3,10.116.0.4",
        "port 6432 (pgbouncer)",
        "+------------------------------------------------+"
    ]
}

PLAY RECAP *****************************************************************************************************************************************************************************************************
10.116.0.2                 : ok=104  changed=56   unreachable=0    failed=0    skipped=348  rescued=0    ignored=0   
10.116.0.3                 : ok=89   changed=53   unreachable=0    failed=0    skipped=328  rescued=0    ignored=0   
10.116.0.4                 : ok=89   changed=53   unreachable=0    failed=0    skipped=328  rescued=0    ignored=0   
localhost                  : ok=20   changed=5    unreachable=0    failed=0    skipped=70   rescued=0    ignored=0

image

passed

@vitabaks
Copy link
Owner Author

Test: Hetzner

export HCLOUD_API_TOKEN=******

ansible-playbook deploy_pgcluster.yml \
  --user=root --private-key=$HOME/.ssh/id_rsa --extra-vars \
    "provision='hetzner' \
     servers_count=3 \
     server_type='CCX13' \
     server_image='ubuntu-22.04' \
     server_location='ash' \
     volume_size=100 \
     ssh_key_name=vitaliy \
     patroni_cluster_name=vitaliy-test-pgcluster"

Result:


PLAY [Provision of cloud resources (virtual machine + disk) for PostgreSQL HA Cluster (on HETZNER)] ************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Ensure that 'hcloud' dependency is present on controlling host] ************************************************************************************************************************
ok: [localhost -> 127.0.0.1]

TASK [cloud-resources : Hetzner Cloud: Gather information about SSH key 'vitaliy'] *****************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Set variable: ssh_key_names] ***********************************************************************************************************************************************************
ok: [localhost] => (item=None)
ok: [localhost]

TASK [cloud-resources : Hetzner Cloud: Gather information about network zones] *********************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Hetzner Cloud: Extract network zone for server_location] *******************************************************************************************************************************
ok: [localhost] => (item=network_zone: us-east)

TASK [cloud-resources : Hetzner Cloud: Gather information about networks] **************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Hetzner Cloud: Create a network 'postgres-cluster-network'] ****************************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : Hetzner Cloud: Create a subnetwork in the network 'postgres-cluster-network'] **********************************************************************************************************
changed: [localhost]

TASK [cloud-resources : Set variable: server_network] **********************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Hetzner Cloud: Create or modify SSH firewall] ******************************************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : Hetzner Cloud: Create or modify private firewall] **************************************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : Hetzner Cloud: Gather information about firewalls] *************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Hetzner Cloud: Extract firewall names for 'vitaliy-test-pgcluster'] ********************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Hetzner Cloud: Create or modify server] ************************************************************************************************************************************************
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode01)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode02)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode03)

TASK [cloud-resources : Hetzner Cloud: Add server to network 'postgres-cluster-network'] ***********************************************************************************************************************
ok: [localhost] => (item=vitaliy-test-pgcluster-pgnode01)
ok: [localhost] => (item=vitaliy-test-pgcluster-pgnode02)
ok: [localhost] => (item=vitaliy-test-pgcluster-pgnode03)

TASK [cloud-resources : Hetzner Cloud: Create or modify volume] ************************************************************************************************************************************************
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode01-storage)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode02-storage)
changed: [localhost] => (item=vitaliy-test-pgcluster-pgnode03-storage)

TASK [cloud-resources : Wait for host to be available via SSH] *************************************************************************************************************************************************
ok: [localhost] => (item=5.161.228.76)
ok: [localhost] => (item=5.161.63.15)
ok: [localhost] => (item=5.161.229.14)

TASK [cloud-resources : Show Server info] **********************************************************************************************************************************************************************
ok: [localhost] => (item=5.161.228.76) => {
    "msg": [
        "ID: 37095542",
        "Name: vitaliy-test-pgcluster-pgnode01",
        "Image: ubuntu-22.04",
        "Type: ccx13",
        "Volume Size: 100 GB",
        "Public IP: 5.161.228.76",
        "Private IP: 10.0.1.1"
    ]
}
ok: [localhost] => (item=5.161.63.15) => {
    "msg": [
        "ID: 37095550",
        "Name: vitaliy-test-pgcluster-pgnode02",
        "Image: ubuntu-22.04",
        "Type: ccx13",
        "Volume Size: 100 GB",
        "Public IP: 5.161.63.15",
        "Private IP: 10.0.1.2"
    ]
}
ok: [localhost] => (item=5.161.229.14) => {
    "msg": [
        "ID: 37095551",
        "Name: vitaliy-test-pgcluster-pgnode03",
        "Image: ubuntu-22.04",
        "Type: ccx13",
        "Volume Size: 100 GB",
        "Public IP: 5.161.229.14",
        "Private IP: 10.0.1.3"
    ]
}

TASK [cloud-resources : Inventory | Initialize ip_addresses variable] ******************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Inventory | Extract IP addresses] ******************************************************************************************************************************************************
ok: [localhost] => (item=public_ip: 5.161.228.76, private_ip: 10.0.1.1)
ok: [localhost] => (item=public_ip: 5.161.63.15, private_ip: 10.0.1.2)
ok: [localhost] => (item=public_ip: 5.161.229.14, private_ip: 10.0.1.3)

TASK [cloud-resources : Inventory | Add host to 'postgres_cluster', 'master' groups] ***************************************************************************************************************************
ok: [localhost] => (item=5.161.228.76)

TASK [cloud-resources : Inventory | Add host to 'postgres_cluster', 'replica' groups] **************************************************************************************************************************
ok: [localhost] => (item=5.161.63.15)
ok: [localhost] => (item=5.161.229.14)

TASK [cloud-resources : Inventory | Add host to 'etcd_cluster' group] ******************************************************************************************************************************************
ok: [localhost] => (item=5.161.228.76)
ok: [localhost] => (item=5.161.63.15)
ok: [localhost] => (item=5.161.229.14)

PLAY [Deploy PostgreSQL HA Cluster (based on "Patroni" and "etcd")] ********************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.2]
ok: [10.0.1.3]

TASK [System information] **************************************************************************************************************************************************************************************
ok: [10.0.1.1] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "AMD EPYC Processor, count: 1, cores: 1",
        "Disk space total": "75.04 GB",
        "Kernel": "5.15.0-79-generic",
        "Memory": "7.56 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "vServer",
        "Virtualization type": "kvm"
    }
}
ok: [10.0.1.2] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "AMD EPYC Processor, count: 1, cores: 1",
        "Disk space total": "75.04 GB",
        "Kernel": "5.15.0-79-generic",
        "Memory": "7.56 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "vServer",
        "Virtualization type": "kvm"
    }
}
ok: [10.0.1.3] => {
    "system_info": {
        "Architecture": "x86_64",
        "CPU model": "AMD EPYC Processor, count: 1, cores: 1",
        "Disk space total": "75.04 GB",
        "Kernel": "5.15.0-79-generic",
        "Memory": "7.56 GB",
        "OS": "Ubuntu 22.04",
        "Product name": "vServer",
        "Virtualization type": "kvm"
    }
}

TASK [pre-checks : Set max_connections from vars or use default] ***********************************************************************************************************************************************
ok: [10.0.1.1] => (item={'option': 'max_connections', 'value': '500'})

TASK [pre-checks : PgBouncer | Calculate pool_size] ************************************************************************************************************************************************************
ok: [10.0.1.1] => (item={'name': 'postgres', 'dbname': 'postgres', 'pool_parameters': ''})

TASK [pre-checks : PgBouncer | Calculate total pool_size] ******************************************************************************************************************************************************
ok: [10.0.1.1]

TASK [pre-checks : PgBouncer | Show total pool_size] ***********************************************************************************************************************************************************
ok: [10.0.1.1] => {
    "pgbouncer_total_pool_size": "20"
}

TASK [pre-checks : PostgreSQL | check that data directory "/var/lib/postgresql/15/main" is not initialized] ****************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

TASK [Update apt cache] ****************************************************************************************************************************************************************************************
changed: [10.0.1.2]
changed: [10.0.1.3]
changed: [10.0.1.1]

TASK [Make sure the gnupg and apt-transport-https packages are present] ****************************************************************************************************************************************
changed: [10.0.1.3]
changed: [10.0.1.1]
changed: [10.0.1.2]

TASK [Make sure that the iproute is installed] *****************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.2]
ok: [10.0.1.3]

PLAY [Configure etcd Cluster and System Settings] **************************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.2]
ok: [10.0.1.3]

TASK [Update apt cache] ****************************************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.2]
ok: [10.0.1.3]

TASK [Make sure the gnupg and apt-transport-https packages are present] ****************************************************************************************************************************************
ok: [10.0.1.3]
ok: [10.0.1.2]
ok: [10.0.1.1]

TASK [sysctl : Build a sysctl_conf dynamic variable] ***********************************************************************************************************************************************************
ok: [10.0.1.1] => (item=etcd_cluster)
ok: [10.0.1.1] => (item=master)
ok: [10.0.1.2] => (item=etcd_cluster)
ok: [10.0.1.1] => (item=postgres_cluster)
ok: [10.0.1.2] => (item=postgres_cluster)
ok: [10.0.1.3] => (item=etcd_cluster)
ok: [10.0.1.2] => (item=replica)
ok: [10.0.1.3] => (item=postgres_cluster)
ok: [10.0.1.3] => (item=replica)

TASK [sysctl : Setting kernel parameters] **********************************************************************************************************************************************************************
changed: [10.0.1.3] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [10.0.1.2] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [10.0.1.1] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
changed: [10.0.1.1] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [10.0.1.3] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [10.0.1.2] => (item={'name': 'vm.swappiness', 'value': '1'})
changed: [10.0.1.1] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [10.0.1.3] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [10.0.1.2] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
changed: [10.0.1.1] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [10.0.1.3] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [10.0.1.2] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
changed: [10.0.1.1] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [10.0.1.3] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [10.0.1.2] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
changed: [10.0.1.1] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [10.0.1.3] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [10.0.1.2] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
changed: [10.0.1.1] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [10.0.1.3] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [10.0.1.2] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
changed: [10.0.1.1] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [10.0.1.3] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [10.0.1.2] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
changed: [10.0.1.1] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [10.0.1.3] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [10.0.1.2] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
changed: [10.0.1.1] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [10.0.1.3] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [10.0.1.2] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
changed: [10.0.1.1] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [10.0.1.3] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [10.0.1.2] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
changed: [10.0.1.1] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [10.0.1.3] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [10.0.1.2] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
changed: [10.0.1.1] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [10.0.1.3] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [10.0.1.2] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
changed: [10.0.1.1] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [10.0.1.3] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [10.0.1.2] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
changed: [10.0.1.1] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [10.0.1.3] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [10.0.1.2] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
changed: [10.0.1.1] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
changed: [10.0.1.3] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
changed: [10.0.1.2] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})

TASK [etcd : Make sure the unzip/tar packages are present] *****************************************************************************************************************************************************
changed: [10.0.1.2]
changed: [10.0.1.3]
changed: [10.0.1.1]

TASK [etcd : Download "etcd" package] **************************************************************************************************************************************************************************
changed: [10.0.1.1] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)
[WARNING]: Module remote_tmp /tmp/root/ansible did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct
permissions manually
changed: [10.0.1.2] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)
changed: [10.0.1.3] => (item=https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-amd64.tar.gz)

TASK [etcd : Extract "etcd" into /tmp] *************************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [etcd : Copy "etcd" and "etcdctl" binary files to /usr/local/bin/] ****************************************************************************************************************************************
changed: [10.0.1.1] => (item=etcd)
changed: [10.0.1.3] => (item=etcd)
changed: [10.0.1.2] => (item=etcd)
changed: [10.0.1.1] => (item=etcdctl)
changed: [10.0.1.3] => (item=etcdctl)
changed: [10.0.1.2] => (item=etcdctl)

TASK [etcd : Add etcd user] ************************************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [etcd : Create etcd conf directory] ***********************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [etcd : Create etcd data directory] ***********************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [etcd : Generate conf file "/etc/etcd/etcd.conf"] *********************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [etcd : Copy systemd service file] ************************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [etcd : Enable and start etcd service] ********************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [etcd : Wait for port 2379 to become open on the host] ****************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

TASK [etcd : Wait until the etcd cluster is healthy] ***********************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

TASK [etcd : cluster health] ***********************************************************************************************************************************************************************************
ok: [10.0.1.1] => {
    "msg": "http://10.0.1.1:2379 is healthy: successfully committed proposal: took = 2.963209ms"
}
ok: [10.0.1.2] => {
    "msg": "http://10.0.1.2:2379 is healthy: successfully committed proposal: took = 3.411725ms"
}
ok: [10.0.1.3] => {
    "msg": "http://10.0.1.3:2379 is healthy: successfully committed proposal: took = 2.991831ms"
}

PLAY [consul.yml | Consul Playbook] ****************************************************************************************************************************************************************************

PLAY [consul.yml | Configure Consul instances] *****************************************************************************************************************************************************************
skipping: no hosts matched

PLAY [deploy_pgcluster.yml | Postgres Cluster Configuration] ***************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.2]
ok: [10.0.1.3]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.2]
ok: [10.0.1.3]

TASK [add-repository : Add repository apt-key] *****************************************************************************************************************************************************************
changed: [10.0.1.3] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})
changed: [10.0.1.1] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})
changed: [10.0.1.2] => (item={'key': 'https://www.postgresql.org/media/keys/ACCC4CF8.asc'})

TASK [add-repository : Add repository] *************************************************************************************************************************************************************************
changed: [10.0.1.2] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})
changed: [10.0.1.3] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})
changed: [10.0.1.1] => (item={'repo': 'deb https://apt.postgresql.org/pub/repos/apt/ jammy-pgdg main'})

TASK [packages : Install system packages] **********************************************************************************************************************************************************************
ok: [10.0.1.1] => (item=['python3'])
ok: [10.0.1.3] => (item=['python3'])
ok: [10.0.1.2] => (item=['python3'])
ok: [10.0.1.1] => (item=python3)
ok: [10.0.1.3] => (item=python3)
ok: [10.0.1.2] => (item=python3)
changed: [10.0.1.3] => (item=python3-dev)
changed: [10.0.1.2] => (item=python3-dev)
changed: [10.0.1.1] => (item=python3-dev)
changed: [10.0.1.3] => (item=python3-psycopg2)
changed: [10.0.1.2] => (item=python3-psycopg2)
changed: [10.0.1.1] => (item=python3-psycopg2)
ok: [10.0.1.3] => (item=python3-setuptools)
ok: [10.0.1.1] => (item=python3-setuptools)
ok: [10.0.1.2] => (item=python3-setuptools)
changed: [10.0.1.3] => (item=python3-pip)
ok: [10.0.1.3] => (item=curl)
changed: [10.0.1.1] => (item=python3-pip)
ok: [10.0.1.3] => (item=less)
ok: [10.0.1.1] => (item=curl)
changed: [10.0.1.2] => (item=python3-pip)
ok: [10.0.1.3] => (item=sudo)
ok: [10.0.1.1] => (item=less)
ok: [10.0.1.2] => (item=curl)
ok: [10.0.1.3] => (item=vim)
ok: [10.0.1.1] => (item=sudo)
ok: [10.0.1.2] => (item=less)
ok: [10.0.1.3] => (item=gcc)
ok: [10.0.1.1] => (item=vim)
ok: [10.0.1.2] => (item=sudo)
ok: [10.0.1.1] => (item=gcc)
ok: [10.0.1.2] => (item=vim)
changed: [10.0.1.3] => (item=jq)
ok: [10.0.1.2] => (item=gcc)
ok: [10.0.1.3] => (item=iptables)
changed: [10.0.1.1] => (item=jq)
ok: [10.0.1.3] => (item=acl)
changed: [10.0.1.2] => (item=jq)
ok: [10.0.1.1] => (item=iptables)
ok: [10.0.1.2] => (item=iptables)
ok: [10.0.1.1] => (item=acl)
changed: [10.0.1.3] => (item=dnsutils)
ok: [10.0.1.2] => (item=acl)
changed: [10.0.1.1] => (item=dnsutils)
changed: [10.0.1.2] => (item=dnsutils)

TASK [packages : PostgreSQL | ensure postgresql database-cluster manager package] ******************************************************************************************************************************
changed: [10.0.1.2]
changed: [10.0.1.3]
changed: [10.0.1.1]

TASK [packages : PostgreSQL | disable initializing of a default postgresql cluster] ****************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [packages : PostgreSQL | disable log rotation with logrotate for postgresql] ******************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [packages : Install PostgreSQL packages] ******************************************************************************************************************************************************************
changed: [10.0.1.1] => (item=postgresql-15)
changed: [10.0.1.2] => (item=postgresql-15)
changed: [10.0.1.3] => (item=postgresql-15)
ok: [10.0.1.1] => (item=postgresql-client-15)
ok: [10.0.1.2] => (item=postgresql-client-15)
ok: [10.0.1.3] => (item=postgresql-client-15)
ok: [10.0.1.1] => (item=postgresql-contrib-15)
ok: [10.0.1.2] => (item=postgresql-contrib-15)
ok: [10.0.1.3] => (item=postgresql-contrib-15)
changed: [10.0.1.2] => (item=postgresql-server-dev-15)
changed: [10.0.1.3] => (item=postgresql-server-dev-15)
changed: [10.0.1.1] => (item=postgresql-server-dev-15)

TASK [sudo : Add user to /etc/sudoers.d/] **********************************************************************************************************************************************************************
changed: [10.0.1.1] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})
changed: [10.0.1.3] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})
changed: [10.0.1.2] => (item={'name': 'postgres', 'nopasswd': 'yes', 'commands': 'ALL'})

TASK [swap : Ensure swap exists] *******************************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.2]
ok: [10.0.1.3]

TASK [swap : Create swap file] *********************************************************************************************************************************************************************************
changed: [10.0.1.2]
changed: [10.0.1.3]
changed: [10.0.1.1]

TASK [swap : Set permissions on swap file] *********************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [swap : Make swap file if necessary] **********************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [swap : Run swapon on the swap file] **********************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [swap : Manage swap file entry in fstab] ******************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [sysctl : Build a sysctl_conf dynamic variable] ***********************************************************************************************************************************************************
ok: [10.0.1.1] => (item=etcd_cluster)
ok: [10.0.1.1] => (item=master)
ok: [10.0.1.2] => (item=etcd_cluster)
ok: [10.0.1.1] => (item=postgres_cluster)
ok: [10.0.1.2] => (item=postgres_cluster)
ok: [10.0.1.3] => (item=etcd_cluster)
ok: [10.0.1.2] => (item=replica)
ok: [10.0.1.3] => (item=postgres_cluster)
ok: [10.0.1.3] => (item=replica)

TASK [sysctl : Setting kernel parameters] **********************************************************************************************************************************************************************
ok: [10.0.1.1] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [10.0.1.2] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [10.0.1.3] => (item={'name': 'vm.overcommit_memory', 'value': '2'})
ok: [10.0.1.1] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [10.0.1.2] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [10.0.1.3] => (item={'name': 'vm.swappiness', 'value': '1'})
ok: [10.0.1.1] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [10.0.1.2] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [10.0.1.3] => (item={'name': 'vm.min_free_kbytes', 'value': '102400'})
ok: [10.0.1.1] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [10.0.1.2] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [10.0.1.3] => (item={'name': 'vm.dirty_expire_centisecs', 'value': '1000'})
ok: [10.0.1.1] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [10.0.1.2] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [10.0.1.3] => (item={'name': 'vm.dirty_background_bytes', 'value': '67108864'})
ok: [10.0.1.1] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [10.0.1.2] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [10.0.1.3] => (item={'name': 'vm.dirty_bytes', 'value': '536870912'})
ok: [10.0.1.1] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [10.0.1.2] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [10.0.1.3] => (item={'name': 'vm.zone_reclaim_mode', 'value': '0'})
ok: [10.0.1.1] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [10.0.1.2] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [10.0.1.3] => (item={'name': 'kernel.numa_balancing', 'value': '0'})
ok: [10.0.1.1] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [10.0.1.2] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [10.0.1.3] => (item={'name': 'kernel.sched_autogroup_enabled', 'value': '0'})
ok: [10.0.1.1] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [10.0.1.2] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [10.0.1.3] => (item={'name': 'net.ipv4.ip_nonlocal_bind', 'value': '1'})
ok: [10.0.1.1] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [10.0.1.2] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [10.0.1.3] => (item={'name': 'net.ipv4.ip_forward', 'value': '1'})
ok: [10.0.1.1] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [10.0.1.2] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [10.0.1.3] => (item={'name': 'net.ipv4.ip_local_port_range', 'value': '10000 65535'})
ok: [10.0.1.1] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [10.0.1.3] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [10.0.1.2] => (item={'name': 'net.core.netdev_max_backlog', 'value': '10000'})
ok: [10.0.1.1] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [10.0.1.3] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [10.0.1.2] => (item={'name': 'net.ipv4.tcp_max_syn_backlog', 'value': '8192'})
ok: [10.0.1.1] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [10.0.1.3] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [10.0.1.2] => (item={'name': 'net.core.somaxconn', 'value': '65535'})
ok: [10.0.1.1] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
ok: [10.0.1.3] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})
ok: [10.0.1.2] => (item={'name': 'net.ipv4.tcp_tw_reuse', 'value': '1'})

TASK [transparent_huge_pages : Create systemd service "disable-transparent-huge-pages.service"] ****************************************************************************************************************
changed: [10.0.1.2]
changed: [10.0.1.1]
changed: [10.0.1.3]

RUNNING HANDLER [transparent_huge_pages : Start disable-transparent-huge-pages service] ************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [pam_limits : Linux PAM limits | add or modify nofile limits] *********************************************************************************************************************************************
changed: [10.0.1.1] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [10.0.1.3] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [10.0.1.2] => (item={'limit_type': 'soft', 'limit_item': 'nofile', 'value': 65536})
changed: [10.0.1.1] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})
changed: [10.0.1.3] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})
changed: [10.0.1.2] => (item={'limit_type': 'hard', 'limit_item': 'nofile', 'value': 200000})

TASK [locales : Generate locales] ******************************************************************************************************************************************************************************
ok: [10.0.1.1] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})
ok: [10.0.1.3] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})
ok: [10.0.1.2] => (item={'language_country': 'en_US', 'encoding': 'UTF-8'})

TASK [locales : Set locale "en_US.utf-8" into /etc/default/locale] *********************************************************************************************************************************************
changed: [10.0.1.1] => (item=LANG=en_US.utf-8)
changed: [10.0.1.2] => (item=LANG=en_US.utf-8)
changed: [10.0.1.3] => (item=LANG=en_US.utf-8)
changed: [10.0.1.1] => (item=LANGUAGE=en_US.utf-8)
changed: [10.0.1.2] => (item=LANGUAGE=en_US.utf-8)
changed: [10.0.1.3] => (item=LANGUAGE=en_US.utf-8)
changed: [10.0.1.1] => (item=LC_ALL=en_US.utf-8)
changed: [10.0.1.3] => (item=LC_ALL=en_US.utf-8)
changed: [10.0.1.2] => (item=LC_ALL=en_US.utf-8)

PLAY [balancers.yml | Configure load balancers] ****************************************************************************************************************************************************************
skipping: no hosts matched

PLAY [deploy_pgcluster.yml | Install and configure pgBackRest] *************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.2]
ok: [10.0.1.3]

PLAY [deploy_pgcluster.yml | PostgreSQL Cluster Deployment] ****************************************************************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************************************************************************
ok: [10.0.1.2]
ok: [10.0.1.1]
ok: [10.0.1.3]

TASK [Include OS-specific variables] ***************************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.2]
ok: [10.0.1.3]

TASK [cron : Make sure that the cron package is installed] *****************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

TASK [pgbouncer : Install pgbouncer package] *******************************************************************************************************************************************************************
changed: [10.0.1.2]
changed: [10.0.1.3]
changed: [10.0.1.1]

TASK [pgbouncer : Ensure config directory "/etc/pgbouncer" exist] **********************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [pgbouncer : Check if pgbouncer systemd service file exists] **********************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

TASK [pgbouncer : Stop and disable standard init script] *******************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [pgbouncer : Configure pgbouncer systemd service file] ****************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [pgbouncer : Ensure pgbouncer service is enabled] *********************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [pgbouncer : Enable log rotation with logrotate] **********************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [pgbouncer : Configure pgbouncer.ini] *********************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [pgpass : Configure a password file (/var/lib/postgresql/.pgpass)] ****************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

RUNNING HANDLER [pgbouncer : Restart pgbouncer service] ********************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

RUNNING HANDLER [pgbouncer : Wait for port "6432" to become open on the host] **********************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

RUNNING HANDLER [Restart pgbouncer service] ********************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

RUNNING HANDLER [Wait for port "6432" to become open on the host] **********************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

TASK [patroni : Copy patroni requirements.txt file] ************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [patroni : Install setuptools] ****************************************************************************************************************************************************************************
changed: [10.0.1.2]
changed: [10.0.1.1]
changed: [10.0.1.3]

TASK [patroni : Install requirements] **************************************************************************************************************************************************************************
changed: [10.0.1.2]
changed: [10.0.1.3]
changed: [10.0.1.1]

TASK [patroni : Install patroni] *******************************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [patroni : Create conf directory] *************************************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [patroni : Generate conf file "/etc/patroni/patroni.yml"] *************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [patroni : Copy systemd service file "/etc/systemd/system/patroni.service"] *******************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [patroni : Prepare PostgreSQL | make sure the postgresql log directory "/var/log/postgresql" exists] ******************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [patroni : Prepare PostgreSQL | make sure PostgreSQL data directory "/var/lib/postgresql/15/main" exists] *************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [patroni : Prepare PostgreSQL | make sure the postgresql config files exists] *****************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.2]
ok: [10.0.1.3]

TASK [patroni : Prepare PostgreSQL | generate default postgresql config files] *********************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [patroni : Prepare PostgreSQL | make sure the data directory "/var/lib/postgresql/15/main" is empty] ******************************************************************************************************
changed: [10.0.1.1] => (item=absent)
changed: [10.0.1.2] => (item=absent)
changed: [10.0.1.3] => (item=absent)
changed: [10.0.1.1] => (item=directory)
changed: [10.0.1.2] => (item=directory)
changed: [10.0.1.3] => (item=directory)

TASK [patroni : Start patroni service on the Master server] ****************************************************************************************************************************************************
changed: [10.0.1.1]

TASK [patroni : Wait for port 8008 to become open on the host] *************************************************************************************************************************************************
ok: [10.0.1.1]

TASK [patroni : Check PostgreSQL is started and accepting connections on Master] *******************************************************************************************************************************
ok: [10.0.1.1]

TASK [patroni : Wait for the cluster to initialize (master is the leader with the lock)] ***********************************************************************************************************************
ok: [10.0.1.1]

TASK [patroni : Prepare PostgreSQL | generate pg_hba.conf] *****************************************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [patroni : Prepare PostgreSQL | reload for apply the pg_hba.conf] *****************************************************************************************************************************************
changed: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

TASK [patroni : Start patroni service on Replica servers] ******************************************************************************************************************************************************
changed: [10.0.1.2]
changed: [10.0.1.3]

TASK [patroni : Wait for port 8008 to become open on the host] *************************************************************************************************************************************************
ok: [10.0.1.2]
ok: [10.0.1.3]

TASK [patroni : Check that the patroni is healthy on the replica server] ***************************************************************************************************************************************
ok: [10.0.1.2]
ok: [10.0.1.3]

TASK [patroni : Turning off postgresql autostart from config "start.conf" (will be managed by patroni)] ********************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [patroni : Disable "postgresql@15-main" service] **********************************************************************************************************************************************************
ok: [10.0.1.3]
ok: [10.0.1.1]
ok: [10.0.1.2]

TASK [patroni : Add PATRONICTL_CONFIG_FILE environment variable into /etc/environment] *************************************************************************************************************************
changed: [10.0.1.1]
changed: [10.0.1.3]
changed: [10.0.1.2]

TASK [postgresql-users : Make sure the PostgreSQL users are present] *******************************************************************************************************************************************
changed: [10.0.1.1] => (item=pgbouncer)

TASK [pgbouncer/config : Ensure config directory "/etc/pgbouncer" exist] ***************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.2]
ok: [10.0.1.3]

TASK [pgbouncer/config : Update pgbouncer.ini] *****************************************************************************************************************************************************************
ok: [10.0.1.1]
ok: [10.0.1.3]
ok: [10.0.1.2]

TASK [pgbouncer/config : Create function 'user_search' for pgbouncer 'auth_query' option in all databases] *****************************************************************************************************
changed: [10.0.1.1]

TASK [deploy-finish : Get postgresql users list] ***************************************************************************************************************************************************************
ok: [10.0.1.1]

TASK [deploy-finish : PostgreSQL list of users] ****************************************************************************************************************************************************************
ok: [10.0.1.1] => {
    "users_result.stdout_lines": [
        "                                    List of roles",
        " Role name  |                         Attributes                         | Member of ",
        "------------+------------------------------------------------------------+-----------",
        " pgbouncer  |                                                            | {}",
        " postgres   | Superuser, Create role, Create DB, Replication, Bypass RLS | {}",
        " replicator | Replication                                                | {}"
    ]
}

TASK [deploy-finish : Get postgresql database list] ************************************************************************************************************************************************************
ok: [10.0.1.1]

TASK [deploy-finish : PostgreSQL list of databases] ************************************************************************************************************************************************************
ok: [10.0.1.1] => {
    "dbs_result.stdout_lines": [
        "   name   |  owner   | encoding |   collate   |    ctype    |  size   | tablespace ",
        "----------+----------+----------+-------------+-------------+---------+------------",
        " postgres | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 7453 kB | pg_default",
        "(1 row)"
    ]
}

TASK [deploy-finish : Check postgresql cluster health] *********************************************************************************************************************************************************
ok: [10.0.1.1]

TASK [deploy-finish : PostgreSQL Cluster health] ***************************************************************************************************************************************************************
ok: [10.0.1.1] => {
    "patronictl_result.stdout_lines": [
        "+ Cluster: vitaliy-test-pgcluster +----------+---------+-----------+----+-----------+",
        "| Member                          | Host     | Role    | State     | TL | Lag in MB |",
        "+---------------------------------+----------+---------+-----------+----+-----------+",
        "| vitaliy-test-pgcluster-pgnode01 | 10.0.1.1 | Leader  | running   |  1 |           |",
        "| vitaliy-test-pgcluster-pgnode02 | 10.0.1.2 | Replica | streaming |  1 |         0 |",
        "| vitaliy-test-pgcluster-pgnode03 | 10.0.1.3 | Replica | streaming |  1 |         0 |",
        "+---------------------------------+----------+---------+-----------+----+-----------+"
    ]
}

TASK [deploy-finish : Create list of nodes] ********************************************************************************************************************************************************************
ok: [10.0.1.1]

TASK [deploy-finish : PostgreSQL Cluster connection info] ******************************************************************************************************************************************************
ok: [10.0.1.1] => {
    "msg": [
        "+------------------------------------------------+",
        "address 10.0.1.1,10.0.1.2,10.0.1.3",
        "port 6432 (pgbouncer)",
        "+------------------------------------------------+"
    ]
}

PLAY RECAP *****************************************************************************************************************************************************************************************************
10.0.1.1                   : ok=104  changed=58   unreachable=0    failed=0    skipped=348  rescued=0    ignored=0   
10.0.1.2                   : ok=89   changed=55   unreachable=0    failed=0    skipped=328  rescued=0    ignored=0   
10.0.1.3                   : ok=89   changed=55   unreachable=0    failed=0    skipped=328  rescued=0    ignored=0   
localhost                  : ok=24   changed=6    unreachable=0    failed=0    skipped=107  rescued=0    ignored=0

image

passed

vitabaks and others added 27 commits May 24, 2024 15:42
@vitabaks
Add variables: PGBACKREST_BACKUP_HOUR and WALG_BACKUP_HOUR
2418305
@vitabaks
Create an inventory.json file if the ANSIBLE_INVENTORY_JSON variable …
f98c4a7 
…is defined
@vitabaks
Merge branch 'master' into cloud
47e3f8e
@vitabaks
Check if SSH_PRIVATE_KEY_CONTENT is set and create the SSH private ke…
dddddcb 
…y file if it is
@vitabaks
Add 'int' filter for postgresql_version variable
ba0ba94
@vitabaks
Add variable 'server_spot'
13ebf1b
@vitabaks
Azure: Add support for Spot instance
b8ef090
@vitabaks
Add azure_admin_username variable
c25748a
@vitabaks
Azure: Add eviction_policy option
c1a0d89
@vitabaks
Add "json_log" callback plugin
853d6b1
@vitabaks
Update json_log.py
a920bce
@vitabaks
Update json_log.py
3d7a12e
@vitabaks
Add timestamp to json_log
1306891
@vitabaks
Do not record the result of a skipped task item
57d6f18
@vitabaks
json_log.py: Add DETAIL level
dd99f1a
@vitabaks
fix flake8 linter warning
5cd3a36
@vitabaks
cloud-resources: Update "Server info" task
51b6927
@vitabaks
rename task "System information" to "System info"
ee43914
@vitabaks
Update task "System info"
260db30
@vitabaks
Update main.yml
2b96de2
@vitabaks
Update main.yml
6101e33
@vitabaks
Update main.yml
da2c3a1
@vitabaks
Update main.yml
a8a29ac
@vitabaks
deploy-finish: fix typo for Hetzner "Connection info" task
b7d7205
@vitabaks
Merge branch 'master' into cloud
7b4be48
@vitabaks
GCP: Load Balancer
2b98634
@vitabaks
Add gcp_compute_backend_service_log_enable variable
0fdb300
@vitabaks
Copy link
Owner Author

Test: GCP Load Balancer

Commit: 2b98634

Note: Currently, only Global External classic proxy Network Load Balancer is supported.

ansible-playbook deploy_pgcluster.yml --extra-vars \
  "ansible_user=root \
   cloud_provider='gcp' \
   cloud_load_balancer=true \
   server_spot=true \
   server_count=3 \
   server_type='e2-medium' \
   server_image='projects/ubuntu-os-cloud/global/images/family/ubuntu-2204-lts' \
   server_location='us-east1' \
   system_volume_size=50 \
   volume_size=50 \
   postgresql_version=16 \
   pgbouncer_install=true \
   patroni_cluster_name=vitabaks-pgcluster \
   ssh_public_keys='ssh-rsa AAAAB3NzaC1yc2EAAA*****3wX2uPq35NlVL6Bn/whzcMINzKKCc7AVGbki'"

Result:


PLAY [Deploy PostgreSQL HA Cluster (based on "Patroni")] ********************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Include main variables] ***********************************************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Generate a unique temporary SSH key name] ***********************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Generate a new temporary SSH key to access the server for deployment] *******************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : Set variable: ssh_key_name and ssh_key_content] *****************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Ensure that 'google-auth' dependency is present on controlling host] ********************************************************************************************************************
ok: [localhost -> 127.0.0.1]

TASK [cloud-resources : Lookup the GCP_SERVICE_ACCOUNT_CONTENTS environmental variable] *************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Set variable: gcp_service_account_contents] *********************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : GCP: Gather information about project] **************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : Set variable: gcp_network_name] *********************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : GCP: Gather information about network] **************************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : GCP: Extract ip_range for network 'default'] ********************************************************************************************************************************************
ok: [localhost]

TASK [cloud-resources : GCP: Create or modify SSH public firewall rule] *****************************************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : GCP: Create or modify Postgres cluster firewall rule] ***********************************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : GCP: Create health checks and LB firewall rule] *****************************************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : GCP: Create or modify VM instance] ******************************************************************************************************************************************************
changed: [localhost] => (item=vitabaks-pgcluster-pgnode01)
changed: [localhost] => (item=vitabaks-pgcluster-pgnode02)
changed: [localhost] => (item=vitabaks-pgcluster-pgnode03)

TASK [cloud-resources : GCP: [Load Balancer] Create instance group] *********************************************************************************************************************************************
changed: [localhost]

TASK [cloud-resources : GCP: [Load Balancer] Create health check] ***********************************************************************************************************************************************
changed: [localhost] => (item=vitabaks-pgcluster-primary-hc)
changed: [localhost] => (item=vitabaks-pgcluster-replica-hc)

TASK [cloud-resources : GCP: [Load Balancer] Create backend service] ********************************************************************************************************************************************
changed: [localhost] => (item=vitabaks-pgcluster-primary)
changed: [localhost] => (item=vitabaks-pgcluster-replica)

TASK [cloud-resources : GCP: [Load Balancer] Create target TCP proxy] *******************************************************************************************************************************************
changed: [localhost] => (item=vitabaks-pgcluster-primary-proxy)
changed: [localhost] => (item=vitabaks-pgcluster-replica-proxy)

TASK [cloud-resources : GCP: [Load Balancer] Reserve static IP address] *****************************************************************************************************************************************
changed: [localhost] => (item=vitabaks-pgcluster-primary-ip)
changed: [localhost] => (item=vitabaks-pgcluster-replica-ip)

TASK [cloud-resources : GCP: [Load Balancer] Create forwarding rule] ********************************************************************************************************************************************
changed: [localhost] => (item=vitabaks-pgcluster-primary-fr)
changed: [localhost] => (item=vitabaks-pgcluster-replica-fr)

TASK [cloud-resources : Wait for host to be available via SSH] **************************************************************************************************************************************************
ok: [localhost] => (item=34.75.150.237)
ok: [localhost] => (item=34.74.24.56)
ok: [localhost] => (item=34.148.172.232)

TASK [cloud-resources : Server info] ****************************************************************************************************************************************************************************
ok: [localhost] => (item=34.75.150.237) => {
    "msg": {
        "id": "7188753037207374462",
        "image": "ubuntu-2204-lts",
        "name": "vitabaks-pgcluster-pgnode01",
        "private_ip": "10.142.0.16",
        "public_ip": "34.75.150.237",
        "type": "e2-medium",
        "volume_size": "50 GB"
    }
}
ok: [localhost] => (item=34.74.24.56) => {
    "msg": {
        "id": "4278690618588367475",
        "image": "ubuntu-2204-lts",
        "name": "vitabaks-pgcluster-pgnode02",
        "private_ip": "10.142.0.18",
        "public_ip": "34.74.24.56",
        "type": "e2-medium",
        "volume_size": "50 GB"
    }
}
ok: [localhost] => (item=34.148.172.232) => {
    "msg": {
        "id": "2008758823073974856",
        "image": "ubuntu-2204-lts",
        "name": "vitabaks-pgcluster-pgnode03",
        "private_ip": "10.142.15.229",
        "public_ip": "34.148.172.232",
        "type": "e2-medium",
        "volume_size": "50 GB"
    }
}
...

TASK [deploy-finish : Postgres list of users] *******************************************************************************************************************************************************************
ok: [10.142.0.16] => {
    "msg": [
        "                              List of roles",
        " Role name  |                         Attributes                         ",
        "------------+------------------------------------------------------------",
        " pgbouncer  | ",
        " postgres   | Superuser, Create role, Create DB, Replication, Bypass RLS",
        " replicator | Replication"
    ]
}

TASK [deploy-finish : Postgres list of databases] ***************************************************************************************************************************************************************
ok: [10.142.0.16] => {
    "msg": [
        "                                                       List of databases",
        "   Name    |  Owner   | Encoding | Locale Provider |   Collate   |    Ctype    | ICU Locale | ICU Rules |   Access privileges   ",
        "-----------+----------+----------+-----------------+-------------+-------------+------------+-----------+-----------------------",
        " postgres  | postgres | UTF8     | libc            | en_US.UTF-8 | en_US.UTF-8 |            |           | ",
        " template0 | postgres | UTF8     | libc            | en_US.UTF-8 | en_US.UTF-8 |            |           | =c/postgres          +",
        "           |          |          |                 |             |             |            |           | postgres=CTc/postgres",
        " template1 | postgres | UTF8     | libc            | en_US.UTF-8 | en_US.UTF-8 |            |           | =c/postgres          +",
        "           |          |          |                 |             |             |            |           | postgres=CTc/postgres",
        "(3 rows)"
    ]
}

TASK [deploy-finish : Postgres Cluster info] ********************************************************************************************************************************************************************
ok: [10.142.0.16] => {
    "msg": [
        "+ Cluster: vitabaks-pgcluster (7380334332110398205) ----+-----------+----+-----------+",
        "| Member                      | Host          | Role    | State     | TL | Lag in MB |",
        "+-----------------------------+---------------+---------+-----------+----+-----------+",
        "| vitabaks-pgcluster-pgnode01 | 10.142.0.16   | Leader  | running   |  1 |           |",
        "| vitabaks-pgcluster-pgnode02 | 10.142.0.18   | Replica | streaming |  1 |         0 |",
        "| vitabaks-pgcluster-pgnode03 | 10.142.15.229 | Replica | streaming |  1 |         0 |",
        "+-----------------------------+---------------+---------+-----------+----+-----------+"
    ]
}

TASK [deploy-finish : Connection info] **************************************************************************************************************************************************************************
ok: [10.142.0.16] => {
    "msg": {
        "address": {
            "primary": "34.160.248.138",
            "replica": "34.160.84.41"
        },
        "password": "j0C2fajhB9NFCrGlePz7r0DwOk7TzaYG",
        "port": "6432",
        "superuser": "postgres"
    }
}

PLAY RECAP ******************************************************************************************************************************************************************************************************
10.142.0.16                : ok=123  changed=62   unreachable=0    failed=0    skipped=419  rescued=0    ignored=0   
10.142.0.18                : ok=104  changed=59   unreachable=0    failed=0    skipped=381  rescued=0    ignored=0   
10.142.15.229              : ok=104  changed=59   unreachable=0    failed=0    skipped=381  rescued=0    ignored=0   
localhost                  : ok=30   changed=11   unreachable=0    failed=0    skipped=186  rescued=0    ignored=0

Screenshots:

image

image

  • 1 healthy only - primary

image

  • 2 healthy - replicas

Check Primary

root@vitabaks-pgcluster-pgnode01:~# PGPASSWORD=j0C2fajhB9NFCrGlePz7r0DwOk7TzaYG psql -h 34.160.248.138 -p 6432 -U postgres -d postgres -c "select pg_is_in_recovery()"
 pg_is_in_recovery 
-------------------
 f
(1 row)

Check Replica

root@vitabaks-pgcluster-pgnode01:~# PGPASSWORD=j0C2fajhB9NFCrGlePz7r0DwOk7TzaYG psql -h 34.160.84.41 -p 6432 -U postgres -d postgres -c "select pg_is_in_recovery()"
 pg_is_in_recovery 
-------------------
 t
(1 row)

Switchover

postgres@vitabaks-pgcluster-pgnode01:~$ patronictl switchover vitabaks-pgcluster
Current cluster topology
+ Cluster: vitabaks-pgcluster (7380334332110398205) ----+-----------+----+-----------+
| Member                      | Host          | Role    | State     | TL | Lag in MB |
+-----------------------------+---------------+---------+-----------+----+-----------+
| vitabaks-pgcluster-pgnode01 | 10.142.0.16   | Leader  | running   |  1 |           |
| vitabaks-pgcluster-pgnode02 | 10.142.0.18   | Replica | streaming |  1 |         0 |
| vitabaks-pgcluster-pgnode03 | 10.142.15.229 | Replica | streaming |  1 |         0 |
+-----------------------------+---------------+---------+-----------+----+-----------+
Primary [vitabaks-pgcluster-pgnode01]: 
Candidate ['vitabaks-pgcluster-pgnode02', 'vitabaks-pgcluster-pgnode03'] []: vitabaks-pgcluster-pgnode02
When should the switchover take place (e.g. 2024-06-14T13:33 )  [now]: 
Are you sure you want to switchover cluster vitabaks-pgcluster, demoting current leader vitabaks-pgcluster-pgnode01? [y/N]: y
2024-06-14 12:33:20.17774 Successfully switched over to "vitabaks-pgcluster-pgnode02"
+ Cluster: vitabaks-pgcluster (7380334332110398205) ----+---------+----+-----------+
| Member                      | Host          | Role    | State   | TL | Lag in MB |
+-----------------------------+---------------+---------+---------+----+-----------+
| vitabaks-pgcluster-pgnode01 | 10.142.0.16   | Replica | stopped |    |   unknown |
| vitabaks-pgcluster-pgnode02 | 10.142.0.18   | Leader  | running |  1 |           |
| vitabaks-pgcluster-pgnode03 | 10.142.15.229 | Replica | running |  1 |         0 |
+-----------------------------+---------------+---------+---------+----+-----------+

result:

              now              | pg_is_in_recovery 
-------------------------------+-------------------
 2024-06-14 12:33:15.424919+00 | f
(1 row)

              now              | pg_is_in_recovery 
-------------------------------+-------------------
 2024-06-14 12:33:16.487728+00 | f
(1 row)

             now              | pg_is_in_recovery 
------------------------------+-------------------
 2024-06-14 12:33:17.54989+00 | f
(1 row)

psql: error: connection to server at "34.160.248.138", port 6432 failed: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
psql: error: connection to server at "34.160.248.138", port 6432 failed: FATAL:  server login has been failing, try again later (server_login_retry)
psql: error: connection to server at "34.160.248.138", port 6432 failed: FATAL:  server login has been failing, try again later (server_login_retry)
psql: error: connection to server at "34.160.248.138", port 6432 failed: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
psql: error: connection to server at "34.160.248.138", port 6432 failed: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
psql: error: connection to server at "34.160.248.138", port 6432 failed: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
psql: error: connection to server at "34.160.248.138", port 6432 failed: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
              now              | pg_is_in_recovery 
-------------------------------+-------------------
 2024-06-14 12:33:35.953383+00 | f
(1 row)

             now              | pg_is_in_recovery 
------------------------------+-------------------
 2024-06-14 12:33:37.01567+00 | f
(1 row)
  • the connection was automatically redirected to the new Primary server

passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@vitabaks vitabaks

Labels
in progress new feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@vitabaks