pgbackrest host is not getting used for multiple clusters

[kumarashish071092]

I have multiple environments such as dev, qa, and UAT . All the environments have two node clusters but pgbackrest host is shared by all.

When I execute the deploy_cluster.yml playbook, the global is set to repo1-path and it follows for all environments hence only one environment is visible and the rest are giving the below errors:

postgres@pgbackrest:~$ pgbackrest backup --type=full --stanza=supplier-connect-qa
WARN: configuration file contains invalid option 'include-path'
2024-02-15 10:35:37.779 P00   INFO: backup command begin 2.49: --archive-check --no-archive-copy --backup-standby --exec-id=2477007-fbadf106 --log-level-console=info --log-level-file=detail --log-path=/var/log/pgbackrest --pg1-host=10.X.X.X --pg2-host=10.X.X.X --pg1-path=/mnt/Postgres/postgresql/15/supplier-connect-qa/data --pg2-path=/mnt/Postgres/postgresql/15/supplier-connect-qa/data --pg1-port=35432 --pg2-port=35432 --repo1-block --repo1-bundle --repo1-path=/mnt/Postgres/pgbackrest/supplier-connect-dev --repo1-retention-archive=4 --repo1-retention-full=4 --repo1-type=posix --no-resume --stanza=supplier-connect-qa --start-fast --stop-auto --type=full
ERROR: [055]: unable to load info file '/mnt/Postgres/pgbackrest/supplier-connect-dev/backup/supplier-connect-qa/backup.info' or '/mnt/Postgres/pgbackrest/supplier-connect-dev/backup/supplier-connect-qa/backup.info.copy':
       FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-dev/backup/supplier-connect-qa/backup.info' for read
       FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-dev/backup/supplier-connect-qa/backup.info.copy' for read
       HINT: backup.info cannot be opened and is required to perform a backup.
       HINT: has a stanza-create been performed?
2024-02-15 10:35:37.780 P00   INFO: backup command end: aborted with exception [055]

how can I set up my backrest config which can create separate repos for all environments such as :

/mnt/Postgres/pgbackrest/supplier-connect-dev
/mnt/Postgres/pgbackrest/supplier-connect-qa
/mnt/Postgres/pgbackrest/supplier-connect-uat

[vitabaks]

The repo1-path must be the same in all configurations of the database server and backup repository. I don't see any problems with repo1-path being for example /mnt/data/pgbackrest

Backups will be distributed to different directories according to the stanza name, so it is enough to have a unique stanza name for each cluster (see pgbackrest_stanza variable).

[vitabaks]

I usually use different pgbackrest servers for different environments.

[kumarashish071092]

Hi Vitaliy, I am attaching my pgbackrest configurations and parameters. I have tried all the options but still getting errors . The error with the attached configuration is below : ``` ***@***.***:/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev$ pgbackrest backup --type=full --stanza=supplier-connect-dev WARN: configuration file contains invalid option 'include-path' 2024-02-16 06:20:34.253 P00 INFO: backup command begin 2.49: --archive-check --no-archive-copy --backup-standby --exec-id=2665452-851c74ab --log-level-console=info --log-level-file=detail --log-path=/var/log/pgbackrest --pg1-host=10.X.X.X --pg2-host=10.X.X.X --pg1-path=/mnt/Postgres/postgresql/15/supplier-connect-dev/data --pg2-path=/mnt/Postgres/postgresql/15/supplier-connect-dev/data --pg1-port=35432 --pg2-port=35432 --repo1-block --repo1-bundle --repo1-path=/mnt/Postgres/pgbackrest/supplier-connect-qa --repo1-retention-archive=4 --repo1-retention-full=4 --repo1-type=posix --no-resume --stanza=supplier-connect-dev --start-fast --stop-auto --type=full ERROR: [055]: unable to load info file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info' or '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info.copy': FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info' for read FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info.copy' for read HINT: backup.info cannot be opened and is required to perform a backup. HINT: has a stanza-create been performed? 2024-02-16 06:20:34.254 P00 INFO: backup command end: aborted with exception [055] ***@***.***:/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev$ ``` why it is looking for supplier-connect-dev stanza info in supplier-connect-qa directory? Also repo1-path is pointing to QA directory. what I am missing here? Regards, Ashish

…

________________________________ From: Vitaliy Kukharik ***@***.***> Sent: Friday, February 16, 2024 3:33 AM To: vitabaks/postgresql_cluster ***@***.***> Cc: Kumar Ashish ***@***.***>; Author ***@***.***> Subject: Re: [vitabaks/postgresql_cluster] pgbackrest host is not getting used for multiple clusters (Issue #576) [Email from a non-Nagarro source: please exercise caution with links and attachments] The repo1-path must be the same in all configurations of the database server and backup repository. I don't see any problems with repo1-path being for example /mnt/data/pgbackrest Backups will be distributed to different directories according to the stanza name, so it is enough to have a unique stanza name for each cluster. — Reply to this email directly, view it on GitHub<#576 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVXUU3VDD3JDHDATQAHBDDTYT2AZXAVCNFSM6AAAAABDKPNZ2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBXGQYTCNZVGU>. You are receiving this because you authored the thread.Message ID: ***@***.***> pgbackrest_install: true # or 'true' pgbackrest_install_from_pgdg_repo: true # or 'false' pgbackrest_stanza: "{{ patroni_cluster_name }}" # specify your --stanza pgbackrest_repo_type: "posix" # or "s3", "gcs", "azure" pgbackrest_repo_host: "10.10.9.122" # dedicated repository host (optional) pgbackrest_repo_user: "postgres" pgbackrest_conf_file: "/etc/pgbackrest/pgbackrest.conf" # see more options https://pgbackrest.org/configuration.html pgbackrest_conf: global: # [global] section - { option: "log-level-file", value: "detail" } - { option: "log-path", value: "/var/log/pgbackrest" } - { option: "repo1-host", value: "{{ pgbackrest_repo_host }}" } - { option: "repo1-host-user", value: "{{ pgbackrest_repo_user }}" } - { option: "repo1-type", value: "{{ pgbackrest_repo_type | lower }}" } - { option: "repo1-path", value: "/mnt/Postgres/pgbackrest/{{ patroni_cluster_name }}" } - { option: "repo1-retention-full", value: "4" } - { option: "repo1-retention-archive", value: "4" } - { option: "start-fast", value: "y" } - { option: "stop-auto", value: "y" } - { option: "resume", value: "n" } - { option: "link-all", value: "y" } # - { option: "", value: "" } stanza: # [stanza_name] section - { option: "pg1-path", value: "{{ postgresql_data_dir }}" } - { option: "recovery-option", value: "recovery_target_action=promote" } - { option: "log-level-console", value: "info" } - { option: "process-max", value: "4" } - { option: "repo1-path", value: "/mnt/Postgres/pgbackrest/{{ patroni_cluster_name }}" } # - { option: "", value: "" } # (optional) dedicated backup server config (if "repo_host" is set) pgbackrest_server_conf: global: - { option: "log-level-file", value: "detail" } - { option: "log-level-console", value: "info" } - { option: "log-path", value: "/var/log/pgbackrest" } - { option: "repo1-type", value: "{{ pgbackrest_repo_type | lower }}" } - { option: "repo1-path", value: "/mnt/Postgres/pgbackrest/{{ patroni_cluster_name }}" } - { option: "repo1-retention-full", value: "4" } - { option: "repo1-retention-archive", value: "4" } - { option: "repo1-bundle", value: "y" } - { option: "repo1-block", value: "y" } - { option: "start-fast", value: "y" } - { option: "stop-auto", value: "y" } - { option: "resume", value: "n" } - { option: "link-all", value: "y" } - { option: "archive-check", value: "y" } - { option: "archive-copy", value: "n" } - { option: "backup-standby", value: "y" } # - { option: "port", value: "{{ postgresql_port }}" } # the stanza section will be generated automatically pgbackrest_archive_command: "pgbackrest --stanza={{ pgbackrest_stanza }} archive-push %p"

[kumarashish071092]

It worked with below config in both pgbackrest_server_conf and pgbackrest-conf for global 😂 repo1-path : '/mnt/postgres/pgbackrest' removing cluster name form repo1-path solved the issue. We can close this now . Thanks , Ashish

…

________________________________ From: Kumar Ashish ***@***.***> Sent: Friday, February 16, 2024 4:57 PM To: vitabaks/postgresql_cluster ***@***.***>; vitabaks/postgresql_cluster ***@***.***> Cc: Author ***@***.***> Subject: Re: [vitabaks/postgresql_cluster] pgbackrest host is not getting used for multiple clusters (Issue #576) Hi Vitaliy, I am attaching my pgbackrest configurations and parameters. I have tried all the options but still getting errors . The error with the attached configuration is below : ***@***.***:/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev$ pgbackrest backup --type=full --stanza=supplier-connect-dev WARN: configuration file contains invalid option 'include-path' 2024-02-16 06:20:34.253 P00 INFO: backup command begin 2.49: --archive-check --no-archive-copy --backup-standby --exec-id=2665452-851c74ab --log-level-console=info --log-level-file=detail --log-path=/var/log/pgbackrest --pg1-host=10.X.X.X --pg2-host=10.X.X.X --pg1-path=/mnt/Postgres/postgresql/15/supplier-connect-dev/data --pg2-path=/mnt/Postgres/postgresql/15/supplier-connect-dev/data --pg1-port=35432 --pg2-port=35432 --repo1-block --repo1-bundle --repo1-path=/mnt/Postgres/pgbackrest/supplier-connect-qa --repo1-retention-archive=4 --repo1-retention-full=4 --repo1-type=posix --no-resume --stanza=supplier-connect-dev --start-fast --stop-auto --type=full ERROR: [055]: unable to load info file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info' or '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info.copy': FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info' for read FileMissingError: unable to open missing file '/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev/backup.info.copy' for read HINT: backup.info cannot be opened and is required to perform a backup. HINT: has a stanza-create been performed? 2024-02-16 06:20:34.254 P00 INFO: backup command end: aborted with exception [055] ***@***.***:/mnt/Postgres/pgbackrest/supplier-connect-qa/backup/supplier-connect-dev$ why it is looking for supplier-connect-dev stanza info in supplier-connect-qa directory? Also repo1-path is pointing to QA directory. what I am missing here? Regards, Ashish

________________________________ From: Vitaliy Kukharik ***@***.***> Sent: Friday, February 16, 2024 3:33 AM To: vitabaks/postgresql_cluster ***@***.***> Cc: Kumar Ashish ***@***.***>; Author ***@***.***> Subject: Re: [vitabaks/postgresql_cluster] pgbackrest host is not getting used for multiple clusters (Issue #576) [Email from a non-Nagarro source: please exercise caution with links and attachments] The repo1-path must be the same in all configurations of the database server and backup repository. I don't see any problems with repo1-path being for example /mnt/data/pgbackrest Backups will be distributed to different directories according to the stanza name, so it is enough to have a unique stanza name for each cluster. — Reply to this email directly, view it on GitHub<#576 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVXUU3VDD3JDHDATQAHBDDTYT2AZXAVCNFSM6AAAAABDKPNZ2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBXGQYTCNZVGU>. You are receiving this because you authored the thread.Message ID: ***@***.***>