S3 location is missing UID when configured against custom S3 provider

[bo0ts]

We are running:

• registry.opensource.zalan.do/acid/postgres-operator:v1.10.1
• OpenShift 4.12
• S3 is provided through MinIO
• Operator is installed through Helm Charts

We use the following configmap to configure our postgres pods:

kind: ConfigMap
apiVersion: v1
metadata:
  name: postgres-pod-config
data:
  AWS_ACCESS_KEY_ID: ...
  AWS_SECRET_ACCESS_KEY: ...
  AWS_S3_FORCE_PATH_STYLE: "true"
  AWS_REGION: us-east-1
  AWS_ENDPOINT: http://s3-backup.intern:9000
  USE_WALG_BACKUP: "true"
  USE_WALG_RESTORE: "true"
  WALG_DISABLE_S3_SSE: "true"
  WAL_S3_BUCKET: backup

After that all backups end up in s3://backup/spilo/${NAMESPACE_NAME}-${INSTANCE_NAME}/wal/${PG_VERSION} and the cluster UID is missing.

Is this intentional and why is this the case? Backup/Restore works except the restore in-place case, which we would like to see working as well.

[jhoelzel]

i can confirm this to be the case with a digitialocean bucket too but i suspect its not the storage but my conf:

apiVersion: v1
kind: ConfigMap
metadata:
  name: postgres-operator-conf
data:
  WAL_S3_BUCKET: prod-postgres
  WAL_BUCKET_SCOPE_PREFIX: ""
  WAL_BUCKET_SCOPE_SUFFIX: "production"
  USE_WALG_BACKUP: "true"
  USE_WALG_RESTORE: "true"
  CLONE_USE_WALG_RESTORE: "true"
  BACKUP_SCHEDULE: '00 1 * * *' # 1 am at night
  AWS_S3_FORCE_PATH_STYLE: "true" # needed for MinIO
  AWS_ENDPOINT: https://nyc3.digitaloceanspaces.com # Endpoint URL to your S3 Endpoint; DO in this example
  AWS_REGION: us-east-1
  BACKUP_NUM_TO_RETAIN: "5"

[c0deaddict]

I also hit this issue, but with S3.. Looked in the source of the operator and found out that if you set the S3 bucket in the env vars of the postgresql resource, it does not configure the env variable WAL_BUCKET_SCOPE_SUFFIX with the uid of the cluster. The operator can be tricked into injecting the env variable by setting in the Helm values:

configAwsOrGcp:
  wal_s3_bucket: configured-per-instance

Now the postgres pods have a uid in the env var WAL_BUCKET_SCOPE_SUFFIX (prepended with a slash) while the bucket can still be overriden via the env.

[FxKu]

Closing this issue, but I leave #2858 as a reminder to improve docs.