Unable to add pgbouncer using Helm chart

[jayon-niravel]

Official Helm Chart version

1.10.0 (latest released)

Apache Airflow version

2.6.0

Kubernetes Version

1.23

Helm Chart configuration

VALUES.YAML

# Otherwise pass connection values in metadataConnection: user: postgres pass: postgres protocol: postgresql host: airflow-postgresql port: 5432 db: airflow sslmode: disable /# resultBackendConnection defaults to the same database as metadataConnection resultBackendConnection: ~ /# or, you can use a different database /# resultBackendConnection: /# user: postgres /# pass: postgres /# protocol: postgresql /# host: ~ /# port: 5432 /# db: postgres /# sslmode: disable /# Note: brokerUrl can only be set during install, not upgrade brokerUrl: ~

pgbouncer:

Enable PgBouncer

enabled: true

Number of PgBouncer replicas to run in Deployment

replicas: 1

Max number of old replicasets to retain

revisionHistoryLimit: ~

Command to use for PgBouncer(templated).

command: ["pgbouncer", "-d", "/etc/pgbouncer/pgbouncer.ini"]

Args to use for PgBouncer(templated).

args: ~
auth_type: md5
auth_file: /etc/pgbouncer/users.txt

annotations to be added to the PgBouncer deployment

annotations: {}

podAnnotations: {}

Create ServiceAccount

serviceAccount:
# Specifies whether a ServiceAccount should be created
create: true
# The name of the ServiceAccount to use.
# If not set and create is true, a name is generated using the release name
name: ~

# Annotations to add to worker kubernetes service account.
annotations: {}

Additional network policies as needed

extraNetworkPolicies: []

Pool sizes

metadataPoolSize: 10
resultBackendPoolSize: 5

Maximum clients that can connect to PgBouncer (higher = more file descriptors)

maxClientConn: 100

supply the name of existing secret with pgbouncer.ini and users.txt defined

you can load them to a k8s secret like the one below

apiVersion: v1

kind: Secret

metadata:

name: pgbouncer-config-secret

data:

pgbouncer.ini: <base64_encoded pgbouncer.ini file content>

users.txt: <base64_encoded users.txt file content>

type: Opaque

configSecretName: pgbouncer-config-secret

configSecretName: airflow-pgbouncer-config

PgBouncer pod disruption budget

podDisruptionBudget:
enabled: false

# PDB configuration
config:
  maxUnavailable: 1

Limit the resources to PgBouncer.

When you specify the resource request the k8s scheduler uses this information to decide which node to

place the Pod on. When you specify a resource limit for a Container, the kubelet enforces those limits so

that the running container is not allowed to use more of that resource than the limit you set.

See: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/

Example:

resource:

limits:

cpu: 100m

memory: 128Mi

requests:

cpu: 100m

memory: 128Mi

resources: {}

service:
extraAnnotations: {}

https://www.pgbouncer.org/config.html

verbose: 0
logDisconnections: 0
logConnections: 0

sslmode: "prefer"
ciphers: "normal"

ssl:
ca: ~
cert: ~
key: ~

Add extra PgBouncer ini configuration in the databases section:

https://www.pgbouncer.org/config.html#section-databases

extraIniMetadata: ~
extraIniResultBackend: ~

Add extra general PgBouncer ini configuration: https://www.pgbouncer.org/config.html

extraIni: ~

Mount additional volumes into pgbouncer.

extraVolumes: []
extraVolumeMounts: []

Select certain nodes for PgBouncer pods.

nodeSelector: {}
affinity: {}
tolerations: []
topologySpreadConstraints: []

priorityClassName: ~

uid: 1001180000

metricsExporterSidecar:
resources: {}
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
sslmode: "disable"

livenessProbe:
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 1

readinessProbe:
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 1``

Docker Image customizations

pgbouncer.ini

`[databases]
airflow-metadata = host=airflow-postgresql dbname=airflow port=5432 pool_size=10 auth_user=postgres
airflow-result-backend = host=airflow-postgresql dbname=airflow port=5432 pool_size=5 auth_user=postgres

[pgbouncer]
logfile = /var/log/pgbouncer/pgbouncer.log
pidfile = /var/run/pgbouncer/pgbouncer.pid
pool_mode = transaction
listen_port = 6543
listen_addr = *
auth_type = md5
auth_file = /etc/pgbouncer/users.txt
ignore_startup_parameters = extra_float_digits
max_client_conn = 100
admin_users = postgres
verbose = 0
log_disconnections = 0
log_connections = 0

server_tls_sslmode = prefer
server_tls_ciphers = normal`

users.txt
users.txt

What happened

Pgbouncer container does not start and there are no error logs.

If I use below command
command: ["pgbouncer", "-u", "postgres", "/etc/pgbouncer/pgbouncer.ini"]
then I would get "FATAL Error: Unable switch to user postgres"

How can I fix this issue?

What you think should happen instead

Pgbouncer container does not start and there are no error logs.

If I use below command
command: ["pgbouncer", "-u", "postgres", "/etc/pgbouncer/pgbouncer.ini"]
then I would get "FATAL Error: Unable switch to user postgres"

How to reproduce

Using above mentioned YAML and values

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

Can you please re-format your issue in the way that it will be possible to read? I will close it, but please create a new one that will be readable. If in doubt - use preview in the field when you enter issue details. This will give you a hint on how the issue looks like for those who look at it.

[jayon-niravel]

@potiuk : Apologies for the formatting
My last minute editing messed it up and I forgot to preview it.

Created a new one as requested: #33206