[patroni] Support updating database credentials via chart upgrade

[phyrwork]

Is this a request for help?:
Yes

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
Bug report

Version of Helm and Kubernetes:
Helm 2.7.0
Kubernetes 1.8.1-gke1

Which chart:
patroni

What happened:
helm install/upgrade of release succeeds and all pods start correctly, but connection to postgres using "to connect to your database" post-helm-install text fails with

psql: FATAL:  password authentication failed for user "postgres"
FATAL:  password authentication failed for user "postgres"

when attempting to connect as superuser.

This occurs when installing/upgrading a release name which has already has this chart installed to it currently/previously (i.e. purged)

What you expected to happen:
I expected to be able to connect to postgres

How to reproduce it (as minimally and precisely as possible):

# fresh install to not previously used release name
helm upgrade --install patroni-1 incubator/patroni

# login as superuser
PGPASSWORD_SUPERUSER=$(kubectl get secret --namespace default patroni-1 -o jsonpath="{.data.password-superuser}" | base64 --decode)
kubectl run -i --tty --rm psql --image=postgres --env "PGPASSWORD=$PGPASSWORD_SUPERUSER" --command -- psql -U postgres -h patroni-1.default.svc.cluster.local postgres

# auth is successful
> postgres=# 

# logout and upgrade release with identical values (i.e. password not changed)
helm upgrade --install patroni-1 incubator/patroni

# attempt to login again
kubectl run -i --tty --rm psql --image=postgres --env "PGPASSWORD=$PGPASSWORD_SUPERUSER" --command -- psql -U postgres -h patroni-1.default.svc.cluster.local postgres

# authentication fails
> psql: FATAL:  password authentication failed for user "postgres"
> FATAL:  password authentication failed for user "postgres"

# now try completely removing release and redeploying
helm del --purge patroni-1
helm upgrade --install patroni-1 incubator/patroni

# login continues to fail
kubectl run -i --tty --rm psql --image=postgres --env "PGPASSWORD=$PGPASSWORD_SUPERUSER" --command -- psql -U postgres -h patroni-1.default.svc.cluster.local postgres

> psql: FATAL:  password authentication failed for user "postgres"
> FATAL:  password authentication failed for user "postgres"

# now deploy to a new release name on the same cluster and login when ready
helm upgrade --install patroni-2 incubator/patroni
PGPASSWORD_SUPERUSER=$(kubectl get secret --namespace default patroni-2 -o jsonpath="{.data.password-superuser}" | base64 --decode)
kubectl run -i --tty --rm psql --image=postgres --env "PGPASSWORD=$PGPASSWORD_SUPERUSER" --command -- psql -U postgres -h patroni-2.default.svc.cluster.local postgres

> postgres=# 

Anything else we need to know:
n/a

[lchabert]

Do you find any workaround for this issue ?
I have the same behaviour using mongodb or rabbitmq server.

No password change, and if i upgrade my application, following request has been refused: 'authentication failed'

[gak]

I'm having the same issue with both postgres and redis.

Here is a shorter reproduction showing the password changing in the secrets store after an upgrade. I'm using redis in this example because the password is saved as clear text in the config, so it's easier to see what's going on.

> helm install --name password-bug stable/redis
...

> get secret password-bug-redis -o json | jq -r '.["data"]["redis-password"]' | base64 -d
c8pz98ACsD

> kubectl exec password-bug-redis-5484c8d64-dmw5k cat /opt/bitnami/redis/conf/redis.conf  | grep ^requirepass
requirepass c8pz98ACsD

> helm upgrade password-bug stable/redis
...

> kubectl get secret password-bug-redis -o json | jq -r '.["data"]["redis-password"]' | base64 -d
LBXTLIoN3U  # <-- newly generated password

> kubectl exec password-bug-redis-5484c8d64-dmw5k cat /opt/bitnami/redis/conf/redis.conf  | grep ^requirepass
requirepass c8pz98ACsD  # <-- original password

A not so optimal fix is not using the random password generation, and instead hardcode a password in the password parameter, i.e. postgresPassword and redisPassword.

[gak]

Just found the related issue in helm/helm#3053

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[patrickleet]

/remove-lifecycle stale

[stale]

This issue is being automatically closed due to inactivity.