K8SPSMDB-796: Fix PBM connection if replset is exposed

[egegunes]

If replset is exposed, the operator uses ClusterIPs in the replset config. When PBM tries to open connection to replset using TLS certificates, it fails since these IPs are not included in the certificate. Ideally we want to move away from using unsafe options from TLS but it needs careful thinking.

[JNKPercona]

Test name	Status
storage	passed
self-healing	passed
self-healing-chaos	passed
operator-self-healing	passed
one-pod	passed
init-deploy	passed
operator-self-healing-chaos	passed
monitoring-2-0	passed
limits	passed
arbiter	passed
scaling	passed
service-per-pod	passed
security-context	passed
liveness	passed
rs-shard-migration	passed
cross-site-sharded	passed
smart-update	passed
version-service	passed
users	passed
data-sharded	passed
non-voting	passed
demand-backup-eks-credentials	passed
data-at-rest-encryption	passed
upgrade-consistency	passed
demand-backup	passed
scheduled-backup	passed
demand-backup-sharded	passed
upgrade	passed
upgrade-sharded	passed
pitr	passed
pitr-sharded	passed
We run 31 out of 30

commit: 2ae3023
image: perconalab/percona-server-mongodb-operator:PR-1060-2ae30234

[jonathon2nd]

I'm replying here as my post keeps getting hidden on the forum
https://forums.percona.com/t/basic-cluster-with-tls-not-working-operator-k8s-helm-setup/17844/7

Installed operator with
kubectl apply --server-side -f https://raw.githubusercontent.com/percona/percona-server-mongodb-operator/main/deploy/bundle.yaml

Then installed this cluster

apiVersion: psmdb.percona.com/v1
kind: PerconaServerMongoDB
metadata:
  name: minimal
  namespace: default
spec:
  image: percona/percona-server-mongodb:latest
  replsets:
  - affinity:
      antiAffinityTopologyKey: kubernetes.io/hostname
    name: rs0
    size: 3
    volumeSpec:
      persistentVolumeClaim:
        resources:
          requests:
            storage: 3Gi
  secrets:
    users: minimal
  sharding:
    enabled: false
  upgradeOptions:
    apply: disabled
    schedule: 0 2 * * *

And this cert gets made

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: minimal-ssl
  namespace: default
  ownerReferences:
  - apiVersion: psmdb.percona.com/v1
    controller: true
    kind: PerconaServerMongoDB
    name: minimal
    uid: 6271257d-6ece-4da5-8d51-54ecdebc6ad1
  resourceVersion: "18897670"
  uid: 32945f74-5399-4c6a-9375-a23dcf547986
spec:
  commonName: minimal
  dnsNames:
  - localhost
  - minimal-rs0
  - minimal-rs0.default
  - minimal-rs0.default.svc.cluster.local
  - '*.minimal-rs0'
  - '*.minimal-rs0.default'
  - '*.minimal-rs0.default.svc.cluster.local'
  - minimal-rs0.default.svc.clusterset.local
  - '*.minimal-rs0.default.svc.clusterset.local'
  - '*.default.svc.clusterset.local'
  - minimal-mongos
  - minimal-mongos.default
  - minimal-mongos.default.svc.cluster.local
  - '*.minimal-mongos'
  - '*.minimal-mongos.default'
  - '*.minimal-mongos.default.svc.cluster.local'
  - minimal-cfg
  - minimal-cfg.default
  - minimal-cfg.default.svc.cluster.local
  - '*.minimal-cfg'
  - '*.minimal-cfg.default'
  - '*.minimal-cfg.default.svc.cluster.local'
  - minimal-mongos.default.svc.clusterset.local
  - '*.minimal-mongos.default.svc.clusterset.local'
  - minimal-cfg.default.svc.clusterset.local
  - '*.minimal-cfg.default.svc.clusterset.local'

here are the full log files
minimal-rs0-0_mongod.log
minimal-rs0-1_mongod.log
minimal-rs0-2_mongod.log