hubble-relay is not working. Its in ContainerCreating state

[prashantchitta]

Environment:

• Cloud provider or hardware configuration:
  Baremetal

• OS (printf "$(uname -srm)\n$(cat /etc/os-release)\n"):
  RHEL8

• Version of Ansible (ansible --version):
  ansible 2.10.8

• Version of Python (python --version):
  Python 3.9.2

Kubespray version (commit) (git rev-parse --short HEAD):
release-2.21

Network plugin used:
Cilium

Full inventory with variables (ansible -i inventory/sample/inventory.ini all -m debug -a "var=hostvars[inventory_hostname]"):

Command used to invoke ansible:

Output of ansible run:

Anything else do we need to know:

As part of #9613, hubble UI configuration issue was fixed #9567

But hubble-relay is not working. It stays in ContainerCreating status

 "kube-system   hubble-relay-5d9598db84-vd75z        0/1     ContainerCreating   0               7m9s    <none>           instance-2   <none>           <none>",

This is because cilium 1.12 has changed and hubble relay is missing the updates

[habibullinrsh]

  Warning  FailedMount  24m (x10 over 28m)   kubelet            MountVolume.SetUp failed for volume "tls" : configmap "hubble-ca-cert" not found
  Warning  FailedMount  12m (x4 over 24m)    kubelet            Unable to attach or mount volumes: unmounted volumes=[config tls], unattached volumes=[hubble-sock-dir config tls kube-api-access-7xhpw]: timed out waiting for the condition
  Warning  FailedMount  8m22s (x2 over 26m)  kubelet            Unable to attach or mount volumes: unmounted volumes=[tls config], unattached volumes=[tls kube-api-access-7xhpw hubble-sock-dir config]: timed out waiting for the condition
  Warning  FailedMount  2m1s (x21 over 28m)  kubelet            MountVolume.SetUp failed for volume "config" : configmap "hubble-relay-config" not found