PODNAMES as shown from argocli does not match with actual generated pod names

[andyshieh]

Summary

The Argo CLI's watch command should display the Argo generated pod names correctly, which in the new versions that use POD_NAMES=v2 have the format of [defined prefix]-[task template name]-[random number string]

However currently the Argo CLI show pod names as [defined prefix]-[random number string], missing the task template name component.

Example:
I have a workflow with generateName: test-addition and its first step is load-input.
When using argo submit, a workflow ID test-addition-zkzwx was generated.
The step load-input results in an actual pod named test-addition-zkzwx-load-input-2972163892 which shows up from kubectl get pods
But from the argo watch screen, its pod name is shown as test-addition-zkzwx-2972163892

##Environment
v3.2.6 on argoexec, workflow-controller, and argocli
v1.23.1-2+9e8d1bf4219080 on Kubernetes
Kubernetes initialized using microk8s and using the containerd runtime
Environmental variable POD_NAMES=v2 is set for both the argocli deployment and workflow-controller deployment

Diagnostics

To reproduce this bug, use any workflow .yaml file and submit with argo submit workflow.yaml --watch. The pod names shown in the watch screen would be incorrect.

Using kubectl to check confirms that the actual pods do have the V2 name format [defined prefix]-[task template name]-[random number string]. So the pod name generation is working fine. It's just the Argo CLI watch screen not "retrieving" or showing these names right. Perhaps the Argo CLI was implemented in a way that doesn't actually retrieve the actual pod names, but instead using a pre-defined template to display the pod names, and that template wasn't updated together with the V2 POD NAME update.

Aside from the pod names, the workflows run perfectly fine with no errors.

Assigning to @JPZ13 Thank you.

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

[JPZ13]

Thanks @andyshieh! Will have a look at this today