upload-artifact has a race condition

[mthaddon]

Currently upload-artifact (v4) has a race condition. Per https://github.com/actions/upload-artifact?tab=readme-ov-file#v4---whats-new:
"""
The contents of an Artifact are uploaded together into an immutable archive. They cannot be altered by subsequent jobs unless the Artifacts are deleted and recreated (where they will have a new ID).
"""
However, if you have multiple jobs running at the same time they will all delete the artifact and then try to upload and potentially fail. We saw this on the jenkins-k8s-operator worfklow. Two out of three failed, because they were all trying to upload at exactly "Thu, 21 Mar 2024 10:18:28 GMT".

We should consider adding a retry.

[mthaddon]

Note that there is an upstream issue that would work around this actions/upload-artifact#530