Docker updates cause subsequent builds to fail

[malfet]

🐛 Describe the bug

Most recent example:
https://hud.pytorch.org/pytorch/pytorch/commit/8a5d9843ff5d5dd865fc922853a15b3e7e459fdb (underlying workflows trunk and pull )

And subsequent commits after this one started to fail as well until I've manually re-run docker image build in https://github.com/pytorch/pytorch/actions/runs/2637584738

We are having those regressions periodically, despite numerous claims that race conditions have been fixed

Versions

CI

cc @ezyang @gchanan @zou3519 @seemethere @malfet @pytorch/pytorch-dev-infra

[seemethere]

Oh interesting. These are failures on test jobs where we don't run a similar calculate-docker-image step meaning we don't even give them the opportunity to build the image locally.

We can add that step to the test jobs as well to make it so they'll build the docker image as well if it's not already built.

As well, it's weird to me that the docker-builds workflow did not run on the trunk push. I'll investigate why that might be the case.

[seemethere]

Submitted:

• ci: Add docker image build to linux test step #81227
• ci: Add push trigger for docker-builds #81228

This should hopefully resolve issues where docker images are not pushed for master commits that do affect docker builds.