prefect deploy --all only needs to run build steps once - does it for each deployment

[jonaslb]

First check

• I added a descriptive title to this issue.
• I used the GitHub search to find a similar request and didn't find it.
• I searched the Prefect documentation for this feature.

Prefect Version

2.x

Describe the current behavior

I'm trying to set up a project with multiple flows/deployments which will use the prefect_docker.projects.steps.build_docker_image build step in the project level prefect.yaml file.

But as I am testing it now, it seems that the build (and push) of the docker image is performed for each deployment, in spite of being project level defined?

Fortunately docker does know one or two things about caching, but it still seems to take several seconds to go through all the steps and attempt to do a push to the registry - for each deployment. This hardly makes sense since the image is project-wide, so it'll be the same for each deployment.

Describe the proposed behavior

Currently the build steps are executed in prefect.cli.deploy._run_single_deploy which is executed for every deployment. Here I would probably move it into prefect.cli.deploy.deploy and call it only once either before or after actually creating the deployments.

Example Use

This would just be a pain relief, saving time for anyone (e.g. CD) running prefect deploy --all

Additional context

No response

[zhen0]

@serinamarie - one for you to consider

[jacob-r-g]

+1 I am facing this same annoyance

edit: I have also noticed a strange behavior (which may be on the docker side? or the AWS ECR side?) where when i run the prefect deploy --all for the first time after a code change, the image is actually pushed once per flow deployment (to AWS ECR). After this initial run, if i run prefect deploy --all again, there are no uploads (Even if i go into AWS ECR and delete all the different images except the first one). Interestingly, if I cancel out of a prefect deploy --all run half way through then start again, there will be no attempt to push the images for the first half of the flow deployments but there will be a push of separate (though identical) images for each flow that has not run through the deployment process.

@jonaslb have you encountered this? Even if prefect wants to attempt to rebuild and repush the docker image for each flow, i wish i could figure out what is causing it to sometimes actually push that image, vs realize it has already been pushed so pushing an identical image is not necessary. Actually pushing each identical image wastes a lot of time...

[jonaslb]

I decided the prefect build/push etc steps were inappropriate for docker-based projects due to this bug, so build/push in my project now happens with separate commands and a Dockerfile before running prefect deploy --all (those steps are just left empty in prefect). So I haven't seen the behaviour you describe, since I stopped using that part of it.