Add support for using Cloud Build in breeze build-image

[turbaszek]

Description

I would like to build airflow images using external services like Google Cloud Build. In this way I don't have to run the build on local machine and have to design custom CI pipelines to build it (especially for dev purposes).

Use case / motivation

Building production image can take time and running this on notebooks sounds like not an optimal way of doing it. We can take advantage of system dedicated to do this task like Google Cloud Build.

Related Issues

N/A

[turbaszek]

WDYT @potiuk ?

[potiuk]

Why not. Might be a good option to add.

[potiuk]

However It might not be needed . I have found one reason why the images after pulling might run a bit longer (docker-context-files were not empty when I built it last time), I am going to prevent that problem in a next PR and then it shoudl be much faster next time. I think when we finally fix it building image via ci build will be much longer than locally all the time (no matter how fast machine you will have) - it will be network-bound. Sending context and pulling image will take far longer than rebuilding it in all circumstances.

[potiuk]

Fix is in #12534. Once we merge it and I push the images it shoudl be rather fast to rebuild next time

[potiuk]

The 3.6 version (default) should be now fast to pull/build please try it out @turbaszek and let's observe it for a while to see if this is really working in a stable way I 🤞 . Now when rebuild is done it should be pretty much always reasonably fast. Let me know if it is not. I hope I fixed last errors within this part and do not plan any more structural changes that could break it.

[potiuk]

3.7 out

[potiuk]

And 3.8