Add matrix and caching to CI image Building

[ranchodeluxe]

Background

Given that our CI unit-test job takes ~5 minutes to run we need to make it faster by using docker-layer caching and might as well build a dynamic matrix.

AC

• build a dynamic matrix of images to build via ls -1 dockerimages/
• the base image tags to pull from should also be dynamically generated from helm-chart/eoapi/values.yaml resources so we don't ever have to update the images if base image tags change for tiitler/tipg/pgstac/stac
• add docker layer caching so we get this really fast

[ividito]

Gave the matrix idea a shot (main...ci/unit-test-optimizations) but didn't have much success. The concurrent jobs still spend a lot of time setting up minikube, and as far as I can tell there isn't a way to take those built images and bring them to another step with a shared minikube cluster.

[ranchodeluxe]

Gave the matrix idea a shot (main...ci/unit-test-optimizations) but didn't have much success. The concurrent jobs still spend a lot of time setting up minikube, and as far as I can tell there isn't a way to take those built images and bring them to another step with a shared minikube cluster.

Thanks for giving this a look @ividito ❤️ I love the & background solution so much b/c I love bash 😄 I'm not sure there's anyway to speed up minikube startup unfortunately and that will always be a limitation. Let me think if there's another route we can take

[ividito]

I think something like what elastic does might work well. If Devseed has an EKS cluster available and ready, we could use OIDC to authorize our actions to helm install to that cluster and run unit tests there, instead of being reliant on minikube.

[ranchodeluxe]

I think something like what elastic does might work well. If Devseed has an EKS cluster available and ready, we could use OIDC to authorize our actions to helm install to that cluster and run unit tests there, instead of being reliant on minikube.

Yeah nice, not opposed to this. It's a question of whether we want to swallow the costs of keeping something running just for unit tests

[ranchodeluxe]

Thanks @ividito for giving me the idea. Seems Atlantis pod already had a GKE cluster set up for this and with the addition of #37 we are now using it and don't need to any of these shenanigans 🥳

It still takes ~3 minutes to run the pipeline for unittests but we can call that a huge speedup for now and focus on other things until later