Preloading images using load_base docker image resource as well as docker in docker

[scottillogical]

I am currently working on prefetching various docker images to speed up 2 sections of our pipelines

1. A push to a docker registry using the docker_image resource and load_bases https://github.com/concourse/docker-image-resource
2. A docker-in-docker job that uses docker-compose to run integration tests https://blog.concourse-ci.org/2020/12/29/running-docker-in-concourse.html

In both of these uses cases, especially #2, we want to know ahead of time what docker images are going to be used so we can preload them via concourse GETs and leverage docker caching/avoid external network calls

My main issue with the preloading is that the only way to find out which specific docker images are used is to look at the Dockerfile and docker-compose yml for an individual application. But that can change on a commit to commit basis, while the pipelines are generally not changed along with the application code or Dockerfile.

So my question is how do we leverage docker caching in concourse pipelines while not coupling the pipeline to various individual application dockerfiles and docker-compose files?.