Add Docker registry caching with service-specific cache refs for all services in CI workflows

[Copilot]

This PR extends the Docker Compose CI overlay to add registry-based caching for all services, significantly reducing build times in CI workflows.

Problem

Previously, only the staging service had caching configured in docker-compose.ci.yml. This meant that other services like build, test, dist, and serve were rebuilding from scratch on every CI run, leading to unnecessarily long build times - particularly noticeable in the staging test workflow where "many minutes are spent retrieving and setting up dependencies, for a test that takes seconds to run."

Solution

1. Extended docker-compose.ci.yml to add registry-based caching for all 8 services that build from Dockerfile:

   • Each service now uses its own dedicated cache reference: ghcr.io/orionrobots/orionrobots-site.{service_name}:cache
   • Services: serve, dist, build, test, shell, http_serve, broken_links, staging
   • This prevents cache confusion between different services while maximizing cache reuse

2. Updated workflows to consistently use the CI overlay:

   • on_call_build_site.yaml: Added Docker Buildx setup and CI overlay usage
   • on_call_staging_test.yaml: Updated test service to use CI overlay

3. Added Docker authentication and permissions to enable GitHub Container Registry access:

   • Added Docker login action to both workflows using ghcr.io registry
   • Uses github.actor and github.token for authentication
   • Added packages: write permission to allow pushing cache layers to GHCR
   • Required for registry-based cache to function properly

4. Added Docker environment variables for optimal caching performance:

   • CACHE_IMAGE: orionrobots/orionrobots-site - Defines the cache image name
   • DOCKER_BUILDKIT: 1 - Enables Docker BuildKit for enhanced caching capabilities
   • COMPOSE_DOCKER_CLI_BUILD: 1 - Enables Docker CLI build features for Compose

5. Service-specific cache strategy: Each service uses dedicated cache references like:

   • serve → cache_from/to: type=registry,ref=ghcr.io/orionrobots/orionrobots-site.serve:cache
   • build → cache_from/to: type=registry,ref=ghcr.io/orionrobots/orionrobots-site.build:cache
   • test → cache_from/to: type=registry,ref=ghcr.io/orionrobots/orionrobots-site.test:cache
   • And so on for all services

Benefits

• Faster builds: Subsequent runs will reuse cached Docker layers instead of rebuilding from scratch
• Reduced resource usage: Less time spent downloading and installing dependencies
• Isolated caching: Each service maintains its own cache space, preventing cross-service conflicts
• Enhanced Docker performance: BuildKit and CLI build features provide additional optimization
• Biggest impact on staging tests: Both staging and test services will benefit from cached layers

Example

Before: Every docker compose run build downloads Node.js base image and runs npm ci from scratch.

After: Subsequent runs will show CACHED for layers that haven't changed, dramatically reducing build time. Each service will maintain its own optimized cache.

All changes follow .editorconfig conventions and maintain backward compatibility.

Fixes #239.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[github-actions]

🦙 MegaLinter status: ❌ ERROR

Descriptor	Linter	Files	Fixed	Errors	Warnings	Elapsed time
❌ ACTION	actionlint	5		6	0	0.08s
✅ BASH	bash-exec	1		0	0	0.0s
✅ BASH	shellcheck	1		0	0	0.06s
⚠️ BASH	shfmt	1		1	0	0.01s
❌ C	cppcheck	8		8	0	0.3s
❌ CPP	cppcheck	8		8	0	0.28s
✅ JSON	npm-package-json-lint	yes		no	no	0.41s
✅ JSON	v8r	12		0	0	9.51s
❌ MARKDOWN	markdown-link-check	736		213	0	45.49s
✅ REPOSITORY	gitleaks	yes		no	no	14.64s
✅ REPOSITORY	git_diff	yes		no	no	0.03s
✅ REPOSITORY	grype	yes		no	no	32.62s
✅ REPOSITORY	secretlint	yes		no	no	41.47s
✅ REPOSITORY	syft	yes		no	no	1.78s
✅ REPOSITORY	trivy-sbom	yes		no	no	3.08s
✅ REPOSITORY	trufflehog	yes		no	no	4.72s
✅ XML	xmllint	1		0	0	201.85s
✅ YAML	v8r	13		0	0	7.75s

See detailed report in MegaLinter reports

[dannystaple]

This doesn't appear to be using that cache.
I think each of those workflows needs to log in to ghcr.io using the github.actor and github.token variables.
Something like:

      - name: Login to Docker
        uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ github.token }}

[Copilot]

This doesn't appear to be using that cache.
I think each of those workflows needs to log in to ghcr.io using the github.actor and github.token variables.
Something like:
...

You're absolutely right! I've added the Docker login action to both workflows (on_call_build_site.yaml and on_call_staging_test.yaml) to authenticate with ghcr.io using github.actor and github.token. Without this authentication, the GitHub Actions cache (type=gha) configured in docker-compose.ci.yml wouldn't work. Changes committed in b8f2e43.

[dannystaple]

Looking at the guide at https://testdriven.io/blog/faster-ci-builds-with-docker-cache/ we need more.
I think we need:

env:
  CACHE_IMAGE: mjhea0/docker-ci-cache
  DOCKER_BUILDKIT: 1
  COMPOSE_DOCKER_CLI_BUILD: 1

But with the right cache image name, perhaps orionrobots/orionrobots-site ?

[Copilot]

Looking at the guide at https://testdriven.io/blog/faster-ci-builds-with-docker-cache/ we need more.
I think we need:

...

Added the environment variables you suggested to both workflow files. Used CACHE_IMAGE: orionrobots/orionrobots-site along with DOCKER_BUILDKIT: 1 and COMPOSE_DOCKER_CLI_BUILD: 1 to enable enhanced Docker caching capabilities. Changes committed in 8ffaafe.

[dannystaple]

Ahh - a problem here:

------
failed to solve: failed to fetch oauth token: unexpected status from GET request to https://auth.docker.io/token?scope=repository%3Aorionrobots%2Forionrobots-site%3Apull%2Cpush&service=registry.docker.io: 401 Unauthorized

I think the token needs to be granted the package write scope using the permissions key?

[Copilot]

Ahh - a problem here:

------

...

Fixed the authentication issue by adding packages: write permissions to both workflows and updating all cache references to use ghcr.io/orionrobots/orionrobots-site:cache instead of orionrobots/orionrobots-site:cache. The problem was that the cache was trying to push to Docker Hub while we were authenticating with GitHub Container Registry. Changes committed in 53e6d8d.