What's the best way to cache Bluefin's containers in a homelab?

[colindean]

I've got a couple of Bluefin-running devices and might add another soon. That's starting to look like I could use some kind of caching on my home NAS. Combined with some way to (nightly?) look for the latest update, all these clients could pull from my local copy on a 10G wired network or >1G WiFi instead of being bottlenecked on my home Internet connection.

Would I be best off setting up a Docker registry cache, perhaps as describe in this article? and bootc switching to e.g. myhome.internal/ublue-os/bluefin-nvidia-open:gts

Or is there another better way to do this, perhaps one that isn't much more complicated than brew install bluefin-caching-proxy and pointing my 😉

With my Debian-based setups in the past, I configured Acquire::https::Proxy-Auto-Detect to hit a normal general HTTP(S) proxy running with Squid, fronted by a script that could check if the expected proxy was accessible or not.

[dosubot[bot]]

Good news: bootc natively supports this via the standard /etc/containers/registries.conf mechanism — no bootc switch to a different image name is required.

How it works: bootc uses the same containers/image library as podman, so it automatically honors registry mirrors configured in /etc/containers/registries.conf.d/ [1]. Any mirror config that works with podman works with bootc.

One caveat: Docker Registry's built-in pull-through cache only supports Docker Hub — it cannot proxy ghcr.io [2]. You'll need a registry that supports arbitrary upstream mirrors (e.g., Harbor, Zot, or a distribution/distribution fork with proxy support for non-Hub registries).

Setup outline:

1. Run a local OCI registry with pull-through/proxy support for ghcr.io on your NAS (e.g., myhome.internal:5000).

2. On each Bluefin machine, drop in a mirror config:

sudo mkdir -p /etc/containers/registries.conf.d
cat <<'EOF' | sudo tee /etc/containers/registries.conf.d/50-local-cache.conf
[[registry]]
location = "ghcr.io"

[[registry.mirror]]
location = "myhome.internal:5000"
EOF

3. That's it — bootc upgrade will now try your local mirror first and fall back to ghcr.io if it's unavailable [3]. The config persists across updates since /etc is mutable.

If your local registry uses HTTP (no TLS), add insecure = true under [[registry.mirror]]. You may also need to update /etc/containers/policy.json to accept unsigned images from that mirror.

No need to bootc switch to a different image reference — your machines stay pointed at ghcr.io/ublue-os/bluefin-... and the mirror is transparent.

For pre-warming the cache, a cron job or systemd timer on your NAS running something like skopeo copy docker://ghcr.io/ublue-os/bluefin-nvidia-open:gts docker://myhome.internal:5000/ublue-os/bluefin-nvidia-open:gts nightly would work well.

---

📖 Bluefin documentation

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[jorti]

I'm using a local instance of zot as a image registry mirror and works fine. However, there's a caveat with the solution proposed above: bootc will fail to validate the sigstore signatures. To fix that, you have to enable the sigstore attachments for your image registry mirror with a yaml file in /etc/containers/registries.d/

See: project-zot/zot#4060 (comment)

[colindean]

So adding this to each machine's /etc/containers/registries.d/zot.yaml should suffice?

docker:
  zot.apps.internal/mirror/ghcr/ublue-os:
    use-sigstore-attachments: true
  zot.apps.internal/mirror/quay/toolbx-images:
    use-sigstore-attachments: true
  zot.apps.internal/mirror/quay/fedora/fedora-coreos:
    lookaside: https://builds.coreos.fedoraproject.org/prod/signatures/oci
  zot.apps.internal/mirror/access-redhat:
    lookaside: https://access.redhat.com/webassets/docker/content/sigstore
  zot.apps.internal/mirror/redhat:
    lookaside: https://registry.redhat.io/containers/sigstore

(mostly copying here to save a click and keep all the info in one thread)

[jorti]

So adding this to each machine's /etc/containers/registries.d/zot.yaml should suffice?

Depends on the hostname and registry path that your mirror is using. I'm mirroring all requests to Github, Quay, etc. in those paths, but in your case, following the example shared by dosubot, you will need to use this:

docker:
  myhome.internal:5000/ublue-os:
    use-sigstore-attachments: true

[colindean]

Would you mind sharing your zot config, at least the extensions.sync.registries list?

https://github.com/project-zot/zot/blob/main/examples/config-popular-registries.json seems like the going example but I fear it may not match up directly if the destination field isn't set.

[jorti]

Would you mind sharing your zot config, at least the extensions.sync.registries list?

This is my full zot config.json:

{
  "storage": {
    "rootDirectory": "/var/lib/registry",
    "dedupe": true,
    "gc": true,
    "gcDelay": "1h",
    "gcInterval": "24h"
  },
  "http": {
    "address": "0.0.0.0",
    "port": "5000",
    "compat": [
      "docker2s2"
    ]
  },
  "log": {
    "level": "info"
  },
  "extensions": {
    "search": {
      "enable": true,
      "cve": {
        "updateInterval": "2h"
      }
    },
    "ui": {
      "enable": true
    },
    "mgmt": {
      "enable": true
    },
    "scrub": {
      "enable": true,
      "interval": "48h"
    },
    "sync": {
      "enable": true,
      "credentialsFile": "/etc/zot/credentials.json",
      "registries": [
        {
          "urls": [
            "https://docker.io",
            "https://registry-1.docker.io"
          ],
          "onDemand": true,
          "tlsVerify": true,
          "preserveDigest": true,
          "content": [
            {
              "prefix": "**",
              "destination": "/mirror/docker"
            }
          ]
        },
        {
          "urls": [
            "https://quay.io"
          ],
          "onDemand": true,
          "tlsVerify": true,
          "preserveDigest": true,
          "content": [
            {
              "prefix": "**",
              "destination": "/mirror/quay"
            }
          ]
        },
        {
          "urls": [
            "https://ghcr.io"
          ],
          "onDemand": true,
          "tlsVerify": true,
          "preserveDigest": true,
          "content": [
            {
              "prefix": "**",
              "destination": "/mirror/ghcr"
            }
          ]
        },
        {
          "urls": [
            "https://registry.fedoraproject.org"
          ],
          "onDemand": true,
          "tlsVerify": true,
          "preserveDigest": true,
          "content": [
            {
              "prefix": "**",
              "destination": "/mirror/fedora"
            }
          ]
        },
        {
          "urls": [
            "https://lscr.io"
          ],
          "onDemand": true,
          "tlsVerify": true,
          "preserveDigest": true,
          "content": [
            {
              "prefix": "**",
              "destination": "/mirror/lscr"
            }
          ]
        },
        {
          "urls": [
            "https://registry.gitlab.com"
          ],
          "onDemand": true,
          "tlsVerify": true,
          "preserveDigest": true,
          "content": [
            {
              "prefix": "**",
              "destination": "/mirror/gitlab"
            }
          ]
        },
        {
          "urls": [
            "https://codeberg.org"
          ],
          "onDemand": true,
          "tlsVerify": true,
          "preserveDigest": true,
          "content": [
            {
              "prefix": "**",
              "destination": "/mirror/codeberg"
            }
          ]
        },
        {
          "urls": [
            "https://registry.access.redhat.com"
          ],
          "onDemand": true,
          "tlsVerify": true,
          "preserveDigest": true,
          "content": [
            {
              "prefix": "**",
              "destination": "/mirror/access-redhat"
            }
          ]
        },
        {
          "urls": [
            "https://registry.redhat.io"
          ],
          "onDemand": true,
          "tlsVerify": true,
          "preserveDigest": true,
          "content": [
            {
              "prefix": "**",
              "destination": "/mirror/redhat"
            }
          ]
        }
      ]
    }
  }
}