Fail to push to ghcr.io due to expired token

[kwohlfahrt]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug, and:

• The documentation does not mention anything about my problem
• There are no open or closed issues that are related to my problem

Description

I have a Docker image that takes ~15 minutes to build. It fails to push due to what seems to be an expired token, this same workflow worked four months ago.

Expected behaviour

I expect the image to be pushed without issues.

Actual behaviour

I see the error below.

ERROR: failed to solve: failed to push ghcr.io/kwohlfahrt/etcdmon:pr-17: failed to copy: GET https://productionresultssa0.blob.core.windows.net/actions-cache/65d-285998874
--------------------------------------------------------------------------------
RESPONSE 403: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
ERROR CODE: AuthenticationFailed
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:575f15c3-601e-007f-3360-bb586a000000
Time:2025-05-02T12:51:41.2053839Z</Message><AuthenticationErrorDetail>Signature not valid in the specified time frame: Start [Fri, 02 May 2025 12:35:49 GMT] - Expiry [Fri, 02 May 2025 12:45:54 GMT] - Current [Fri, 02 May 2025 12:51:41 GMT]</AuthenticationErrorDetail></Error>

Note the expiry time is 10 minutes (and 5 seconds) after the start time, which is also approximately when the run started. However, the current time is about 15 minutes after the start time, as the build takes about 15 minutes.

Repository URL

ghcr.io/kwohlfahrt/etcdmon

Workflow run URL

https://github.com/kwohlfahrt/etcdmon/actions/runs/14795220088/job/41540969947?pr=17

YAML workflow

The publish step fails in the workflow.

name: Build & Push Image
on:
  push:
    branches:
      - master
    tags:
      - v*
  # All external PRs require approval to run actions
  pull_request: ~

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/build-push-action@v6
        with:
          context: .
          load: true
          tags: etcdmon:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max
      - name: Create kind cluster
        run: |
          kind create cluster --config tests/e2e/kind.yaml --kubeconfig ./tests/e2e/kubeconfig.yaml
      - name: Load operator images
        run: |
          kind load docker-image etcdmon:latest
      - uses: actions/setup-go@v5
        with:
          go-version: "^1.23.4"
      - run: go test -timeout 5m ./...

  publish:
    runs-on: ubuntu-latest
    needs: test
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4
        # For aarch64 builder support
      - uses: docker/setup-qemu-action@v3
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - uses: docker/metadata-action@v5
        id: meta
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
      - uses: docker/build-push-action@v6
        with:
          context: .
          platforms: linux/amd64,linux/arm64
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

Workflow logs

job-logs.txt

BuildKit logs

Additional info

This same workflow succeeded ~4 months ago: https://github.com/kwohlfahrt/etcdmon/actions/runs/12726203195

[kwohlfahrt]

If I run the build-push-action first with push: false, then the second time run it with push: true, this works around the issue. The second build only takes a few seconds (because it is cached) so the credential doesn't expire.

[rquintana-axelspace]

I've been having this exact same issue since beginning of April.
It was really hard to spot because like you mentioned in the beginning, it just kind of blindsides you. Also, the error itself is a bit ambiguous and I thought it was something to do with our credentials for the longest time.

I really hope something is added to extend the token duration or something, because right now I've just had to disable caching which is really annoying.

[rquintana-axelspace]

Also, modifying the cache-from, cache-to properties does not change the outcome.
The only time the build passes is when a week is passed and the cache has been cleared out, so this error doesn't occur. (Also causing more confusion).