Error: "unauthorized: authentication required` in the middle of push process

[ChrisB85]

I'm recieveing unauthorized: authentication required error in the middle of image push process:

root@RYZEN:/home/user/# docker push registry.mydomain.com/someimage:latest
The push refers to repository [registry.mydomain.com/someimage]
5f70bf18a086: Layer already exists
865c2d40902c: Layer already exists
d6df4d8f1d2b: Layer already exists
814b91d8ce2a: Pushing [==================================================>]  427.3MB
243673d2c35b: Layer already exists
e0e5c14f683a: Pushing [=======================>                           ]  212.8MB/459.5MB
3e8300fe133b: Layer already exists
cd6c2464dc51: Layer already exists
f7f303b50df0: Pushing [==================================================>]  240.8MB
d5fcc5cffc99: Pushing [==================================================>]  268.5MB
1fdbf5f06e1e: Layer already exists
eb5e1abd9327: Pushed
1059c10ff87a: Pushed
3cdaf4f3899c: Pushed
57d046864aa6: Pushed
3eb0486809d0: Pushed
1efc5401b6f1: Pushed
b31fe8530467: Pushed
c151effcd197: Pushed
0b0b8e9d2e04: Pushed
a9ed1f92fa62: Pushing [==================================================>]  13.28MB
6e28a572644e: Waiting
67b5ce3064ab: Waiting
88cd9b949e2e: Waiting
e2ef8a51359d: Waiting
unauthorized: authentication required

In logs I found:

registry_1        | time="2023-08-12T11:07:55.859035163Z" level=warning msg="error authorizing context: invalid token" go.version=go1.19.9 http.request.host=registry.mydomain.com http.request.id=1e427dd3-f10a-4be5-8eff-d7fac425db04 http.request.method=PATCH http.request.remoteaddr=XX.XX.XX.XX http.request.uri="/v2/someimage/blobs/uploads/8c879921-b2e4-43eb-982e-e41eb42e174c?_state=S26m-OteMyJ3enZXXezp-PB6ZJCrGYnk-PxV_pxbtTJ7Ik5hbWUiOiJjb3JvZmxleCIsIlVVSUQiOiI4Yzg3OTkyMS1iMmU0LTQzZWItOTgyZS1lNDFlYjQyZTE3NGMiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjMtMDgtMTJUMTE6MDU6MzguODQ1MzI0MjA2WiJ9" http.request.useragent="docker/24.0.2 go/go1.20.4 git-commit/659604f kernel/5.15.90.1-microsoft-standard-WSL2 os/linux arch/amd64 UpstreamClient(Docker-Client/24.0.2 \(linux\))" vars.name=someimage vars.uuid=8c879921-b2e4-43eb-982e-e41eb42e174c

I'm using NGINX Proxy Manager, so I'm not sure if it's actually registry-admin problem.

[zebox]

Did you docker login before push from docker client host where you try push?

[ChrisB85]

Yes of course, otherwise there would be no Layer already exists messages.

[ChrisB85]

I noticed one more thing that is iportant here:
token not to be used after 2023-08-12 18:49:18 +0000 UTC - currently 2023-08-12 18:50:04.450141787 +0000 UTC m=+1137.976484753

registry_1        | time="2023-08-12T18:50:04.450025916Z" level=debug msg="authorizing request" go.version=go1.19.9 http.request.host=registry.example.com http.request.id=36ba250b-1f75-4e4f-82ef-20505c478ba7 http.request.method=PATCH http.request.remoteaddr=89.67.20.253 http.request.uri="/v2/someimage/blobs/uploads/96cdc261-78c2-413f-bddb-444561dce03b?_state=54yDQGIsRt40E4223ROMISjILe5wVTzsTZ5PCmcQu-N7Ik5hbWUiOiJjb3JvZmxleCIsIlVVSUQiOiI5NmNkYzI2MS03OGMyLTQxM2YtYmRkYi00NDQ1NjFkY2UwM2IiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjMtMDgtMTJUMTg6NDc6MTkuMTc1NjY4OTMzWiJ9" http.request.useragent="docker/24.0.2 go/go1.20.4 git-commit/659604f kernel/5.15.90.1-microsoft-standard-WSL2 os/linux arch/amd64 UpstreamClient(Docker-Client/24.0.2 \(linux\))" vars.name=someimage vars.uuid=96cdc261-78c2-413f-bddb-444561dce03b
registry_1        | time="2023-08-12T18:50:04.450157953Z" level=info msg="token not to be used after 2023-08-12 18:49:18 +0000 UTC - currently 2023-08-12 18:50:04.450141787 +0000 UTC m=+1137.976484753"
registry_1        | time="2023-08-12T18:50:04.450243068Z" level=warning msg="error authorizing context: invalid token" go.version=go1.19.9 http.request.host=registry.example.com http.request.id=36ba250b-1f75-4e4f-82ef-20505c478ba7 http.request.method=PATCH http.request.remoteaddr=89.67.20.253 http.request.uri="/v2/someimage/blobs/uploads/96cdc261-78c2-413f-bddb-444561dce03b?_state=54yDQGIsRt40E4223ROMISjILe5wVTzsTZ5PCmcQu-N7Ik5hbWUiOiJjb3JvZmxleCIsIlVVSUQiOiI5NmNkYzI2MS03OGMyLTQxM2YtYmRkYi00NDQ1NjFkY2UwM2IiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjMtMDgtMTJUMTg6NDc6MTkuMTc1NjY4OTMzWiJ9" http.request.useragent="docker/24.0.2 go/go1.20.4 git-commit/659604f kernel/5.15.90.1-microsoft-standard-WSL2 os/linux arch/amd64 UpstreamClient(Docker-Client/24.0.2 \(linux\))" vars.name=someimage vars.uuid=96cdc261-78c2-413f-bddb-444561dce03b
registry_1        | 172.27.0.2 - - [12/Aug/2023:18:50:04 +0000] "PATCH /v2/someimage/blobs/uploads/96cdc261-78c2-413f-bddb-444561dce03b?_state=54yDQGIsRt40E4223ROMISjILe5wVTzsTZ5PCmcQu-N7Ik5hbWUiOiJjb3JvZmxleCIsIlVVSUQiOiI5NmNkYzI2MS03OGMyLTQxM2YtYmRkYi00NDQ1NjFkY2UwM2IiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjMtMDgtMTJUMTg6NDc6MTkuMTc1NjY4OTMzWiJ9 HTTP/1.1" 401 218 "" "docker/24.0.2 go/go1.20.4 git-commit/659604f kernel/5.15.90.1-microsoft-standard-WSL2 os/linux arch/amd64 UpstreamClient(Docker-Client/24.0.2 \\(linux\\))"

So I suppose token is expiring during push operation.

[zebox]

So I suppose token is expiring during push operation.

I think you are right, because registry-admin use default token TTL (60 seconds). It can be occurred when image has big size and a docker client pushing with chunked upload. PATCH method indicate about it in registry log.

I added TokenTTL option to registry section for registry-admin config

You can try it with updated docker image with tag master

docker pull zebox/registry-admin:master

and add token_ttl option with you TTL in the token-ra-config.yml config file:

# registry-admin config file
... 
registry:
  ...
  token_ttl: 3600 # value in seconds
  ...
...

I will add release tag later, after thoroughly test of all changes.

[ChrisB85]

It works, thank you. I think we can close this ticket.