[docker-slim renovate image] replace the default index.docker.io

[misterkramer]

Which Renovate are you using?

Renovate Open Source CLI

Which platform are you using?

Bitbucket Server

Have you checked the logs? Don't forget to include them if relevant

DEBUG: Found poetry version constraint - checking for a compatible renovate/python image to use (repository=~kramer/cz, branch=renovate/all)
       "constraint": "^3.7"
DEBUG: getLabels(https://index.docker.io, renovate/python, latest) (repository=~kramer/cz, branch=renovate/all)
DEBUG: Updated 1 package files (repository=~kramer/cz, branch=renovate/all)
       }
         "stack": "Error: registry-failure\n    at getAuthHeaders (/usr/src/app/node_modules/renovate/dist/datasource/docker/index.js:200:19)\n    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
         "message": "registry-failure",
         },
           }
             "gotTimeout": {"request": 60000}
             "hostType": "docker",
             "method": "GET",
             "useElectronNet": false,
             "cache": false,
             "json": false,
             "form": false,
             "stream": false,
             "followRedirect": true,
             "throwHttpErrors": false,
             "decompress": true,
             "retry": {"methods": {}, "statusCodes": {}, "errorCodes": {}},
             },
               "afterResponse": []
               "beforeRetry": [],
               "beforeRedirect": [null],
               "beforeRequest": [],
               "init": [],
               "beforeError": [],
             "hooks": {
             },
               "accept-encoding": "gzip, deflate"
               "user-agent": "https://github.com/renovatebot/renovate",
             "headers": {
             "href": "https://index.docker.io/v2/",
             "pathname": "/v2/",
             "query": null,
             "search": null,
             "hash": null,
             "hostname": "index.docker.io",
             "port": null,
             "host": "index.docker.io",
             "auth": null,
             "slashes": true,
             "protocol": "https:",
             "path": "/v2/",
           "gotOptions": {
           "url": "https://index.docker.io/v2/",
           "protocol": "https:",
           "path": "/v2/",
           "method": "GET",
           "hostname": "index.docker.io",
           "host": "index.docker.io",
           "code": "ECONNREFUSED",
           "name": "RequestError",
         "err": {
       "err": {
DEBUG: Failed to update poetry.lock file (repository=~kramer/cz, branch=renovate/all)

What would you like to do?

I want to use a slim docker image instead of a full one. In our corporate network, access to https://index.docker.io closed (only registry-mirrors), so an error occurs when executing poetry update. Can I redefine index.docker.io to my registry-mirrors url? I tried all the settings(hostRules, registryUrls), but I can't achieve the desired result.
Please help with the solution.

[rarkins]

Renovate attempts to pull renovate/python in order to install Poetry and perform an update. This is an internal feature/image so not covered by user configuration like registryUrls.

You would need to either:

• Get registry-mirrors to work in the slim image: https://docs.docker.com/registry/recipes/mirror/#configure-the-docker-daemon, OR
• Use the slim image to build and host your own customer Renovate with all required binaries preinstalled (it will be smaller than the full version)

[misterkramer]

@rarkins Thank you for your prompt response.
Yes, I have specified mirrors in the docker daemon settings on the host:

docker info

 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors:
  https://docker.myprivate.com/

but this doesn't work. I still get the ECONNREFUSED .
If I do from the console docker pull renovate/python - image is downloaded successfully, i.e. the mirror is in working condition.

[viceice]

@misterkramer Can you try to map the daemon config to the renovate container? Maybe this is required there.

[rarkins]

You need to be able to run docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock renovate/renovate:slim /bin/bash and then at the prompt have docker pull renovate/python succeed. I would have expected that mapping the docker.sock into the slim container would mean Renovate inherits the daemon settings of the host, but maybe not?

[misterkramer]

You need to be able to run docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock renovate/renovate:slim /bin/bash and then at the prompt have docker pull renovate/python succeed.

yes, it works:

root@bbb01f7f78ee:/usr/src/app# docker info
Client:
 Debug Mode: false

Server:
 Containers: 5
  Running: 4
  Paused: 0
  Stopped: 1
 Images: 12
 Server Version: 19.03.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: systemd
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.4.0-131-generic
 Operating System: Ubuntu 16.04.5 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 24
 Total Memory: 62.58GiB
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors:
  https://docker.myprivate.com/

root@bbb01f7f78ee:/usr/src/app# docker pull renovate/python
Using default tag: latest
latest: Pulling from renovate/python
23884877105a: Already exists
bc38caa0f5b9: Already exists
2910811b6c42: Already exists
36505266dcc6: Already exists
c9adff1658d6: Already exists
e70d623011d0: Already exists
77e39e75cf26: Already exists
413c3363d128: Already exists
1ae6719fa721: Already exists
aa7666ef27e0: Already exists
240e3ada0af3: Already exists
66c14940b1cd: Already exists
c4a0a3c59e5c: Already exists
b34e30f8e827: Pull complete
aedf3c05ef5f: Pull complete
9117966f47d3: Pull complete
cf4ebfaf79a5: Pull complete
Digest: sha256:107b9e0b0c21009d759b42d32ab94a0ae3360bc7805157744cbe50694766dd72
Status: Downloaded newer image for renovate/python:latest
docker.io/renovate/python:latest
root@bbb01f7f78ee:/usr/src/app#

but when I run command root@bbb01f7f78ee:/usr/src/app# renovate in this container I still get the ECONNREFUSED :

DEBUG: Found poetry version constraint - checking for a compatible renovate/python image to use (repository=~kramer/cz, branch=renovate/all)
       "constraint": "^3.7"
DEBUG: Failed to update poetry.lock file (repository=~kramer/cz, branch=renovate/all)
       "err": {
         "err": {
           "name": "RequestError",
           "code": "ECONNREFUSED",
           "host": "index.docker.io",
           "hostname": "index.docker.io",
           "method": "GET",
           "path": "/v2/",
           "protocol": "https:",
           "url": "https://index.docker.io/v2/",
           "gotOptions": {
             "path": "/v2/",
             "protocol": "https:",
             "slashes": true,
             "auth": null,
             "host": "index.docker.io",
             "port": null,
             "hostname": "index.docker.io",
             "hash": null,
             "search": null,
             "query": null,
             "pathname": "/v2/",
             "href": "https://index.docker.io/v2/",
             "headers": {
               "user-agent": "https://github.com/renovatebot/renovate",
               "accept-encoding": "gzip, deflate"
             },
             "hooks": {
               "beforeError": [],
               "init": [],
               "beforeRequest": [],
               "beforeRedirect": [null],
               "beforeRetry": [],
               "afterResponse": []
             },
             "retry": {"methods": {}, "statusCodes": {}, "errorCodes": {}},
             "decompress": true,
             "throwHttpErrors": false,
             "followRedirect": true,
             "stream": false,
             "form": false,
             "json": false,
             "cache": false,
             "useElectronNet": false,
             "method": "GET",
             "hostType": "docker",
             "gotTimeout": {"request": 60000}
           }
         },
         "message": "registry-failure",
         "stack": "Error: registry-failure\n    at getAuthHeaders (/usr/src/app/node_modules/renovate/dist/datasource/docker/index.js:200:19)\n    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
       }

[rarkins]

OK, I think I understand now where the problem is.

Our docker datasource is called to get a list of compatible tags:

renovate/lib/datasource/docker/index.ts

Line 617 in 342bfd4

export async function getReleases({

It's called from here:

renovate/lib/util/exec/docker/index.ts

Line 86 in 342bfd4

const imageReleases = await getReleases({ lookupName });

Because no custom registryUrls are passed, it fails. We'll need to think of a new way to configure this, moving to the main repo.

[misterkramer]

May be to use hostRules?

[viceice]

I think we can pass registryUrls. This would possible solve other own hosting renovate images scenarios.

[rarkins]

The challenge is if you passed the registryUrls value from poetry artifacts -> docker exec -> docker datasource then you'd end up with the poetry registryUrls being passed to docker datasource.

[viceice]

Right, what about passing it while calling setDirectories. Of cause renaming the function then an passing all admin config.

[rarkins]

Can we detect the changed default host using a Docker command? If so then we can lazily detect it the first time there’s a Docker exec

[viceice]

Sure, we could call docker info to return a json object and filter the mirror config, or only use the format argument to print the mirror config.

[rarkins]

In that case let's do it that way and then pass registryUrls to docker datasource if necessary.

[viceice]

@misterkramer would it be ok for you to configure dockerImagePrefix=docker.myprivate.com/renovate, which should solve this issue with pr #7164

[renovate-release]

🎉 This issue has been resolved in version 23.20.0 🎉

The release is available on:

• GitHub release
• 23.20.0

Your semantic-release bot 📦🚀