Latest images tags from Redhat public registry

[javaxiss]

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us what version of Renovate you run.

35.110.2

If you're self-hosting Renovate, select which platform you are using.

GitLab self-hosted

Was this something which used to work for you, and then stopped?

I never saw this working

Wanted end result.

Hello,

I'm using renovate to update Docker images from Redhat public registry : https://registry.access.redhat.com/ which is a Docker V2 registry. The wanted result would be to get the latest images tag returned by renovate bot

What you tried so far.

Renovate is not able to detect latest images version for ubi8-minimal by example. I tried to curl the registry bu running :
curl -sL https://registry.access.redhat.com/v2/ubi8-minimal/tags/list | jq
Only 50 random (not sorted) tags are returned. The response header includes a link for fetching futher results (by adding next_page query parameter to the same request) if required. It seems that the n query parameter is ignored if a value > 50 is set.
I think Renovate is using the same API to get latest tagsfor Docker images But I'm not sure if it is supposed to query multiple times the registry and setting next_page parameter in order to get the right latest tag.

Relevant debug logs

Logs

curl result :

{
  "name": "redhat-prod/ubi8-minimal",
  "tags": [
    "8.4-205.1626828526-source",
    "8.1-409-source",
    "8.0-127",
    "8.2-349",
    "8.6-854-source",
    "8.6-994-source",
    "8.2-339",
    "8.5-204-source",
    "8.3-201-source",
    "8.2-301.1592810506",
    "8.6-902-source",
    "8.5-218-source",
    "8.5-218",
    "8.5-243.1651231653",
    "8.1-398",
    "8.6-854",
    "8.5-240",
    "8.5-243",
    "8.3-298.1618432845",
    "8.2-345-source",
    "8.2",
    "8.3",
    "8.6-994",
    "8.1-407",
    "8.2-339-source",
    "8.0-159",
    "8.1",
    "8.4-200.1622548483",
    "8.6-941",
    "8.2-345",
    "8.4-205-source",
    "8.6-985",
    "8.7-1031-source",
    "8.6-985-source",
    "8.2-301.1592810506-source",
    "8.2-301-source",
    "8.3-291",
    "8.1-409",
    "8.3-298",
    "8.3-201",
    "8.0-213",
    "8.7-923",
    "8.4",
    "8.4-205",
    "8.1-328",
    "8.2-301",
    "8.6",
    "8.5-240.1648458092",
    "8.6-902.1661794353",
    "8.5-240-source"
  ]
}

Renovate logs the following as a latest tag

DEBUG: getLabels(https://registry.access.redhat.com, ubi8-minimal, 8.7-1031-source) (repository=my-repo, baseBranch=feature/my-feature)

[lmilbaum]

@javaxiss Are you a Red Hatter?

[javaxiss]

@lmilbaum I'm not 😇

[lmilbaum]

I am. That is why I've asked.

[viceice]

it probably needs a fix like we do for AWS

renovate/lib/modules/datasource/docker/index.ts

Line 544 in 9aefc20

const limit =

maybe we need to allow more pages then

[javaxiss]

the curl is outputting exactly 50 results even if n query parameter points to more than 50. I think it is needed to use paging to fetch the rest (next_page query param):
tags/list?n=50&next_page=XXXXXXX
That is how skopeo returns all tags from redhat repository, for example.

GET https://registry.access.redhat.com/v2/ubi8/ubi-minimal/tags/list
GET https://registry.access.redhat.com/v2/ubi8/ubi-minimal/tags/list?n=50&next_page=gAAAAABklBPLmkFsJjvQ_ibsAzPpiNW13CyINNgfzxV2HNYfOJQUDeFM1LiiVoA-E8-DI8dytvJYHo8KsyDzkpYf_QAC6TkI84uwsLVbxdeoLM0pvkFnLYY%3D
GET https://registry.access.redhat.com/v2/ubi8/ubi-minimal/tags/list?n=50&next_page=gAAAAABklBPLr2Rj9dQCYuYlzwr7hFITaoTEQnXNVNdliMCvnB2IIPyZYmG4waxQ5g8LvBa7XHTeEUBwp_Ha8BfgNHdBT4c8yHvNHXVKA3Ml1LpbdM3aNrY%3D

[viceice]

this should help:

• feat(datasource/docker): allow override max pages (experimental /self-hosted) #23700

[lmilbaum]

I'm sure that quay.io is the backend. It worth asking if the v1 api is accessible.

[cclaes-dsi]

Is there any update on this? Having the same issue.

[lmilbaum]

I can bring this issue to the Red Had folks who maintain the registry

[cclaes-dsi]

Sorry for the delayed reply. I found the issue.

What was causing the problem for me was that not every public image was added to https://docs.renovatebot.com/presets-workarounds/#workaroundssupportredhatimageversion .

We were using the registry.access.redhat.com/ubi9/ubi-minimal packages.

Copying the workaround config and adding the following 2 lines (not sure which one was the correct fix) fixed the issue:

        "registry.access.redhat.com/ubi9/**",
        "registry.access.redhat.com/ubi9/ubi-minimal",

[javaxiss]

Hi,

@lmilbaum I've created an issue in RedHat support board. They confirmed that problem, and that the response should be parsed to check if another page can be requested (next_page parameter in the response). That is how Skopeo actually gets the latest tags by : skopeo list-tags.

Meanwhile, a new thread has been created in RedHat knowledge base, on the above matters : https://access.redhat.com/solutions/7022895

@cclaes-dsi I think with your last answer you are focusing on ubi9-minimal images, I assume that all the published images would not be more thant 50, that's why all of them return in one page. Could you confirm that ?

[cclaes-dsi]

@javaxiss I'm not sure that's the case. When I added authentication to the public repository config to test something (this was before I added the lines mentioned above), I received the following debug log message:

DEBUG: Using queue: host=registry.access.redhat.com, concurrency=16
DEBUG: No registry auth required for https://registry.access.redhat.com/v2/ubi9/ubi-minimal/tags/list?n=10000

I'm assuming this is the same call that's used for the last solution, since this only lists the ubi-minimal packages. However, as you can see in the link, there's 92 packages available, so I don't think my issue had something to do with the 50+ packages.

I'm pretty sure that for my specific case, Renovatebot wasn't using the correct versioning method. By adding the above lines, the versioning now mentions redhat, before that it would say docker.

[javaxiss]

So, good news.
That means that the query parameter n, is correctly handled. That was not the case when I created the issue.