Tag pull time not being set

[JanPretzel]

Expected behavior and actual behavior:

When pulling an image from the registry like e.g. docker pull your-reg.io/dw/testimage:foo the pull time of the tag foo is never being set, instead only the pull time of the artifacts is being set. The project log also shows a pull only on the digest: dw/testimage:sha256:74e5ad84a67e9b8ed7609b32dc2460b76175020923d7f494a73a851446222d18.

Instead I would expect both the artifact pull time AND the tag pull time to be set. Currently none of tags in our registry every show a pull time for any tag.

Could this be a setup problem? I don't see any option that would suggest enabling/disabling this behavior.

Steps to reproduce the problem:

For me it is the following steps:

• push any image to any project and tag it (or use an existing one)
• pull it via a tag and check if the pull time of the tag is being set

Versions:

• harbor version: 2.4.1
• docker engine version: 20.10.12
• docker-compose version: 2.2.2

[zyyw]

1. pull time not updated
   docker push harbor-url-prefix/image:tag
docker pull harbor-url-prefix/image:tag
   the output of docker pull in this scenario is similar to this one below:

latest: Pulling from library/redis
Digest: sha256:e4b315ad03a1d1d9ff0c111e648a1a91066c09ead8352d3d6a48fa971a82922c
Status: Image is up to date for harbor-domain/library/redis:latest
harbor-domain/library/redis:latest

It is probably because the images exists locally and not ACTUALLY being pulled from remotely, so that the pull time is not updated.

2. pull time is updated
   docker push harbor-url-prefix/image:tag
docker rmi harbor-url-prefix/image:tag
docker pull harbor-url-prefix/image:tag
   the output of docker rmi is similar to the one below this time:

Untagged: harbor-domain/library/redis:latest
Untagged: 10.92.99.252/library/redis@sha256:e4b315ad03a1d1d9ff0c111e648a1a91066c09ead8352d3d6a48fa971a82922c
Deleted: sha256:9b188f5fb1e6e1c7b10045585cb386892b2b4e1d31d62e3688c6fa8bf9fd32b5
Deleted: sha256:fe7afb618c11b8be098a10564a9a1682f83915bfdbaaa5af48791950d418b2d5
Deleted: sha256:3a284ce371b3431ba30071057478e2db8fc096232b1a84f092c4df9e06a4a3e4
Deleted: sha256:4396548b331d1b748c8ba1542f8da54e0a8b84102d8205440aac61e3941bdf71
Deleted: sha256:c80de70938af062d3c273f9925641ec672fe182a796bb4a096a37963c92e071a
Deleted: sha256:e807dfe0532b9dae274911841bab81588e9e34591a5b809b8da39471fb75fdbd
Deleted: sha256:556c5fb0d91b726083a8ce42e2faaed99f11bc68d3f70e2c7bbce87e7e0b3e10

the output of docker pull is similar to this:

latest: Pulling from library/redis
8ec398bc0356: Pull complete
da01136793fa: Pull complete
cf1486a2c0b8: Pull complete
a44f7da98d9e: Pull complete
c677fde73875: Pull complete
727f8da63ac2: Pull complete
Digest: sha256:e4b315ad03a1d1d9ff0c111e648a1a91066c09ead8352d3d6a48fa971a82922c
Status: Downloaded newer image for harbor-domain/library/redis:latest
harbor-domain/library/redis:latest

In short, the update time is only updated when docker pull actually pulls every blob and manifest.

[JanPretzel]

I just tried with an image that I never pulled from that registry:

docker pull harbor-url-prefix/dw/redis:build-nr-20220111.1
build-nr-20220111.1: Pulling from dw/redis
a2abf6c4d29d: Pull complete 
c7a4e4382001: Pull complete 
4044b9ba67c9: Pull complete 
c8388a79482f: Pull complete 
413c8bb60be2: Pull complete 
1abfd3011519: Pull complete 
5b3aa525800d: Pull complete 
a2d802d18593: Pull complete 
5658245142cf: Pull complete 
2b57b672d325: Pull complete 
7ea5e64e7d80: Pull complete 
b7b0d25c2d99: Pull complete 
0138ca56c254: Pull complete 
31764a755c5a: Pull complete 
0ee268ff3c56: Pull complete 
7bea4a3dd341: Pull complete 
23e2c7efabed: Pull complete 
52bd92ee93a1: Pull complete 
73b0aa53f60d: Pull complete 
ed34bd4a4e3e: Pull complete 
a78ccaa1710c: Pull complete 
e15b95fa1e08: Pull complete 
Digest: sha256:288aae6d8de600250a39721ac3169b478626527ba4e5f9c24e8c1fce0b10fd78
Status: Downloaded newer image for harbor-url-prefix/dw/redis:build-nr-20220111.1
harbor-url-prefix/dw/redis:build-nr-20220111.1

The pull time is still empty:

The log also shows an update of the pull time on the digest instead of the tag:

The API response for the corresponsing tag:

{
    "artifact_id": 1125,
    "id": 1260,
    "immutable": false,
    "name": "build-nr-20220111.1",
    "pull_time": "0001-01-01T00:00:00.000Z",
    "push_time": "2022-01-11T06:20:03.217Z",
    "repository_id": 51,
    "signed": false
  }

[vizv]

I just noticed the OCI index's artifact list has pull_time updated

[rafis]

Maybe you have set environment variable

PULL_TIME_UPDATE_DISABLE: "true"

somethere?

[vizv]

Maybe you have set environment variable

PULL_TIME_UPDATE_DISABLE: "true"

somethere?

I don't (installed with helm chart), for me it's not update artifact's pull_time however pull_time of OCI index's artifacts got updated, see #15591 (comment)

[EugenMayer]

I'am a collegue of @JanPretzel - we are using the helm-chart (https://artifacthub.io/packages/helm/harbor/harbor) and did not use such a setting.

Checked the harbor-core pod and indeed nothing is set

env | grep PULL_TIME_UPDATE_DISABLE

I would even say, the helm chart has no way of setting this at all anyway - neither it is part of the deployment https://github.com/goharbor/harbor-helm/blob/master/templates/core/core-dpl.yaml - so for us it is 'whatever the application default is', fairly sure

[EugenMayer]

Just wanted to ensure this is somehow still on the radar.

I'am not sure if there is a misunderstanding of somewhat, but this should be happening for every harbor installation out there.

I try to repeat what happens here, maybe to sort all all other misunderstandings.

We use the harbor go-harbor/go-client library and when we fetch an models.Artifact the following can be seen

models.Artifact.PullTime is set properly
models.Artifact.Tags.PullTime is always 0

When we check the UI, same result. Looking at the /artifacts-tab the pull time is listed (that will be the artifact pull time), clicking on any of that artifacts and seeing the list of tags, those have no pull time.

I could imagine we might have this special environments to make this bug present, if it is not simply always present
a) use minio-s3 as registry storage (latest minio version, non distributed variant)
b) use any s3 storage
c) have artifacts with multiple tags applied to them

E.g. when somewhat a s3-property is used to get the pull time of a tag, a) might matter

Thank you for considering this issue as a little more important

[wy65701436]

Please make ensure that whether the request sent by your client is pull by tag, then once Harbor receives that request, it will try to update the pull time of that particular tag. However, if the request is pull by digest, Harbor cannot know which tag to update, it will only update the pull time of that specific artifact, as well as the repository update time.

[EugenMayer]

We are exclusively pulling by tag, 100/100 times.

[wy65701436]

We are exclusively pulling by tag, 100/100 times.

Please provide the incremental proxy.log on pull, let’s see the actual request.

[EugenMayer]

Happy to provide those.

A couple of questions to ensure I give you what you asked for:

• I assume you mean harbors Nguni as a proxy, not the k8s ingress?
• how does the segment start/ end you are looking for? I guess something that proves that the tag was actually used
• Is any specific verbosity level required? If yes, how to set it

Hint:
I would obfuscate the domain part consistently using example.com, leaving anything else untouched though.

[EugenMayer]

I would love to provide the logs you asked for if you could tell me which proxy you mean, the one being the API gateway inside the harbor stack or our own ingress - thanks

[github-actions]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[github-actions]

This issue was closed because it has been stalled for 30 days with no activity. If this issue is still relevant, please re-open a new issue.

[larsschwegmann]

Any updates on this?

[EugenMayer]

@larsschwegmann all i can tell you is, it is still broken, more nearly close to 2 years later. Not sure why @wy65701436 has no stakes or interest in fixing this issue, especially considering how important this field is for layer-cleanup tasks.

Maybe @wy65701436 wants to drop a comment why this is not interesting/not on the roadmap or simply not important enough, would be great