Unused container image layers aren't always deleted after removing image from package registry

[rom4nik]

Description

Steps to hopefully reproduce the issue:

1. Build some container image
   podman build . -t localhost:3000/przemek/test:latest
2. Login to registry
   podman login --tls-verify=false localhost:3000
3. Push image to registry
   podman push --tls-verify=false localhost:3000/przemek/test:latest
4. Delete it from user accessible packages list or from packages list in site administration
5. Total size used (usually) doesn't change, layer files are still present in /var/lib/gitea/packages
6. Running "Cleanup expired packages" sometimes deleted some of the remaining files, but most of the time it didn't.

Can't verify if it's reproducible on try.gitea.io due to verification email not arriving and possibly lack of site administration access afterwards.

Gitea Version

1.17.0

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

https://gist.github.com/rom4nik/a8bd467b43ae2692bf437db7356bfc00

Screenshots

No response

Git Version

2.36.2

Operating System

Arch Linux

How are you running Gitea?

For testing: podman run -p 127.0.0.1:3000:3000 docker.io/gitea/gitea:1.17.0-rootless

Database

SQLite

[KN4CK3R]

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Cleanup expired packages
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;[cron.cleanup_packages]
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Whether to enable the job
;ENABLED = true
;; Whether to always run at least once at start up time (if ENABLED)
;RUN_AT_START = true
;; Whether to emit notice on successful execution too
;NOTICE_ON_SUCCESS = false
;; Time interval for job to run
;SCHEDULE = @midnight
;; Unreferenced blobs created more than OLDER_THAN ago are subject to deletion
;OLDER_THAN = 24h

The OLDER_THAN setting is active too if you run the task from the admin page. So only blobs older than 24h are removed by default.

[rom4nik]

I wasn't aware of this, sorry for making a bogus bug report then. It'd be great if this could be mentioned somewhere on the site admin pages and/or in documentation.