Cleanup functionality - e.g. keep "latest" and things newer than <n> days

[DonMartin76]

Is a functionality on the roadmap which is able to automatically purge images via some criteria (to be discussed)?

We are quickly approaching some hard limits (500 GB in our case) and need to find a way to clean things up.

Our requirement in the first iteration would be something like: "Keep any manifest marked latest, and additionally anything newer than n days."

We are considering scripting this (somehow, e.g. in Python or node), but wouldn't bother if you say it's just around the corner. Or, even better, do you know of anybody who has already done something similar?

The data is readily available over the API, as it seems; e.g. az acr repository show-manifests -n <name> --repository <repo> returns exactly the kind of data which would be needed to implement such a functionality - each manifest has a list of tags, and additionally a timestamp.

[SteveLasker]

Hi Don,
As you noted, image policy management is listed on our roadmap; https://aka.ms/acr/roadmap
We have a conceptual design that allows you to set an expiration policy on specific tags. This way you can keep your released images for x days, while a default policy will delete them after fewer days.
We are currently working on the OS & Framework patching scenarios. But, have image management queued up soon after.
Don't worry about the perceived hard limits. Here's a post on what's coming: https://blogs.msdn.microsoft.com/stevelasker/2018/02/02/relaxing-acr-storage-limits-with-tools-to-self-manage/

A core feature needed for most of our future work is a metadata api, to track things, like policies. A very brief doc is here: https://github.com/SteveLasker/acr/tree/master/preview/metadata

If you'd like to help us shape ACR, I'd like to invite you to the preview group: https://aka.ms/acr/preview/signup
Steve

[DonMartin76]

Thanks! The relaxed limits is a good thing - but we still will need to do some housekeeping earlier than you have finished the autopurge or policies, I fear. I whipped up something real quick, which seems to work, but ran into #83. I will consider joining the preview group, thanks for the pointer!

[SteveLasker]

The metadata API is also used for us to do some indexing, to speed up requests on manifest listing. I'm just checking when that rolls out and confirming it will solve the specific manifest listing issue.
We did improve the delete API as the previous api was a bit confusing. I couldn't find the issue that tracked it. I think it was in the az cli repo. But, here's the cleaned up command:
az acr repository delete -n MyRegistry --image hello-world:latest

Delete an image by sha256-based manifest digest. This deletes all tags referencing the manifest and any associated layer data.

[DonMartin76]

I found the handling of the manifests and corresponding image tags fairly intuitive and flexible. If it just were a little faster, I would be very happy with it actually.

[SteveLasker]

Thanks @DonMartin76 ,
I'll reply to #83

[jeffbeagley]

We use an auto incrementing build ID generated from Azure Pipelines, so we needed a quick way just to purge older images based on the tag ID. I wrote this quick shell script to purge images older than the 10th image. Could easily be modified just to require a "delete before x date" reference.

https://github.com/jeffbeagley/azure_cr_purge/blob/master/retention.sh

[SteveLasker]

Hi @jeffbeagley,
Here's the auto-purge spec feedback area: AzureCR/specs#1

[sajayantony]

Closing this item for now since we could potentially use things like tag locking and - https://docs.microsoft.com/en-us/azure/container-registry/container-registry-auto-purge
Feel free to open new items to discuss details of these implementations if needed.