Open discussion on Auto-Purge Policies design

[SteveLasker]

Please provide any feedback to the Auto-Purge design: here

[brobichaud]

This goes way beyond what I had anticipated for feature set and fully meets all of the scenario needs I had considered for our systems. So my comments are both "bravo", and "make it so!".

[powareverb]

Looks brilliant. Interested to know how exactly images would be marked under each policy (e.g. released). Possible API calls for this? PowerShell? 😁

[dany74q]

Looks awesome guys, great work !

[SteveLasker]

we haven't finalized the cli, but it would look something like: az acr purge-policy set --image foo:333 --policy 'release'
There will be an equivalent REST call as well.

[tgarlot]

Would that be possible to define the policy as what is done in Preview for storage (https://docs.microsoft.com/en-us/azure/storage/common/storage-lifecycle-managment-concepts) e.g. with a JSON payload? Would be great.

[Lampei]

Is there any kind of potential release roadmap to this feature? This also meets all of my scenarios for registry maintenance, so if it is a "coming very soon" vs. a coming 2020, then it would save me from rolling my own.

[SteveLasker]

Hi Lampei, we are actively working on Auto purge. We’ll have some of the basics for a recycle bin and purging un-tagged images. We hope to have the full spec done by Q1 of 2019.

[bpoland]

Hi there, this looks great. Any plans to include helm charts for automated cleanup, since ACR supports those too?

Thanks!

[SteveLasker]

Hi @brobichaud
Yup, we'll add these as well. Although, we'll have to think about this a bit more. We know an image is still running, to defer the image purge for [configured days] past the last known usage. For images, we plan to monitor all managed nodes in Azure to report back the image is still in use.
Helm charts are a bit different. Once a chart is used to do a deployment, it may continue to run for 6 months. If the policy is 6 months, and the chart hadn't been used for 6 months, what happens on 6 months and a day, when you may need to do a new chart deployment.
We'll think more and update in January.
Thanks for the issue. It's a great catch.
Happy holidays,
Steve

[phil-lgr]

I'm heavily using the Azure UIs, the CLI not so much—setting the policies via a UI in Azure would be much appreciated

[SteveLasker]

Thanks Phil,
Yup, we’ll have UI for these as well.

[bpoland]

Helm charts are a bit different. Once a chart is used to do a deployment, it may continue to run for 6 months. If the policy is 6 months, and the chart hadn't been used for 6 months, what happens on 6 months and a day, when you may need to do a new chart deployment.

Hi @SteveLasker,

Not sure how in-depth AKS can get but I believe Helm info is stored as ConfigMaps inside of Kubernetes. So there would probably be a way to check that and figure out whether a chart is still in use.

[jeffbeagley]

We use an auto incrementing build ID generated from Azure Pipelines, so we needed a quick way just to purge older images based on the tag ID. I wrote this quick shell script to purge images older than the 10th image. Could easily be modified just to require a "delete before x date" reference.

https://github.com/jeffbeagley/azure_cr_purge/blob/master/retention.sh

I believe an ideal retention policy would follow a behavior similiar to
purge where (tagCount>10 && timestamp <=time_retention) so that you can still maintain images in order to rollback regardless of how long its been since that image has been developed against

[ellieayla]

That's similar to the policy approach taken by https://github.com/fraunhoferfokus/deckschrubber and I think it works really well for common usecases.

[mkstephenson]

Is there any update to when this would be available in preview?

[adusumillipraveen]

@SteveLasker Is this still planned to be released in Q1 of 2019 ?

[rotata]

We will have some more concrete updates for you in the next few days as we are currently going through a detailed analysis of our Feature asks and potential deliverables.

[adusumillipraveen]

Thanks, will be looking forward for an update.

[mustafababil]

Here is a Powershell script to delete old images by some limit (count or create date) until this policy is released.

[SudoBrendan]

For images, we plan to monitor all managed nodes in Azure to report back the image is still in use.

Will there be an endpoint we can hit to mark an image as in-use outside of Azure?

[SteveLasker]

Yes. All the APIs will be available on ACR. We'll simply provide some experiences that automate these.

[TraGicCode]

I also am very anxious for this feature!

[yiyangfcv]

It's almost end of Q2 now. Any updates on the preview / roadmap?

[brobichaud]

Indeed, it's been so long I'd almost forgotten about this. Until I had to do my monthly manual purge, which is so very cumbersome and tedious...

[mkstephenson]

Any news on the progress of this?

[mustafababil]

you can look here: https://docs.microsoft.com/en-us/azure/container-registry/container-registry-auto-purge

[mkstephenson]

Awesome thanks! Do we report bugs here as well? The filter parameter doesn't seem to work

[flipchart]

Can I delete only the last n tags? I don't want a time window but a number of tags window. Like AWS has.

Also, will this eventually make it into the portal UI like AWS has had for ages?

[SteveLasker]

Hi folks,
This is just the beginning. The full spec is in our plans including portal UI. We’ll be releasing untagged manifest purging in the portal next.
For issues, please use https://aka.ms/acr/issues
@flipchart, we discussed last n tags, but felt uncomfortable that n is not a steady state of reference as new tags could be pushed in. And, wouldn’t you want the bottom n tags? Can you elaborate a bit more on the scenario?
For the roadmap, and background on why auto-purging has been a slower pace: https://aka.ms/acr/roadmap

[flipchart]

@SteveLasker we do CI and tag our release images as e.g. build241, build242, etc. We want to keep the last e.g. 5 builds on the registry to roll back a version or two if necessary. So for us, it's sort by tag name descending and keep the top 5, delete everything else.

This is the AWS UI, which I use for another project. It's not explicit, but it does exactly what I describe above. Maybe they do it by date. That would also work.

To be honest, I'd even settle for doing it manually through the portal UI once a month if I could just select multiple images to be deleted. The one by one delete is insanely painful.

[SteveLasker]

ok, so you want to delete all but the most recent n. That makes much more sense. Please add a request at https://aka.ms/acr/uservoice for tracking.
We’ve also considered the multi-option in the portal for delete. I can’t remember if we agreed to keep it or thought it wouldn’t scale for the number of tags that might exist. Please add a user voice request for tracking that as well.
Thanks for the feedback

[brobichaud]

I've got auto-purge hooked up now through the tasks feature and it's working well for us so far. Totally saves me a bunch of manual effort! I'm pretty happy with this simple feature set for now, I'd just love to see it exposed more in the Azure Portal. Such as being able to view the scheduled tasks, all of their details, be able to add/update/delete them, and toggle them on/off.

So far, well done!

[oysterkwok]

It seems right now auto-purge can only target a specific repository? Could we use wildcard for it?
Basically we are looking for:

1. Able to purge all repositories and all tags which are last updated X days ago, except locked tags
2. If possible, purge all repositories and all tags which never pulled Y days ago, except locked tags

[SteveLasker]

Thanks @oysterkwok
What you don't yet see is the repo based permissions work we're getting ready to release. https://aka.ms/acr/roadmap

The concern is a wildcard would need the identity running the delete to have multi-repo permissions. Not a bad thing, but a concerning thing. If the identity doesn't have permissions, do we just report the failure?

The other is accidental deletes. What if you accidentally delete something with too much scope permissions. We actually spend a lot of time answering requests to restore deleted images, repos and registries. We have a backlog item for a recycle bin so customers can do this themselves.

But, you're point is valid, that you'd want broader control to delete things, without having to specify each repo. We're just taking a slower pace to this, to be more deliberate.

We did just release a new untagged manifest policy, that isn't repo specific.