-
Notifications
You must be signed in to change notification settings - Fork 317
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ECR] [request]: Support regular expression matching for tags in lifecycle policies #1213
Comments
This would be nice. The ability to use regular expressions instead of being restricted to just a "prefix" string for the tag filter on lifecycle policies, that is. I'm currently trying to maintain thousands of images and the way they're currently named, I need to filter by a prefix and a suffix, and a regex option would solve that problem and, probably, countless others. In the meantime, I guess I'll just have to use lots and lots of ... uhhh, other commands. I'll figure something else out, I'm sure. |
This would be great, we're using Java-Spring style versioning of releases (1.0.0.RELEASE) and currently migrating our Docker registry to ECR, which atm forces me to change the versioning to RELEASE-1.0.0 or something like that, so we can create lifecycle policies on how many version of certain release type we want to keep in the registry. Such version is not SemVer compatible (and looks terrible), which breaks other tooling in our pipeline. |
I wish I could give this 10 thumbs up. When using AWS SAM with the image type for lambda functions, by default it creates an image tag with a combination of the name of the lambda + the dockertag metadata of the image. If your lambda is something like |
Another thump up for me. We have a on commit deployment with argocd. Each commit triggers a build that triggers a new image that is tagged with the commit hash. These images are only used very short as the tests are executed against it but there are tons of images now in ecr |
it will be grate to have some "not equal" and/or "not content" patterns and parce all existed tags on the image. I'm OK to add an extra tag for images I need and delete all other withount this extra tags. image [1.0.0-dev.123, 1.0.0-dev, dev] <-- keep images with "dev" tag |
thumb up. does anyone actually put "dev" or "prod" in the front of the tag? the matching should support SemVer style tags. |
In order to reduce image bloat, we had to create our tags prefixed with an |
Big thumbs up from me. We have tens of thousands of images across hundreds of repositories. And not having this required us to develop a custom solution. Its been painful and still is painful not having this. |
word! ability to use prefix only is pretty limiting and offering regex seems reasonable. 🐰 |
Hi all! We're looking at making enhancements to LCP rules and we've been tracking this issue. As we start looking into it, we'd like to know if wildcards would meet most needs, or if a more complete regex experience is required. Understood that prefixes are limited, but typically some sort of schema is used in tagging. It might be simpler to introduce wildcards, but we'd want to know that it would get customers a meaningful value. What do folks think? |
For our use case, wildcards are not enough, since we need full regex expressions to distinguish dev and prod tags since we only want to delete older dev tags and keep all prod tags. So, we would regex expressions like the following: |
Wildcards will work for some of our tag formats e.g. we have inflight release candidates which have SNAPSHOT in the name, so However, images with semantic version tags, are considered released images and therefore fall under separate lifecycle rules. Elsewhere we have tooling which pushes images which we have been able to change to use prefixes. So a mixed bag. Bottomline, wildcards wont work for us entirely, and regex would be a better fit. |
I'd prefer a regex if possible. Currently we have to force a prefix into our tags to denote dev images but ideally we just use the |
Please also support by age and since last pulled. |
I think it is very important do change the way Lifecycle Policy rules work. They don't provide the same quality as the handling of other aws services.
But in general we can say that ECR lifecycle should allow to handle complex tagging strategies, while being able to be understandable and easy to set up. It seems like the whole functionality could need a very generous rework with some good features like f.e. sinceLastPulled, keep image. Btw. The way of testing rules and apply them is a very good feature that also prevent accidentally data loss. This is something i want to especially praise! |
I also second the comment above around pruning based on last pull date. Although, we have seen some bizarre behaviour of last pull date. Will reach out separately on this issue. Cheers, |
I don't think this has been put forward, but for me a useful regex scheme would have each match be treated individually. My usecase is to be able to keep 1 tag for an image per pull request. E.g. a tag would have a regex prefix like
I then want to keep
So the latest tag per regex match of the prefix and not end up with only |
Hi 👋🏼 Wildcards would be a huge win comparing to prefixes. It would give me some additional flexibility. Putting it in perspective: Assuming that last pulled and image age are supported. |
This would definitely be great. Our use case is for suffixes on tagged images. We often use Agree with @joaocfernandes
Fwiw, one specific example is a busy frontend ECR repo where images need to be tagged with |
Hi, the ability to enforce a Tag naming-convention (vN.N.N-env or whatever) on image upload is likely a fundamental requirement for most CICD/Pipeline-based build systems. The current lack of this feature on AWS ECR is a key differentiator ...- please implement this !! |
A lot of nice ideas mentioned here but personally I would say if one were to go for an easy win with minimal effort regular expressions would be the one. In my use case suffixes would suffice (I have never noticed prefix versioning personally so this drove me crazy :-) ) but I feel it is not too different to have wildcard and pattern matching (effort wise for this is an assumption) and the second is more complete as a solution. So pattern matching for the win |
Hello, in our case we use sementic versioning to tag our images. Since this is not supported yet LCP. we now have lot of old images to clean up. We'll probably build a lambda to do that on regular basis. This is not ideal at all, since we need to extra work/compute to handle it and will force to have images lifecycles in different tools (in ECR and in custom lambda). @hsejour do you know if this is issue is planned, if yes, is there any ETA (a quarter or semester timeframe is enough). |
We would love if AWS can solve the problem for all customers. We have repositories in lot of AWS regions replicated. |
The typical tagging schema is semantic versioning. This means that prefix matching is simply inadequate since the distinction between release and build is in the suffix (or specifically the lack of a suffix). Even if use cases could be met with multiple wildcard rules, only 50 rules are allowed in each policy. Using regex would cover more use cases per rule and would help users stay within the rule quota. |
Please prioritise this ticket the ECR policies are almost useless as they are now for us we have this awful Terraform code to build even like that when a project goes over
At the moment this is a pain and we would soon be building up our own script to manage the image lifecycle |
@ingshtrom Hi Alex, any updates on this? Our need for this continues to grow - it would be incredibly helpful in managing a fairly large number of ECR repos in AWS. Ah excuse me, I see the label changed to in progress a few days ago - great to see and thank you |
also need this |
👀 |
@hsejour Can you provide a status/ETA on this? Thanks! |
No one is assigned? Does anyone know the status? |
Hi everyone. Just a quick notice from the ECR team as I notice that we went silent on this issue for quite a bit. We continue to working on support for wildcards in lifecycle policies, and we plan release it before the end of the year! As you all are probably used to, I'm not committing to this, but I'm quite confident that it will be out there soon. I have definitely heard that wildcards are not enough for everyone, and we want to continue working on improving LCPs, including RegEx support, SemVer support, and last date pulled. However, I do not have more to share about what will be done or when. Just note that feedback is being heard and we are using it to adjust our roadmap! |
We were expecting a big announcement at Re:Invent. |
Hi all - wildcards for LCP are live. A "just-after-reinvent" launch! I'm keeping this item open since the original ask is for regular expressions, which is more than wildcards. But I hope this will help some of you make progress! Thanks for your patience |
@rafavallina / @HaroonSaid When I tried to use aws console to import the policy and got this: So wildcards doesn't work well (it worked only if you add rules one by one, not via json import) |
@vchirikov Can you try to use 'tagged' in the 'tagStatus' field? That field is only to specify if the images must be tagged. You use the 'tagPatternList' field to indicate that we are doing a wildcard matching. |
Please, prioritize this issue. |
please prioritize this issue this should be like a few lines of code to enable regex filtering and this is open since 2021! |
please prioritize this issue, it's a real pain for us |
Community Note
Tell us about your request
What do you want us to build?
I to use regular expressions to match on tags in my lifecycle policies. For example, we have an image tagged
<account_id>.dkr.ecr.us-east-1.amazonaws.com/<image_name>:v1.1.0_test-4db22261f6a2ca5de2cb7eae3382dba32b3676da
.Right now, we need to tag the image as
<account_id>.dkr.ecr.us-east-1.amazonaws.com/<image_name>:test-4db22261f6a2ca5de2cb7eae3382dba32b3676da_v1.1.0
so that we can use prefix matching in the lifecycle policy.Which service(s) is this request for?
ECR
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
We want to be able to have lifecycle policies that are more dynamic. Some things are in image tags like Git SHAs, Semver, etc. which are dynamic and we cannot match on dynamic strings in lifecycle policies.
Are you currently working around this issue?
For existing images, we cannot change. For new images, we can start tagging with a prefix we can filter on to be able to match in our lifecycle policies.
Additional context
Anything else we should know?
Attachments
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)
The text was updated successfully, but these errors were encountered: