Support filtering by multiple labels/resources ("OR" logic)

[tillepille]

Describe the solution you'd like

Currently

velero create backup mulit -l 'backup=foo' -l 'backup=bar'

or

velero create backup multi --include-resources 'secrets' --include-resources 'configmaps'

means, that the flag gets overwritten.
So only configmaps / things with the label backup=bar will be backed up.

It would be great if i could specify multiple resource types or label combinations.

Anything else you would like to add:
I know that i can specify multiple labels in one string like -l 'backup=foo,shouldbackup=true'
but this is an AND not an OR

In my usecase its not possible to add one label to all the things i want to include into the backup.

Environment:

• Velero version (use velero version): v1.0.0
• Kubernetes version (use kubectl version):v1.14.2
• Kubernetes installer & version:v1.14.0
• Cloud provider or hardware configuration: hetzner (backup into aws s3)
• OS (e.g. from /etc/os-release): Client debian:stretch-slim (container) / Server Ubuntu:18.04

[skriss]

@tillepille thanks for reporting the issue, this is a known limitation. If it's not possible to specify in one set of flags, for now your best bet is to set up multiple backups, each with their own filter.

[tillepille]

@skriss Exactly, multiple backups are my current setup, but this kind of eliminates the benefits of using velero.

Do you have a starting point in the point for me?
I'm probably to bad at Go but i would like to see if i can do something, or at least better understand the limitation.

Thanks for your fast reply!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

Closing the stale issue.

[birsanv]

I am really happy to see the request being reopened @reasonerjt , I browsed this issue a few days ago and was disappointed to see this thread was closed.

I am also trying to find a solution to use multiple label selectors to group resources in one backup file. I spent some time to figure out if there is any workable option other then creating separate backups for each labelSelector rule.

In my scenario I want to backup all secrets with a label selector labels in a subset.
The only option I could get to was to have one label set on all these secrets. I started on this route but is not as easy as it looks and I am not even sure if it's feasible in reality.

These secrets are created by various teams, some external, and is extremely difficult to create a consensus among all groups to use one label. Also, we would have to take care of existing secret resources when defining this one label solution; unless they are updated to add the new label we agree upon using, the backup will miss existing resources.

Is there a plan to address this issue in velero ?..

I have in mind a simple solution :

The velero labelSelector option should have a new flag, label-selector-criteria with 2 values: (INCLUDE, EXCLUDE)
This is similar with the INCLUDE/EXCLUDE option available now when backing up namespaces and resources.

If label-selector--criteria=INCLUDE, the labelSelector is used to include all resources matching the rules
If label-selector--criteria=EXCLUDE, the labelSelector is used to exclude all resources matching the rules

So with the current implementation, you have a default label-selector-criteria=INCLUDE ( velero includes in backup all resources matching the labelSelector rules )

How is this solving our issue :

We want to be able to define a label selector using OR for the rules; but OR is a !AND, right ? So an OR rule can be simply created by defining an AND rule and negating it.
For example, to create this OR rule : label-foo in (A) OR label-bar exists ( include in backup all resources with label-foo equal to A or having a label-bar )
Use : label-selector--criteria=EXCLUDE , label-foo not in (A) AND label-bar does not exist ( basically get all resources matching the !(rules) and exclude them from the backup, instead of including them; add them to meta.ExtractList ..)

The current velero implementation works with a label-selector--criteria=INCLUDE, we just need to add the label-selector--criteria=EXCLUDE option
You already have an EXCLUDE option for namespaces and resource names, we need something for the labelSelector too

[nirutgupta]

It would be better if we just go with how kubectl handles the labels. We can refer https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#list-and-watch-filtering

[birsanv]

@nirutgupta the issue is that you can't define a kubectl filter criteria for an OR condition, for two different label selectors, like :
label-foo in (A) OR label-bar exists

[dymurray]

After discussing with a few folks, I think that this solution isn't incredibly complex but there is the downside of having to introduce a new API field which essentially duplicates existing behavior.

To solve the use cases @birsanv described above, we could simply add a new API LabelSelectors (notice plural) of type []*metav1.LabelSelector. And update the code that grabs the set of resources for a given label selector to instead loop over this list and concatenate them essentially giving us an OR functionality. This does get a little confusing as we would have LabelSelector and LabelSelectors and we would have to explicitly define what takes precedence (or just concatenate them all).

I suggest this as I don't believe we will be able to get OR functionality at the Kube level, so this would be a good workaround on our end. Thoughts?