Multiple "label!" filters for prune are ORed unlike all other filter combos which are ANDed.

[callumgare]

TL;DR

When running a prune with multiple negated label filters the result is not what you would expect, they are ORed together unlike other filters which are ANDed.

Description

This is either a bug or an undocumented "feature" depending on your perspective. The intuitive understanding of filters is that the more filters you add the more specific you make the filtering and the smaller the list of things that should be selected by the filtering process. Essentially the common convention is that multiple filters are ANDed together when a logical operator isn’t explicitly specified. That's how filters in docker prune work (expect for this corner case) and it is also the most useful behavior for pruning in docker because if you want filters to work as an OR you can just simply run the prune command multiple times each with a different filter that you want to be ORed. If the filters were by default ORed together then you couldn't easily replicate the effect of ANDing them. The documentation doesn't explicitly spell out that filters are ANDed together but that is definitely how most people understand them to work.

So for most cases:

• --filter "label=foo" --filter "label=bar" translates to hasLabel(foo) && hasLabel(bar).
• --filter "label=foo" --filter "label!=bar" translates to hasLabel(foo) && !hasLabel(bar).

However when specifying multiple negated label filters:

• --filter "label!=foo" --filter "label!=bar" you would expect to translate to !hasLabel(foo) && !hasLabel(bar) but actually translates to !hasLabel(foo) || !hasLabel(bar) (or written as it is implemented in the code: !(hasLabel(foo) && hasLabel(bar))).

Use Case

The situation which I ran into it (and which I can imagine other people running into similar cases) is that I want to prune all containers that aren't database containers (containers which aren't labeled "db", "mysql" or "postgres").

Proposed Solution

Although painful in the short-term because it includes breaking backwards compatibility, I believe the cleanest long-term solution is to change the behavior of negated filters so that they are ANDed together. As it was never garenteed that negated label filters would be ORed and I believe most people aren't aware of this corner case and assume all filters are ANDed this would bring behaviour a lot more in line with expected behaviour than it would break expected behaviour in the ocational case where someone has become aware of this quirk and learnt to expect it.

There are situations when you might want the current behavior. As mentioned that can be easily replicated by running the pruning command multiple times. Potentially you could expand how filters work to allow negating the result of multiple ended filters thus allowing you to essentially Replicate the current behavior without having to run a command twice but I don’t think the increased complexity is worth the small benefit that would give.

I’m not quite sure how adverse the doctor project is to breaking backwards compatibility and in the case where it does how much warning it provides before behavior changes etc so maybe someone else can chip in on that? If breaking backwards compatibility is considered too costly than one option might be adding a new filter something like --filter "label!and=bar" which would allow negated label filters to be entered together. Definitely very ugly but at least it would provide that functionality without disturbing any scripts relying on current behavior and it doesn't make the currently functionality uglier so if you're not using multiple negated label filters you don't need to be aware of the ugliness.

I am very happy to submit a merge request once there is consensus on how this should be dealt with. If there is strong opposition to making any changes then the current behavior should at the very least to be documented.

Workaround For People Currently Experiencing This Issue

bash -c '{ docker ps -aq --filter "status=exited" && docker ps -aq --filter "label=foo" && docker ps -aq --filter "label=bar"; }' | sort | uniq -u | xargs --no-run-if-empty docker rm

Add or remove && docker ps -aq --filter "label=<label exclude from pruning>" in the command to add or remove negated label filters. Other non-negated filters should be added to the first instance of docker ps (eg docker ps -aq --filter "status=exited" --filter "until=24h"). bash -c is only necessary for users of shells which don't support the bash specific syntax used above (like fish).

How To Reproduce Unexpected Behavior

I've only tested this for docker container prune but I believe applies to all objects which can be pruned (containers, images, volumes, networks).

docker run --label=foobar --name=foobar busybox && docker run --label=foo --name=foo busybox && docker run --label=bar --name=bar busybox

$ docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                     PORTS                              NAMES
711d3bf8faf8        busybox             "sh"                     3 seconds ago       Exited (0) 3 seconds ago                                      bar
02eedafde893        busybox             "sh"                     4 seconds ago       Exited (0) 3 seconds ago                                      foo
bf5cc8fb74d1        busybox             "sh"                     5 seconds ago       Exited (0) 4 seconds ago                                      foobar

$ # Looking at this command I would expect it to remove just "bar" but it removes all containers.
$ docker container prune --force --filter "label!=foo" --filter "label!=foobar"
Deleted Containers:
711d3bf8faf87645e47796933eb8b1ec12458af7d302e02b65bbdd5c7140f566
02eedafde89371f8956e44c6430372fb63bd89560519cf400ae57cdb428521f8
bf5cc8fb74d1a51aa83839167013fd0c496baa417a4c44f5e4589e087469e19f

Total reclaimed space: 0B

$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

Offending Code

The matchLabels function in daemon/prune.go and daemon/images/image_prune.go, and the byLabelFilter function in volume/service/by.go is the cause. Again I'm happy to write a patch + tests.