Skip to content
This repository

Allow exclude filtering #524

Open
vesterbaek opened this Issue June 13, 2012 · 11 comments
Jeppe Vesterbæk

Sometimes it is usefull to be able to exclude instead of just filtering. I've added a simple solution, where prepending a field name with '!' in the url results in an exclude

For example, to filter(id=42)

/api/v1/model/?id=42

and to exclude(id=42)

/api/v1/model/?!id=42

I'm not sure it's the best syntax, but it's what I've come up with that's simple and easy to implement. I've overridden the following in my own resources to do the filtering

    def build_filters(self, filters=None):
        if not filters:
            return filters

        applicable_filters = {}
        # Normal filtering
        filter_params = dict([(x, filters[x]) for x in filter(lambda x: not x.startswith('!'), filters)])
        applicable_filters['filter'] = super(MyResource, self).build_filters(filter_params)
        # Exclude filtering
        exclude_params =  dict([(x[1:], filters[x]) for x in filter(lambda x: x.startswith('!'), filters)])
        applicable_filters['exclude'] = super(MyResource, self).build_filters(exclude_params)

        return applicable_filters

    def apply_filters(self, request, applicable_filters):
        objects = self.get_object_list(request)

        f = applicable_filters.get('filter')
        if f:
            objects = objects.filter(**f)
        e = applicable_filters.get('exclude')
        if e:
            objects = objects.exclude(**e)
        return objects

Input is welcome

Josh Bohde
Collaborator

Why not /api/v1/model?id__ne=42?

Jeppe Vesterbæk

Didn't know that would work? When I try, I get "The 'id' field does not support relations.".

I need to be able to negate all filter types, e.g. exclude(date__gte=...)

marauder37

ne doesn't work for most field types. I need something like vesterbaek's code so I can exclude based on text fields.

Deepan

Would anyone have a code sample showing how this works? Can't figure our where I would add the exclude filter...

Mickaël Falck

/api/v1/model/?!id=42

The main problem of this syntax appears once you want to use that in an API library such as Slumber, which takes directly pythonic kwargs (like .filters() and .exclude() methods of Django's ORM) and thus will throw an invalid syntax.

I do think that using __ne filter (from the url-side) and, then, apply such filters in the .exclude() method is better.

/api/v1/model?id__ne=42

Here is my snippet which handles __ne filter :

    def apply_filters(self, request, applicable_filters, applicable_excludes={}):
        return self.get_object_list(request).filter(**applicable_filters).exclude(**applicable_excludes)

    def obj_get_list(self, request=None, **kwargs):
        filters = {}

        if hasattr(request, 'GET'):
            # Grab a mutable copy.
            filters = request.GET.copy()

        # Update with the provided kwargs.
        filters.update(kwargs)

        # Splitting out filtering and excluding items
        new_filters = {}
        excludes = {}
        for key, value in filters.items():
            # If the given key is filtered by ``not equal`` token, exclude it
            if key.endswith('__ne'):
                key = key[:-4] # Striping out trailing ``__ne``
                excludes[key] = value
            else:
                new_filters[key] = value

        filters = new_filters

        # Building filters
        applicable_filters = self.build_filters(filters=filters)
        applicable_excludes = self.build_filters(filters=excludes)

        try:
            base_object_list = self.apply_filters(request, applicable_filters, applicable_excludes)
            return self.apply_authorization_limits(request, base_object_list)
        except ValueError:
            raise BadRequest("Invalid resource lookup data provided (mismatched type).")

(Please note that only obj_get_list is affected, since it was the only method where I needed such filtering in my Ressource)

This way any field with __ne token will be exluded from final request directly by the django's ORM, without breaking the usual behaviour of apply_filter.

Krister Svanlund

Here is my suggestion to how we could solve it. It excludes when you add ! infront of the value. The thing that is missing from this solution is for it to work in lists (if that is even useful) and there probably has to be some way of escaping the exclamation point if that actually is the first character.

Craig Labenz

vesterbaek's code was very close for me. I just had to give it a few tweaks:

    def build_filters(self, filters=None):
        if not filters:
            return filters

        applicable_filters = {}

        # Normal filtering
        filter_params = dict([(x, filters[x]) for x in filter(lambda x: not x.endswith('!'), filters)])
        applicable_filters['filter'] = super(ReviewResource, self).build_filters(filter_params)

        # Exclude filtering
        exclude_params = dict([(x[:-1], filters[x]) for x in filter(lambda x: x.endswith('!'), filters)])
        applicable_filters['exclude'] = super(ReviewResource, self).build_filters(exclude_params)

        return applicable_filters

    def apply_filters(self, request, applicable_filters):
        objects = self.get_object_list(request)

        f = applicable_filters.get('filter')
        if f:
            objects = objects.filter(**f)
        e = applicable_filters.get('exclude')
        if e:
            objects = objects.exclude(**e)
        return objects
Joshua Gourneau

A workaround is to use a iregex with inverse matching

http://HOST/api/v1/resource/?format=json&thing__iregex=^((?!notThis).)*$
Nicholas Pappas

@lastmikoi Have you updated your function since apply_authorization_limits was depreciated? I am hunting around to try and figure out how to implement your solution with the new security implementation, but have not yet found the right solution.

Mickaël Falck

@EvilClosetMonkey Sadly, no, I didn't came across this issue since then.

Mark Garro

I'm using a variant of the code of @craiglabenz and @vesterbaek

Instead of !=, since this is encoded and decoded by various URL libraries and also does funky things when put into a curl statement in terminal, I've done it for __ne, since that's closest to the Django ORM. Also, stringing multiple "excludes", doesn't have similar behavior with the above code.

Consider,

objects.filter(field1__ne=a, field2__ne=b)

You'd expect returned object that were both not equal to a and not equal to b. This can similarly be written as excludes, and they'd be different exclude statements, i.e.

objects.exclude(field1=a).exclude(field2=b)

So, apply them in sequence, not within the same exclude statement:

def build_filters(self, filters=None):
    """
    First, separate out normal filters and the __ne operations
    """
    if not filters:
        return filters

    applicable_filters = {}

    # Normal filtering
    filter_params = dict([(x, filters[x]) for x in filter(lambda x: not x.endswith('__ne'), filters)])
    applicable_filters['filter'] = super(type(self), self).build_filters(filter_params)

    # Exclude filtering
    exclude_params = dict([(x[:-4], filters[x]) for x in filter(lambda x: x.endswith('__ne'), filters)])
    applicable_filters['exclude'] = super(type(self), self).build_filters(exclude_params)

    return applicable_filters

def apply_filters(self, request, applicable_filters):
    """
    Distinguish between normal filters and exclude filters
    """
    objects = self.get_object_list(request)

    f = applicable_filters.get('filter')
    if f:
        objects = objects.filter(**f)
    e = applicable_filters.get('exclude')
    if e:
        for exclusion_filter, value in e.items():
            objects = objects.exclude(**{exclusion_filter: value})
    return objects
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.