Filter foreign key field by null

[mick88]

I can't seem to find documentation on filtering fields by null value. By logic, Django's isnull=True should work, but it doesn't work for me:
/api/houses/?ownerid__isnull=True doesn't filter houses with no owner

[miki725]

hmm. ownerid__isnull=True should work since I overwrite isnull to use BooleanField - https://github.com/miki725/django-url-filter/blob/master/url_filter/filters.py#L20

can you provide example how you use your filterset?

[mick88]

I simply added ownerid as one of filterables field in my viewset:

class HouseViewSet(viewsets.ReadOnlyModelViewSet):
    queryset = models.House.objects.all().select_related('ownerid')
    serializer_class = serializers.HouseSerializer
    filter_fields = [
        'price',
        'parking',
        'ownerid',
        'location',
    ]

The query should return 1000 objects, but instead returns all 1500. I can even filter houses by owners with 0 or more score /api/houses/?ownerid__score__gte=0, effectively returning houses with ownerid not null (500 results).
Other filters, such as /api/houses/?ownerid__name=mick88 work correctly and return one item.

Settings:

REST_FRAMEWORK = {
    'DEFAULT_FILTER_BACKENDS': (
        'url_filter.integrations.drf.DjangoFilterBackend',
    ),
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.PageNumberPagination',
    'PAGE_SIZE': API_ITEMS_PER_PAGE,
}

[miki725]

Thanks for the info. Ill try to take a look when I can.

[mick88]

I stepped through the code and I think I found the issue in the library. get_spec appears to raise SkipFilter, because "isnull" is not in self.filters in this line:

# url_filter/filtersets/base.py:349
if name not in self.filters:  # name = u'isnull'; self = PlayerFilterSet
    raise SkipFilter

Further, I found that this issue only manifests itself on foreign key fields. Filtering by other fields with isnull works correctly, i.e. /api/houses/?price__isnull=True

[miki725]

oh. that makes sense for foreign keys. I have (had) some updates coming up however didnt get a change to push it in yet due to load at work. Ill try to fix it soon or contributions are welcome.

[mick88]

I put in a PR there. Not sure if it's the right way to do it, but it's best I could do and seems to work. I haven't tested it with my project yet, but I added a test which tests my case and it passes.

[mick88]

I also discovered that it's impossible to filter by isnull=False (unless using workaround isnull!=true), because isnull boolean field was required. I put in a fix for this in my PR.

I tested this now with my project and works as intended. Please review my changes and merge.

[miki725]

Thanks for the reports and the PR.

I actually had isnull=False fixed (https://github.com/miki725/django-url-filter/pull/9/files#diff-1c4aee14792f7a8c40bc2256837aa2c5R22) a while ago however never finished writing tests and so never finished #9. My plan is to catch up on my open-source project these holidays so hopefully will push a new release with some fixes and new features.

[miki725]

@mick88 I solved your issue in 6330135 as part of #9 .

now all of these work on the test project:

/one-to-one/restaurants/?place__isnull=True
/one-to-one/restaurants/?place_id__isnull=True

I also added tests for your explicit scenario of using FKs integer field directly (in your case ownerid). They were added in da65e93

The only issue now is that ModelFilterSet does not add these automatically. In the test project project example of Places and Restaurants only Restaurant.place filter is added however Restaurant.place_id will not. For that I added a separate ticket #14

Let me know if this ticket is good for closing now. Thanks