Upgrade to 2.0 - weird ForeignKey filtering issue

[zmallen]

Hey there,

I am upgrading to Django 2 and using django-filter==2.2.0 after upgrading from 1.1.0

I have the following model:

class Config(models.Model):
    company_id = models.ForeignKey(Company, on_delete=models.CASCADE, to_field='company_id')
    rule_id = models.IntegerField()
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

with the following filter:

class ConfigFilter(django_filters.rest_framework.FilterSet):
    created_at_gte = IsoDateTimeFilter(field_name='created_at', lookup_expr='gte')
    created_at_lte = IsoDateTimeFilter(field_name='created_at', lookup_expr='lte')

    updated_at_gte = IsoDateTimeFilter(field_name='updated_at', lookup_expr='gte')
    updated_at_lte = IsoDateTimeFilter(field_name='updated_at', lookup_expr='lte')

    class Meta:
        model = Config

        fields = (
            'id',
            'company_id',
            'rule_id',
            'created_at',
            'updated_at',
        )

This table is queried frequently for ?company_id=FOO, and in the previous version, if FOO did not exist, Django would return empty results array.

With the new upgrade, Im getting tests failing on querying for company_id that doesnt exist. Specifically, Im getting a ChoiceField error when someone queries this table with a company_id that does not exist:

{'company_id': ['Select a valid choice. That choice is not one of the available choices.']}

And im getting similar errors on filters for ForeignKeys that dont exist. Is this new behavior, or am I upgrading incorrectly?

[rpkilby]

Is this just a change to company_id, or is it a general change in validation error handling? e.g., if you submitted ?created_at_gte=foo, did you get a validation error in 1.1.0, or was it just ignored?

If it's the former, I'm not really sure. ChoiceFilter and ModelChoiceFilter didn't change substantially between 1.1.0 and 2.2.0, but if it's the latter, it's probably related to #788. If you want validation errors (because they inform the user that they've made a mistake), then there's nothing to do. If you want validation errors to be ignored, you need to subclass the filter backend and set raise_exception = False.

[zmallen]

Thanks @rpkilby , ill review what you posted.

I actually dug up this issue #84 and we have the reverse problem - we have an error being thrown if nothing exists. I switched this to a numberfilter (overriding the ForeignKey default for the ChoiceFilter) and it worked. Will have to see what works best here.

[rpkilby]

To clarify, does the NumberFilter work with FOO, or are you providing a valid number, but a number that doesn't correspond with a company_id?

[zmallen]

Sorry - FOO specifically is a Number that does not exist in a separate table. I should have changed this to -1 or 9999 (the latter in your response)

[rpkilby]

Okay - that makes sense. A NumberFilter is just going to check that the value is indeed a number, while the purpose of the ModelChoiceFilter is to guarantee that the value is a member of the queryset.

[evgenybf]

ModelChoiceFilter in django-filter 0.13.0 returns empty list if PK value doesn't exist in the database. Now we get either validation error or the filter returns everything as if the PK value wasn't specified.

[Bergiu]

I had the same problem and using this factory to solve the problem:

import django_filters
from django_filters.constants import ALL_FIELDS
from django.db import models


class FixedReferenceFilter(django_filters.FilterSet):
    @classmethod
    def filter_for_lookup(cls, f, lookup_type):
        if isinstance(f, models.ForeignKey) and lookup_type == 'exact':
            return django_filters.NumberFilter, {}
        # use default behavior otherwise
        return super().filter_for_lookup(f, lookup_type)


def filterset_factory(model, fields=ALL_FIELDS):
    meta = type(str('Meta'), (object,), {'model': model, 'fields': fields})
    filterset = type(str('%sFilterSet' % model._meta.object_name),
                     (FixedReferenceFilter,), {'Meta': meta})
    return filterset