Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception when using a regex for a related field #258

Closed
AlexDaniel opened this issue Jan 12, 2021 · 12 comments
Closed

Exception when using a regex for a related field #258

AlexDaniel opened this issue Jan 12, 2021 · 12 comments
Labels
bug Something isn't working fix confirmation pending issue has been fixed and confirmation from issue reporter is pending

Comments

@AlexDaniel
Copy link

AlexDaniel commented Jan 12, 2021

In DRF I'm trying to register a path like this (the id in this case is UUID):

router.register(r'user/(?P<user_id>[0-9a-f-]{36})/foo', mystuff.views.FooViewSet)

This is roughly what I have in the model:

class Foo(models.Model):
    …
    user = models.ForeignKey(MyCustomUser, on_delete=models.PROTECT, editable=False)

I'm pretty sure it works, but I get an exception when I'm trying to open swagger-ui:

Internal Server Error: /api/schema/
Traceback (most recent call last):
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/django/core/handlers/base.py", line 179, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/drf_spectacular/views.py", line 61, in get
    return self._get_schema_response(request)
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/drf_spectacular/views.py", line 65, in _get_schema_response
    return Response(generator.get_schema(request=request, public=self.serve_public))
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/drf_spectacular/generators.py", line 188, in get_schema
    paths=self.parse(request, public),
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/drf_spectacular/generators.py", line 165, in parse
    operation = view.schema.get_operation(path, path_regex, method, self.registry)
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 62, in get_operation
    parameters = self._get_parameters()
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 185, in _get_parameters
    **dict_helper(self._resolve_path_parameters(path_variables)),
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 324, in _resolve_path_parameters
    schema = self._map_model_field(model_field, direction=None)
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 389, in _map_model_field
    return self._map_serializer_field(field, direction)
  File "/home/alex/.cache/pypoetry/virtualenvs/foobarbaz-6qdzD22P-py3.9/lib/python3.9/site-packages/drf_spectacular/openapi.py", line 461, in _map_serializer_field
    model_field = field.parent.Meta.model._meta.pk
AttributeError: 'NoneType' object has no attribute 'Meta'

For what it's worth, both user and user_id give an exception.

@tfranzel
Copy link
Owner

hi @AlexDaniel, i suspect this somehow related to your serializer. field.parent is the serializer instance and Meta is expected to be there on a ModelSerializer. i also believe foreignkey on a "normal" serializer (not ModelSerializer) will throw in DRF when used.

we do similar things a lot and it has not produced any problems yet. also this is covered in the tests. i would need more information to reproduce this.

@tfranzel
Copy link
Owner

and my guess is the regex plays no role here

@AlexDaniel
Copy link
Author

AlexDaniel commented Jan 12, 2021

@tfranzel thanks! That's reassuring. However, my serializer isn't really that special either:

class FooSerializer(serializers.ModelSerializer):
    class Meta:
        model = Foo
        fields = '__all__'

Although… wait, which serializer are we talking about exactly? Could it be that my custom user model is affecting it? But there is no other serializer that can be relevant here, so where exactly should I be looking at?

I'll try to provide a full example if I have time.

@tfranzel
Copy link
Owner

that is really strange. it could be the model, but i highly doubt it. we have a custom user model too and it works fine. this is the test i have written with your example

# tests/test_regressions.py
def test_foo(no_warnings):
   # custom user
    class M3(models.Model):
        pass  # pragma: no cover

   # Foo
    class M4(models.Model):
        field = models.ForeignKey(M3, on_delete=models.PROTECT, editable=False)

    class M4Serializer(serializers.ModelSerializer):
        class Meta:
            fields = '__all__'
            model = M4

    class XViewset(viewsets.ModelViewSet):
        serializer_class = M4Serializer
        queryset = M4.objects.none()

    # should throw because of the bug
    schema = generate_schema('x', XViewset)

so this should be your example but it works. we need more context to find the problem. ideally you would create a reproduction with this snippet.

@AlexDaniel
Copy link
Author

Good! I'll try the snippet as soon as I wake up. However, something seems to be missing in your snippet?

Did I already mention that this:

router.register(r'user/(?P<user_id>[0-9a-f-]{36})/foo', mystuff.views.FooViewSet)

Throws an exception like I have shown above. But this:

router.register(r'user/(?P<blahblah>[0-9a-f-]{36})/foo', mystuff.views.FooViewSet)

doesn't. I'm not entirely sure where in your example you're actually using your field.

@tfranzel
Copy link
Owner

sry, the error message threw me off. it is in fact the URL parameter that is causing problems. i have a reproduction. will look into it

@tfranzel
Copy link
Owner

@AlexDaniel so i debugged it. it wasn't the regex itself, but the related field resolution for the path variable. there were 4 other ways to do this and you chose the only one that didn't work 😄 i won't even try to explain it, because it's just whacky internal DRF details.

please confirm that it works and close. thanks!

@tfranzel tfranzel added bug Something isn't working fix confirmation pending issue has been fixed and confirmation from issue reporter is pending labels Jan 12, 2021
@AlexDaniel
Copy link
Author

there were 4 other ways to do this and you chose the only one that didn't work

Hey, I'm actually wondering, what are the other ways to do it? When it comes to implementing nested paths like this, there are:

  • drf-nested-routers (ok but can be difficult to work with)
  • appending to urlpatterns directly (doesn't really work with ViewSets)
  • doing what I just did here

But you're saying there are more ways to do it with router.register? For example, I did try using path converters, which avoids the regex and drf-spectacular seems to be picking them up perfectly, but drf itself does not support them: encode/django-rest-framework#6789.

@tfranzel
Copy link
Owner

its not what you did, it is how you did it. nesting is fine, but you created very special circumstances for which the code path was broken.

these examples would have worked without the fix:

router.register(r'user/<uuid:user>/foo', mystuff.views.FooViewSet)

OR

    class M4(models.Model):
        # primary_key changes DRF interM4nals that make discovery easier
        field = models.ForeignKey(M3, primary_key=True, on_delete=models.PROTECT, editable=False)

would have taken different code paths, that would have worked.

@AlexDaniel
Copy link
Author

@tfranzel I'm like 99% certain that the first snippet does not work. It's about the drf ticket I mentioned earlier. Yes, drf-spectacular will generate the schema perfectly, but that route is unusable in DRF itself. Try making a request to it and it won't match any routes. It could also be that I did something wrong in my code though, but I'm pretty sure I tried real hard to make that work.

@tfranzel
Copy link
Owner

ahh sry, you are right. after just reading the ticket i realized that. however, the second one would have most likely worked. there is a small difference between what drf-spectacular and DRF will accept. in some cases like the first one, spectacular will allow more than DRF.

well we fixed it so it should now work for you. would be nice if you could confirm that with the master state.

@AlexDaniel
Copy link
Author

Just installed 0.13 and indeed, it works now! Both <user> and <user_id> seem to function correctly, the schema is generated without exceptions and it detects that it's a uuid field. Fantastic!

Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working fix confirmation pending issue has been fixed and confirmation from issue reporter is pending
Projects
None yet
Development

No branches or pull requests

2 participants