Exception raised when generating OpenAPI schema after adding new serializer fields

[jeremystretch]

Deployment Type

NetBox Cloud

NetBox Version

v3.7.4

Python Version

3.10

Steps to Reproduce

I'm opening this issue to document a suspected bug uncovered during work on #15464. That FR entails defining new ManyToManyFields on the customer User and Group models introduced in NetBox v4.0 to track ObjectPermission assignments. Consequently, the API serializers for these models must each also have a field added to track these relationships. This is what has been added to the UserSerializer class:

    permissions = SerializedPKRelatedField(
        source='object_permissions',
        queryset=ObjectPermission.objects.all(),
        serializer=ObjectPermissionSerializer,
        nested=True,
        required=False,
        many=True
    )

Note that the field defines a source attribute different from its name, as the underlying relationship is referenced as object_permissions rather than permissions.

Expected Behavior

Adding this field to the model should not break the schema generation.

Observed Behavior

An exception is raised when generating the OpenAPI schema:

Traceback (most recent call last):
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper
    return view_func(request, *args, **kwargs)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/drf_spectacular/views.py", line 84, in get
    return self._get_schema_response(request)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/drf_spectacular/views.py", line 92, in _get_schema_response
    data=generator.get_schema(request=request, public=self.serve_public),
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/drf_spectacular/generators.py", line 281, in get_schema
    paths=self.parse(request, public),
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/drf_spectacular/generators.py", line 252, in parse
    operation = view.schema.get_operation(
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 99, in get_operation
    request_body = self._get_request_body()
  File "/home/jstretch/projects/netbox/netbox/core/api/schema.py", line 198, in _get_request_body
    return super()._get_request_body(direction)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 1295, in _get_request_body
    schema, partial_request_body_required = self._get_request_for_media_type(serializer, direction)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 1327, in _get_request_for_media_type
    component = self.resolve_serializer(serializer, direction)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 1605, in resolve_serializer
    component.schema = self._map_serializer(serializer, direction, bypass_extensions)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 927, in _map_serializer
    schema = self._map_basic_serializer(serializer, direction)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/drf_spectacular/openapi.py", line 1020, in _map_basic_serializer
    for field in serializer.fields.values():
  File "/usr/lib/python3.10/functools.py", line 981, in __get__
    val = self.func(instance)
  File "/home/jstretch/projects/netbox/netbox/netbox/api/serializers/base.py", line 54, in fields
    return super().fields
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/django/utils/functional.py", line 47, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 356, in fields
    for key, value in self.get_fields().items():
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 1076, in get_fields
    field_class, field_kwargs = self.build_field(
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 1222, in build_field
    return self.build_unknown_field(field_name, model_class)
  File "/home/jstretch/projects/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 1340, in build_unknown_field
    raise ImproperlyConfigured(

Exception Type: ImproperlyConfigured at /api/schema/
Exception Value: Field name `permissions` is not valid for model `User`.

With some effort, I was able to track this down to the following lines in NetBoxAutoSchema.get_writable_class():

netbox/netbox/core/api/schema.py

Lines 159 to 160 in 69c0aac

	elif isinstance(child, ManyRelatedField) and isinstance(child.child_relation, SerializedPKRelatedField):
	properties[child_name] = None

Removing these lines results in the schema being generated successfully and, as far as I have been able to discern, accurately. Several of the request definitions are renamed (e.g. PatchedWritableProviderRequest becomes PatchedProviderRequest) but otherwise unmodified, and the title and/or description are removed from a few fields. I believe this also removes the erroenous designation of several fields as required (e.g. vdcs on Interface).