Replace NestedFooSerializer and brief=True pattern with DRF built-in Meta.depth

[jathanism]

Proposed Changes

The proposed change is:

• Remove all nested_serializers modules and NestedFooSerializer classes
• Remove all code to process them along with the concept of ?brief=True and setting self.brief on viewsets
• Remove all overloaded foo = NestedFooSerializer() fields on API serializers and allow them to fall back to the default field from the model
• Leverage the built-in nested serialization feature in DRF e.g. depth=0 is default (showing PKs), depth=1 shows first-level objects nested representations, etc.
• Find a way to provide desired nested depth using request query parameters (e.g. /api/dcim/sites/?depth=1)

Justification

This bespoke implementation is hindering our ability to leverage existing community extensions to Django for things like JSON schema generation, API import/export using CSV, among others. Additionally it is extra boilerplate code that must be written for each new model, and is custom code that must continue to be maintained and tested.

We will be better off in the long run 1.) using existing features of DRF, 2.) leveraging community resources where reasonably possible.

[bryanculver]

What is the impact to plugins and integrators to the schemas as well as things like pynautobot?

[glennmatthews]

https://www.linkedin.com/pulse/friday-quick-tip-dynamic-depth-serialization-django-rest-neidinger is probably relevant as far as dynamic depth based on request query parameters.

One thing we might lose in making this change (and should seek to preserve) is the ability to identify related objects by natural keys or otherwise by properties other than PK, e.g.:

POST api/dcim/devices/

{
    "name": "MyNewDevice",
    "rack": {
        "site": {
            "name": "Equinix DC6"
        },
        "name": "R204"
    },
    ...
}

instead of requiring:

POST /api/dcim/devices/

{
    "name": "MyNewDevice",
    "rack": "7f3ca431-8103-45cc-a9ce-b94c1f784a1d",
    ...
}

[jathanism]

https://www.linkedin.com/pulse/friday-quick-tip-dynamic-depth-serialization-django-rest-neidinger is probably relevant as far as dynamic depth based on request query parameters.

One thing we might lose in making this change (and should seek to preserve) is the ability to identify related objects by natural keys or otherwise by properties other than PK, e.g.:

POST api/dcim/devices/

{
    "name": "MyNewDevice",
    "rack": {
        "site": {
            "name": "Equinix DC6"
        },
        "name": "R204"
    },
    ...
}

instead of requiring:

POST /api/dcim/devices/

{
    "name": "MyNewDevice",
    "rack": "7f3ca431-8103-45cc-a9ce-b94c1f784a1d",
    ...
}

This is writable nested serialization, which I think could still be supported. See: https://www.django-rest-framework.org/api-guide/serializers/#writable-nested-representations

[jathanism]

What is the impact to plugins and integrators to the schemas as well as things like pynautobot?

I think we should still marry this with other work to provide PK or Natural Key across the board as best we can. There are some other options here to consider as well. See:

• https://github.com/AltSchool/dynamic-rest
• https://github.com/rsinger86/drf-flex-fields
• https://github.com/dbrgn/drf-dynamic-fields
• https://github.com/miki725/django-url-filter
• https://github.com/evenicoulddoit/django-rest-framework-serializer-extensions

[glennmatthews]

https://www.linkedin.com/pulse/friday-quick-tip-dynamic-depth-serialization-django-rest-neidinger is probably relevant as far as dynamic depth based on request query parameters.
One thing we might lose in making this change (and should seek to preserve) is the ability to identify related objects by natural keys or otherwise by properties other than PK, e.g.:

POST api/dcim/devices/

{
    "name": "MyNewDevice",
    "rack": {
        "site": {
            "name": "Equinix DC6"
        },
        "name": "R204"
    },
    ...
}

instead of requiring:

POST /api/dcim/devices/

{
    "name": "MyNewDevice",
    "rack": "7f3ca431-8103-45cc-a9ce-b94c1f784a1d",
    ...
}

This is writable nested serialization, which I think could still be supported. See: https://www.django-rest-framework.org/api-guide/serializers/#writable-nested-representations

I may be misunderstanding but that seems different - that seems to be about e.g. creating the nested-serialized objects when creating the base object (although that's also a cool idea!), whereas what we have currently is the ability to reference existing nested-serialized objects by their natural key(s).

[jathanism]

https://www.linkedin.com/pulse/friday-quick-tip-dynamic-depth-serialization-django-rest-neidinger is probably relevant as far as dynamic depth based on request query parameters.
One thing we might lose in making this change (and should seek to preserve) is the ability to identify related objects by natural keys or otherwise by properties other than PK, e.g.:

POST api/dcim/devices/

{
    "name": "MyNewDevice",
    "rack": {
        "site": {
            "name": "Equinix DC6"
        },
        "name": "R204"
    },
    ...
}

instead of requiring:

POST /api/dcim/devices/

{
    "name": "MyNewDevice",
    "rack": "7f3ca431-8103-45cc-a9ce-b94c1f784a1d",
    ...
}

This is writable nested serialization, which I think could still be supported. See: https://www.django-rest-framework.org/api-guide/serializers/#writable-nested-representations

I may be misunderstanding but that seems different - that seems to be about e.g. creating the nested-serialized objects when creating the base object (although that's also a cool idea!), whereas what we have currently is the ability to reference existing nested-serialized objects by their natural key(s).

We can do both in fact. Being able to support nested retrieval and updates using either primary keys, natural keys, or nested representations.

[bryanculver]

Done with #3500