Annotate related objects counts on a REST API queryset only when the field is active on the serializer

[jeremystretch]

NetBox version

v3.7.2

Feature type

New functionality

Proposed functionality

Many REST API endpoints annotate the count of certain related objects. For example, the sites endpoint includes counts for assigned racks, devices, racks, etc.:

{
    ...
    "circuit_count": 1,
    "device_count": 2,
    "prefix_count": 0,
    "rack_count": 1,
    "virtualmachine_count": 0,
    "vlan_count": 0
}

These values are annotated on the queryset statically under the view class:

class SiteViewSet(NetBoxModelViewSet):
    queryset = Site.objects.annotate(
        device_count=count_related(Device, 'site'),
        rack_count=count_related(Rack, 'site'),
        prefix_count=count_related(Prefix, 'site'),
        vlan_count=count_related(VLAN, 'site'),
        circuit_count=count_related(Circuit, 'terminations__site'),
        virtualmachine_count=count_related(VirtualMachine, 'cluster__site')
    )

This issue proposes moving these annotations to the serializer class, effected as discrete fields which can be optionally included or omitted per the new functionality being introduced under #15087. (The current counts are retained by default to preserve existing behavior, but are omitted when the API client has requested a specific subset of fields which does not include the related object counts.)

This will require the introduction of a new field type which can be employed to automatically attach an annotation to the underlying queryset (similar to how we prefetch only relevant fields under #15087). I imagine something like this:

class SiteSerializer(NetBoxModelSerializer):
    ...
    rack_count = ObjectCountField(model='dcim.Rack', related_field='site')
    device_count = ObjectCountField(model='dcim.Device', related_field='site')
    circuit_count = ObjectCountField(model='circuits.Circuit', related_field='terminations__site')

The view class will likely need to be extended to automatically annotate the queryset for ObjectCountFields which are included for the request.

Use case

Moving these annotations to a serializer class and annotating their values only when necessary will greatly optimize the underlying querysets.

For instance, the devices list currently annotates counts for six different related object types: racks, devices, prefixes, VLANs, circuits, and virtual machines. These annotations are always computed and attached to the API results, even if their respective values are not conveyed in the response (due to the use of brief mode or because a subset of fields was requested). This can lead to significant unnecessary overhead when dealing with large numbers of objects.

Database changes

None; this change does not impact the underlying database schema in anyway, just the manner in which object relations are queried.

External dependencies

None

[jeremystretch]

Tagging this for v4.0 as it will provide ancillary benefit to FR #15087