Skip to content

Improve the test coverage for trainable_backbone_layers of object detection models #4592

@datumbox

Description

@datumbox

🚀 The feature

The trainable_backbone_layers parameter supported by all the Object Detection factory methods is not covered properly by unit-tests.

We should improve the coverage by testing the various permitted values of each model. One idea would be to test the number of trainable parameters for all possible values:

max_trainable=5
n_trainable_params = []
for trainable_layers in range(0, max_trainable+1):
    model = torchvision.models.detection.ssd300_vgg16(pretrained=False,
            pretrained_backbone=False,
            trainable_backbone_layers=trainable_layers)

    n_trainable_params.append(len([p for p in model.parameters() if p.requires_grad]))

assert n_trainable_params == [45, 51, 57, 63, 67, 71]

Unfortunately the above generates as many models as the number of permitted values so it is slow, so it's worth seeking alternative ways to test this.

All detection mode building methods need to be covered.

  • fasterrcnn_resnet50_fpn (max_trainable==5)
  • fasterrcnn_mobilenet_v3_large_320_fpn (max_trainable==6)
  • fasterrcnn_mobilenet_v3_large_fpn (max_trainable==6)
  • maskrcnn_resnet50_fpn (max_trainable==5)
  • keypointrcnn_resnet50_fpn (max_trainable==5)
  • retinanet_resnet50_fpn (max_trainable==5)
  • ssd300_vgg16 (max_trainable==5)
  • ssdlite320_mobilenet_v3_large (max_trainable==6)

Motivation, pitch

The lack of proper testing led to undetected bugs in the past. See #4548

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions