Clarify documentation/defaults for trainable_backbone_layers

[ethanwhite]

📚 The doc issue

The defaults for trainable_backbone_layers are currently set to None (used in faster_rcnn, keypoint_rcnn, mask_rcnn, and retinanet) and the description in Args: is a version of:

trainable_backbone_layers (int): number of trainable (not frozen) resnet layers starting from final block.
   Valid values are between 0 and 5, with 5 meaning all backbone layers are trainable.

This makes it unclear what value is used in the default case. One reasonable reading of this is that none of the backbone layers are trained. The actual value used if this optional argument isn't change is 3, which is set by _validate_trainable_layers, and can only be determined by reading the source.

Suggest a potential alternative/fix

Either the defaults should be changed to 3 to clearly indicate the default behavior or the docstrings in faster_rcnn, keypoint_rcnn, mask_rcnn, and retinanet should be updated to something like:

trainable_backbone_layers (int): number of trainable (not frozen) resnet layers starting from final block.
   Valid values are between 0 and 5, with 5 meaning all backbone layers are trainable. 3 is used if this argument is not set.

I'd be happy to submit a PR with either of these approaches, but wanted to check if this is useful and which approach is preferred before proceeding.

[datumbox]

@ethanwhite Thanks for bringing this up.

This specific parameter has been a source of confusion for quite a few users. There is also a related issue with the proposal of replacing the default None values with actual numerics at #4625. Note that this will have to be done in a BC way (continue to support None) because it can affect users who make use of this "feature".

Our reference script actually make use of it:

vision/references/detection/train.py

Lines 115 to 117 in 3300692

	parser.add_argument(
	"--trainable-backbone-layers", default=None, type=int, help="number of trainable layers of backbone"
	)

Supporting Nones allows us to have different defaults for different models (SSD doesn't use 3 for example). If the user doesn't specify the argument, None is assumed and then the builder method is left to decide its value. Of course this could have been developed in a different way (don't pass the keyword argument if None) but it was chosen because the specific idiom of setting defaults to None and later overwriting them has been used in TorchVision for a while (though have been efforts to move away from it lately).

Given there is a separate issue related to general refactoring, I would recommend for the time being to update the documentation for every model and provide which value is assumed if None is passed. Please note that in addition to the model families you listed, the ssd models also use the same logic/method bu have different defaults. A PR that updates the documentation is welcome, please tag me once you send it to help you on the review.