Exploring vit features_only via new forward_intermediates() API, inspired by #2131

[rwightman]

No description provided.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[rwightman]

design changed a bit,

making the focal method forward_intermediates() ... it's a variant of forward_features() that focuses on collecting relevant intermediates and processing them for consisten NCHW spatial feature output.

Instead of trying to build this in to foward_features() I opted to keep it separate for now, to lower risk of regressions and prevent some complexities with torchscript / typing, etc.

FeatureGetterNet is a new feature extraction wrapper that uses forward_intermediates to return feature maps using the features_only=True attribute. This wrapper calls forward_intermediates() with the features_only=True arg to just return the intermediate features.

Models that have forward_intermediates() can also be used as below, splitting forward to extract both the intermediates and get the class predictions.

final_feat, intermediates = model.forward_intermediates(input) 
output = model.forward_head(final_feat)  # pooling + classifier head

[isaaccorley]

Looking forward to testing this!