Any plans to support anchor distillation

[yarkable]

Describe the feature

Distill the anchor distribution.

Motivation

Not only distill nn.Module features, but also detection results, e.g., scores of positive and negative anchors.

Ex1. AAAI 2022 Anchor Distillation

[pppppM]

Hi, @yarkable , this is a good issue.
In mmrazor, you can not only add hooks to nn.Module, but also add hooks to ordinary python functions. Adding hooks to ordinary python functions is achieved through a context manager, which can be found in BaseDistiller.

There is an example to align the student's _preprocess to teacher in YOLOX.
You can modify align_methods according to your needs, such as aligning the result of label assign or the output of other python function.

distiller=dict(
    type='SingleTeacherDistiller',
    teacher=teacher,
    teacher_trainable=False,
    align_methods=[
        dict(method='YOLOX._preprocess', import_module='mmdet.models')
    ],
    components=[
        ...
    ]
)

[pppppM]

Because the paper seems not to release code, I'm unsure if this context manager can fully meet your needs.
If you are interested in reproducing this paper, welcome to make a PR, and we can improve the context manager together. @yarkable

[yarkable]

OK, thanks for your reply. I'll check it.