pruning transformer-based model

[huanghmingyue]

Hi. Thank you for your contribution.
Can OTO be used for pruning Transformers? I want to apply it to a DETR-like object detection model(DINO:https://github.com/IDEA-Research/DINO). Is this feasible?
I tried
oto = OTO(model, example_inputs)
and got this error:

[iamanigeeit]

@huanghmingyue I think you are using a very old version of pytorch. Try upgrading python and pytorch.

[tianyic]

@huanghmingyue I agreed with @iamanigeeit that you need to consider use a newer pytorch for getting tracing graph. Transformer is supported such as bert and LLM. But some new transformers may require implement some unseen operators.