A question about pretrained model

[galaxyWisdom]

hi, I have a question that if I want to extract the feature in pretrained model,should I only to inference the encoder with all mask=0?

[pengzhiliang]

It's a good question, and I will test it later.

[pengzhiliang]

I have test it. You can set the --mask_ratio=0. in the command.
But you will get an ERROR when model calculates the mse loss.
To avoid it, you can add a evaluate_one_epoch fuction in engine_for_pretraining.py to extract the feature of pretrained model. And maybe def evaluate(data_loader, model, device) can help you.