knn_eval() with resnet50 has missing keys in state_dict

[mgpadalkar]

While the fc layer is not needed when extracting features from ResNet50, the following command

$ python eval_knn.py --dump_features resnet50_features --arch resnet50 --data_path imagenet1k_folder

generates this error:

RuntimeError: Error(s) in loading state_dict for ResNet:
        Missing key(s) in state_dict: "fc.weight", "fc.bias".

Here is the complete output:

Will run the code on one GPU.
| distributed init (rank 0): env://
fatal: Not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
git:
  sha: N/A, status: clean, branch: N/A

arch: resnet50
batch_size_per_gpu: 128
checkpoint_key: teacher
data_path: imagenet1k_folder
dist_url: env://
dump_features: resnet50_features
gpu: 0
load_features: None
local_rank: 0
nb_knn: [10, 20, 100, 200]
num_workers: 10
patch_size: 16
pretrained_weights: 
rank: 0
temperature: 0.07
use_cuda: True
world_size: 1
/home/user/anaconda3/envs/vissl/lib/python3.7/site-packages/torchvision/transforms/transforms.py:258: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
  "Argument interpolation should be of type InterpolationMode instead of int. "
/home/user/anaconda3/envs/vissl/lib/python3.7/site-packages/torch/utils/data/dataloader.py:477: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 8, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  cpuset_checked))
Data loaded with 1281167 train and 50000 val imgs.
Please use the `--pretrained_weights` argument to indicate the path of the checkpoint to evaluate.
Since no pretrained weights have been provided, we load the reference pretrained DINO weights.
Traceback (most recent call last):
  File "eval_knn.py", line 227, in <module>
    train_features, test_features, train_labels, test_labels = extract_feature_pipeline(args)
  File "eval_knn.py", line 70, in extract_feature_pipeline
    utils.load_pretrained_weights(model, args.pretrained_weights, args.checkpoint_key, args.arch, args.patch_size)
  File "/home/user/codes/dino-main/utils.py", line 107, in load_pretrained_weights
    model.load_state_dict(state_dict, strict=True)
  File "/home/user/anaconda3/envs/vissl/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1224, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for ResNet:
        Missing key(s) in state_dict: "fc.weight", "fc.bias". 

[woctezuma]

• Issue with dino_resnet50 #11

I wonder why this happens again.

[woctezuma]

You run:

$ python eval_knn.py --dump_features resnet50_features --arch resnet50 --data_path imagenet1k_folder

Then the error occurs at line 107:

dino/utils.py

Lines 102 to 110 in de9ee3d

	elif model_name == "resnet50":
	url = "dino_resnet50_pretrain/dino_resnet50_pretrain.pth"
	if url is not None:
	print("Since no pretrained weights have been provided, we load the reference pretrained DINO weights.")
	state_dict = torch.hub.load_state_dict_from_url(url="https://dl.fbaipublicfiles.com/dino/" + url)
	model.load_state_dict(state_dict, strict=True)
	else:
	print("There is no reference weights available for this model => We use random weights.")

You could try to fix the error by setting strict=False as in the fix for #11: 1d06521

[mgpadalkar]

Hi @woctezuma,

Thanks for a quick response. I tried your suggested solution by setting strict=False and the error is gone. However, this leads to the following unexpected result for --arch resnet50. ❌

Features are ready!
Start the k-NN classification.
10-NN classifier result: Top1: 0.1, Top5: 0.5
20-NN classifier result: Top1: 0.1, Top5: 0.5
100-NN classifier result: Top1: 0.1, Top5: 0.5
200-NN classifier result: Top1: 0.1, Top5: 0.5

Nevertheless, not changing strict=True but adding model.fc=nn.Identity() after line 103 in

dino/utils.py

Lines 102 to 107 in de9ee3d

	elif model_name == "resnet50":
	url = "dino_resnet50_pretrain/dino_resnet50_pretrain.pth"
	if url is not None:
	print("Since no pretrained weights have been provided, we load the reference pretrained DINO weights.")
	state_dict = torch.hub.load_state_dict_from_url(url="https://dl.fbaipublicfiles.com/dino/" + url)
	model.load_state_dict(state_dict, strict=True)

provided the expected result as follows. ✔️

Features are ready!
Start the k-NN classification.
10-NN classifier result: Top1: 67.388, Top5: 83.964
20-NN classifier result: Top1: 67.49, Top5: 86.004
100-NN classifier result: Top1: 65.642, Top5: 87.74
200-NN classifier result: Top1: 64.266, Top5: 87.476

[woctezuma]

Good job! :D

You get Top1 67.49% with 20-NN, which is as reported in Table 2 in the article.

[mgpadalkar]

Updated this change to my fork.

[mathildecaron31]

It should be fixed now 9bebc3f