"training" parameter missing from "meta" section of config file cifar10_snn.yaml

[priyamdey]

Hi! I was trying to finetune a pretrained model for cifar10 subsets. Regarding this, I have got 2 questions:

1. In the file "src/fine_tune.py", we are importing "training" parameter from "meta" section of args (line no. 65). But this parameter is missing from the config file "configs/paws/cifar10_snn.yaml". Am I referring to the wrong config file here?
2. Why have we kept training=True while creating validation dataset of cifar10 (line no. 154 in src/fine_tune.py)?

Thanks!

[MidoAssran]

Hi @priyamdey,

1. The config file is correct, but the cifar10 models are fine-tuned with src/snn_fine_tune.py and evaluated with snn_eval.py
2. This is just a subtlety of the variable naming, since the validation set is a subset of the train set. This line here tells the model to use the validation set, not the train set.

[priyamdey]

Ah ok. Thanks for the clarification!

[priyamdey]

Sorry, maybe I got a bit confused here. In the appendix of the paper, under the section of "Fine-tuning details", the following is mentioned:
Following [1], we fine-tune a linear classifier from the first layer of the projection head in the pretrained encoder fθ, and initialize the weights of the linear classifier to zero. Specifically, we simultaneously fine-tune the encoder/classifier weights by optimizing a supervised cross-entropy loss on the small set of available labeled samples.

As you mentioned out that for fine-tuning, src/snn_fine_tune.py script is used. However, I see suncet loss and 3-layer MLP as the projection head being used in that script. src/fine_tune.py has the right loss (CE) and MLP (1-layer followed by linear clf) based on what is mentioned in the appendix.

[MidoAssran]

Yes that's true for ImageNet. But for CIFAR10 we just do nearest-neighbours classification/fine-tuning (see appendix C)

[priyamdey]

Oh I see. I missed that. Now it's clear. Thank you for pointing that out!