It might use unlabeled data to train "fullysup"

[DoctorKey]

Thank you for your code!
When I run the code by using table-1-cifar10-4000-fullysup.yml, I find it might use unlabeled data to train "fullysup". A batch will contain labeled data and unlabeled data. Since the Wide ResNet has BN layer, it will use unlabeled data to compute the variables of BN.

[avital]

Hi @DoctorKey, thanks for the careful analysis. I believe you're right.

We used to have a "supervised_only" flag for this purpose but we removed it to simplify our data loading pipeline.

I wonder if there's a simple change that can resolve this problem, without introducing a full new data loading path...

[avital]

Happy to hear any suggestions if you have any. If not, I'll take a closer look at this, but I won't have time to do that this week.